Python Web Crawling/Scraping

  • Introduction of Python
  • Introduction of Interpreter and Compiler language
  • Downloading & Installation of Python and Visual Studio IDE.
  • Setting up path.
  • Use of Print function, Escape Sequence Character.
  • Comments, Multiline Comments.
  • Python Keywords.

Variable & Data Type

  • Introduction of Variables.
  • Naming of Variable.
  • Typecasting in Python.
  • Swapping variables.
  • Re-declaring variable.
  • Delete Variable.
  • Introduction of Data Type.
  • Learn about SET.

Operator in Python

  • Arithmetic Operators
  • Comparison (Relational) Operators
  • Assignment Operators
  • Logical Operators
  • Bitwise Operators
  • Membership Operators
  • Identity Operators

Conditional logic in python

  • Use of if, else, elif condition in python.
  • Use of AND or OR in python.

Loops in Python

  • Use of While loop
  • Use of For loop.
  • Use Range function.
  • Nested loop.
  • Use of Break and Continue with loop

Working on String Data Type

  • Operations on String
  • Slicing String
  • Ord() and chr() Functions.
  • String Function in Python.
  • In and not in Operators.
  • String Comparison.
  • Use FString.
  • Iterating string using for loop.

Working on Number

  • Number types (int, float)
  • Number type conversion.
  • Built- in function on number type.
  • Built- in function on string type.

Working on List

  • Creating List in Python.
  • Accessing elements from List.
  • Use List method.
  • Use for loop in list.
  • List Slicing.
  • Operators in list.
  • In or not in Operators in List.
  • List Comprehension.

Working on Dictionary

  • Creating Dictionary in Python.
  • Retrieving, modifying, adding elements in Dictionary.
  • Looing in Dictionary Element.
  • Use Dictionary method.
  • Dictionary Comprehension.

Working on Tuples

  • Creating tuples.
  • Use Function in Tuples.
  • Iterating through tuples.
  • Slicing Tuples.
  • In or not in Operators in Tuples.

Working on Set

  • Creating Set.
  • Use Set Method.
  • Use loop in Set.

Function in Python

  • Creating Function.
  • Function with return.
  • Use of Global, Local and Nonlocal Variable in function.
  • Creating Function with Parameters.
  • Return with Multiple values.
  • Creating Recursive function.
  • Working on Nested function.

Args and Kwargs in Python

  • Passing parameter as *args and **kwargs in function.

Anonymous/Lambda Function

  • Use lambda function in python.
  • Use lambda function in Map or Filter function.

Built-in function in Python

  • Map
  • Filter
  • Enumerate
  • Next

Python Files I/O

  • Reading and Writing Files
  • Managing Directories and Files
  • The OS Module
  • Copying Files with Python
  • Renaming Files with Python
  • Zipping Files with Python
  • Working on CSV library
  • Writing data in csv file
  • Accessing data from csv

Error Handling in Python

  • Use of Try and Except in Python
  • Errors and Exceptions
  • Exception Handling

Date Time Module

  • Use of strftime
  • Use of datetime function
  • Formatting of date time

Advance Topic in Python

  • Python Iterator
  • Python Generator
  • Python Decorator
  • Python Regex

OOP in Python

  • What is a class?
  • Object and object instantiation
  • Class constructor
  • Class methods
  • Inheritance in Python Class
  • Encapsulation
  • Polymorphism
  • Data abstraction

Modules and Packages

  • Understanding about Python module
  • Understanding about Python packages
  • Import functions and classes from module and packages.

Installing Packages with pip

  • Installing and uninstalling of python packages with pip.
  • Installing specific version of library

Web Crawling/Scraping

  • Understanding of Robots.txt and Terms & Condition of the websites.
  • Understanding of HTML Page.
  • Inspect the elements from Chrome/Firefox developer
  • Understand the Chrome/Firefox developer options.
  • Understanding the html tags, class name, id, xpath, css selector.
  • Creating the xpath and Css Selector
  • Understanding of Request and Post Method.

Web Crawling/Scraping with Selenium

  • Web crawling with Selenium Webdriver.
  • Fetch the attributes of element
  • Fetch data from the website
  • Select value from dropdown
  • Infinite Scroll/Pagination of the web page (Window, Container)
  • Click on the website
  • Get Href/URL/Link/Hyperlink from the website.
  • Execute JavaScript code in python.
  • Upload file into the website
  • Download the files
  • Print webpage as pdf.
  • Pull data in excel, csv using pandas, csv library.

Create undetected crawler in Selenium

  • Open chrome browser in localhost
  • Use existing browser in selenium code
  • Crawl data from already opened browser

Web Scraping – BeautifulSoup Python

  • Web crawling with BeautifulSoup.
  • Fetch the attributes of element
  • Use Request method in BeautifulSoup
  • Pull data in excel, csv using pandas, csv library.
  • Use API.
  • Read data from xml, json format and pull into csv, and excel.

Web Scraping – Scrapy Python

  • Web crawling with Scrapy.
  • Create spider in scrapy.
  • How to use css selector and xpath in scrapy.
  • Fetch the attributes of element
  • Pull data in excel, csv using pandas, csv library.
  • Use API.
  • Read data from xml and json format. 

Web Scraping Projects

  • Google Maps – fetch google review.
  • Ecommerce website – fetch data from ecommerce website eg. Amazon, Flipkart.