Pradumna Milind Panditrao

Web scraping: It is a data mining or data extraction process in which data from different websites is extracted and stored into a local system or database. Web scraping is also known as crawling or spidering.

Web scraping can be done with the help of programming and using small pieces of codes/functions which are known as scrapers.

Behavioral patterns offer repeatable solutions to commonly occurring problems. It involves the following methods:
Iterators
Observers
Templates
Visitors
Commands

Selenium is used for web application testing and is very useful for browsing data collections. Its major advantage is that it's free and requires no additional licenses. It is supported on all web browsers and only needs a web driver. (Google Chrome, Safari, Firefox, IE, etc.)

JSON wire protocol: It is used to transfer data between the server and the client on web. It is a REST API which transfers over an HTTP protocol.

To write software for data extraction or crawling, some basic knowledge of HTML, DOM, and other programming languages is a must.

Read through the target website's terms and conditions to understand how you can legally use the data. Most websites prohibit you from using their data for commercial purposes.
Make sure you are not downloading data at too rapid a rate because this may break the website. You could potentially get blocked from the website as well.

DOM (Document Object Model) parsing: It is an interface that is used for modifying and updating the structure and content of XML documents.

HTML parsing: In this particularly, take the code and extract relevant information as per the point of view of users.

Web scraping software: Nowadays there are many web scraping software available so directly need to provide the headers as input parameters and automatically relevant data get available in the comfortable format in CSV, JSON.