Email Scraper

The challenge

The strategic use of email marketing can inform leads of new products, promotions, resources, services, and company growth. Therefore, the client wanted to generate email leads for a large set of companies and people. Generally, emails are made available in personal portfolios, and company websites, and a person can google the name of the company or person and collect the respective emails if it is present on the open web. But this manual process isn’t feasible for a huge set of leads.

The Solution

So, we took advantage of the efficient keyword searching capabilities of the google search engine and automated the process of searching for emails on google search.

A string is generated for each of the leads given. For example, for a company XYZ, the ‘emails of the company: XYZ’ string will be generated and searched on google. The Google search engine’s algorithm will go through the pages and will show the maximum number of emails that could be found on any website related to the company. The scraper will then scrape through the entire google search result and extract all the valid possible emails. The scraper rejects all invalid emails such as [email protected], and ***@xxx.com and only collects the emails which have a sensible format.

Google prevents any bot from misusing its searching service, so we have used selenium, an automated framework, to ensure the most human-like behavior possible for a bot script. We have also integrated services for solving captchas if any appear during the scraping process. All the results are then stored in a CSV file in the format required by the client.

Advantages

• Thousands of emails can be scraped within a very short period of time which can help the company increase its reach.
• Bot detection and captcha problem are solved by the scraper during the scraping process.
• A lot of time and effort is saved with this scraping tool.