Our client was looking for options to automate job listing based on keywords and organisations across different job boards, including google job search and welcometothejungle.com. The client wanted to scrap three main data points, Job title, Job URL, and date when the job was posted, from the job websites.
When the client approached us, their significant problems were, listing out the job manually from job portals according to the keywords was extensively time taking and tiring, and it required human resources exclusively for this work, which became an added cost.
So, to simplify their needs, we planned to perform web scraping on the websites mentioned by the clients in order to gather data on job listings in a simple, time and cost-efficient way. Firstly, We created a job scraper bot to perform all the manual processes from searching for the companies and keywords on the listed job portals. We also built an API, which acts as a trigger that initiates the process.
Along with that, we integrated an n8n automation tool to give the process and environment a smooth and uninterrupted run. When the client clicks start in the n8n tool, it will initiate the process, and the scrapper bot will run through the website and gather the required data.
When the scrapper set was ready, the web crawlers started providing the data in the client’s required format. If the client provides the company name and keyword, the scrapper will collect the job title, URL and data posted. If the company is not found, then it will give the result otherwise.
- It took a week to design the technical aspects and set up the web crawlers, allowing the client to gather data in a shorter time.
- Our hands-on experience in web scraping helped us design a solution that can quickly perform all the manual processes and control a vast amount of data.
- The job scrapper will be more affordable in terms of cost and time than the manual listing.