Data Crawling Engineer – Python (WFH)
Experience: 2–4 Years
Location: Remote (Work from Home)
Mode of Engagement: Full-time
No. of Positions: 2–4
Educational Qualification: Bachelor's degree in Computer Science, IT, or related field
Industry: IT / Software Services / Data & AI
Notice Period: Immediate Joiners Preferred
What We Are Looking For
- Strong hands-on experience in Python-based data crawling, data extraction, and web automation using Requests, Scrapy, Selenium, and Playwright.
- Experience working with JavaScript-heavy and dynamic websites using Selenium, Playwright, or similar browser automation frameworks.
- Good understanding of HTTP protocols, cookies, sessions, headers, tokens, authentication, and browser storage.
- Experience handling pagination, APIs, AJAX requests, and structured data extraction from websites.
- Knowledge of proxy integration, IP rotation, rate limiting, retries, and anti-bot handling.
- Experience working with SQL/NoSQL databases for storing and managing crawled data.
- Familiarity with REST APIs, JSON, XPath, CSS Selectors, HTML parsing, and browser debugging tools.
- Strong debugging, analytical, and problem-solving skills with the ability to work independently.
- Good understanding of Git, Linux, and modern development workflows.
Responsibilities
- Develop and maintain enterprise-grade data crawling and extraction pipelines using Python.
- Build reliable crawlers using Requests, Scrapy, Selenium, and Playwright.
- Crawl and extract structured data from static and dynamic websites.
- Handle authenticated crawling workflows using cookies, sessions, tokens, and browser automation.
- Optimize crawling performance while ensuring data quality and reliability.
- Store, validate, and transform extracted data for downstream systems.
- Troubleshoot crawling failures and improve scraper stability.
- Collaborate with AI, Data Engineering, and Product teams to deliver high-quality datasets.
- Follow coding standards, documentation, and engineering best practices.
Qualifications
- 2–4 years of hands-on experience in Python-based Data Crawling and Web Automation.
- Strong experience with Requests, Scrapy, Selenium, and/or Playwright.
- Experience crawling JavaScript-heavy and dynamic websites.
- Good knowledge of SQL/NoSQL databases.
- Understanding of proxies, browser automation, cookies, sessions, and HTTP protocols.
- Strong debugging and problem-solving skills.
- Experience working in Agile environments will be an added advantage.
Pay: ₹400,040.69 - ₹600,000.22 per year
Benefits:
Application Question(s):
- Years of experience in Python-based Data Crawling and Web Automation?
- Years of experience using Scrapy, Selenium, or Playwright?
- Years of experience crawling JavaScript-heavy and dynamic websites?
- Years of experience handling proxies, cookies, sessions, and browser automation?
- Years of experience working with SQL/NoSQL databases?
- Notice Period ( days )
- Current CTC ( monthly )
Work Location: Remote
