Experience Required: 2+ Years
Work Mode: On-site / WFO
Employment Type: Full-time
6 day work-week (Sat WFH usually)
Role Overview
We need a developer experienced in e-commerce data extraction to build and maintain crawlers that collect product-level data at scale from online retail websites. The role focuses on accurate category mapping, pricing extraction, and handling modern front-end challenges (SPA, lazy loading, anti-bot).
Key Responsibilities
Build and maintain web crawlers for e-commerce websites using Python and/or VBA
Extract structured product data
Handle category listing pages + product detail pages (PDPs)
Work with dynamic sites (React, Angular, AJAX calls)
Identify and use network APIs (XHR/fetch) instead of DOM scraping where possible
Export clean datasets into CSV / TSV / Excel / JSON feeds
Ensure crawlers are resilient to frequent DOM/UI changes
Optimize scraping speed while avoiding blocks (rate limiting, retries)
Required Skills
Strong experience in Python scraping (mandatory) and/or VBA automation
Hands-on with:
o
Selenium / Playwright (for dynamic pages)
o
BeautifulSoup / Requests (for static/API scraping)
Strong understanding of:
o
E-commerce page structures (PLP vs PDP)
o
XPath / CSS selectors / regex
Familiarity with:
o
Lazy loading, infinite scroll, variant selection logic
Preferred Skills
Experience with Scrapy or Playwright advanced features
Knowledge of affiliate/product feed structures (like Admitad, CJ, etc.)
Experience with proxy rotation, headers spoofing, session persistence
Familiarity with Excel-based scraping workflows (VBA macros)
Exposure to data pipelines or feed generation systems
Pay: Up to ₹800,000.00 per year
Application Question(s):
- Open to a 6 day work week ?
Experience:
- Web Crawling/Scraping: 2 years (Preferred)
Work Location: In person
