Experience
* 4+ years of professional experience as a Python Developer
* Strong experience building production-grade web scraping systems
---
## Key Responsibilities
* Design, develop, and maintain scalable web scraping and data extraction pipelines.
* Build robust scrapers capable of handling dynamic websites, anti-bot protections, CAPTCHAs, and rate limiting.
* Develop high-performance distributed scraping infrastructure using Docker and Kubernetes.
* Optimize scraper performance, reliability, and fault tolerance for large-scale deployments.
* Build and maintain APIs using Python web frameworks such as FastAPI or Django.
* Develop asynchronous and distributed task processing pipelines using Celery.
* Monitor production systems using observability tools such as Grafana and Prometheus.
* Collaborate with engineering teams to build resilient, maintainable, and scalable data collection platforms.
* Continuously research and adopt new scraping techniques, browser automation methods, and AI-powered extraction technologies.
* Learn and contribute to AI/ML and Agentic AI initiatives for intelligent web automation.
---
## Required Skills
* Strong Python programming skills with solid understanding of:
* Object-Oriented Programming (OOP)
* Python best practices
* Clean code principles
* Extensive experience with:
* Selenium
* Undetected ChromeDriver (UC)
* Requests
* CAPTCHA solving techniques
* Browser automation
* Strong understanding of modern anti-bot mechanisms and bypass strategies.
* Experience working with Docker.
* Experience deploying applications on Kubernetes.
* Hands-on experience with Celery.
* Experience with at least one Python web framework:
* FastAPI
* Django
* Familiarity with monitoring and observability tools such as Grafana and Prometheus.
* Experience with Git and Linux environments.
---
## Preferred Qualifications
Candidates will receive higher priority if they have experience with:
* Building large-scale distributed scraping infrastructure.
* Kubernetes administration and orchestration.
* Social media scraping or automation/bot development.
* AI-assisted or agentic web scraping systems.
* Proxy management, browser fingerprinting, and anti-detection techniques.
* Message queues, distributed workers, and scalable backend architectures.
---
## What We Offer
* Opportunity to work on large-scale cyber intelligence products.
* Exposure to cutting-edge AI and Agentic AI technologies.
* Challenging engineering problems involving distributed systems and web automation.
* Collaborative engineering culture with opportunities for continuous learning and innovation.
MUST HAVE :-
- Python scraping
- Celery
- Docker
- Good communication
Pay: Up to ₹2,000,000.00 per year
Work Location: In person
