Job Overview:
We are looking for a Senior Data Engineer with strong expertise in PySpark, Apache Spark, and Google Cloud Dataproc to build and optimize scalable data processing pipelines.
Key Responsibilities:
- Develop and maintain large-scale ETL/ELT pipelines using PySpark and Spark.
- Build and optimize data processing jobs on GCP Dataproc.
- Work with BigQuery, GCS, Cloud Composer/Airflow, and other GCP services.
- Optimize Spark jobs for performance and scalability.
- Troubleshoot production issues and ensure data quality and reliability.
Required Skills:
- 5–10 years of experience in Data Engineering/Big Data.
- Strong hands-on experience with PySpark, Spark, Python, and SQL.
- Strong experience with Google Cloud Dataproc and GCP.
- Good understanding of Spark architecture and performance tuning.
- Experience with ETL/ELT pipelines and cloud-based data platforms.
Pay: ₹100,000.00 - ₹120,000.00 per month
Benefits:
Work Location: In person
