Location: Bengaluru, Kolkata, Gurugram, Pune,
Notice period: Immediate to Serving
Experience: 6-12 years
Job Summary
We are looking for an experienced AWS Data Engineer with expertise in building scalable data pipelines and modern data lake solutions on AWS. The ideal candidate should have hands-on experience with Amazon EMR, Apache Iceberg, and AWS data services to design, develop, and optimize batch and streaming data processing solutions.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines on AWS.
- Build and optimize ETL/ELT workflows using Python, PySpark, and Apache Spark.
- Develop and manage data lake solutions using Apache Iceberg on Amazon S3.
- Configure and manage Amazon EMR clusters for large-scale data processing.
- Process streaming data using Amazon Kinesis (preferred).
- Work with AWS services such as Glue, Athena, Redshift, Lambda, and S3.
- Optimize data processing performance, partitioning, and query execution.
- Ensure data quality, reliability, security, and governance.
- Collaborate with data analysts, data scientists, and application teams to deliver business requirements.
- Monitor, troubleshoot, and enhance existing data pipelines.
Required Skills
- 6+ years of experience as a Data Engineer.
- Strong programming skills in Python.
- Hands-on experience with Apache Spark / PySpark.
- Experience with Amazon EMR.
- Strong knowledge of Apache Iceberg.
- Experience with Amazon S3 and AWS data services.
- Good understanding of SQL and relational databases.
- Experience with ETL/ELT pipeline development.
- Familiarity with Git and CI/CD practices.
Preferred Skills
- Experience with Amazon Kinesis.
- Knowledge of AWS Glue, Athena, Redshift, and Lambda.
- Experience with orchestration tools such as Apache Airflow or Amazon MWAA.
- Knowledge of data lake and lakehouse architectures.
- Understanding of distributed data processing and performance tuning.
Pay: ₹2,000,000.00 - ₹2,300,000.00 per year
Work Location: Hybrid remote in Bengaluru, Karnataka (Bengaluru, Bengaluru Urban District)
