Skill : Python, Pyspark, AWS
Exp : 6 to 12 years
Location : Pune
Job Summary
We are seeking a skilled Python + PySpark + AWS Developer to design, develop, and maintain scalable data engineering solutions. The role involves building high-performance ETL pipelines, processing large datasets, and integrating with AWS cloud services to support enterprise analytics and reporting needs.
Key Responsibilities
Design, develop, and optimize ETL pipelines using PySpark and Python .
Work with large-scale datasets to perform data cleansing, transformation, and aggregation.
Integrate with AWS services such as S3, Glue, EMR, Lambda, Redshift, Athena, Kinesis .
Implement data lake and data warehouse solutions on AWS.
Ensure data quality, consistency, and security across all processing stages.
Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions.
Monitor, troubleshoot, and optimize data workflows for performance and cost efficiency.
Follow best practices for code versioning, CI/CD, and cloud resource management.
Write unit tests and maintain documentation for all developed solutions.
Required Skills & Qualifications
Strong programming skills in Python (data manipulation, APIs, automation).
Hands-on experience with PySpark for distributed data processing.
Proficiency in AWS cloud services (S3, Glue, EMR, Lambda, Redshift, Athena, IAM).
Solid understanding of SQL and relational database concepts.
Experience with data lake and data warehouse architectures.
Knowledge of ETL/ELT design patterns and best practices.
Familiarity with Git and CI/CD tools (e.g., Jenkins, GitHub Actions).
Strong problem-solving and debugging skills.
Bachelor’s degree in Computer Science, Information Technology, or related field.
Preferred Skills
Experience with streaming data (Kafka, Kinesis).
Knowledge of Docker and containerized deployments.
Familiarity with Terraform or CloudFormation for infrastructure as code.
Exposure to machine learning pipelines in a big data environment.
Understanding of data governance and compliance standards.
Work Environment
Location: Onsite / Remote / Hybrid (as per company policy)
Team: Works closely with data engineers, analysts, and cloud architects.
Tools: AWS Cloud, Spark, Python, SQL, Git, CI/CD pipelines.
