This position requires a sound knowledge on specializing in AWS EMR, AWS Glue, PySpark, Python, and SQL. Databricks and Airflow etc. Strong foundation on database concepts and SQL is also required.
Experience: Minimum 6 years
Key Result Areas and Activities:
- Technology Assessment and Design
- Documentation and Stakeholder Communication
- Process Improvement and Automation
- Training and Knowledge Sharing
Essential Skills:
- In-depth knowledge of the following AWS services: S3, EC2, EMR, Athena, AWS Glue, Lambda
- Experience with at least one MPP database: AWS Redshift, Snowflake, SingleStore
- Proficiency in Big Data technologies: Apache Spark, Databricks
- Must have strong programming skills in Python
- Responsible for building data pipelines in AWS And Databricks
- Experience with Big Data table formats, such as Delta Lake (open source)
- Must have very strong SQL skills
- Experience with orchestration tools like Apache Airflow
- Expertise in developing ETL workflows with complex transformations such as SCD, deduplications, aggregations, etc.
- Should be a quick and self-learner, ready to adapt to new AWS services or Big Data technologies as required
- Strong understanding of data warehousing concepts
Pay: ₹180,000.00 - ₹200,000.00 per month
Work Location: In person
