Data Engineer – ETL
Location: Remote
Experience: 7–8 Years
Employment Type: C2C / C2H
Shift: 11:00 AM – 9:00 PM
About the Role
We are hiring a skilled Data Engineer with strong expertise in ETL/ELT development, Azure Data Platform, Apache Spark, Snowflake, and AI/ML integration. The ideal candidate will have hands-on experience building scalable cloud-native data solutions, optimizing enterprise data platforms, and enabling AI/LLM-powered data engineering workflows.
Key Responsibilities
- Design, develop, and optimize scalable ETL/ELT data pipelines.
- Build robust data ingestion, transformation, and orchestration workflows on Azure.
- Develop distributed data processing solutions using PySpark and Spark SQL.
- Build and maintain enterprise data lakes and data warehouses using Azure Databricks and Snowflake.
- Implement secure data governance using Unity Catalog.
- Develop serverless solutions using Azure Functions and Azure Service Bus.
- Write and optimize complex SQL queries and database objects.
- Automate infrastructure provisioning using Terraform.
- Build and maintain CI/CD pipelines using GitHub Actions, Git, Artifactory, and SonarQube.
- Collaborate with AI/ML teams to integrate LLMs and AI models into enterprise data platforms.
- Ensure data quality, scalability, monitoring, and production support.
Required Skills
- 7–8 years of Data Engineering experience.
- Strong expertise in Python.
- Hands-on experience with Apache Spark (PySpark & Spark SQL).
- Advanced SQL programming and query optimization.
- Strong experience with:
- Azure Databricks
- Unity Catalog
- Snowflake
- Azure Functions
- Azure Service Bus
- Experience with Terraform and CI/CD tools.
- Knowledge of Git, GitHub Actions, Artifactory, and SonarQube.
- Hands-on experience integrating AI/ML solutions and Large Language Models (LLMs) into data engineering workflows.
- Experience working with Azure cloud data services and enterprise-scale data platforms.
Preferred Skills
- Azure Data Factory (ADF)
- Azure Synapse Analytics
- Delta Lake
- MLflow
- Azure DevOps
- Kafka/Event Streaming
- Docker & Kubernetes
- Microsoft Fabric
Education
Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
Pay: ₹80,000.00 - ₹110,000.00 per month
Experience:
- Data Engineer-ETL: 7 years (Required)
Work Location: Remote
