Data Engineer (ETL) – Azure Databricks | Snowflake | PySparkJob Title
Data Engineer (ETL)
Experience Required
7–8 Years
Work Model
Hybrid (Pan India)
Work Timings
11:00 AM – 9:00 PM IST
Job Summary
We are looking for a highly skilled Data Engineer with strong expertise in ETL/ELT development, cloud-based data platforms, and modern data engineering practices. The ideal candidate will have hands-on experience with Azure Databricks, Snowflake, Spark, Python, and infrastructure automation. Experience working with AI/ML and Large Language Models (LLMs) will be a strong advantage.
The candidate will be responsible for designing, building, and maintaining scalable data pipelines, optimizing data processing frameworks, and enabling advanced analytics and AI-driven solutions across the organization.
Required Skills
- Data Engineering and ETL/ELT Processes
- Python
- Apache Spark (PySpark, Spark SQL)
- Azure Databricks (Unity Catalog)
- Snowflake
- Azure Functions
- Azure Service Bus
- SQL
- Terraform
- CI/CD Tools and Processes
- GitHub Actions
- Git
- Artifactory
- SonarQube
- AI/ML and Large Language Models (LLM)
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines for enterprise data platforms.
- Build and optimize data processing solutions using Python, PySpark, and Spark SQL.
- Develop and manage data solutions on Azure Databricks and Snowflake.
- Implement and maintain Unity Catalog-based governance and security frameworks.
- Create serverless integrations and event-driven workflows using Azure Functions and Azure Service Bus.
- Develop efficient SQL queries, data models, and performance optimization strategies.
- Automate infrastructure deployment and management using Terraform.
- Implement CI/CD pipelines and DevOps best practices using GitHub Actions, Git, Artifactory, and SonarQube.
- Collaborate with data architects, business analysts, and cross-functional teams to deliver data-driven solutions.
- Support AI/ML initiatives by preparing and engineering datasets for predictive analytics and LLM-based applications.
- Ensure data quality, security, governance, and operational excellence across data platforms.
Required Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- 7–8 years of hands-on experience in Data Engineering and ETL/ELT development.
- Strong expertise in Python, Spark, and SQL.
- Experience working with Azure Databricks and Snowflake in enterprise environments.
- Hands-on experience with Terraform and Infrastructure as Code (IaC).
- Strong understanding of DevOps and CI/CD methodologies.
- Experience with cloud-native services and distributed data processing systems.
- Excellent analytical, troubleshooting, and communication skills.
Preferred Qualifications
- Experience with AI/ML model deployment and MLOps practices.
- Hands-on exposure to Generative AI and Large Language Model (LLM) implementations.
- Knowledge of data governance, data security, and metadata management.
- Azure certifications or Snowflake certifications are a plus.
Key Search Keywords
ETL, ELT, Data Engineering, Python, PySpark, Spark SQL, Azure Databricks, Unity Catalog, Snowflake, Azure Functions, Azure Service Bus, SQL, Terraform, GitHub Actions, Git, Artifactory, SonarQube, CI/CD, AI/ML, Generative AI, LLM.
Pay: ₹1,000,000.00 - ₹1,500,000.00 per year
Work Location: Hybrid remote in Noida, Uttar Pradesh
