Job Summary
We are seeking a skilled and motivated Data Engineer to design, develop, and maintain scalable data pipelines and data solutions. The ideal candidate should have hands-on experience with modern data engineering tools and cloud platforms, enabling efficient data processing, transformation, and integration for business intelligence and analytics purposes.
Key Responsibilities
- Design, develop, and maintain robust ETL/ELT pipelines for data ingestion and transformation.
- Build and optimize scalable data processing solutions using Apache Spark and Databricks.
- Develop and manage data workflows and scheduling using Apache Airflow.
- Integrate data from multiple sources including databases, APIs, and cloud storage platforms.
- Work with Azure Data Factory (ADF), Microsoft Fabric, AWS, and Azure services to build cloud-based data solutions.
- Write complex SQL queries, stored procedures, and optimize database performance.
- Ensure data quality, integrity, security, and governance across data platforms.
- Monitor, troubleshoot, and improve data pipeline performance and reliability.
- Collaborate with Data Analysts, BI Developers, and cross-functional teams to understand data requirements.
- Implement best practices for data architecture, data modeling, and pipeline development.
Required SkillsTechnical Skills
- Strong proficiency in SQL.
- Hands-on experience with Databricks.
- Experience with Apache Spark (PySpark preferred).
- Expertise in ETL and ELT processes.
- Experience with Apache Airflow for workflow orchestration.
- Knowledge of Azure Data Factory (ADF).
- Experience with Microsoft Fabric.
- Experience working with AWS services (S3, Glue, Redshift, Lambda, etc.).
- Experience with Azure cloud services.
- Understanding of data warehousing concepts and data modeling.
- Familiarity with Git and CI/CD practices.
Pay: ₹800,000.00 - ₹1,200,000.00 per year
Work Location: In person
