Data Engineer Pune · Full Time · Hybrid
About the Role This role involves developing and maintaining data pipelines, ensuring data quality, and optimizing performance. The successful candidate will work on Azure / Microsoft Fabric, implementing data ingestion strategies and applying dimensional data modeling. The goal is to ensure reliability, scalability, and efficiency in data processing.
Key Responsibilities
- Develop and maintain data pipelines (ETL/ELT) on Azure / Microsoft Fabric, ensuring reliability and scalability.
- Implement data ingestion strategies (full, incremental, snapshot) with proper error handling, logging, and retries.
- Write and optimize complex SQL queries for efficient data processing and performance.
- Apply dimensional data modeling (Star/Snowflake) and understand Medallion architecture (Bronze–Silver–Gold) for structured data flows.
- Ensure data quality through validations, checks, and basic anomaly detection mechanisms.
Requirements
- Have hands-on experience with Datawarehouse systems using Azure / Data Fabric data platform.
- Understand Data Warehouse Architecture (Medallion: Bronze–Silver–Gold).
- Design ETL/ELT pipeline (error handling, logging, retry, incremental loads).
- Implement data ingestion strategies (full load, delta, snapshot).
- Manage metadata (timestamps, source tagging, lineage).
- Apply data modeling (Dimensional modeling – Star/Snowflake, Kimball).
- Perform data transformation & conformance (deduplication, standardization, business rules).
- Utilize SQL Server (advanced querying, performance tuning).
- Implement data quality frameworks (validation, quality gates, anomaly detection).
- Optimize performance (query optimization, pipeline efficiency).
- Have basic PowerBI skills.
- Understand governance frameworks (naming standards, reusable models, KPI standardization).
Good to Have
- Advanced Power BI skills.
- CI/CD for data pipelines.
- Oracle Primavera P6.
