Job Description – Data Engineer (Microsoft Fabric)
Role: Data Engineer – Microsoft Fabric
Experience: 8 Years
Start Date: 13th July
Duration: 3 Months (Contract)
Work Mode: Remote
Work Timings: 1:30 PM – 10:30 PM (IST)
Job Summary
We are seeking an experienced Data Engineer with Microsoft Fabric expertise to design, develop, and optimize scalable data engineering solutions. The ideal candidate should have strong hands-on experience in building modern ETL/ELT pipelines, implementing Medallion Architecture (Bronze, Silver, Gold), and working with PySpark, SQL, and Microsoft Fabric to deliver production-grade data solutions.
Key Responsibilities
- Perform data analysis to understand source systems and business requirements.
- Translate business requirements into scalable data models and pipeline designs.
- Design, develop, and maintain data ingestion pipelines from multiple source systems into Microsoft Fabric.
- Build batch and incremental (delta) data ingestion pipelines.
- Develop scripts to extract data from databases, flat files, and APIs.
- Ingest and land raw data into the Bronze layer, ensuring reliability, scalability, and traceability.
- Transform raw data into standardized Silver layer datasets through data cleansing, validation, deduplication, and conformance.
- Design and develop Gold layer data models, including fact tables, dimension tables, and aggregated datasets for reporting and analytics.
- Design end-to-end ETL/ELT pipelines following Medallion Architecture best practices.
- Support the development of semantic models and reporting-ready datasets.
- Implement data quality checks to ensure completeness, accuracy, freshness, and reconciliation of data.
- Develop robust error handling mechanisms including logging, retry logic, exception handling, and alerting.
- Build self-healing data pipelines with checkpointing, idempotent processing, auto-retry, and failure recovery mechanisms.
- Optimize pipeline performance to ensure efficient and reliable production data processing.
- Monitor data pipelines and proactively troubleshoot failures or performance bottlenecks.
- Implement CI/CD and DevOps practices for deployment across Development, Testing, and Production environments.
- Write, modify, and debug scripts using PySpark and SQL.
- Follow software engineering best practices, coding standards, and design principles.
- Collaborate with business stakeholders, BI teams, architects, and cross-functional teams to deliver end-to-end data solutions.
Required Skills
- 8 years of experience in Data Engineering.
- 2-4 years of Strong hands-on experience with Microsoft Fabric .
- Experience in designing and implementing ETL/ELT pipelines.
- Strong knowledge of Medallion Architecture (Bronze, Silver, Gold).
- Proficiency in PySpark and SQL.
- Experience working with multiple data sources, including databases, APIs, and flat files.
- Strong understanding of data modeling concepts (Fact & Dimension modeling).
- Experience implementing data quality, validation, and reconciliation processes.
- Knowledge of pipeline monitoring, logging, and performance optimization.
- Experience with CI/CD pipelines and DevOps practices.
- Strong debugging, troubleshooting, and problem-solving skills.
- Good understanding of software design standards and best coding practices.
- Excellent communication and stakeholder management skills.
Preferred Skills
- Experience with Microsoft Fabric Lakehouse and Data Factory.
- Exposure to semantic models and analytics-ready datasets.
- Experience working in enterprise-scale cloud data platforms.
- Familiarity with Agile development methodologies.
Pay: ₹80,000.00 - ₹100,000.00 per month
Work Location: Remote
