Job Description:
Experience (6 - 9 years)
We are seeking a highly skilled Data Engineer with a strong background , Databricks, Python, Microsoft Fabric, Azure Data Stack, and SQL . The ideal candidate will have extensive experience in automating data pipelines and managing complex data integration scenarios using Fabric/Databricks and is known for taking initiative and delivering top-quality work and documentation.
Key Responsibilities:
Develop and maintain scalable data pipelines and data integrations to support continuing increases in data volume and complexity.
Utilize Databricks, Python, SQL for data extraction, transformation, and loading (ETL) in complex scenarios.
Collaborate with stakeholders and agile teams to understand and address complex data requirements.
Deliver top-quality work and documentation, demonstrating a strong attention to detail and a commitment to accuracy and clarity.
Implement robust measures to ensure data accuracy, quality, and accessibility.
Monitor data performance, identify bottlenecks, and modify infrastructure as needed.
Work at a high pace in an agile team environment, taking initiative to solve problems and improve systems.
Technical Skills:
Microsoft Fabric
Azure Data Factory
Azure Databricks
Azure Functions
Azure DevOps including CI/CD pipelines
Good understanding of Infrastructure as Code
Data modelling
SQL
