- We are looking for an experienced Azure Databricks Developer Data Engineer to design build and optimize large scale data processing solutions on the Microsoft Azure cloud
- The ideal candidate will have strong hands on experience with Azure Databricks data engineering frameworks and modern data lake architectures to support analytics and business intelligence initiatives
- Design develop and optimize data pipelines using Azure Databricks PySpark Spark SQL
- Build and manage scalable data ingestion transformation and processing workflows
- Implement ETL ELT pipelines using Databricks Azure Data Factory and related tools
- Develop data solutions using Azure Data Lake Storage ADLS Gen2
- Optimize Spark jobs for performance cost and scalability
- Implement best practices for data modeling partitioning and schema management
- Work with structured semi structured and unstructured data
- Collaborate with business analytics and downstream BI teams
- Ensure data quality reliability and monitoring of data pipelines
- Support CI CD version control and automated deployments
- Primary skills Technology Data On Cloud Platform Azure Data Lake ADL
- 5 9 years of overall experience in data engineering big data development
- 3 years of hands on experience with Azure Databricks
- Strong expertise in PySpark Spark SQL
- Experience with Microsoft Azure services including
- Azure Data Factory ADF
- Azure Data Lake Storage ADLS
- Azure SQL Synapse good to have
- Solid understanding of data lake lakehouse architectures
- Strong SQL skills for data transformation and analytics
- Experience with workflow orchestration and scheduling
- Knowledge of data warehousing and data modeling concepts
- Familiarity with Git and CI CD pipelines
- Excellent problem solving and communication skills
- Databricks Certified Data Engineer Associate Professional
- Experience with Delta Lake
- Knowledge of streaming frameworks Structured Streaming Event Hubs
- Exposure to BI tools Power BI Tableau
- Experience working in Agile Scrum teams
- Basic understanding of Python best practices and libraries
Technology->Cloud Platform->Azure Analytics Services->Azure Databricks
