Key Responsibilities
- Design, develop, and maintain end-to-end data pipelines using Azure services.
- Build and optimize ETL/ELT workflows for data integration and transformation.
- Work with large-scale data platforms on Azure Cloud.
- Lead development activities and provide technical guidance to team members.
- Ensure data quality, governance, security, and compliance across Azure environments.
- Collaborate with business stakeholders to gather and translate requirements into technical solutions.
- Optimize data processing performance and cost efficiency.
- Implement and maintain CI/CD pipelines using Azure DevOps.
- Monitor, troubleshoot, and support production data systems.
- Work closely with data scientists, analysts, and architects to deliver data solutions.
Mandatory Skills
- Strong hands-on experience with Python and SQL.
- Expertise in Azure Data Services:
- Azure Data Factory (ADF)
- Azure Databricks
- Azure Synapse Analytics
- Azure Data Lake Storage Gen2 (ADLS)
- Experience in ETL/ELT pipeline development.
- Strong understanding of Data Warehousing and Data Modeling concepts.
- Experience with Apache Spark and Big Data processing.
- Knowledge of Azure DevOps and CI/CD implementation.
- Experience with REST APIs and data integration patterns.
Preferred Skills
- Azure Event Hub and/or Kafka.
- Power BI reporting and visualization.
- Delta Lake and Lakehouse Architecture.
- Azure Data Engineer Certification (DP-203).
- Knowledge of Data Security and Governance on Azure.
Required Competencies
- Strong leadership and mentoring skills.
- Excellent communication and stakeholder management abilities.
- Strong analytical and problem-solving skills.
- Ability to manage multiple priorities in a fast-paced environment.
Primary Skills
Azure Data Engineering, Azure Data Factory (ADF), Azure Databricks, Python, SQL
Secondary Skills
Azure Synapse Analytics, ADLS Gen2, Apache Spark, Azure DevOps, Kafka, Event Hub, Power BI, Delta Lake.
data engineering,azure development,databricks,azure data factory,
