Role description
Job Role: Azure Data Engineer
Job Location: Any UST Location
Notice Period: Immediate Joiners Only
Experience Required: 5+ years
Job Summary
We are seeking a highly skilled Azure Databricks Data Engineer with strong experience in designing, developing, and optimizing scalable data integration and analytics solutions on the Microsoft Azure platform. The ideal candidate will have expertise in Databricks, PySpark, Spark SQL, Python, SQL, Azure Data Factory (ADF), and Azure data services, along with a solid understanding of modern data architectures, data warehousing, and cloud-based ETL/ELT frameworks.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks, PySpark, Spark SQL, Python, SQL, and Azure Data Factory (ADF).
- Build and optimize data integration solutions for batch and real-time data processing across enterprise data platforms.
- Develop and manage data ingestion, transformation, cleansing, enrichment, and orchestration processes from multiple structured and unstructured data sources.
- Implement and support modern Lakehouse architectures leveraging Delta Lake and Azure data services.
- Design and maintain enterprise-grade data models, data warehouses, and analytical data structures.
- Utilize Azure services including Azure Data Lake Storage (ADLS), Azure Synapse Analytics, Azure Functions, Event Hub, and related Azure components.
- Collaborate with Data Architects, Business Analysts, QA teams, and business stakeholders to translate business requirements into scalable technical solutions.
- Ensure data quality, governance, security, compliance, and metadata management best practices are followed across data platforms.
- Monitor, troubleshoot, and optimize Spark jobs, SQL queries, data pipelines, and cloud-based workloads for performance and cost efficiency.
- Develop reusable frameworks, automation scripts, and best practices to improve platform reliability and operational efficiency.
- Implement CI/CD pipelines and deployment automation using Git, Azure DevOps, and DevOps best practices.
- Participate in Agile ceremonies including sprint planning, backlog grooming, code reviews, and release activities.
- Support enterprise reporting, business intelligence, advanced analytics, and data transformation initiatives.
Required Skills & Qualifications
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
- 4+ years of experience in Data Engineering, ETL/ELT development, and cloud-based data platforms.
- Hands-on expertise in Azure Databricks, PySpark, Spark SQL, Python, SQL, and Azure Data Factory (ADF).
- Experience with Azure Data Lake Storage (ADLS), Azure Synapse Analytics, Azure Functions, and Event Hub.
- Strong understanding of Data Warehousing, Lakehouse Architecture, Delta Lake, Data Modeling, Data Governance, and Data Quality.
- Experience in performance tuning, troubleshooting, and optimizing data pipelines and Spark workloads.
- Familiarity with Git, Azure DevOps, and CI/CD practices.
- Strong analytical, problem-solving, communication, and stakeholder management skills.
- Experience working in Agile development environments.
Skills
Data Engineering, Microsoft Azure Services, Cloud Computing, Data Processing
About UST
UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.
