Role description
UST is looking for a highly skilled Azure Data Engineer with 5 to 10 yrs of exp to join our team and play a key role in designing and implementing efficient data solutions on the Azure cloud platform. The ideal candidate should have a strong background in data engineering, possess expertise in Azure services, and demonstrate proficiency in data processing technologies, particularly PySpark.
Responsibilities:
Data Pipeline Development:
- Design, implement, and optimize end-to-end data pipelines on Azure, focusing on scalability and performance.
- Develop and maintain ETL workflows for seamless data processing.
Azure Cloud Expertise:
- Utilize Azure services such as Azure Data Factory, Azure SQL Database, and Azure Databricks for effective data engineering.
- Implement and manage data storage solutions on Azure.
Data Transformation with PySpark:
- Leverage PySpark for advanced data transformations, ensuring high-quality and well-structured output.
- Implement data cleansing, enrichment, and validation processes using PySpark.
Performance Optimization:
- Optimize data pipelines, queries, and PySpark jobs to enhance overall performance and scalability.
- Identify and address performance bottlenecks within data processing workflows.
Requirements:
- Proven experience as a Data Engineer, emphasizing expertise in Azure.
- Proficiency in Azure Data Factory and other relevant Azure services.
- Expertise in PySpark for data processing and analytics is a must.
- Experience with data modeling, ETL processes, and data warehousing.
Skills
Azure Data Factory, Pyspark, Azure Databricks
About UST
UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.
