Job Description
UST is looking for a skilled AWS Data Lead to join team with 8 to 10 years of experience and contribute to the development and maintenance of robust data pipelines and systems on the AWS platform, with a focus on leveraging PySpark for efficient data processing. As a Data Engineer, role will play a crucial role in ensuring the efficient flow, storage, and processing of data for our organization.
Responsibilities:
Data Pipeline Development: Design, implement, and maintain scalable and efficient data pipelines on the AWS platform using tools such as AWS Glue, Apache Spark, and PySpark.
AWS Services Utilization: Leverage AWS services like S3, Glue, Athena, and others to build end-to-end data solutions.
ETL Processing with PySpark: Develop and optimize ETL processes using PySpark to facilitate seamless data extraction, transformation, and loading.
Requirements:
Experience: Proven experience as a Data Lead with a focus on AWS data solutions, including hands-on experience with PySpark.
Programming Skills: Proficient in Python, with advanced experience in PySpark for efficient data processing.
ETL Tools: Hands-on experience with AWS Glue or other ETL tools for data processing.
data engineering,aws,aws glue,pyspark,etl,
