Core Responsibilities:
Lead the design and optimization of large-scale, cloud-based data pipelines and systems.
Develop infrastructure to collect, transform, combine, and publish/distribute customer data
Mentor junior engineers and contribute to best practices for data engineering standards.
Collaborate closely with data architects, analysts, and stakeholders to deliver reliable data solutions.
Implement and optimize ETL/ELT processes for performance and scalability.
Ensure data quality, integrity, and security across all environments.
Research and implement new tools, frameworks, and automation strategies to enhance productivity.
Support DevOps initiatives through CI/CD pipeline management and infrastructure-as-code practices.
Qualifications:
Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
4+ years’ experience as a data engineer or relevant role
Understanding of Big Data technologies, DataLake, DeltaLake, Lakehouse architectures
Advanced proficiency in Python, SQL, and Apache Spark.
Deep understanding of data modeling, ETL/ELT processes, and data warehousing concepts.
Experience with cloud data platforms such as Azure Data Factory, AWS Glue, or GCP Dataflow.
Strong background in performance tuning and handling large-scale datasets.
Familiarity with version control, CI/CD pipelines, and agile development practices.
Proficient with handling different file formats: JSON, Avro, and Parquet
Knowledge of one or more database technologies (e.g. PostgreSQL, Redshift, Snowflake, NoSQL databases).
Preferred Qualifications:
Cloud certification from a major provider (AWS, Azure, or GCP).
Hands-on experience with Snowflake, Databricks, Apache Airflow, or Terraform.
Exposure to data governance, observability, and cost optimization frameworks.
