We are seeking an experienced Senior Data Engineer to lead the design, development, and delivery of scalable enterprise data solutions. This role will build and optimize batch and real-time data pipelines, reusable integration frameworks, and governed data platforms that support analytics, AI, and self-service data access.
The ideal candidate has deep expertise in Databricks, Apache Spark, AWS, data modeling, governance, and production operations. The role also provides technical leadership, defines engineering standards, mentors other engineers, and partners with architecture, business, analytics, data science, and DevOps teams.
Experience in manufacturing, biotechnology, pharmaceutical, life sciences, or another regulated industry is preferred.
Roles and Responsibilities
Integrate structured, semi-structured, and unstructured data from enterprise, manufacturing, API, and third-party sources.
Optimize Spark workloads, Databricks compute, SQL queries, partitioning, storage, and caching for performance and cost.
Implement workflow orchestration, monitoring, alerting, data-quality controls, and recovery processes.
Implement metadata management, lineage, cataloging, governance, RBAC, and data-security controls.
Collaborate with architects, analysts, data scientists, product teams, and DevOps teams.
