Must have skills
GCP, BigQuery, Cloud Storage, Apache Airflow, dbt, Python, SQL, data pipelines, ingestion, transformation, Git/Bitbucket, data quality checks
Good to have skills
Dataplex, Collibra integration, CI/CD, Grafana, Splunk, Power BI data preparation, financial services data
Core Technical Requirements
Cloud & Platform Engineering:
- Strong proficiency across AWS and GCP.
- On AWS: IAM roles and policies, Lambda, S3, RDS, Redshift.
- On GCP: BigQuery, workload optimisation, cost management.
Data Engineering Tooling:
- Hands‑on experience building data pipelines for data warehouse environments.
- Strong SQL capability including analytical and window functions.
- Advanced dbt Cloud experience (models, SCDs, tests, macros, release management).
- Airflow workflow design including idempotency, backfills and monitoring.
DevOps, CI/CD & IaC:
- Experience with CI/CD (GitHub, Bitbucket, GitHub Actions, Bamboo).
- Infrastructure as Code using Terraform or CloudFormation.
- Familiarity with data quality frameworks and automated testing of pipelines.
Programming & Modelling:
- Proficiency in Python.
- Strong data modelling expertise: facts/dimensions, SCD patterns, performance optimisation.
Security, Data Governance & Operational Resilience:
- Solid grounding in cloud security best practices and least‑privilege IAM.
- Understanding of data access controls, regulated environments and governance expectations.
Delivery Approach & Ways of Working- Experience working within Agile delivery frameworks.
- Use of JIRA and Confluence for delivery management and documentation.
- Proven track record collaborating effectively in multi‑disciplinary teams across engineering, product and analytics.
Specialisations - BigQuery performance tuning and cost optimisation.
- DBT Cloud governance (contracts, tests, macros, release processes).
- Airflow workflow optimisation and monitoring.
- Cross‑cloud pipelines and data movement (AWS GCP).
- Security, IAM and data‑access design for regulated financial services environments.
Experience & Other- 8 or more years experience in data engineering or equivalent technical roles.
- Strong problem‑solving capability, comfortable troubleshooting pipeline failures and performance issues.
- Strong communication and stakeholder management skills.
- Self‑starter able to lead components independently while partnering closely with internal teams.
- Experience supporting production environments, including incident response and root‑cause analysis.
- All intellectual property rights retained by client
