Job Description: GCP Data Engineering (Support Engineer) – Lead Programmer Analyst.
We are looking for a skilled and experienced GCP Data Engineer to join our team. In this role, you will be responsible for monitoring, troubleshooting, and maintaining the health of [production workflows built on PGSQL, BigQuery, Python, Apace Airflow. Additionally, you will manage deployments, hotfixes and continuous improvements using our Jenkins Devx pipeline, blending Data engineering support with strong DevOps practices.
Required Skills and Qualifications
-
Experience: 5+ years of experience in Application Support, Production Support, Development with heavy focus on data pipelines, DevOps.
-
Orchestration & Code: Strong experience troubleshooting Apache Airflow (identifying DAG failures, log analysis) and reading/debugging Python code.
-
Databases: Proficiency in writing and optimizing SQL queries for both PostgreSQL and GCP BigQuery (analytical/Datawarehouse).
-
Cloud Platform: Familiarity with GCP code services (BQ, Cloud Storage, Cloud Logging/monitoring, and IAM)
-
DevOps/CI-CD: hands-on experience using Jenkins pipelines for application deployment and version control.
-
Incident Management: Strong understanding of SRE, ITIL practices, incident management and SLA tracking.
-
Communications: Excellent problem-solving and communication skills to coordinate with business stakeholders and internal teams during high priority incidents.
-
Knowledge of containerization (Docker/Kubernetes)
-
Must be Google Cloud Certified Professional Data Engineer.
Key Responsibilities:
1) Production Support & Application Monitoring
-
Monitor and maintain the health, performance, and reliability of core batch processing pipelines.
-
Troubleshoot, debug and resolve failures in Airflow DAGS and python scripts within SLA.
-
Conduct RCA for production incidents & implement permanent fixes, workarounds.
2) Code Fixes & Development
-
Develop, test and implement code fixes for bugs, minor enhancements, data parsing errors or performance issues in production
3) Database & Data Integrity Support
-
Monitor and optimize query performance in Google Cloud BigQuery and PostgreSQL.
-
Troubleshoot data discrepancies or performance bottlenecks in database layers.
-
Assist with Data Recovery, Patch runs and manual data fixes when batch processing fails.
4) Release Management & DevOps Support
-
Handle Code Deployments, Infrastructure updates and hotfixes.
-
Maintain & Improve CI/CD pipeline scripts to ensure reliable and automated environment promotions.
-
Infrastructure as Code (IaC) - Solid experience with Terraform for managing and provisioning GCP resources.
-
Troubleshooting GCP Pub/Sub topics, subscriptions, quotas and message backlogs.
