-
Strong expertise in GCP (BigQuery, Composer, Dataproc, Dataflow, Pub/Sub)
-
Hands-on experience with Databricks for large-scale data processing
-
Strong programming skills in SQL and PySpark
-
Experience with Oracle and SQL Server databases
-
Strong understanding of data modelling (dimensional, medallion, UDM)
-
Experience in batch and streaming pipeline development
-
Knowledge of data ingestion, CDC, and orchestration frameworks
-
Familiarity with data governance, quality, and lineage
-
Exposure to CI/CD and DevOps practices
-
Strong leadership and stakeholder management skills
Detailed Responsibilities-
Design and build scalable batch and streaming data pipelines
-
Develop data ingestion and transformation frameworks
-
Implement CDC and incremental data loading strategies
-
Work on GCP platforms including BigQuery, Dataflow, Dataproc, and Pub/Sub
-
Build and manage workflows using Cloud Composer (Airflow)
-
Implement metadata-driven and reusable pipeline frameworks
-
Ensure data quality, validation, and monitoring
-
Drive migration from Oracle/SQL Server to cloud platforms
-
Convert legacy SQL and ETL logic to BigQuery/PySpark
-
Collaborate with analytics and reporting teams
-
Lead and mentor data engineering teams
-
Optimise pipelines for performance and cost
-
Troubleshoot and resolve pipeline/data issues
-
Support advanced analytics and AI/ML use cases
Location: Chennai / Bengaluru / Hyderabad
Experience: 8+ Years
