Designation: Assistant Manager - Data Engineering
Experience: 4 to 8 years
Location: Chennai
Job Description:
We are looking for a highly motivated Assistant Manager – Data Engineer with strong background in GCP-based data solutions. This ideal candidate will be responsible for designing scalable data architectures, building optimized data pipelines, mentoring junior engineers, and driving performance improvements across our data ecosystem. This role offers the opportunity to work on cutting-edge cloud data platforms and collaborate closely with cross-functional stakeholders to deliver enterprise-grade solutions.
Responsibilities:
1. Solution Architecture & Design
- Assist in designing scalable, secure, and cost-efficient data architectures using GCP-native services.
- Hands-on design experience in BigQuery, DataForm, DataFlow, Cloud SQL, Cloud Composer, Vertex AI, GBQ Workflow, DataPrep, Data Fusion
- Collaborate with business and technology stakeholders to define end-to-end data solutions.
2. Data Modelling & Implementation
- Design and maintain conceptual, logical, and physical data models for analytics and reporting.
- Build and manage ETL/ELT pipelines using GCP services like Dataflow, Dataproc, and BigQuery.
- Ensure data governance, lineage, and quality controls are integrated into solutions.
3. Performance Tuning & Optimization
- Optimize data pipelines, queries, and storage layers for scalability and performance.
- Fine-tune BigQuery cost performance, partitioning, and clustering strategies.
- Implement monitoring and alerting for proactive performance management.
4. Leadership, Mentoring
- Lead, guide, and mentor a team of data engineers, enabling technical upskills and growth.
- Conduct code reviews, design validations, and architecture discussions.
- Champion best practices for data engineering, governance, and cloud-native development.
Skills:
• 4+ years of hands-on experience with GCP data engineering services, including: o BigQuery, DataFlow, Cloud Run, Cloud Composer, Data Stream, Data Fusion, Cloud SQL, Vertex AI, GBQ Workflow, DataPrep, Data Fusion, DataForm, GKE, Spanner • Strong expertise in SQL and Python / PySpark for data transformations. • Hands-on experience in ETL/ELT pipeline development and data modelling. • Familiarity with CI/CD for data pipelines using tools like Cloud Build, Git, Jenkins.
• Experience with real-time streaming frameworks (e.g., Pub/Sub, Kafka, Spark Streaming). • Knowledge of data lakehouse architectures and open-source tools like Apache Iceberg, Delta Lake, or Hudi. • Understanding of data security, IAM roles, DLP policies, and compliance frameworks in GCP. • Familiarity with modern orchestration tools like Airflow, dbt, or Dagster. • Hands-on experience working in Agile/Scrum environments
