Job Description – Senior Cloud Data Engineer
Job Title: Senior Cloud Data Engineer
Experience: 5+ Years
Employment Type: Full-Time
Role Summary
We are seeking a highly skilled Senior Cloud Data Engineer to support a large-scale Cloud Data Modernization initiative. The ideal candidate will have strong experience in migrating enterprise data platforms from on-premises environments to Azure Cloud and Snowflake, along with hands-on expertise in designing, developing, and optimizing modern cloud-based data solutions.
The role requires deep technical expertise in Azure Data Services, Databricks, Snowflake, ETL/ELT development, DevOps practices, and data engineering best practices. The successful candidate will collaborate with architects, developers, business teams, and engineering teams to deliver secure, scalable, and high-performing data platforms.
Key Responsibilities
- Lead and execute migration of enterprise ETL workloads from on-premises SQL Server-based data warehouses to Azure Cloud and Snowflake platforms.
- Analyze, redesign, and modernize existing ETL processes developed using:
- SSIS
- T-SQL
- SQL Server-based data workflows
- Design, develop, and implement scalable cloud data engineering solutions using:
- Azure Data Factory (ADF)
- Self-hosted Integration Runtime (SHIR)
- Azure Data Lake Storage Gen2 (ADLS Gen2)
- Azure Blob Storage
- Azure Logic Apps
- Azure Databricks
- Snowflake
- Develop and optimize ETL/ELT pipelines for large-scale data processing and analytics workloads.
- Build data transformation workflows using Databricks and Python-based processing frameworks.
- Implement data ingestion, integration, transformation, and orchestration solutions across enterprise data platforms.
- Configure and manage CI/CD pipelines and DevOps practices using GitHub Actions.
- Collaborate with cross-functional teams including data architects, analysts, application teams, and business stakeholders.
- Troubleshoot data pipeline failures, performance issues, and integration challenges.
- Optimize data workflows for scalability, reliability, and performance.
- Implement security controls, governance practices, and compliance requirements across cloud data platforms.
- Support Agile delivery practices, including sprint planning, technical discussions, code reviews, and release activities.
- Participate in architecture discussions and provide recommendations for cloud data modernization strategies.
- Apply AI/ML capabilities and tools to enhance data processing workflows and ETL automation where applicable.
Required QualificationsEducation
- Bachelor's degree in Computer Science, Information Technology, Engineering, Data Science, or related field.
Experience
- Minimum 5+ years of experience as a Cloud Data Engineer.
- Strong experience delivering enterprise-scale data engineering solutions.
- Hands-on experience migrating data workloads from on-premises platforms to cloud environments.
- Experience working with modern cloud data architectures and analytics platforms.
Required Technical SkillsCloud Data Engineering
Strong hands-on experience with:
- Microsoft Azure Cloud
- Azure Data Factory (ADF)
- Self-hosted Integration Runtime (SHIR)
- Azure Data Lake Storage Gen2 (ADLS Gen2)
- Azure Blob Storage
- Azure Logic Apps
- Azure Databricks
Data Platforms
- Snowflake
- Databricks
- SQL Server
- Cloud data warehouse concepts
- Data lake and lakehouse architectures
ETL/ELT Development
Strong experience with:
- ETL pipeline design and development
- SSIS migration and modernization
- T-SQL development
- Data transformation frameworks
- Data ingestion processes
- Data integration patterns
Programming & Scripting
Experience with:
- Python
- SQL
- Scripting for data processing and automation
DevOps & CI/CD
Experience with:
- GitHub Actions
- CI/CD pipeline implementation
- Source control practices
- Automated deployment processes
Data Engineering Practices
Strong understanding of:
- Data modeling
- Database design principles
- Data quality frameworks
- Data validation
- Performance optimization
- Pipeline monitoring and troubleshooting
Preferred Qualifications
- Experience developing data engineering solutions using Databricks for analytics and large-scale processing workloads.
- Experience with data governance, metadata management, and data quality best practices.
- Knowledge of additional cloud platforms such as:
- AWS
- Google Cloud Platform
- Experience applying AI/ML tools and models to:
- Data processing workflows
- ETL automation
- Data analytics solutions
- Azure or Snowflake certifications preferred.
- Experience working in Agile/Scrum environments.
Key Competencies
- Cloud Data Engineering
- Azure Data Platform
- Snowflake Development
- Databricks Engineering
- ETL/ELT Modernization
- Data Migration
- Python & SQL Development
- Data Pipeline Optimization
- DevOps & CI/CD Practices
- Data Governance
- Problem Solving
- Analytical Thinking
- Technical Leadership
- Collaboration & Communication
Role Outcomes
The successful candidate will:
- Modernize legacy data platforms through successful cloud migration.
- Build scalable and reliable Azure-based data solutions.
- Enable advanced analytics through Snowflake and Databricks platforms.
- Improve data processing efficiency through automation and optimization.
- Establish robust engineering practices supporting enterprise data transformation initiatives.
Work Location: Hybrid remote in Noida, Uttar Pradesh (Noida)
