Cloud Platform Engineer
Location
Noida, Uttar Pradesh
Work Mode
Remote
Employment Type
Full-time
Job Overview
We are seeking a highly skilled Cloud Platform Engineer with strong experience in Databricks, Azure Cloud, and Snowflake to join our team for a Cloud Data Modernization project.
The ideal candidate will have hands-on experience in cloud data platforms, ETL migration, data engineering, cloud modernization, CI/CD, and data platform solutions.
Databricks is the primary data skill, Snowflake is the secondary data skill, and strong Azure and/or AWS cloud infrastructure experience is required.
Key Responsibilities
- Contribute to the migration of ETL workloads from on-premises platforms to Azure Cloud, Databricks, and Snowflake.
- Develop and implement scalable data platform solutions using Databricks, Azure Data Factory (ADF), SHIR, Logic Apps, ADLS Gen2, Blob Storage, and Snowflake.
- Review and analyze existing on-premises ETL processes and support their modernization.
- Develop and optimize data pipelines using Apache Spark and PySpark.
- Implement DevOps practices and CI/CD pipelines using GitHub Actions.
- Collaborate with cross-functional teams to ensure seamless integration and reliable data flow.
- Tune and optimize workloads for performance, scalability, and cost efficiency.
- Implement data quality validation, monitoring, reconciliation, and observability processes.
- Support automated testing and data validation frameworks.
- Ensure data security, governance, and compliance with industry standards.
- Participate in code reviews and follow engineering best practices.
- Work effectively within Agile delivery methodologies.
- Leverage AI-assisted development tools such as GitHub Copilot, Databricks Assistant, ChatGPT, or Claude to improve engineering productivity.
Required Qualifications
- Strong hands-on experience with Databricks.
- Experience with Azure Cloud and/or AWS cloud infrastructure.
- Hands-on experience with Snowflake.
- Strong knowledge of Apache Spark and PySpark.
- Experience with Azure Data Factory (ADF), SHIR, Logic Apps, ADLS Gen2, and Blob Storage.
- Strong experience in ETL/ELT development using on-premises databases and cloud technologies.
- Strong SQL development skills, including complex queries, stored procedures, views, and ETL transformations.
- Experience with Python or other scripting languages.
- Experience with GitHub, branching, pull requests, and GitHub Actions.
- Understanding of CI/CD, automated testing, data quality, monitoring, and observability.
- Experience with code reviews and engineering best practices.
- Experience working in Agile environments.
- Strong analytical, problem-solving, communication, and collaboration skills.
- Ability to work independently, manage ambiguity, and meet tight deadlines.
Preferred Qualifications
- Experience with Delta Lake and Lakehouse architectures.
- Knowledge of Bronze/Silver/Gold (Medallion) architecture.
- Experience with Databricks and Snowflake performance optimization.
- Knowledge of data modeling and database design.
- Experience with Airflow or other orchestration frameworks.
- Knowledge of data governance and data quality best practices.
- Experience with healthcare payer data domains such as Claims, Membership, Enrollment, Provider, Clinical, and Financial data.
- Experience using AI/ML solutions for data engineering workflows and automation.
- Azure or Databricks certification is a plus.
- Experience with additional cloud platforms and modern data technologies is an advantage.
Candidate Profile
- Strong technical and analytical problem-solving skills.
- Excellent communication and collaboration abilities.
- Self-motivated and comfortable working in a fully remote environment.
- Ability to quickly learn and adapt to new technologies.
- Strong ownership and attention to quality.
- Ability to identify, communicate, and escalate critical issues effectively.
- Willingness to travel occasionally, if required by business needs.
Key Skills
Primary Skills: Databricks, Azure Cloud / AWS, Apache Spark, PySpark
Secondary Skills: Snowflake, Azure Data Factory, SHIR, Logic Apps, ADLS Gen2, Blob Storage, Python, SQL, GitHub Actions, CI/CD, Delta Lake, Airflow
Work Location: Hybrid remote in Remote
