Noida, Uttar Pradesh
Job Summary
\"Seeking a hybrid Azure Databricks Platform Support & Data Engineer capable of managing day-to-day platform operations (50%) including job monitoring, workflow support, access management, Unity Catalog, clusters, Data Lake and Synapse support, while dedicating the remaining capacity (50%) to developing data ingestions, enhancing PySpark/Spark SQL pipelines, and creating or optimizing data assets and tables.\" Senior Azure Databricks Platform Support & Data Engineer ________________________________________ Role Summary Seeking an experienced Azure Databricks Platform Support & Data Engineer responsible for ensuring the operational stability, governance, and support of the Azure data platform while also contributing to data engineering initiatives including data ingestion, pipeline development, table creation, and solution enhancements. The role is expected to operate in a hybrid capacity: • ~50% Platform Operations & Support • ~50% Data Engineering & Enhancements The ideal candidate will have strong expertise in Azure Databricks, Unity Catalog, PySpark, Spark SQL, Azure Data Lake, Synapse Analytics, access governance, production support, and data pipeline development. The individual will be responsible for maintaining platform reliability, monitoring workflows, managing security and access controls, resolving integration issues, and developing scalable data solutions. Unity Catalog governance, access controls, identities, and administration are core capabilities for modern Databricks environments. ________________________________________ Role Allocation 50% Platform Support & Operations Responsible for maintaining day-to-day health and stability of the Azure data platform. Platform Monitoring & Operations • Monitor Databricks jobs, workflows, and scheduled processes • Ensure jobs are operational, enabled, and meeting SLA expectations • Proactively identify and resolve workflow failures • Investigate production incidents and perform root cause analysis • Coordinate issue resolution with upstream and downstream application teams • Monitor platform performance, utilization, and operational health • Support release validation and production deployment activities Databricks Platform Administration • Manage Azure Databricks workspaces • Configure and maintain Unity Catalog • Manage catalogs, schemas, tables, external locations, and storage credentials • Administer clusters and compute resources • Support job scheduling and workflow orchestration • Optimize cluster utilization and platform performance Access & Security Management • Manage RBAC and Unity Catalog permissions • Provision and deprovision user access • Manage service principals and managed identities • Support security reviews and compliance requirements • Troubleshoot authentication and authorization issues Integration & Platform Support • Support Azure Data Lake Storage Gen2 • Support Synapse Analytics integrations • Troubleshoot database connectivity issues • Support Tableau integration and data availability issues • Resolve API and application integration challenges • Coordinate with infrastructure, application, and analytics teams ________________________________________ 50% Data Engineering & Platform Enhancements Responsible for ongoing development and enhancement activities. Data Pipeline Development • Develop and enhance PySpark-based data pipelines • Create and optimize Spark SQL processes • Build and enhance ETL/ELT workflows • Improve performance and scalability of existing solutions • Troubleshoot and remediate pipeline issues Data Ingestion • Build and maintain ingestion frameworks • Onboard new source systems • Support batch and incremental data loads • Support ingestion from databases, files, APIs, and enterprise applications Data Modeling & Table Management • Create and maintain Delta Lake tables • Create managed and external Unity Catalog tables • Support Bronze, Silver, and Gold architecture layers • Implement data quality, validation, and reconciliatio
Key Responsibilities
\"Seeking a hybrid Azure Databricks Platform Support & Data Engineer capable of managing day-to-day platform operations (50%) including job monitoring, workflow support, access management, Unity Catalog, clusters, Data Lake and Synapse support, while dedicating the remaining capacity (50%) to developing data ingestions, enhancing PySpark/Spark SQL pipelines, and creating or optimizing data assets and tables.\" Senior Azure Databricks Platform Support & Data Engineer ________________________________________ Role Summary Seeking an experienced Azure Databricks Platform Support & Data Engineer responsible for ensuring the operational stability, governance, and support of the Azure data platform while also contributing to data engineering initiatives including data ingestion, pipeline development, table creation, and solution enhancements. The role is expected to operate in a hybrid capacity: • ~50% Platform Operations & Support • ~50% Data Engineering & Enhancements The ideal candidate will have strong expertise in Azure Databricks, Unity Catalog, PySpark, Spark SQL, Azure Data Lake, Synapse Analytics, access governance, production support, and data pipeline development. The individual will be responsible for maintaining platform reliability, monitoring workflows, managing security and access controls, resolving integration issues, and developing scalable data solutions. Unity Catalog governance, access controls, identities, and administration are core capabilities for modern Databricks environments. ________________________________________ Role Allocation 50% Platform Support & Operations Responsible for maintaining day-to-day health and stability of the Azure data platform. Platform Monitoring & Operations • Monitor Databricks jobs, workflows, and scheduled processes • Ensure jobs are operational, enabled, and meeting SLA expectations • Proactively identify and resolve workflow failures • Investigate production incidents and perform root cause analysis • Coordinate issue resolution with upstream and downstream application teams • Monitor platform performance, utilization, and operational health • Support release validation and production deployment activities Databricks Platform Administration • Manage Azure Databricks workspaces • Configure and maintain Unity Catalog • Manage catalogs, schemas, tables, external locations, and storage credentials • Administer clusters and compute resources • Support job scheduling and workflow orchestration • Optimize cluster utilization and platform performance Access & Security Management • Manage RBAC and Unity Catalog permissions • Provision and deprovision user access • Manage service principals and managed identities • Support security reviews and compliance requirements • Troubleshoot authentication and authorization issues Integration & Platform Support • Support Azure Data Lake Storage Gen2 • Support Synapse Analytics integrations • Troubleshoot database connectivity issues • Support Tableau integration and data availability issues • Resolve API and application integration challenges • Coordinate with infrastructure, application, and analytics teams ________________________________________ 50% Data Engineering & Platform Enhancements Responsible for ongoing development and enhancement activities. Data Pipeline Development • Develop and enhance PySpark-based data pipelines • Create and optimize Spark SQL processes • Build and enhance ETL/ELT workflows • Improve performance and scalability of existing solutions • Troubleshoot and remediate pipeline issues Data Ingestion • Build and maintain ingestion frameworks • Onboard new source systems • Support batch and incremental data loads • Support ingestion from databases, files, APIs, and enterprise applications Data Modeling & Table Management • Create and maintain Delta Lake tables • Create managed and external Unity Catalog tables • Support Bronze, Silver, and Gold architecture layers • Implement data quality, validation, and reconciliatio
Skill Requirements
strong expertise in Azure Databricks, Unity Catalog, PySpark, Spark SQL, Azure Data Lake, Synapse Analytics, access governance, production support, and data pipeline development
Other Requirements
strong expertise in Azure Databricks, Unity Catalog, PySpark, Spark SQL, Azure Data Lake, Synapse Analytics, access governance, production support, and data pipeline development
#body.unify div.unify-button-container .unify-apply-now: focus, #body.unify div.unify-button-container .unify-apply-#body.unify div.unify-button-container .unify-apply-now: focus, #body.unify div.unify-button-container .unify-apply-
