Project Role : Custom Software Engineer
Project Role Description : Design, build and configure applications to meet business process and application requirements.
Must have skills : Microsoft Azure Data Services, Microsoft Azure Databricks
Good to have skills : NA
Minimum
3 year(s) of experience is required
Educational Qualification : 15 years full time education
Summary:
We re seeking a visionary Senior Data Engineer to design and build on our Azure-based data platform. This role requires deep expertise in Databricks, Medallion Architecture, and Delta Lake, combined with strong orchestration skills using Azure Data Factory (ADF). You ll also leverage data virtualization technologies including Denodo and Snowflake to enable unified, efficient access to data across the enterprise. Experience with Veeva Vault for life sciences data integration adds significant value. You will lead a team building scalable, secure, and high-performance data solutions that power analytics and AI initiatives.
Roles & Responsibilities:
Design and implement scalable data pipelines using Databricks and Azure Data Factory
Design and maintain Medallion Architecture (Bronze, Silver, Gold layers) for structured and semi-structured data
Develop and optimize Delta Lake tables for ACID-compliant, performant data storage and querying
Lead integration and virtualization of diverse data sources leveraging Denodo and Snowflake platforms
Collaborate with teams to ingest, harmonize, and manage life sciences data from Veeva Vault and related systems
Drive data lake and warehouse development on Azure Synapse and Data Lake Storage Gen2
Ensure robust data governance, lineage, and security across the platform
Mentor junior engineers and promote engineering best practices
Drive CI/CD automation and infrastructure-as-code for data workflows
Professional & Technical Skills:
5+ years of experience in data engineering, including 3+ years in databricks.
Deep hands-on experience with Databricks (PySpark, Delta Lake, etc.)
Strong understanding of Medallion Architecture and practical implementation
Expertise with Delta Tables: schema evolution, time travel, compaction, and performance tuning
Proficiency in Azure Data Factory for orchestration and automation
Solid knowledge of Azure services: Data Lake Storage Gen2, Synapse Analytics, Azure Functions
Advanced Python and SQL skills for data processing, scripting, and modeling
Exposure to data virtualization technologies such as Denodo and Snowflake for unified data access
Experience working with Veeva Vault data integration and lifecycle
Familiarity with Git, CI/CD pipelines, and DevOps practices
Additional Information:
- The candidate should have minimum 3 years of experience in Microsoft Azure Data Services.
- This position is based at our Bengaluru office.
- A 15 years full time education is required.
