Job Opportunity:
We are looking for an experienced Databricks Data Engineer with strong hands-on expertise in Scala
and the Azure ecosystem to design, build, and optimize scalable data ingestion and processing
frameworks. The role involves close collaboration with clients, architects, and cross-functional
teams to translate business requirements into high-quality technical solutions.
Key Responsibilities:
- Understand functional and business requirements and translate them into detailed technical
specifications
- Design and develop scalable frameworks and reusable components for data ingestion and
processing using Azure Databricks
- Build, optimise, and maintain ETL/ELT pipelines to process large-scale and complex datasets
- Apply Spark/Scala optimisation techniques to ensure performance, scalability, and reliability
- Work closely with Client Data Engineering Managers to align technical solutions with business
goals
- Collaborate with customer Architecture teams on solution design, code reviews, and best
practices
- Participate in Agile sprint-based development and deliver high-quality outputs as per project
plans
- Ensure code quality, performance tuning, and adherence to data engineering standards
Required Skills & Experience:
- Strong hands-on design and development experience on the Databricks platform
- Proficiency in Scala with Apache Spark
- Experience working with the Azure ecosystem, including:
o Azure Data Lake Storage (ADLS Gen2)
o Azure Data Factory (ADF)
o Azure Key Vault
- Proven experience in Spark performance tuning and optimization for large datasets
- Experience designing and implementing ETL/ELT pipelines in Databricks
- Strong problem-solving skills with the ability to handle complex business logic
- Good communication skills and experience working directly with clients
