We are looking for an experienced Databricks Engineer with 12+ years of overall IT experience to join our team on a long-term contract. The ideal candidate should have deep expertise in Databricks, Apache Spark, PySpark, Python, SQL, and cloud platforms (AWS, Azure, or GCP), along with a proven track record of delivering enterprise-scale data engineering solutions.
This is a customer-facing role requiring strong technical leadership, solution architecture capabilities, and hands-on experience building scalable cloud-based data platforms.
-
Databricks Certified Data Engineer Professional (Mandatory)
-
Lead end-to-end Databricks implementation projects from solution design through deployment and production support.
-
Design, develop, and optimize scalable data pipelines using Databricks, Apache Spark, and PySpark.
-
Build cloud-native data engineering solutions using AWS, Microsoft Azure, or Google Cloud Platform (GCP).
-
Design and implement modern Lakehouse architectures for enterprise-scale data platforms.
-
Translate business requirements into scalable, high-performance technical solutions.
-
Integrate Databricks solutions with enterprise applications, APIs, and third-party systems.
-
Configure, monitor, troubleshoot, and optimize Databricks environments for performance, reliability, and scalability.
-
Implement CI/CD pipelines and DevOps best practices for data engineering workflows.
-
Collaborate with architects, business stakeholders, and cross-functional teams to deliver high-quality solutions.
-
Provide technical leadership, mentor engineering teams, and promote best practices.
-
Participate in customer workshops, architecture discussions, and technical consulting sessions.
-
Ensure solution quality, security, governance, and operational excellence throughout project delivery.
-
Databricks
-
Apache Spark
-
PySpark
-
Python
-
SQL
-
Data Engineering
-
ETL/ELT Pipeline Development
-
Lakehouse Architecture
-
AWS / Microsoft Azure / Google Cloud Platform (GCP)
-
Solution Architecture
-
Systems Integration
-
CI/CD Pipelines
-
Performance Optimization
-
Data Modeling & Data Warehousing
-
MLOps
-
Git & DevOps Practices
-
12+ years of overall IT experience.
-
Hands-on expertise with Databricks and modern data engineering platforms.
-
Experience delivering 10+ end-to-end Databricks implementation projects.
-
Strong experience with distributed data processing using Apache Spark and PySpark.
-
Proven experience designing and implementing enterprise-scale cloud-based data platforms.
-
Strong understanding of Lakehouse Architecture, data integration, governance, and data modeling.
-
Experience implementing CI/CD pipelines and deployment automation.
-
Excellent troubleshooting, debugging, and performance tuning skills.
-
Strong client-facing consulting and stakeholder management experience.
-
Experience with Delta Lake, Unity Catalog, MLflow, and Databricks Workflows.
-
Knowledge of modern data architecture patterns and cloud-native services.
-
Experience working in Agile/Scrum environments.
-
AWS, Azure, or GCP certifications are an added advantage.
-
Excellent communication and presentation skills.
-
Strong analytical and problem-solving abilities.
-
Customer-first mindset with strong consulting skills.
-
Ability to manage multiple priorities in a fast-paced environment.
-
Leadership, ownership, and mentoring capabilities.
