Job Title: Senior Databricks Engineer & Solutions Architect
Role Overview
We are looking for a Senior Databricks Engineer & Solutions Architect to lead the strategy, design, and implementation of our Enterprise Data Intelligence Platform. In this hybrid role, you will act as the principal architect shaping our cloud lakehouse vision while staying hands-on to build, optimize, and secure high-performance data pipelines. You will take complete ownership of end-to-end data product delivery and mentor our engineering squads.
Key Responsibilities
- Lakehouse Architecture: Design end-to-end cloud platforms leveraging the Databricks Medallion paradigm (Bronze, Silver, Gold). Optimize Delta Lake storage using Liquid Clustering, Z-Ordering, and data skipping.
- Pipeline Engineering: Build and debug highly scalable batch and streaming pipelines using dbt, PySpark, Spark SQL, and Delta Live Tables (DLT). Define robust data contracts and strictly enforce SLA metrics.
- Governance & Security: Architect enterprise data governance layouts using Databricks Unity Catalog. Implement Row/Column-Level security, dynamic data masking, and automated lineage tracking.
- Collaboration & Mentorship: Partner with DevOps, Analytics, and ML squads (via MLflow). Lead code/design reviews, establish development standards, and elevate team capabilities.
- Visualization: Excellent in Power BI visualization and with extensive SQL knowledge.
Technical Qualifications & Skills
- Experience: 5+ years in Data Engineering/Architecture, with 3+ years of dedicated, hands-on experience building and scaling production Databricks environments.
- Core Tech: Advanced proficiency in dbt, Python (PySpark), Spark SQL, and distributed computing internals (memory tuning, shuffle mitigation, execution plans), Power BI, MS BI, SQL.
- Cloud & IaC: Hands-on experience with native cloud data ecosystems (e.g. MS Fabric, Azure Data Factory, AWS Glue, IAM, cloud storage) and infrastructure automation using Terraform.
- Preferred Certifications: Databricks Solutions Architect, Databricks Data Engineer (Associate/Professional), or Associate Cloud Architect certifications.
Exp: 8+ yrs
Loc: Bangalore
Work: Hybrid (3 days a week WFO)
Work Location: In person
