Senior Data Engineer – Databricks
Location: Bengaluru, Karnataka – Near Lalbagh
Work Mode: Work From Office / Onsite
Experience: 4–8 Years
Employment Type: Full-Time
Salary: ₹8–12 LPA
Interview: Face-to-Face
Preferred: Bengaluru-local candidates
About the Role
We are looking for a Senior Data Engineer – Databricks with 4+ years of professional experience to build scalable, reliable, and high-performance data platforms using Databricks and modern cloud technologies.
The ideal candidate should have strong hands-on experience in Databricks, Apache Spark/PySpark, Python, SQL, Delta Lake, and cloud data platforms. The role involves designing, developing, optimizing, and supporting production-grade data engineering solutions.
Key Responsibilities
- Design, develop, and maintain scalable batch and streaming data pipelines using Databricks and Apache Spark.
- Build ETL/ELT pipelines using PySpark, Python, and SQL.
- Develop data ingestion frameworks from databases, APIs, files, and other enterprise data sources.
- Build data transformation and processing pipelines using Delta Lake.
- Implement incremental processing, CDC, partitioning, and efficient data-loading strategies.
- Develop and optimize Databricks notebooks, jobs, workflows, and production pipelines.
- Work with Delta Lake, Unity Catalog, Databricks Workflows, and Databricks SQL.
- Troubleshoot and optimize Spark jobs for performance and cost.
- Work with cloud platforms such as Azure, AWS, or GCP.
- Integrate Databricks with cloud storage, databases, data warehouses, APIs, and messaging platforms.
- Build production-ready pipelines with monitoring, logging, alerting, and error handling.
- Implement CI/CD practices and work with Git and DevOps tools.
- Participate in production support, incident resolution, and root-cause analysis.
- Mentor junior data engineers and contribute to technical decisions.
Required Technical Skills
- 4+ years of professional experience in Data Engineering.
- Strong hands-on experience with Databricks.
- Strong experience with Apache Spark / PySpark.
- Strong programming skills in Python and SQL.
- Hands-on experience with Delta Lake.
- Experience designing and developing enterprise-grade data pipelines.
- Strong understanding of ETL/ELT, data modeling, batch processing, incremental processing, CDC, partitioning, data quality, and performance optimization.
- Experience with at least one major cloud platform: Azure, AWS, or GCP.
- Experience with Git and CI/CD.
- Strong troubleshooting and analytical skills.
Good to Have
- Unity Catalog
- Databricks Workflows
- Delta Live Tables / Lakeflow
- Databricks SQL
- Kafka or other streaming technologies
- Real-time/streaming data pipelines
- Azure Data Factory / AWS Glue / GCP Dataflow
- Terraform or Infrastructure as Code
- Data governance and cataloging tools
- Docker/Kubernetes
- ML/AI or GenAI data pipelines
Ideal Candidate
4–8 years of experience | Databricks | PySpark | Python | SQL | Delta Lake | Cloud | Data Pipelines | CI/CD
Candidates should have a strong ownership mindset, excellent problem-solving abilities, and the ability to handle data engineering projects from requirement → design → development → deployment → production support.
Education
Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related discipline. Equivalent practical experience may also be considered.
What We Offer
- Opportunity to work on modern Databricks and cloud data platforms.
- Exposure to large-scale data engineering and AI/ML/GenAI platforms.
- Opportunity to work with modern data architecture, governance, and cloud technologies.
- Career growth opportunities toward Lead Data Engineer, Data Architect, or Data Engineering Manager roles.
- Collaborative and engineering-focused work environment.
Pay: ₹800,000.00 - ₹1,200,000.00 per year
Work Location: In person
