Job Description:
We are looking for a Senior Data Engineer with 4+ years of experience to build scalable and high-performance data platforms using Databricks and modern cloud technologies.
Key Responsibilities:
- Design, develop, and maintain scalable batch and streaming data pipelines using Databricks and Apache Spark.
- Build ETL/ELT pipelines using PySpark, Python, and SQL.
- Develop data ingestion and transformation frameworks from databases, APIs, files, and event streams.
- Work extensively with Delta Lake, Databricks Workflows, Unity Catalog, and Databricks SQL.
- Optimize Spark jobs, queries, partitioning, joins, and data processing for performance and cost.
- Implement incremental processing, CDC, data quality, and governance practices.
- Build production-ready pipelines with monitoring, logging, alerting, and error handling.
- Work with cloud platforms such as Azure, AWS, or GCP.
- Participate in CI/CD, production support, troubleshooting, and root-cause analysis.
Required Skills:
- 4+ years of professional experience in Data Engineering.
- Strong hands-on experience with Databricks.
- Strong experience with Apache Spark / PySpark.
- Strong programming skills in Python and SQL.
- Hands-on experience with Delta Lake.
- Experience in ETL/ELT, data pipelines, CDC, batch/incremental processing, and data optimization.
- Experience with at least one cloud platform – Azure / AWS / GCP.
- Experience with Git and CI/CD.
- Strong analytical and troubleshooting skills.
Good to Have:
- Unity Catalog / Databricks Workflows
- Delta Live Tables / Lakeflow
- Kafka / Streaming
- Azure Data Factory / AWS Glue / GCP Dataflow
- Terraform / Docker / Kubernetes
- Data Governance
- ML/AI or GenAI data pipelines
Pay: ₹800,000.00 - ₹1,200,000.00 per year
Benefits:
- Paid time off
- Provident Fund
Application Question(s):
- How many years of hands-on experience do you have with Databricks?
- Are you willing and available to attend a direct Face-to-Face interview in Bengaluru?
- What is your current CTC?
- What is your expected CTC?
Experience:
- Data Engineering: 5 years (Preferred)
Work Location: In person
