Data Engineer

Archelons Solutions Private Limited
Bengaluru, Karnataka

Quick apply

Job details

Full-time
₹38,00,000 a year
7 days ago

Qualifications

CI/CD
Azure
Law
Data migration
Spark
Encryption
Git
Master's degree
SQL
AWS
Continuous integration
Mentoring
Unity
Leadership
Python
Identity & access management

Full job description

Please find the IAS sheet attached.

S No

Skill Area

Description

Assessment Criteria

Data Engineering Experience & Technical Leadership

10+ years of hands-on experience designing, developing, and supporting enterprise data platforms, along with proven experience leading technical initiatives, mentoring engineers, driving architecture decisions, and delivering large-scale data platform projects.

Must Have

Databricks Lakehouse Platform

hands-on experience with Databricks, Delta Lake, Databricks Workflows, DLT, Unity Catalog, and Medallion Architecture in production environments.

Must Have

Apache Spark / PySpark

strong expertise in Spark architecture, transformations/actions, partitioning, performance tuning, caching, joins, and handling large-scale distributed workloads.

Must Have

Python

proficient in Python for developing production-grade data pipelines, transformations, error handling, modular coding, and automation.

Must Have

SQL

advanced SQL including joins, CTEs, window functions, aggregations, optimization, and data validation.

Must Have

ETL / ELT Pipeline Development

designed and implemented scalable batch and streaming ETL/ELT pipelines in production environments.

Must Have

Data Modeling

understand dimensional modeling, star/snowflake schemas, lakehouse modeling, and schema design principles.

Must Have

Data Lake / Lakehouse Architecture

experience designing and implementing Bronze/Silver/Gold architectures and managing large-scale data lakes.

Must Have

Cloud Platform (AWS/Azure/GCP)

strong hands-on experience with at least one major cloud platform and its core data services.

Must Have

Workflow Orchestration (Airflow/Databricks Workflows)

able to design, schedule, monitor, and troubleshoot complex workflow dependencies and retries.

Must Have

Distributed Data Processing & Performance Optimization

expert in partitioning, skew handling, optimization, cluster sizing, and cost optimization.

Must Have

Data Governance & Security

understand IAM, access control, lineage, encryption, cataloging, and governance frameworks such as Unity Catalog.

Must Have

Technical Leadership & Mentoring

experience leading teams, conducting code reviews, establishing standards, mentoring engineers, and driving technical decisions.

Must Have

CI/CD & Software Engineering Practices

Git, branching strategies, code reviews, automated deployments, and CI/CD pipelines.

Must Have

Large-scale Data Migration

experience migrating data platforms from legacy/on-prem/cloud environments to modern architectures.

Must Have

GenAI / LLM Concepts

LLM fundamentals, embeddings, vector databases, prompts, and RAG concepts. Production experience is preferred but not mandatory.

Must Have

Quick apply

Jobseeker tools

Employer Tools

Browse

Stay Connected