Big Data Engineer
Experience: 6–12 Years | Relevant
Location: Bengaluru
Key Responsibilities:
Design, develop, and maintain scalable Big Data solutions.
Build and optimize ETL/ELT pipelines for processing large volumes of structured and
unstructured data.
Develop data processing applications using Apache Spark (PySpark/Scala).
Work with Hadoop ecosystem components such as HDFS, Hive, and YARN.
Integrate data from multiple sources including databases, APIs, and cloud platforms.
Optimize data pipelines for performance, scalability, and reliability.
Collaborate with data architects, analysts, and business stakeholders to deliver data
solutions.
Perform troubleshooting, testing, deployment, and production support.
Ensure data quality, security, and governance standards are maintained.
Create technical documentation and follow Agile development practices.
Required Skills:
6–12 years of experience in Big Data/Data Engineering.
Strong hands-on experience with Apache Spark (PySpark/Scala).
Experience with Hadoop, Hive, HDFS, and YARN.
Strong SQL and data modeling skills.
Experience with Kafka or other streaming technologies.
Hands-on experience with Databricks is preferred.
Knowledge of cloud platforms such as AWS, Azure, or GCP.
Experience with ETL tools and workflow orchestration (Airflow/Oozie or similar).
Familiarity with Git and CI/CD practices.
Excellent analytical, debugging, and problem-solving skills.
Preferred Skills:
Experience with Snowflake, Delta Lake, or Data Lake architectures.
Knowledge of NoSQL databases (MongoDB, Cassandra, HBase).
Experience with containerization tools such as Docker and Kubernetes.
Exposure to Agile/Scrum methodologies.
Strong communication and stakeholder management skills.
Work Location: In person
