Project Role : Application Developer
Project Role Description : Design, build and configure applications to meet business process and application requirements.
Must have skills : Apache Hadoop
Good to have skills : NA
Minimum
3 year(s) of experience is required
Educational Qualification : 15 years full time education
Role Summary:
We are looking for a skilled Data Engineer to design, build, and maintain scalable data ingestion and processing pipelines for a modern data lake environment. The role involves working with Hadoop ecosystem technologies, Spark (Scala), Hive, SQL, and Unix scripting to support enterprise-grade data platforms.
Key Responsibilities
Design and develop data ingestion pipelines to load structured and unstructured data into the enterprise Data Lake.
Own the documentation, design, development, and architecture of Hadoop-based applications.
Develop, manage, and optimize data stored in Hive databases to support analytics and reporting needs.
Build and maintain data ingestion and transformation jobs using Apache Spark with Scala.
Write clean, efficient, and maintainable code test, debug, and optimize programs following best engineering practices.
Develop Unix shell scripts for automation, scheduling, and operational workflows, including SFTP-based file transfers across environments.
Write and optimize complex SQL queries to support data transformations, validations, and business logic.
Collaborate with cross-functional teams to ensure data quality, reliability, and performance.
Required Skills & Experience
Strong hands-on experience with Hadoop ecosystem (HDFS, Hive).
Solid expertise in Apache Spark (Scala) for data processing.
Proficiency in SQL, including complex joins, aggregations, and performance tuning.
Strong experience with Unix/Linux scripting and file transfer protocols (SFTP).
Experience designing and supporting data lake architectures.
Understanding of data engineering best practices, debugging, and performance optimization.
Additional Information:
- The candidate should have minimum 4 years of experience in Apache Hadoop.
- This position is based at our Chennai office.
- A 15 years full time education is required.
