- Key Responsibilities
- Design develop and maintain scalable batch and near real time data pipelines using Scala and PySpark on distributed data platforms
- Lead end to end implementation of data engineering solutions from requirement analysis and design to deployment and support
- Optimize Spark jobs for performance reliability and cost efficiency including partitioning caching and resource tuning
- Implement robust data quality checks validation frameworks and monitoring to ensure accuracy and completeness of data
- Collaborate with architects and stakeholders to define data models data flows and integration patterns aligned with business needs
- Establish and enforce coding standards best practices and review processes for Scala and PySpark development
- Mentor junior engineers provide technical guidance and foster a culture of knowledge sharing and continuous improvement
- Troubleshoot complex production issues perform root cause analysis and implement long term preventive solutions
- Contribute to roadmap planning effort estimation and prioritization of data engineering initiatives as a technology lead
- Primary skills Domain Finacle Core Functional Finacle Core WMS Grand Master Technology Java Apache
- Knowledge of more than one technology
- Basics of Architecture and Design fundamentals
- Knowledge of Testing tools
- Knowledge of agile methodologies
- Understanding of Project life cycle activities on development and maintenance projects
- Understanding of one or more Estimation methodologies Knowledge of Quality processes
- Basics of business domain to understand the business requirements
- Analytical abilities Strong Technical Skills Good communication skills
- Good understanding of the technology and domain
- Ability to demonstrate a sound understanding of software quality assurance principles SOLID design principles and modelling methods
- Awareness of latest technologies and trends
- Excellent problem solving analytical and debugging skills
Technology->Java->Apache->Scala,Technology->Big Data - Data Processing->PySpark
