- Join a fast paced engineering team where you ll build and optimize data driven solutions using Python Apache Spark and Scala
- In this role you ll collaborate closely with data engineers analysts and platform teams to deliver reliable pipelines and scalable processing jobs that turn raw data into trusted insights
- You ll contribute across the development lifecycle design implementation testing deployment and performance tuning while learning from peers and sharing your own expertise
- If you enjoy solving complex data challenges improving runtime efficiency and writing clean maintainable code this is a great opportunity to grow your impact in a collaborative quality focused environment
- Data Engineering Development
- Develop and maintain batch stream data processing jobs using Spark with Scala and or Python
- Build reusable components and utilities to standardize ingestion transformation and validation patterns
- Implement efficient transformations joins aggregations and partitioning strategies for large scale datasets
- Performance Quality Reliability
- Tune Spark jobs for performance e
- g
- caching shuffles partition sizing and troubleshoot failures or bottlenecks
- Write unit integration tests and ensure code quality through reviews refactoring and documentation
- Monitor job runs analyze logs metrics and drive continuous improvements in stability and runtime
- Data SQL
- Use SQL to validate data reconcile outputs and support downstream reporting analytics needs
- Collaborate with stakeholders to clarify requirements and translate them into robust data processing logic
- Bachelor s degree in Computer Science Engineering or a related field
- 2 5 years of hands on experience building solutions with Python
- 2 5 years of experience working with Apache Spark for large scale data processing
- 2 5 years of experience developing with Scala or strong ability to work across Scala and Python codebases
- Working knowledge of SQL for querying validation and troubleshooting datasets
- Strong problem solving skills and ability to collaborate effectively in a team environment
Technology->Analytics - Packages->Python - Big Data,Technology->Big Data - Data Processing->Spark,Technology->Functional Programming->Scala
