Wb Senior spark / pyspark data engineer Jobs – 3 Open Jobs Today

Senior Spark / PySpark Data Engineer
Web Spiders —Kolkata, West Bengal
- Full-time
- Software troubleshooting
- AWS
- AI
1d
Senior AWS EMR Engineer
Web Spiders —Kolkata, West Bengal
- Full-time
- AWS
- AI
1d
Senior Associate - Data Science
mPokket —Kolkata, West Bengal
- Full-time
- Microsoft Excel
- Azure
- Computer Science

More jobs to discover

Find similar jobs that you might be interested in.

Senior AWS EMR Engineer
Web Spiders —Kolkata, West Bengal
5–10 minute initial screening call with the TA team. If AWS EMR + Spark/Hadoop is your core expertise, we'd love to hear from you.
1d
Senior Spark / PySpark Data Engineer
Web Spiders —Kolkata, West Bengal
The ideal candidate will have strong experience designing and developing high-performance ETL/ELT pipelines and distributed data-processing solutions using…
1d

I want to receive the latest job alerts for senior spark / pyspark data engineer in wb

By signing in to your account, you agree to SimplyHired's Terms of Service and consent to our Cookie and Privacy Policy.

Explore jobs in more locations

senior spark / pyspark data engineer jobs near wb

Data Engineer jobs in Kolkata, West Bengal

Data Scientist jobs in Kolkata, West Bengal

Senior Engineer jobs in Kolkata, West Bengal

Civil Engineer jobs in Pune, Maharashtra

Back Office , Bank, Data Entry Operator jobs in Jalgaon, Maharashtra

Data Engineer jobs in Mumbai Central, Mumbai, Maharashtra

Data Engineer jobs in Ahmedabad, Gujarat

Civil Engineer jobs in Chandigarh, Chandigarh

Data Engineer jobs in Orissa

Data Engineer jobs in Hinjewadi, Pune, Maharashtra

Data Scientist jobs in Thiruporur Road, Chennai, Tamil Nadu

Data Scientist jobs in Gandhinagar, Gujarat

Data Scientist jobs in Ambattur, Chennai, Tamil Nadu

Data Science jobs in Vadodara, Gujarat

Data Scientist jobs in Hsr Layout Sector 1, Bengaluru, Karnataka

Data Scientist Banking jobs in Mumbai, Maharashtra

Senior Engineer jobs in Dera Bassi, Punjab

Senior Engineer jobs in Mathura, Uttar Pradesh

Senior Engineer jobs in Puducherry, Puducherry

Senior Engineer jobs in Bahadurgarh, Haryana

Senior Spark / PySpark Data Engineer

Web Spiders -
Kolkata, West Bengal

Apply Now

Job details

Full-time
1 day ago

Qualifications

CI/CD
Software troubleshooting
Software deployment
Spark
AWS Certification
AWS
Distributed systems
Continuous integration
ETL
S3
Apache
Redshift
AI
Data warehouse
Python
Hadoop

Full job description

Location

Kolkata (Rajarhat–Newtown)

Type

Full Time

Department

Technology

Web Spiders is looking for a Senior Spark / PySpark Data Engineer with strong hands-on experience in Apache Spark, PySpark, Python, and large-scale data engineering. The ideal candidate will have strong experience designing and developing high-performance ETL/ELT pipelines and distributed data-processing solutions using Spark/PySpark, along with experience working with cloud-based data platforms and AWS services. If Spark + PySpark + Python is your core expertise and you enjoy solving complex data-processing and scalability challenges, we'd love to hear from you. 5+ Years Experience | Kolkata – Work from Office Core Stack: Apache Spark • PySpark • Python • ETL/ELT • AWS • S3 • Glue • Redshift * Immediate joiners preferred.* Working Hours: Ability to work in the US Eastern Time Zone. Depending on project requirements, this may be adjusted to a half-day IST + half-day US EST schedule. What You'll Do: Design, develop, and optimize large-scale ETL/ELT pipelines using Apache Spark and PySpark. Develop scalable data transformation and processing solutions using PySpark and Python. Build distributed data-processing applications capable of handling large volumes of data. Develop reusable and maintainable Spark/PySpark frameworks and data-processing components. Optimize Spark jobs for performance, scalability, memory utilization, and execution efficiency. Work with complex transformations, joins, aggregations, partitioning, and large datasets. Implement data validation, quality checks, error handling, and monitoring within data pipelines. Work with AWS data services including EMR, Glue, S3, and Redshift. Develop data pipelines supporting data lakes, warehouses, analytics, and downstream applications. Troubleshoot production data pipeline and Spark processing issues. Identify and resolve performance bottlenecks in Spark/PySpark workloads. Collaborate with Data Engineering, Cloud, AI/ML, and Product teams to deliver reliable data solutions. ‍ Must-Have Skills: 5+ years of hands-on experience in Data Engineering. Strong hands-on experience with Apache Spark. Strong hands-on experience with PySpark. Strong programming experience in Python. Proven experience developing and optimizing large-scale ETL/ELT pipelines. Strong understanding of distributed computing and data-processing concepts. Experience working with large datasets and complex data transformations. Strong understanding of Spark performance optimization and tuning. Experience with cloud-based data engineering, preferably AWS. Experience with Amazon S3 and at least one AWS data-processing service such as EMR or Glue. ‍ Good to Have: AWS EMR AWS Glue Apache Airflow / MWAA AWS Step Functions Amazon Redshift Hadoop ecosystem Experience with data lake and data warehouse architectures. Experience with CI/CD and production deployment of data pipelines. AWS Certified Data Engineer or another relevant AWS certification. ‍ Interview Process Application review 5–10 minute initial screening call with the TA team Technical interviews {Domain specific} ‍Practical test conducted in the presence of a panel member ‍Role match & offer

Apply Now

Refine Your Search

senior spark / pyspark data engineer jobs in wb

Senior Spark / PySpark Data Engineer

Senior AWS EMR Engineer

Senior Associate - Data Science

More jobs to discover

I want to receive the latest job alerts for senior spark / pyspark data engineer in wb

Explore jobs in more locations

Location

Type

Department

Jobseeker tools

Employer Tools

Browse

Stay Connected