Spark Developer
Location: Candor Tech Space Sector 21, 8th floor, Tower 6, Gurgaon.
Experience: 4+ Years
Salary: Best in Industry
Employment Type: Full-Time / Contract
Job Summary
We are looking for a skilled Spark Developer to design, build, and optimize scalable batch and real-time data pipelines. The ideal candidate should have hands-on expertise in Apache Spark, Apache Beam, Java/Scala/Python, SQL, and experience with enterprise data platforms such as Teradata and PostgreSQL.
Key Responsibilities
Design, develop, and maintain batch and real-time ETL/ELT pipelines using Apache Spark and Apache Beam.
Build scalable data ingestion pipelines from databases, APIs, and files.
Transform and process large-scale datasets using Java, Scala, Python, PySpark, and Spark Scala.
Develop efficient data pipelines for Teradata, PostgreSQL, and other analytical databases.
Work with multiple data formats including JSON, CSV, Parquet, ORC, XML, and Excel.
Develop streaming applications using Apache Kafka, Confluent Kafka, Spark Streaming, and Apache Flink.
Implement data quality checks, validation, monitoring, and performance optimization.
Collaborate with business stakeholders to translate reporting requirements into scalable data solutions.
Troubleshoot production issues and maintain high-performance data pipelines.
Required Skills
4+ years of experience in Data Engineering with strong expertise in Apache Spark, Apache Beam, and Apache Flink.
Strong programming skills in Java, Scala, Python, SQL, PySpark, and Spark Scala.
Hands-on experience with Teradata, PostgreSQL, and advanced SQL.
Experience in batch and real-time data processing.
Strong understanding of ETL/ELT architecture, data modeling, partitioning, orchestration, and data warehousing concepts.
Experience with Apache Kafka/Confluent Kafka for streaming data pipelines.
Excellent analytical, troubleshooting, and communication skills.
Preferred Skills
Experience with enterprise-scale Big Data platforms.
Knowledge of dimensional modeling, Slowly Changing Dimensions (SCD), and data governance.
Experience with pipeline monitoring, resource optimization, and cluster performance tuning.
Compensation & Work Model
Employment Type: Full-Time / Contract
Location: Candor Tech Space Sector 21, 8th floor, Tower 6, Gurgaon.
Salary: Best in Industry
Interested candidates can share their updated resume at: [email protected]
Pay: Up to ₹2,800,000.00 per year
Work Location: In person
