Experience: 5+ years
Location: Pune, India (Work from Office)
Must Have:
-
Spark development with batch and streaming experience
-
Experience with Pyspark and Data Engineering
-
ETL implementation and migration to spark
-
Programming knowledge on scala, java & python
-
Experience with Kafka and Spark streaming (Dstream and Structured Streaming)
-
Experience with using Jupyter notebooks or any other developer tool
-
Experience with Airflow or other workflow engines
Good to Have Skills:
-
Flink and Kudu streaming
-
Automation of workflows CI/CD
-
Nifi streaming and transformation
-
Informatica workflow migration
