Job Description: Big Data Engineering – Senior Programmer Analyst (Exp: 3-5 Years)
We are looking for a highly skilled and experienced Big Data Engineering to join our team as a Senior Programmer Analyst who is responsible for designing, implementing, and maintaining efficient data ingestion and transformation workflows within the Cloudera cluster. By collaborating with data analysts, project leaders, and technical teams, the role ensures the delivery of high-quality data solutions that meet business needs. The position also focuses on establishing development standards, optimizing data pipelines, and supporting agile project execution to drive continuous improvement and operational excellence.
-
Mastering of Hadoop & Cloudera environment (including understanding insertion, retrieval performance, orchestration with yarn...)
-
Mastering of other Big Data stack components (Kafka, HDFS, Iceberg)
-
Knowledge of Azure Big Data systems and cloud computing.
-
High development skills (Spark, PySpark)
-
Knowledge of agile methodology
-
Very rigorist in incident management procedure
-
Fast adaptability to new technologies
-
Business oriented for fast development (Sens of delivery in time is very important)
-
Team spirit
-
Capability to work under time pressure
-
Experience with business intelligence tools
-
Ability to mentor and guide team members, ensuring adherence to development standards
Roles & Responsibilities:
Collaboration and Requirements Gathering:
- Engage with data analysts and project leaders to understand business needs and data flow requirements for various projects.
- Translate business requirements into technical specifications for data ingestion and transformation processes.
Data Ingestion and Transformation:
- Design and specify data ingestion and transformation workflows in the Cloudera cluster (ELT), ensuring scalability, efficiency, and reliability.
- Monitor data pipelines to ensure smooth operations and address any issues proactively.
Job Implementation and Development:
- Develop and implement Spark-based jobs and orchestration workflows to support data processing and transformation.
- Establish and enforce development standards to accelerate project delivery and maintain code quality.
Agile Methodology and Team Collaboration:
- Actively participate in agile practices, including daily stand-ups, sprint planning, and retrospective meetings, to ensure alignment with project goals.
- Collaborate with technical leads to report progress, share updates, and address challenges.
Code Review and Quality Assurance:
- Conduct code reviews to ensure adherence to best practices, coding standards, and performance optimization.
- Provide constructive feedback to team members to improve code quality and maintain consistency.
Incident Management and Evolution Requests:
- Follow up on incidents or requests for system evolution, ensuring timely resolution and continuous improvement of data pipelines.
- Work closely with stakeholders to implement changes and enhancements as needed.
