About This Role
4 to 7 years of experience
Skills: Cloudera Admin, Cloudera Hadoop, Kafka, Impala, Cloudera Manager
Job Description
- Need a Sr. Cloudera administrator having 4+ to work with our team to manage a solution having 20+ PB of data and develop on CDP 7.1.9 Platform who can ensure the smooth operation of existing system and can help in integration with other system both downstream and upstream.
- Responsible for implementation and ongoing IPDR solution developed on Data platform CDP 1.7.9 infrastructure.
- Manage and support existing cloudera services as well as commission new nodes/services in the ecosystem using tools Cloudera Manager Performance tuning of Hadoop clusters and Hadoop MapReduce routines.
- Employ security using an authentication and authorization system such as Kerberos, . Must have good experience to secure the cluster by implementing user management using Kerberos, LDAP with a good understanding ACLs.
- Screen cluster job performances, capacity planning and rebalancing nodes in the cluster
- Troubleshooting system capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks. Proving logical recommendations and applying Cloudera best practices.
- Monitor cluster connectivity and security.
- Manage and review application & system and log files
- File system management and monitoring.
- Familiarity with DevOps practices, including CI/CD pipelines, monitoring, and logging.
- Experience with tools like Zookeeper, Schema Registry, and Kafka Connect.
- Strong problem-solving skills and the ability to troubleshoot complex issues in a distributed environment
- Design, implement, and manage Kafka-based data pipelines and messaging solutions to support critical business operations and enable real-time data processing.
- Configure, deploy, and maintain Kafka clusters, ensuring high availability and scalability to maximize uptime and support business growth.
- Develop and maintain Kafka connectors such as JDBC, MongoDB, and S3 connectors, along with topics and schemas, to streamline data ingestion from databases, NoSQL data stores, and cloud storage, enabling faster data insights.
- Familiarity with apache open-source configuration management and deployment tools
- Apply Patches and enable new components from the Hadoop ecosystem
- Working experience on Cloudera Distribution.
- Strong knowledge of Impala parameter configuration
- Maintain documentation of Kafka environments, configurations, and processes to ensure knowledge transfer, compliance, and smooth team collaboration.
- Strong knowledge of Object storage Ozone and parameter configuration of Iceberg table configuration.
- Cloudera skills like HDFS, HBase, Hive, Impala, Kudu, Spark 3.x , Iceberg etc.
- Good understanding of Linux Commands and Shell Scripting
- Good understanding of Security, Kerberos etc
Qualifications:
- Bachelor’s degree in computer science, Information Technology, or a related field.
- 4-7 years of experience as a Cloudera Admin/Developer.
- Strong expertise in Cloudera Hadoop, Kafka, Impala, Cloudera Manager, and related big data technologies.
- 4-7 years of experience working with Apache Kafka in a production environment.
- Proficiency in Linux/Unix administration.
- Experience with scripting languages such as Python, Shell, or Perl.
- Solid understanding of Hadoop ecosystem components (HDFS, YARN, MapReduce, Hive, HBase, etc.).
- Proficiency in Java, Scala, or Python for Kafka-related development tasks.
- Experience with data ingestion tools (Sqoop, Flume, Kafka) and data processing frameworks (Spark, Flink).
- Knowledge of database management systems and SQL.
- Excellent problem-solving skills and attention to detail.
- Strong communication and collaboration skills.
