Role Description
•
Design, build, and optimize Kafka-based messaging and streaming pipelines for large-scale, high-throughput applications.
•
Troubleshoot Kafka messaging issues, analyzing broker, producer, and consumer logs to identify and resolve performance bottlenecks.
•
Manage and fine-tune Kafka clusters, including brokers, topics, partitions, replication factors, and consumer groups.
•
Implement and maintain Kafka security features (SSL, SASL, ACLs, Kerberos) and ensure compliance with organizational standards.
•
Integrate Kafka with downstream systems using Kafka Connect, Schema Registry, and REST APIs.
•
Monitor Kafka clusters and pipelines using tools such as Confluent Control Center, Prometheus, Grafana, or Datadog to track consumer lag and key performance metrics.
•
Collaborate with architects and developers to design scalable, fault-tolerant, and resilient Kafka solutions.
•
Contribute to system design and architecture discussions for enterprise data streaming solutions.
Qualifications
•
8–10 years of overall IT experience with 5+ years in Kafka ecosystem.
•
Strong expertise in Kafka core components (brokers, producers, consumers, topics, partitions, replication, offsets).
•
Hands-on experience with Confluent Kafka, Kafka Streams, and Kafka Connect.
•
Proficiency in Java/Scala (Python is a plus).
•
Strong knowledge of log analysis, debugging, and performance tuning in distributed systems.
•
Experience with monitoring and metrics tools (Grafana, Prometheus, Datadog, or similar).
•
Deep understanding of distributed system design principles and event-driven architecture.
Location: , ,
Remotive: Apply Here