Project Metamorphosis: Wir präsentieren die Event-Streaming-Plattform der nächsten GenerationMehr Erfahren

Running Kafka for Maximum Pain

Kafka Summit SF 2017 | Use Case Track

Kafka makes so many things easier to do, from managing metrics to processing streams of data. Yet it seems that so many things we have done to this point in configuring and managing it have been object studies in how to make our lives, as the plumbers who keep the data flowing, more difficult than they have to be. What are some of our favorites?

  • Kafka without access controls
  • Multitenant clusters with no capacity controls
  • Worrying about message schemas
  • MirrorMaker inefficiencies
  • Hope and pray log compaction
  • Configurations as shared secrets
  • One-way upgrades

We’ve made a lot of progress over the last few years improving the situation, in part by focusing some of this incredibly talented community towards operational concerns. We’ll talk about the big mistakes you can avoid when setting up multi-tenant Kafka, and some that you still can’t. And we will talk about how to continue down the path of marrying the hot, new features with operational stability so we can all continue to come back here every year to talk about it.

Todd Palino
Staff Site Reliability Engineer, LinkedIn