Confluent
Confluent at VLDB 2015 | Building a Replicated Logging System with Apache Kafka
Unternehmen

Confluent at VLDB 2015 | Building a Replicated Logging System with Apache Kafka

Guozhang Wang

There has been much renewed interest in using log-centric architectures to scale distributed systems that provide efficient durability and high availability. In this approach, a collection of distributed servers can operate on a replicated log that record state changes in sequential ordering. The log itself can then be treated as the “source-of-truth”: when some of the servers fail and come back, their states can be deterministically reconstructed by replaying this log upon recovery.

Over the past years of developing and operating Kafka, we have envisioned and exercised the idea of extending its commit-log structured architecture into a replicated logging system in order to serve as the underlying data flow backbone for a wide scope of applications, such as data integration, commit log replication, and stream processing, etc. In this year’s Very Large Data Bases conference I will talk about our experience in building such a replicated logging system using Kafka and will present several of its use cases.

If you happen to be attending the VLDB conference and you’re interested in learning more about how to build a replicated log using Kafka, how to deploy it as your commit log replication layer underlying your distributed stores, etc., I invite you to attend my session or find me at the conference.

Building a Replicated Logging System with Apache Kafka
Guozhang Wang, Confluent
10:30am – 12:00pm, Thursday, September 3, 2015
41st International Conference on Very Large Data Bases
Hilton Waikoloa Hotel | Kohala Coast, Hawai’i | August 31 – September 4, 2015

You may also be interested in these blog posts by Jay Kreps (Kafka co-creator):

Putting Apache Kafka To Use: A Practical Guide to Building a Stream Data Platform (Part 1)

Putting Apache Kafka To Use: A Practical Guide to Building a Stream Data Platform (Part 2)

Feel free to share your feedback, questions, and suggestions — about my conference talk or about Kafka in general — with us at any time via https://www.confluent.io/contact or @ConfluentInc on Twitter.

Subscribe to the Confluent Blog

Abonnieren

More Articles Like This

The State of Streams – A European Adventure
Gwen Shapira

The State of Streams – A European Adventure

Gwen Shapira .

I have good news to share! For the first time, this October Confluent will bring to Europe two events dedicated to streaming technology: Confluent Streaming Event in Munich and Confluent ...

Streams and Tables: Two Sides of the Same Coin
Matthias J. Sax

Streams and Tables: Two Sides of the Same Coin

Matthias J. Sax .

We are happy to announce that our paper Streams and Tables: Two Sides of the Same Coin is published and available for free download. The paper was presented at the ...

Bill Bejeck

Predicting Flight Arrivals with the Apache Kafka Streams API

Bill Bejeck .

Kafka Streams makes it easy to write scalable, fault-tolerant, and real-time production apps and microservices. This post builds upon a previous post that covered scalable machine learning with Apache Kafka, ...

Fully managed Apache Kafka as a Service

Try Free

Wir verwenden Cookies, damit wir nachvollziehen können, wie Sie unsere Website verwenden, und um Ihr Erlebnis zu optimieren. Klicken Sie hier, wenn Sie mehr erfahren oder Ihre Cookie-Einstellungen ändern möchten. Wenn Sie weiter auf dieser Website surfen, stimmen Sie unserer Nutzung von Cookies zu.