JAX London, 4–7 October 2021
The Conference for Java & Software Innovation

How to Manage State in Apache Kafka

This talk originates from the archive. To the CURRENT program
Until JAX London starts: ✓ 4-Day-Pass all inclusive ✓ Team discounts Register now
Wednesday, October 9 2019
16:35 - 17:25

Apache Kafka has transformed our ability to move and process data. The old accepted approaches of request/response and RDBMS in system design are no longer the de facto options. However, what does this mean for the way we manage and store the state within our systems?

Event logs often have the full state within them, but that doesn’t necessarily mean that reading the event log from the start to build the current state is the best way to store and manage it. When considering event log based systems, it is vital to think carefully not only about functional considerations, but also non-functional considerations.

This talk is about the importance of considering performance, data consistency and latency from the start when architecturally planning event log systems. It will give examples how developers who were new to Kafka tried various different approaches to managing state in such systems. It shows the successes, the failures and the lessons learned and how an approach to managing state in an event log evolved, and where we hope to take it.

Does Kafka’s long living, persistent event log mean that we can now realise a utopia of fully stateless systems? Look before you leap, says Ben Abramson …

Behind the Tracks

Software Architecture & Design
Software innovation & more
Architecture structure & more
Agile & Communication
Methodologies & more
Emerging Technologies
Everything about the latest technologies
DevOps & Continuous Delivery
Delivery Pipelines, Testing & more
Cloud & Modern Infrastructure
Everything about new tools and platforms
Big Data & Machine Learning
Saving, processing & more