Your goal is simple: take that is happening in your company—every click, every database change, every application log—and made it all available as a real-time stream of well-structured data? No big deal! You’re just taking your decades-old, batch-oriented data integration and data processing and migrating to to real-time streams and real-time processing. In your shop, you call that Tuesday. But of the several challenges to tackle, you’ll have to get data in and out of that stream processing system, and there’s a whole bunch of code there you don’t want to write. This is where Kafka Connect comes in.

Connect is a standard part of Apache Kafka that provides a scalable, fault-tolerant way to stream data between Kafka and other systems. It provides an API for developing standard connectors for common data sources and sinks, giving you the ability to ingest database changes, write streams to tables, store archives in HDFS, and more. We’ll explore the standard connector implementations offered in the Confluent Open Source download, and look at a few operational questions as well. Come to this session to get connected to Kafka!