Writing Stream Processors in Kotlin KotlinConf 2018 - Amsterdam
Who of you uses ?
A quick look over the shoulder 3
Kafk fka in in a Nutshell Source: Wikipedia
What is is so special about Kafka? • Persistent • Time and Log compaction • Guaranteed order per partition 5
Messages • Have a byte[] key • Have a byte[] value • Contain Metadata like a timestamp and the partition
Image from http://kafka.apache.org/documentation/
Event Streams • null-key and a bare or serialized value • Compaction of the topic by time • Examples: Clickstreams, Logs, IoT events
Log Compaction Image from http://cloudurable.com/blog/kafka-architecture-log-compaction/index.html
Stream Processors • Imagine you could do Kotlin Sequences on an infinite list of messages • Stateful • Parallel • And the output is one or more Kafka topic(s)
What now? • I do home automation, a lot • I use homee, which aggregates multiple protocols (Z-Wave, ZigBee, etc) • It has no official API, yet • But … They (unintentionally) made one of their repos public on Github • And now I have an API Blueprint file ☺ • Guess what? It’s a websocket!
Let’s get started • The websocket issues text messages when something happens • Attribute Changes of any connected device (motion, temperature, light, door opens, etc.) • System Events (user created, system update, etc.) • We use Kafka Connect to build what is called a “source” and pipe these events into Kafka.
Let’s look at some code (no live demo, sorry)
Now, why Kotlin? The Java code did not look that bad…
An engineering vie iew • Less verbose, better to read • Extension functions enable composable, high level abstractions • Extensions can be called from the Java folks • One can write libraries with business abstractions, reusable within the company, that blend into the original API
A team le leads vie iew • Onboarding people coming from non-JVM languages to Kotlin is easier than Java • The education material available is very rich, based on current teaching standards • People love the language and the feeling of productivity • Unmodifiable / Immutable helps a lot of people to write more stable code, • resulting in better outcome and higher motivation to go on with learning • because “it just works es expected” • You can write good code even without knowing the language in-depth
Kotlin at tri rivago • More than 80% of our Kafka related source code is Kotlin • Kafka Connect Sinks, Transforms, Converters • Stream Processors • Custom Solutions, based on Spring Boot 2, Spring Kafka, Spring Integration • My current team writes client facing REST and GRPC services based on Spring Boot 2 entirely in Kotlin • Our recently relaunched Android App is written entirely in Kotlin • Some of our B2B focused teams started to rewrite their PHP applications in Kotlin
Curious about how we use Apache Kafka? Kafka and Debezium at trivago (code.talks 2017) https://speakerdeck.com/xenji/kafka-and-debezium-at-trivago-code-dot-talks-2017-edition
Recommend
More recommend
