Java plugins with isolating class loaders

My team's article on how to write Java plugins has been published on the Adevinta Tech Blog. Enjoy!

Making ZIO-Kafka Safer And Faster

My talk "Making ZIO-Kafka Safer And Faster" at Functional Scala 2023 went online!

Explore Erik van Oosten's presentation on improving ZIO-Kafka for better safety and performance. Learn about the modifications introduced in 2023, get insights into the library's internal workings, and uncover useful ZIO techniques and Kafka's lesser-known challenges.

Contents in the video:

2:07 Improvements
9:06 Results
10:29 Rebalances
18:10 The Future

Tips for running Roundcube for years

I have been running a Roundcube instance for about 8 years now. At the beginning I only used it as a backup email client that can be invoked from anywhere. Nowadays, is it so good that I didn't even bother installing Thunderbird on my work laptop.

Unfortunately, I discovered that the docker backup of Roundcube had become quite large, many GBs. This was quite unexpected for a service that is used by only 2 people. The reason was quickly found: the sqlite database was huge!

What did I know? I though ony Postgresql needed scheduled cleanups. Turns out sqlite needs it too! Would this be the reason Android phones tend to fill up over time?

Anyways, the fix was simple: run the vacuum command! So, now I have the following run daily using cron. Problem solved!

sqlite3 /paht/to/roundcubemail.sqlite 'VACUUM;'

Scheduling tasks and sharing state with streams

Recently we built a system that needs to perform 2 tasks. Taks 1 runs every 15 minutes, task 2 runs every 2 minutes. Task 1 kicks off some background jobs (an upload to BigQuery), task 2 checks upon the results of these background jobs and does some cleanup when they are done (delete the uploaded files). The two tasks need to share information back and forth about what jobs are running in the background.

Now think to yourself (ignore the title for now 😉), what would be the most elegant way to implement this? I suspect that most developers will come with a solution that involves some locking, synchronisation and global state. For example by sharing the information through a transactional database, or by using a semaphore to prevent the two tasks from running at the same time plus sharing information in a global variable. This is understandable, most programming environments do not provide better techniques for these kinds of problems at all!

However, if your environment supports streams and has some kind of scheduling, here are two tricks you can use: one for the scheduling of the tasks, the second for sharing information without a global variable.

Here is an example for the first written in Scala using the ZIO streams library. Read on for an explanation.

import zio._ import zio.stream._ def performTask1: Task[Unit] = ??? def performTask2: Task[Unit] = ??? // An enumeration (scala 2 style) for our task. sealed trait BusinessTask object Task1 extends BusinessTask object Task2 extends BusinessTask ZStream.mergeAllUnbounded()( ZStream.fromSchedule(Schedule.fixed(15.minutes)).as(Task1), ZStream.fromSchedule(Schedule.fixed(2.minutes)).as(Task2) ) .mapZIO { case Task1 => performTask1 case Task2 => performTask2 } .runDrain

We create 2 streams, each stream contains sequential numbers, emitted upon a schedule. As you can see, the schedule corresponds directly with the requirements. We do not really care for the sequential numbers, so with stream operator as we convert the stream's emitted values to a value from the BusinessTask enumeration.

Then we merge the two streams. We now have a stream that emits the two enumeration values at the time the corresponding task should run. This is already a big win! Even when the two schedules produce an item at the same time, the tasks will run sequentially. This is because by default streams are evaluated without parallelism.

We are not there yet though. The tasks need to share information. They could access a shared variable but then we still have tightly coupled components and no guarantees that the shared variable is used correctly.

Also, wouldn't it be great if performTask1 and performTask2 are functions that can be tested in isolation? With streams this is possible.

Here is the second part of the idea. Again, read on for an explanation.

case class State(...) val initialState = State(...) def performTask1(state: State): Task[State] = ??? def performTask2(state: State): Task[State] = ??? ZStream.mergeAllUnbounded()( ZStream.fromSchedule(Schedule.fixed(15.minutes)).as(Task1), ZStream.fromSchedule(Schedule.fixed(2.minutes)).as(Task2) ) .scanZIO(initialState) { (state, task) => task match { case Task1 => performTask1(state) case Task2 => performTask2(state) } } .runDrain

We have changed the signatures of the performTask* methods. Also, the mapZIO operator has been replaced with scanZIO. The stream operator scanZIO works much like foldLeft on collections. Like foldLeft, it accepts an initial state, and a function that combines the accumulated state plus the next stream element (of type BusinessTask) and converts those into the next state.

Stream operator scanZIO also emits the new states. This allows further common processing. For example we can persist the state to disk, or collect custom metrics about the state.

Conclusion

Using libraries with higher level constructs like streams, we can express straightforward requirements in a straightforward way. With a few lines of code we have solved the scheduling requirement, and showed an elegant way of sharing information between tasks without global variables.

Discovering scala-cli while fixing my digital photo archive

Over the years I built up a nice digital photo library with my family. It is a messy process. Here are some of the things that can go wrong:

• Digital cameras that add incompatible exif metadata.
• Some files have exif tag CreateDate, others DateTimeOriginal.
• Images shared via Whatsapp or Signal do not have an exif date tag at all.
• Wrong rotation.
• Fuzzy, yet memorable jpeg images wich take 15MB because of their resolution and high quality settings.
• Badly supported ancient movie formats like 3gp and RIFF AVI.
• Old movie formats that need 3 times more disk space than h.265.
• Losing almost all your photos because you thought you could copy an Iphoto library using tar and cp (hint: you can’t). (This took a low-level harddisk scan and months of manual de-duplication work to recover the photos.)
• Another low-level scan of an SD card to find accidentally deleted photos.
• Date in image file name corresponds to import date, not creation date.
• Weird file names that order the files differently than from creation date.
• Images from 2015 are stored in the folder for 2009.
• etc.

I wrote countless bash scripts to mold the collection into order. Unfortunately, to various success. However, now that I am ready to import the library into Immich (please, do sponsor them, they are building a very nice product!), I decided to start cleaning up everything.

So there I was, writing yet another bash script, struggling with parsing a date response from exiftool. And then I remembered the recent articles about scala-cli and decided to try it out.

The experience was amazing! Even without proper IDE support, I was able to crank out scripts that did more, more accurately and faster than I could ever have accomplished in bash.

Here are some of the things that I learned:

• Take the time to learn os-lib.
• If the scala code gets harder to write, open a proper IDE and use code completion. Then copy the code over to your .sc file.
• One well placed .par (using scala-parallel-collections) can more than quadruple the performance of your script!
• You will still spend a lot of time parsing the output from other programs (like exiftoool).
• Scala-cli scripts run very well from Github actions as well.

Conclusions

Next time you open your editor to write a bash file, think again. Perhaps you should really write some scala instead.

Dependabot, Gradle and Scala

Due to a series of unfortunate circumstances we have to deal with a couple of projects that use Gradle as build tool at work. For these projects we wanted automatic PR generation for updated dependencies. Since we use Github Enterprise, using Dependabot seems logical. However, this turned out to be not very straightforward. This article documents one way that works for us.

As we were experimenting with Dependabot, we discovered the following rules:

1. The scala version in the artifact name must not be a variable.
2. A variable for the artifact version is fine, but it must be declared in the same file in the ext block.
3. Versions should follow the Semver specification.
4. You must not use Gradle’s + version range syntax anywhere, Maven’s version range syntax is fine.

In our projects the scala version comes from a plugin. In addition, we sometimes need to cross build for different scala versions, very much at odds with rule no. 1. We solved this with a switch statement.

With these rules and constraints we discovered that the following structure works for us and Dependabot:

ext { jacksonVersion = '2.15.2' scalaTestVersion = '3.0.8' } dependencies { switch(scalaMainVersion) { case "2.12": implementation "com.fasterxml.jackson.module:jackson-module-scala_2.12:$jacksonVersion" testImplementation "org.scalatest:scalatest_2.12:$scalaTestVersion" break case "2.13": implementation "com.fasterxml.jackson.module:jackson-module-scala_2.13:$jacksonVersion" testImplementation "org.scalatest:scalatest_2.13:$scalaTestVersion" break default: break } // implementation 'com.example:library:0.8+' // Don't do this implementation 'com.example:library:[0.8,1.0[' // This is fine }

It took 3 people a month to slowly discover this solution (thank you!). I hope that you, dear reader, will spend your time more productive.

Zio-kafka hacking day

Not long ago I contacted Steven (committer of the zio-kafka library) to get some better understanding of how the library works. April 12, not more than 2 months later I am a committer, and I was sitting in a room together with Steven, Jules Ivanic (another committer) and wildcard Pierangelo Cecchetto (contributor), hacking on zio-kafka.

The meeting was an idea of Jules who was ‘in the neighborhood’. He was traveling from Australia for his company (Conduktor). We were able to get a nice room in the Amsterdam office of my employer (Adevinta). Amsterdam turned out to be a nice middle ground for Steven, me and Pierangelo. (Special thanks to Go Data Driven who also had place for us.)

In the morning we spoke about current and new ideas on how to improve the library. Also, we shared detailed knowledge on ZIO and what Kafka expects from its users. After lunch we started hacking. Having someone to start an ad hoc discussion turned out to be very productive; we were able to move some tough issues forward.

Here are some highlights.

PR #788 — Wait for stream end in rebalance listener is important to prevent duplicates during a rebalance process. This PR was mostly finished for quite some time, but many details made the extensive test suite fail. We were able to solve many of these issues.

In the area of performance we implemented an idea to replace buffering (pre-fetching a fixed number of polls), with pre-fetching based on the stream’s queue size. This resulted in PR #803 — Alternative backpressure mechanism.

We also laid the seeds for another performance improvement implementation: PR #908 — Optimistically resume partitions early.

These last two PRs showed great performance improvements bringing us much closer to direct usage of the Java Kafka client. All 3 PRs are now in review.

All in all it was a lot of fun to meet fellow enthusiasts and hack on the complex machinery that is inside zio-kafka.