Exploring Rkt on Ubuntu

I have been using docker in my home server since 0.4 in 2013. For me the most attractive property of docker is that it provides a way to decrease the amount of stuff one has to install on a server. I have only one server, but it has many different tasks of which some need to be rock solid (my family email) and other are experimental. Containers provide a nice way to clean up experiments. Unfortunately, docker has never been stable. I have had many fights with docker during upgrades and I have never fully understood how docker interacts with the iptables setup from my firewall (Shorewall).

Today, I was trying to run docker-syncthing. Unfortunately, the container went into 'restarting' without any indication what was wrong. Inspired by "Can docker be ousted" and boring and stable containers I decide to give up and try something new: CoreOS' rkt.

Installation

The installation instructions are a bit hidden. This page refers to the script install-rkt.sh. Unfortunately, no link was given. A search on github finally gave the answer. However, in the end, I liked these instructions better: ask ubuntu: Is it possible to install rkt in Ubuntu?.

Because I wanted to run Syncthing and I only had a Docker recipe, I also needed the tool docker2aci. The instructions there are clear except that you need to install golang first (docs are fixed now):

$ sudo apt-get install golang $ git clone git://github.com/appc/docker2aci $ cd docker2aci $ ./build.sh

The docker2aci binary is located in the bin folder. Put it somewhere so you can find it back.

Building and converting a docker image

After building the docker image:

# docker build -t syncthing .

I had the following:

# docker images REPOSITORY TAG IMAGE ID CREATED SIZE syncthing latest 8ea0931f1196 29 hours ago 197.1 MB

Next step is to fetch the image into the rkt image repository. The rkt fetch command can fetch an image directly from a docker repository. However, I found no way to fetch directly from the local docker image repository. This is the work around:

# docker save -o syncthing-docker-image.tar syncthing # docker2aci syncthing-docker-image.tar # rkt --insecure-options=image fetch syncthing-latest.aci # rkt image list ID NAME SIZE IMPORT TIME LAST USED sha512-8161ad07a42e syncthing:latest 168MiB 7 hours ago 7 hours ago # rkt image cat-manifest syncthing:latest | less

Running the image

Now comes the time to run the image:

# rkt --insecure-options=image run --net=host \ --dns=$(awk '/nameserver/ {print $2}' < /etc/resolv.conf) \ --volume=volume-srv-config,kind=host,source=/media/nas/syncthing/config,readOnly=false \ --volume=volume-srv-data,kind=host,source=/media/nas/syncthing/data,readOnly=false \ syncthing

Creating that statement took actually longer then I expected. Here are the highlights:

• Option --dns=... copies the nameserver from you local /etc/resolv.conf to the same file inside the container. This makes DNS work inside the container also. Depending on your DNS setup, you may need to pass more --dns* options.
• The --net=host gives you the easiest access to the network. If you want a bit more security, you will have to dive deeper.
• I also tried to add the options --user=nobody --group=nogroup. However, that resulted in my container not starting up at all with weird error messages.

You now have something like this:

# rkt list UUID APP IMAGE NAME STATE CREATED STARTED NETWORKS 17baa16c syncthing syncthing:latest running 1 minute ago 1 minute ago

Inspecting the container

Inspecting a running container is easy. With rkt enter you can directly open a shell in the container:

# rkt enter 17baa16c enter: no command specified, assuming "/bin/bash" root@rkt-5e9ad759-82b4-4b27-b03a-b6b5074b2ac2:/#

Cleanup

Every time you start a new container, the old one stays around. With rkt gc you cleanup containers that stopped some time ago (more then half an hour?). This command should be run from cron:

# echo -e '#!/bin/sh\nexec rkt gc' > /etc/cron.daily/rkt-gc # chmod +x /etc/cron.daily/rkt-gc

Networking

As a Shorewall user you need to add a rule to open the ports for the syncthing application in /etc/shorewall/rules and that's it. This might get a lot more hairy when you are not using host networking.

Automating startup

Modern Ubuntu's use systemd to start applications. Rkt's systemd manual seems well written but can be used as an introduction as best. Their suggestion to use systemd-run failed me:

# systemd-run --slice=machine /usr/bin/rkt --insecure-options=image run --net=host --dns=192.168.1.1 --volume=volume-srv-config,kind=host,source=/media/nas/syncthing/config,readOnly=false --volume=volume-srv-data,kind=host,source=/media/nas/syncthing/data,readOnly=false syncthing Failed to start transient service unit: Cannot set property ExecStart, or unknown property.

Also, it fails to mention where to put the systemd unit file. After lots of reading (in particular the systemd section of this manual) I created /etc/systemd/system/rkt-syncthing.service with the following content:

[Unit] Description=Rkt syncthing Requires=remote-fs.target After=remote-fs.target [Service] Slice=machine.slice ExecStart=/usr/bin/rkt --insecure-options=image run --net=host --dns=192.168.1.1 --volume=volume-srv-config,kind=host,source=/media/nas/syncthing/config,readOnly=false --volume=volume-srv-data,kind=host,source=/media/nas/syncthing/data,readOnly=false syncthing KillMode=mixed Restart=always [Install] WantedBy=multi-user.target

And finally:

# systemctl daemon-reload # systemctl enable syncthing.service # systemctl start syncthing.service

Ending thoughts

Although I got far in a few hours there are still a few open problems.

1. None of the above software was available as a Ubuntu package. This results in me spending more time to keep everything up to date.
2. Rkt's documentation is okay but not amazing. For example, hyperlinks are missing in crucial places (see above), and in the manual page for rkt-export does not tell you how to indicate which container you want to export.
3. Rkt containers run from an image and store changes as a layer on top. As soon the container exists, you can not start it again with those changes. This means that everything that needs to be persisted must be external to the container. For other changes you can run rkt export to create a new image, or you rebuild your image from scratch.
4. Having to use Docker to build an image for rkt is a bit weird. Next step is to create the image directly with acbuild.

Updates

2017-02-05 Fixed commands for enabling rkt garbage collection via cron.