Crate conmonrs

A pod level OCI container runtime monitor.

The goal of this project is to provide a container monitor in Rust. The scope of conmon-rs encompasses the scope of the c iteration of conmon, including daemonizing, holding open container standard streams, writing the exit code.

However, the goal of conmon-rs also extends past that of conmon, attempting to become a monitor for a full pod (or a group of containers). Instead of a container engine creating a conmon per container (as well as subsequent conmons per container exec), the engine will spawn a conmon-rs instance when a pod is created. That instance will listen over an UNIX domain socket for new requests to create containers, and exec processes within them.

Obtain the latest version

We provide statically linked binaries for every successfully built commit on main via our Google Cloud Storage Bucket. Our provided get script can be used to download the latest version:

> curl https://raw.githubusercontent.com/containers/conmon-rs/main/scripts/get | bash

It is also possible to select a specific git SHA or the output binary path by:

> curl https://raw.githubusercontent.com/containers/conmon-rs/main/scripts/get | \
    bash -s -- -t $GIT_SHA -o $OUTPUT_PATH

The script automatically verifies the created sigstore signatures if the local system has cosign available in its $PATH.

More information about how to use conmon-rs can be found in the usage documentation.

If you want to create a new conmon-rs release, please refer to the release documentation.

Architecture

The whole application consists of two main components:

1. The Rust server: conmon-rs/server (docs)
2. A golang client: pkg/client (docs)

The golang client should act as main interface while it takes care of creating the server instance via the Command Line Interface (CLI) as well as communicating to the server via Cap’n Proto. The client itself hides the raw Cap’n Proto parts and exposes dedicated golang structures to provide a clean API surface.

The following flow chart explains the client and container creation process:

Goals

• Single conmon per pod (post MVP/stretch)
• Keeping RSS under 3-4 MB
• Support exec without respawning a new conmon
• API with RPC to make it extensible (should support golang clients)
• Act as pid namespace init
• Join network namespace to solve running hooks inside the pod context
• Use pidfds (it doesn’t support getting exit code today, though)
• Use io_uring
• Plugin support for seccomp notification
• Logging rate limiting (double buffer?)
• Stats
• IPv6 port forwarding

Future development

In the future, conmon-rs may:

• Be extended to mirror the functionality for each runtime operation.
  • Thus reducing the amount of exec calls that must happen in the container engine, and reducing the amount of memory it uses.
• Be in charge of configuring the namespaces for the pod
  • Taking over functionality that pinns has historically done.

Usage

To use conmon-rs with CRI-O, please ensure that you use at least:

• CRI-O v1.25.2
• conmon-rs v0.4.0

Alternatively, use their latest main versions which are mostly guaranteed to work together.

Configure CRI-O

CRI-O needs to be configured to use conmon-rs. To do this, change the runtime configurations runtime_type and optionally the monitor_path, for example:

> cat /etc/crio/crio.conf.d/99-runtimes.conf

[crio.runtime]
default_runtime = "runc"

[crio.runtime.runtimes.runc]
runtime_type = "pod"
monitor_path = "/path/to/conmonrs"  # Optional, lookup $PATH if not set

CRI-O should now use conmon-rs after a restart, which is being indicated by the debug logs when creating a container:

…
DEBU[…] Using conmonrs version: 0.4.0, tag: none, commit: 130bd1373835cdfef8ae066a87eb4becabbe440a, \
            build: 2022-11-09 10:36:18 +01:00, \
            target: x86_64-unknown-linux-gnu, \
            rustc 1.65.0 (897e37553 2022-11-02), \
            cargo 1.65.0 (4bc8f24d3 2022-10-20)  file="oci/runtime_pod.go:100"
…

Configuring to use with Red Hat OpenShift

OpenShift 4.12 ships the latest version of conmon-rs per default. To use it, just apply the following MachineConfig (for runc):

apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: 99-worker-conmonrs
spec:
  config:
    ignition:
      version: 3.2.0
    storage:
      files:
        - contents:
            source: data:,%5Bcrio.runtime.runtimes.runc%5D%0Aruntime_type%20%3D%20%22pod%22%0A
          mode: 420
          overwrite: true
          path: /etc/crio/crio.conf.d/99-conmonrs.conf

The same can be done for the master role or any other confiured runtime like crun.

Using a custom conmonrs version

All conmonrs commits on main are build via fedora copr. This means that it’s possible to install a custom version by running rpm-ostree, for example for RHCOS 8:

> rpm-ostree override replace https://download.copr.fedorainfracloud.org/results/rhcontainerbot/podman-next/epel-8-x86_64/05025896-conmon-rs/conmon-rs-0.0.git.1551.130d137-1.el8.x86_64.rpm
…
Upgraded:
  conmon-rs 0.4.0-2.rhaos4.12.git.el8 -> 101:0.0.git.1551.130bd137-1.el8
Use "rpm-ostree override reset" to undo overrides
Run "systemctl reboot" to start a reboot
> systemctl reboot

Structs

• Server
  The main server structure.
• Version
  The version structure.