there was a fish in the calculator

Arguments parsing in Guile

2026-05-05T23:50:00Z

While Guile Scheme has a lot of hidden gems, like (ice-9 peg), parsing command line arguments is not one of its strengths in my opinion. Even if there are many powerful approaches to structured argument parsing, there is no API simple enough for me to know it by heart (as opposed for example to Python ArgumentParser).

In this post, I will analyze some of what I believe to be shortcomings of existing command line arguments parsing libraries and present my proposal for a new iteration with a new, small, API for parsing arguments in Guile. It is small enough to fit in my brain, so I think we may be onto something here.

On LLMs

The first commit of my library has been partially created with the help of a large language model. The commit message contains the attribution information and the prompt I used. I am aware that currently LLM provider companies are actively harming people and the environment, enabling racial profiling and supporting governments in making wars against international laws. Nevertheless, I tried one of the commercial LLMs to test its capabilities from a technological standpoint. In general, I believe that there is an ethical way of training and operating LLMs, one that respects content creators, users, workers and the environment. This post is not about LLMs, but if you have any comment or feedback for me, feel free to reach out either on the Fediverse or via email.

(ice-9 getopt-long)

The (ice-9 getopt-long) module is the first API we'll take into consideration, it is modelled after the C library of the same name. According to git log, it is the oldest command line parsing API around, it was added to Guile in 1999!

commit 4925695e21f9150632c4173a7814ce78aacd80bc
Author: Jim Blandy <jimb@red-bean.com>
Date:   Fri Feb 12 10:09:29 1999 +0000

* getopt-long.scm: Remove debugging calls to `pk'.
A new argument-processing package from Russ McManus.
* getopt-long.scm: New file.
* Makefile.am (ice9_sources): Added getopt-long.scm.
* Makefile.in: Regenerated.

Its designed around the idea that it should only take care of short (-s) and long (--long) options, leaving the parsing of positional arguments (optionally delimited by --) to the caller. One would define an option specification:

(define option-spec
  '((help
     (single-char #\h)
     ;; --help does not accept a value
     (value #f))
    (shout
     (single-char #\s)
     ;; --shout does not accept a value
     (value #f))
    (language
     (single-char #\l)
     ;; --shout accepts a value
     (value #t))))

This code defines three options:

-h/--help which is intended to print an help message
-s/--shout and -l/--language which are business logic related flags

getopt-long accepts short and long forms interchangeably, without making distinction. An example is that greet.scm --language it --shout hajar, greet.scm --language en -s hajar and greet.scm -sl it hajar have all the same meaning.

Notice that this specification contains a -h flag, but getopt-long does not handle that by default as some similar APIs do. You are the one that should detect that the 'help option has been passed and emit the help message yourself:

;; Help message
(define (show-help)
  (display "Usage: greet.scm [OPTIONS] NAME

Say hello to someone\n")
  (newline)

  (display "Positional arguments:")
  (newline)
  (display "  NAME")
  (newline)
  (newline)
  (display "Options:")
  (display "
  -h, --help             Show this help message and exit")
  (display  "
  -s, --shout")
  (newline)
  (display "
  -l, --language LEVEL    (default: en)")
  (newline))

;; Parse command line into a structured object
(define options (getopt-long (command-line) option-spec))

;; Use the parsed arguments
(let* ((language (option-ref options 'language "en"))
       (shout? (option-ref options 'shout #f))
       ;; Handle user passing --help
       (help (when (option-ref options 'help #f)
               (show-help)
               (exit 0)))0)))
       ;; Obtain positional arguments list
       (positional-args
        (cdr (first options)))
       (name (if (or (not positional-args)
                     (null? positional-args))
                 (error "NAME argument is required but none was passed")
                 (first positional-args)))
       (msg
        (string-append
         (if (and (string? language) (string=? language "it"))
             "ciao, "
             "hello, ")
         name)))
  (display (if shout? (string-upcase msg) msg))
  (newline))

A note on argument parsing

As you will notice later, all the examined libraries and the one I wrote follow the same general algorithm:
Declare some kind of structured specification
Derive some parsing rules from the specification
Apply the parsing to a list of command line arguments
Provide the user with some key of key-value data structure
In the above example point 1. is the option-spec declaration, points 2. and 3. happen inside getopt-long and point 4. happens in the last let.

With the above code we can run a simple greeter with a basic command line interface with:

$ guile -s greet-getopt-long.scm -s --language it hajar
CIAO, HAJAR
$ guile -s greet-getopt-long.scm -h

Usage: greet-getopt-long.scm [OPTIONS] NAME

Say hello to someone.

Positional arguments:
  NAME

Options:
  -h, --help               Show this help message and exit
  -s, --shout
  -l, --language LANGUAGE   (default: en)

In case you are interested you can also have a look at the complete code for the getopt-long example.

In general (ice-9 getopt-long) is a solid and battle tested API. In case having additional dependencies outside of the Guile runtime is not acceptable, I think it is still a nice option to have, but it could be easier to use.

Default values are specified at option-ref call site, this means that if you read an option with a default value in multiple places you might forget to set the default value in one. Additionally the option specification format (albeit powerful, as it allows more features than I show above, like type validation and more) is not easy to remember (at least for me) and it does not allow describing positional arguments.

One last detail is the need for users to handle --help by themselves. It's good to be able to opt in in that fine grained control but most simple scripts that just want to accept some values (I think of the output of many program-file gexps all over Guix) are usually fine with the library generating the help message and handling scenarios when the user passes -h or --help.

(srfi srfi-37)

(srfi srfi-37) is the second oldest Guile API for parsing command line arguments, it was first added to Guile in 2007:

commit d4c382218de2050de207318e9d8558c0aac6a7b9
Author: Ludovic Courtès <ludo@gnu.org>
Date:   Wed Jul 18 20:40:09 2007 +0000

Revision: lcourtes@laas.fr--2006-libre/guile-core--cvs-head--0--patch-81
Creator:  Ludovic Courtes <ludovic.courtes@laas.fr>

Added SRFI-37, by Stephen Compall.

(See ChangeLogs.)

It is similar to (ice-9 getopt-long) in spirit: positional arguments are left up to the user to be parsed, help messages must be manually crafted and the user must manually handle --help (even if it's easier than with (ice-9 getopt-long)) and options are declared in a specification.

;; Options specification
(define %options
  (list (option '(#\s "shout") #f #f               ;(short long) mandatory? optional?
                ;; Here the API is leaking,
                ;; the end result is simply to
                ;; set 'shout? to #t
                (lambda (opt name arg result)
                  (alist-cons 'shout? #t
                              (alist-delete 'shout? result))))
        (option '(#\h "help") #f #f
                (lambda args
                  (show-help)
                  (exit 0)))
        (option '(#\l "language") #t #f
                (lambda (opt name arg result)
                  (alist-cons 'language arg
                              (alist-delete 'language result))))))

(define %default-options
  ;; Alist of default option values.
  `((shout? . #f)
    (language . "en")))
    
;; Display help message
(define (show-help)
  (display "Usage: greet.scm [OPTIONS] NAME

Say hello to someone\n")
  (newline)

  (display "Positional arguments:")
  (newline)
  (display "  NAME")
  (newline)
  (newline)
  (display "Options:")
  (display "
  -h, --help             Show this help message and exit")
  (display  "
  -s, --shout")
  (newline)
  (display "
  -l, --language LEVEL    (default: en)")
  (newline))

But still, there are some differences. (srfi srfi-37) makes it easier to handle --help with its callback approach in the specification, but the in general the API is leaking, forcing the user to manually manipulate options with alist-cons and alist-delete. Any user error here will weirdly break programs without users not easily noticing.

The second difference of (srfi srfi-37) is args-fold. It is called instead of calling getopt-long, it returns an association list mapping option names to option values. This format has the nice property of being easily introspectable and easy to change programmatically.

(define options
  ;; Derive parsing rules from the options specification,
  ;; and parse the command line into an association list.
  (args-fold
   (cdr (command-line))
   %options
   ;; Called in case of an unrecognized option
   (lambda (opt name arg result)
     (display "~A: unrecognized option~%") name)
   ;; Accumulation procedure for positional arguments
   (lambda (arg result)
     (alist-cons 'argument arg result))
   ;; Start seed
   %default-options))

(let* ((language (assoc-ref options 'language))
       (shout? (assoc-ref options 'shout?))
       (name (assoc-ref options 'argument))
       (msg
        (if (not name)
            (error "NAME argument is required but none was passed")
            (string-append
             (if (and (string? language) (string=? language "it"))
                 "ciao, "
                 "hello, ")
             name))))
  (display (if shout? (string-upcase msg) msg))
  (newline))

This script is completely equivalent to the example for (ice-9 getopt-long), check out its complete code in case you are interested.

(srfi srfi-37) is sligthly easier to use than (ice-9 getopt-long), but the option specification format is even more difficult to remember. I am almost unable to write or modify an option specification without having the Guile manual open. args-fold is super cool, it just misses support for parsing positional arguments to be perfect.

guile-config

This is probably the most complex tool in the list and definitely the one with a broader scope. It has been my default up until now, as it is very handy and follows a batteries included approach. guile-config is a library to declare application configurations specifications that are used to derive command line interfaces and configuration files. It does a lot of stuff, I really invite you to check it out as in my opinion for medium sized projects is perfect. My dream is that the portion of guile-config's code that overlaps with (guix records) would be abstracted from both projects into a common dependency, but this is another story and we will tell it another time.

In guile-config, users declare the configuration of an application with a configuration record:

(use-modules (config))

(define %configuration
  (configuration
   (name 'greet-config)
   (synopsis "The configuration of greet-config.")
   (keywords
    (list
     ;; Command line options or
     ;; configuration file fields
     (switch
      (name 'language)
      (character #\l)
      (default "en")
      (test string?)
      (handler identity)
      (example "it")
      (synopsis "Language to use.")
      (description "The language to use for the output."))
     (switch
      (name 'shout?)
      (character #\s)
      (default #f)
      (test boolean?)
      (handler identity)
      (synopsis "Enable shouting mode")
      (description "Whether to enable shouting mode."))))
   ;; Positional arguments
   (arguments
    (list
     (argument
      (name 'name) (test string?) (handler identity)
      (example "Hajar")
      (synopsis "Name of a friend")
      (description "The name of the friend you want to greet."))))
   (parser sexp-parser)
   (copyright "Giacomo Leidi <therewasa@fishinthecalculator.me>")
   (version "0.1.0")
   (license "GPL3+")
   (author "Giacomo Leidi")))

This record contains all informations to derive command line option names, types and help messages. This allows guile-config to perform validation and help message generation on behalf of the user.

;; Parse options and in case -h/--help is detected
;; generate the help message, display it and exit
(define options (getopt-config-auto (command-line) %configuration))

;; Access configuration values
(let* ((language (option-ref options 'language))
       (shout? (option-ref options 'shout?))
       (name (option-ref options '(name)))
       (msg
        (string-append
         (if (and (string? language) (string=? language "it"))
             "ciao, "
             "hello, ")
         name)))
  (display (if shout? (string-upcase msg) msg))
  (newline))

Here is the complete code for the above script. It is exactly what I need, were it not for the focus on configuration files also, as you can notice from the (parser sexp-parser) line of the configuration specification. This is a specification for something that can be parsed from a file, not a specification of a simple command line interface.

In many cases people just want to write small scripts calling each other and having a structured command line interface that performs validation and is easy to write. My hope with guile-arguments is to create exactly that.

guile-arguments

(arguments) was born due to an itch I often get while writing Guile scripts. In case they are very simple I always resort to calling (first (command-line)), (second (command-line)) and so on, because I am never able to remember any of the above API by heart. In some case where I really needed a structured command line API I even resorted to Python. No parentheses, can you imagine?

(arguments) API is really small and hopefully easy to remember. Users first have to declare a parser specification, which defines all the options the command will have:

(use-modules (arguments))

;; We declare a structured specification of the
;; command line API
(define parser
  (argument-parser
    #:program "greet"
    #:description "Say hello to someone."
    ;; Flags are boolean toggles, #f by default
    (flag
      (name 'shout?)
      (short #\s)
      (help "Shout instead of speaking"))
    ;; Options are arguments that take a value,
    ;; if multiple? is #t they can be passed multiple
    ;; times to represent lists
    (option
      (name 'language)
      (short #\l)
      (default "en")
      (help "Language code"))
    ;; Positional arguments are arguments referenced
    ;; by position. They also can be multiple? but
    ;; only one of them can and it must be the last
    (positional
      (name 'name)
      (help "Name of the friend to greet"))))

Next users must call parse-arguments to get back a structured object representing the command line:

;; Parse command line. parse-arguments parses (cdr (command-line)) by default,
;; but in case you want to pass the command line yourself, you can pass it
;; as the last argument: (parse-arguments parser your-command-line)
(define arguments (parse-arguments parser))

;; Unpack arguments and use them
(match-arguments arguments (shout? language name)
  (let ((msg
         (string-append
          (if (string=? language "it")
              "ciao, "
              "hello, ")
          name)))
    (display (if shout? (string-upcase msg) msg))
    (newline)))

In this example we use match-arguments which is syntax sugar around the arguments-ref primitive to destructure arguments values out of the return value of parse-arguments. In case users only want to access a single argument value they can call (arguments-ref 'name-of-the-argument).

There is not much more, the above would already be equivalent to the earlier examples:

$ guix shell guile -f guix.scm -- guile -s examples/greet.scm -h
Usage: greet [OPTIONS] NAME

Say hello to someone.

Positional arguments:
  NAME              Name of the friend to greet

Options:
  -h, --help               Show this help message and exit
  -s, --shout              Shout instead of speaking
  -l, --language LANGUAGE  Language code (default: en)

(arguments) intercepts -h and --help and automatically emits help messages and exits by default, but by passing #:auto-help? #f users can turn off that behavior and handle the help request themselves.

In case you are interested you can have a look at the code of the (arguments) greeter and test if from the Codeberg repository.

You can get guile-arguments from Guix or soon from openSUSE Tumbleweed. In case you want to play around with the code, you can find it on Codeberg.

Next steps

(arguments) is currently good enough for 80% of my personal use cases. What it currently misses is: first, the ability to structurally declare subcommands and subparsers, probably they can be made to work even with the current API but it'd be hacky, second, internationalization or the ability to handle different output languages such as in auto generated help messages but also in error messages. I really hope it can be of help also to others.

In case you are interested to chat about (arguments), or have any kind of feedback, reach out via email or on the Fediverse!

Bonfire & Guix, a love story

2025-06-04T23:23:00Z

Bonfire is a new framework to build federated applications that just reached RC1. It is written in Elixir, a nice functional language, and allows communities to create custom flavored Fediverse applications, that can be tailored for their specific needs. I have been in touch with the core team and I'm trying to make the experience of running Bonfire on Guix as smooth as possible.

Guix is a general purpose provisioning tool, it implements trustable, functional and reproducible package recipes in the Guile language. It implements also a distro called Guix System, featuring transactional, atomic upgrades and that can be completely be manipulated with Guile. This allows to easily modularize configuration bits and makes it possible to reuse or generate system configurations.

The post is structured in the following way:

Where you start from
DNS Setup
Enabling gocix
Setting up SSL certificates (HTTPS) with Certbot
SSH assumptions
Secrets setup
Automatic database provisioning
Firewall setup
Enabling rootless Podman
Configuring OCI provisioning
Starting Bonfire Social
Running a Reverse proxy
Configuring database backups
Enabling automatic upgrades
Enabling automatic reboots
Summing up

In case you just want to see the final configuration, you can jump from here.

Where you start from

In this post we'll set up a production Bonfire instance with a declarative approach over the Guix System. It will be a secure setup featuring: HTTPS, automatic backups and Guix provisioned secrets. This post also assumes you have a preinstalled Guix machine that you can login into either via console or SSH. If you need help with installing Guix, feel free to reach out, it takes time but I try to help everyone.

If, at any point while following the instructions, you get your system into a broken state you can always go back to the last known system generation with:

# Supposing the last known good generation ID is 120
sudo guix system switch-generation 120

Then reboot. You should now be back again in a functional system. I recommend noting the current generation ID you are in before doing anything, with:

$ guix system list-generations

...

Generation 120  Jun 14 2025 20:57:49    (current)
  file name: /var/guix/profiles/system-120-link
  canonical file name: /gnu/store/lmg382z3zlpi9bcl81srd8dh92nr5aml-system
  label: GNU with Linux-Libre 6.14.11
  bootloader: grub
  root device: UUID: e4a319f6-6552-4d6b-afe8-9988df65173c
  kernel: /gnu/store/awmrxyh7i8phaqniwgmj7v4haxk8g9p2-linux-libre-6.14.11/bzImage
  channels:
    guix:
      repository URL: https://git.guix.gnu.org/guix.git
      branch: master
      commit: c8218094c47482c16f4cdd1e8092c35dab117418

In this case the generation ID is 120 (note the (current)).

DNS setup

Everything in this post assumes you have at least an A DNS record pointing to your machine's public IP address. Even better if you have also an AAAA record. Supposing your domain name is bonfire.fishinthecalculator.me you can check if DNS is working fine on the Guix System with:

$ guix shell bind bind:utils --  dig bonfire.fishinthecalculator.me A

The same goes for AAAA records:

$ guix shell bind bind:utils --  dig bonfire.fishinthecalculator.me AAAA

Enabling gocix

The first piece of configuration you'll need is the gocix channel in your user's .config/guix/channels.scm. gocix is a project I made to bring to your Guix installation cloud native applications, backed by OCI images, that can be configured in Guile. They share most of the nice properties Guix native services have: atomic upgrades, transactions and rollbacks.

After guix pull, in a new shell, your guix describe should look something (commit hashes will probably differ) like this:

$ guix describe
Generation 68   Jun 13 2025 19:20:50    (current)
  guix 4c142ad
    repository URL: https://git.guix.gnu.org/guix.git
    branch: master
    commit: 4c142ad34b5ce32ce9004c3efa90d61d197ce436
  sops-guix 89f46bc
    repository URL: https://github.com/fishinthecalculator/sops-guix
    branch: main
    commit: 89f46bc4686504763f49e6b34c596720d347d8da
  gocix fca34c4
    repository URL: https://github.com/fishinthecalculator/gocix
    branch: main
    commit: fca34c4f871501b252d741fad0522a9fc65b65da

SSL certificates

You want to setup SSL certificates provisioning as soon as possible, since everything from now on presupposes HTTPS. To do so on the Guix system, you have to add the certbot-service-type to your operating-system record (which after installation is available in /etc/config.scm):

(use-modules (gnu services certbot) ...) ;for 'certbot-service-type'

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               (certbot-configuration
                (email "your@email.org")
                (certificates
                 (list
                  (certificate-configuration
                   (domains (list "bonfire.fishinthecalculator.me"))))))))))

Now, after running sudo guix system reconfigure /etc/config.scm, you should be able to check the status of the renew-certbot-certificates Shepherd service with:

$ sudo herd status renew-certbot-certificates
● Status of renew-certbot-certificates:
  It is stopped (one-shot).
  It is enabled.
  Provides: renew-certbot-certificates
  Requires: nginx
  Custom action: configuration
  Will be respawned.

Recent messages (use '-n' to view more or less):
  2025-03-19 22:32:18
  2025-03-19 22:32:18 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  2025-03-19 22:32:18 Certificate not yet due for renewal; no action taken.
  2025-03-19 22:32:18 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  2025-03-19 22:32:18 Certificate successfully acquired: bonfire.fishinthecalculator.me

The command shows that the service has successfully acquired an SSL certificate for your domain. From now on the Guix System will periodically take care of renewing the certificate. You can check the files are actually there with:

$ sudo ls -l /etc/letsencrypt/live/bonfire.fishinthecalculator.me/
total 4
lrwxrwxrwx 1 root root  54 Mar  8 02:51 cert.pem -> ../../archive/bonfire.fishinthecalculator.me/cert1.pem
lrwxrwxrwx 1 root root  55 Mar  8 02:51 chain.pem -> ../../archive/bonfire.fishinthecalculator.me/chain1.pem
lrwxrwxrwx 1 root root  59 Mar  8 02:51 fullchain.pem -> ../../archive/bonfire.fishinthecalculator.me/fullchain1.pem
lrwxrwxrwx 1 root root  57 Mar  8 02:51 privkey.pem -> ../../archive/bonfire.fishinthecalculator.me/privkey1.pem
-rw-r--r-- 1 root root 692 Mar  8 02:51 README

SSH assumptions

If you need SSH access you are encouraged to turn off root login, password authentication and use only SSH keys. You can check the Guix manual on how to do that. Also fail2ban is pretty easy to setup with its default configuration for port 22.

Secrets setup

Next you are going to need to setup some secrets, both for the PostgreSQL database and the Bonfire instance. One way of doing so is with SOPS and sops-guix. You can read a more in depth version of this section at sops-guix.

SOPS allows for two different encyption tools: GnuPG and age. If you don't have any requirement the easiest way is to generate age keys and encrypt secrets with them.

~$ sudo -i
Password: 
~# mkdir -p ~/.config/sops/age
~# guix shell age -- age-keygen -o /root/.config/sops/age/keys.txt
...

Public key: age1m3hcq7d9sl3d0uz6ezxvns4f7mjctksmmf5d8tpptmyz30rk9qnscgzfsa

You'll need one keypair for each user of the secret so, if you intend to be able to update secrets on a different machine than the one you are installing Bonfire on, make sure to generate a keypair there as well. If you are creating secrets on a PC or a laptop and intend to run Bonfire on a server you SSH into, this is your scenario. Create one keypair for your user on your machine at $HOME.config/sops/age/keys.txt and the above on the server.

Next you need to create a SOPS configuration file, named .sops.yaml, in the same directory your configuration file is:

keys:
    - &user_yourself_age age1peu96695en0xrlshkd3j3zzd04payh3cx27yjw6r40z8ekemnuesmkrupn
    - &host_yoursystem age1m3hcq7d9sl3d0uz6ezxvns4f7mjctksmmf5d8tpptmyz30rk9qnscgzfsa

creation_rules:
    - path_regex: .*yoursystem\.yaml$
      key_groups:
          - age:
                - *user_yourself_age
                - *host_yoursystem

You are now ready to create the secrets you need.

PostgreSQL secret

You will need a password to protect the PostgreSQL access from unauthorized users. You can generate a random string with:

$ guix shell openssl -- openssl rand -base64 32
v/hSYQHNCJMYW+U8D3m6ADQ+5382jN9iJ69gfImEISY=

From the same directory where the .sops.yaml and your configuration are stored, run the following command to create a yoursystem.yaml file that will store your encrypted secrets. Unencrypted secrets are supposed to never hit the disk, check out sops-guix README for more information.

$ guix shell sops -- sops yoursystem.yaml

Your default editor will pop up. Replace the SOPS example secrets and add the following content to the file:

postgres:
    bonfire: v/hSYQHNCJMYW+U8D3m6ADQ+5382jN9iJ69gfImEISY=

Save and close the editor. You can now check inside yoursystem.yaml and see that the secrets is effectively encrypted.

meilisearch secret

Bonfire requires an additional service to run searches, meilisearch. Generate another password and add, the same way as before with sops yoursystem.yaml, to your secrets file the following:

meilisearch:
    master: ...

Bonfire secrets

The last three secrets are for Bonfire. You can do that with a one-liner with:

$ sops set yoursystem.yaml '["bonfire"]' "$(echo '{"secret_key_base": "", "signing_salt": "", "encryption_salt": ""}'  | jq  ".secret_key_base = \"$(openssl rand -base64 128)\"" | jq  ".signing_salt = \"$(openssl rand -base64 128)\""  | jq  ".encryption_salt = \"$(openssl rand -base64 128)\"")

This should create correctly sized secrets for Bonfire.

Email secrets

Bonfire needs to be able to send emails to users, both for invites and password change. It supports many ways to do so, I happen to use the free tier of Mailjet but there are many options. Email setup is out of scope for this post, but at the end of the setup process you'll get one or more secret tokens. Add them in yoursystem.yaml the same way you did for the other secrets and save the file. Make sure to remember the list of keys in the YAML file necessary to access the secrets, in my case they are:

mail:
    bonfire:
        key: ...
        private_key: ...

When all secrets are in your yoursystem.yaml file your can add the following to your operating system configuration:

(use-modules (guix gexp)            ;for 'local-file'
             (guix utils)           ;for 'current-source-directory'
             (sops services sops)   ;for 'sops-secrets-service-type' and 'sops-sops-secret->secret-file'
             (sops secrets)         ;for 'sops-secret'
             ...)

(define %project-root
 (current-source-directory))

(define sops.yaml
  (local-file (string-append %project-root "/.sops.yaml")
              "sops.yaml"))

(define-public yoursystem.yaml
  (local-file (string-append %project-root "/yoursystem.yaml")
              "yoursystem.yaml"))

;; PostgreSQL

(define-public bonfire-postgres-password-secret
  (sops-secret
   ;; Each element of this list represents
   ;; one key in yoursystem.yaml.  In this case
   ;; it represents:
   ;;
   ;; postgres:
   ;;      bonfire:
   ;;
   (key '("postgres" "bonfire"))
   (user "oci-container")
   (group "postgres")
   (file yoursystem.yaml)
   (permissions #o440)))

;; Meilisearch

(define-public meilisearch-key-secret
  (sops-secret
   (key '("meilisearch" "master"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))

;; Bonfire

(define-public bonfire-mail-key-secret
  (sops-secret
   (key '("mail" "bonfire" "key"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))
   
(define-public bonfire-mail-private-key-secret
  (sops-secret
   (key '("mail" "bonfire" "private_key"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))

(define-public bonfire-secret-key-base-secret
  (sops-secret
   (key '("bonfire" "secret_key_base"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))

(define-public bonfire-signing-salt-secret
  (sops-secret
   (key '("bonfire" "signing_salt"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))

(define-public bonfire-encryption-salt-secret
  (sops-secret
   (key '("bonfire" "encryption_salt"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               (sops-service-configuration
                (config sops.yaml)
                (secrets
                 (list meilisearch-key-secret
                       bonfire-postgres-password-secret
                       bonfire-mail-key-secret
                       bonfire-mail-private-key-secret
                       bonfire-secret-key-base-secret
                       bonfire-signing-salt-secret
                       bonfire-encryption-salt-secret)))))))

Automatic database provisioning

Bonfire supports the PostgreSQL database engine and requires the postgis extension, this is what is is required to set them up:

(use-modules (gnu packages geo)        ;for postgis
             (gnu packages databases)  ;for 'postgresql'
             (gnu services databases)  ;for 'postgresql-service-type' and 'postgresql-role-service-type'
             ...)


...

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               ...)

      ;; Postgres
      (service postgresql-service-type
               (postgresql-configuration
               (postgresql postgresql)
                ;; Bonfire requires postgis.
                (extension-packages (list postgis))
                (port 5432)))

      (service postgresql-role-service-type
               (postgresql-role-configuration
                (shepherd-requirement
                 ;; Allow database passwords to be provisioned through SOPS secrets 
                 (append
                  %default-postgresql-role-shepherd-requirement
                  '(sops-secrets))))))))

Firewall setup

The iptables-service-type is required by the rootless-podman-service-type, and in general in case the machine is exposed to the Internet having a firewall is a good idea. The following configuration allows only TCP or UDP connections on ports 22, 80 and 443. All ICMP traffic is dropped. All of this both for IPv4 and IPv6.

(use-modules (gnu services networking) ;for 'iptables-service-type'
             (guix gexp)               ;for 'plain-file'
             ...)


...

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               ...)

       ;; Postgres
      (service postgresql-service-type
               ...)

      (service postgresql-role-service-type
               ...)

      (service iptables-service-type
               (iptables-configuration
                (ipv4-rules (plain-file "iptables.rules" "*filter
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT
-A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p tcp --dport 22 -j ACCEPT
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-port-unreachable
COMMIT
"))
                (ipv6-rules (plain-file "ip6tables.rules" "*filter
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT
-A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p tcp --dport 22 -j ACCEPT
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -j REJECT --reject-with icmp6-port-unreachable
COMMIT
")))))))

If the machine is not exposed to the internet then it is sufficient to add to your services:

(use-modules (gnu services networking) ;for 'iptables-service-type'
             ...)

...

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               ...)

       ;; Postgres
      (service postgresql-service-type
               ...)

      (service postgresql-role-service-type
               ...)

      (service iptables-service-type))))

Enabling rootless Podman

If you want to be able to run rootless containers with your own user follow the Guix manual, for the Bonfire OCI image to work it is sufficient to add the rootless-podman-service-type and it DBus dependencies to your configuration:

(use-modules (gnu services containers) ;for 'rootless-podman-service-type'
             (gnu services dbus)       ;for 'dbus-service-type'
             (gnu services desktop)    ;for 'elogind-service-type'
             ...)

...

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               ...)

       ;; Postgres
      (service postgresql-service-type
               ...)

      (service postgresql-role-service-type
               ...)

      (service iptables-service-type
               ...)

      ;; The DBus clique
      (service elogind-service-type)
      (service dbus-root-service-type)

      (service rootless-podman-service-type))))

Configuring OCI provisioning

Bonfire currently runs as an OCI image on the Guix System. This is an advantage in that it's not needed to package and maintain Guix native packages for all the Elixir dependencies, and is a disadvantage since binaries in the OCI image can't be verifiably connected to the source code they and their dependencies are built from. In the past I did some effort to package Bonfire extensions but the work to build the Social flavor is still immense.

To run OCI services with the rootless-podman-service-type, you need to configure the oci-service-type:

(use-modules (gnu services containers) ;for 'rootless-podman-service-type'
             (gnu services dbus)       ;for 'dbus-service-type'
             (gnu services desktop)    ;for 'elogind-service-type'
             ...)

...

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               ...)

       ;; Postgres
      (service postgresql-service-type
               ...)

      (service postgresql-role-service-type
               ...)

      (service iptables-service-type
               ...)

      ;; The DBus clique
      (service elogind-service-type)
      (service dbus-root-service-type)

      (service rootless-podman-service-type)
      
      (service oci-service-type
               (oci-configuration
                (runtime 'podman))))))

Now reconfigure your system and reboot it to finalize service upgrades.

Starting Bonfire Social

To provision a functional Bonfire instance you will need two services, meilisearch and the Bonfire flavour. The (oci services bonfire) module provides a Guix System service able to provision a database, SOPS secrets and the Bonfire instance. Add the following to your operating system configuration, and make sure to change the values based on your actual setup:

(use-modules (oci services bonfire)     ;for 'oci-bonfire-service-type'
             (oci services meilisearch) ;for 'oci-meilisearch-service-type'
             ...)

...

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               ...)

       ;; Postgres
      (service postgresql-service-type
               ...)

      (service postgresql-role-service-type
               ...)

      (service iptables-service-type
               ...)

      ;; The DBus clique
      (service elogind-service-type)
      (service dbus-root-service-type)

      (service rootless-podman-service-type)
      
      (service oci-service-type
               ...)
      
      ;; meilisearch
      (service oci-meilisearch-service-type
               (oci-meilisearch-configuration
                ;; We use the host network since we have a firewall.
                ;; An OCI network could be used between Bonfire and meilisearch.
                (network "host")
                (port "7700")
                ;; Since secrets are provisioned by sops-guix
                ;; sops-secrets must be added as a Shepherd dependency.
                (shepherd-requirement '(user-processes sops-secrets))
                (master-key
                 (sops-secret->secret-file meilisearch-key-secret))))

      ;; Bonfire
      (service oci-bonfire-service-type
               (oci-bonfire-configuration
                (configuration
                 (bonfire-configuration
                  ;; Networking
                  (hostname "bonfire.fishinthecalculator.me")  ;replace with your domain
                  (port "4000")
                  (public-port "443")
                  (network "host")
                  ;; Postgres
                  (postgres-user "bonfire")
                  (postgres-db "bonfire")
                  ;; Mail
                  (mail-domain "bonfire.fishinthecalculator.me")                ;replace with your domain
                  (mail-from "friendlyadmin@bonfire.fishinthecalculator.me")))  ;replace with your domain
                  ;; Shepherd
                  (upload-data-directory "/var/lib/bonfire/uploads")
                  (requirement
                   '(user-processes postgresql postgres-roles sops-secrets podman-meilisearch))
                  (extra-variables
                   `(("MAIL_BACKEND" . "mailjet")
                     ("SERVER_PORT" . "4000")
                     ("SEARCH_MEILI_INSTANCE" . "http://localhost:7700")))
                  ;; Secrets
                  (meili-master-key
                   (sops-secret->secret-file meilisearch-key-secret))
                  (postgres-password
                   (sops-secret->secret-file bonfire-postgres-password-secret))
                  (mail-key
                   (sops-secret->secret-file bonfire-mail-key-secret))
                  (mail-private-key
                   (sops-secret->secret-file bonfire-mail-private-key-secret))
                  (secret-key-base
                   (sops-secret->secret-file bonfire-secret-key-base-secret))
                  (signing-salt
                   (sops-secret->secret-file bonfire-signing-salt-secret))
                  (encryption-salt
                   (sops-secret->secret-file bonfire-encryption-salt-secret)))))))

Now after reconfiguring your system, which might take a while, you should have a podman-bonfire service among your root's herd status:

$ sudo herd status podman-bonfire
Password: 
● Status of podman-bonfire:
  It is running since 11:48:45 AM (12 hours ago).
  Main PID: 26476
  Command: /run/current-system/profile/bin/podman run --rm --replace --name podman-bonfire --entrypoint /bin/sh --env POSTGRES_USER=bonfire --env FLAVOUR=social --env HOSTNAME=bonfire.fishinthecalculator.me --env POSTGRES_HOST=localhost --env POSTGRES_DB=bonfire --env MAIL_DOMAIN=bonfire.fishinthecalculator.me --env MAIL_FROM=friendlyadmin@bonfire.fishinthecalculator.me --env MAIL_PORT=465 --env MAIL_SSL=true --env PORT=4000 --env PUBLIC_PORT=443 --env LANG --env SEEDS_USER=root --env ERLANG_COOKIE=bonfire_cookie --env MIX_ENV=prod --env PLUG_BACKEND=bandit --env APP_NAME=Bonfire --env MAIL_BACKEND=mailjet --env SERVER_PORT=4000 --env SEARCH_MEILI_INSTANCE=http://localhost:7700 --network host -v /var/lib/bonfire/uploads:/opt/app/data/uploads -v /run/secrets/meilisearch:/run/secrets/meilisearch:ro -v /run/secrets/postgres:/run/secrets/postgres:ro -v /run/secrets/bonfire/mail:/run/secrets/bonfire/mail:ro -v /run/secrets/bonfire:/run/secrets/bonfire:ro docker.io/bonfirenetworks/bonfire:1.0.0-rc.1.10-social-amd64 -c "set -e; export MEILI_MASTER_KEY=\"$(cat /run/secrets/meilisearch/master)\"; export POSTGRES_PASSWORD=\"$(cat /run/secrets/postgres/bonfire)\"; export MAIL_KEY=\"$(cat /run/secrets/mail/bonfire/key)\"; export MAIL_PRIVATE_KEY=\"$(cat /run/secrets/mail/bonfire/private_key)\"; export SECRET_KEY_BASE=\"$(cat /run/secrets/bonfire/secret_key_base)\"; export SIGNING_SALT=\"$(cat /run/secrets/bonfire/signing_salt)\"; export ENCRYPTION_SALT=\"$(cat /run/secrets/bonfire/encryption_salt)\"; exec -a ./bin/bonfire ./bin/bonfire start"
  It is enabled.
  Provides: podman-bonfire
  Requires: postgresql postgres-roles sops-secrets podman-meilisearch cgroups2-fs-owner cgroups2-limits rootless-podman-shared-root-fs user-processes podman-volumes
  Custom actions: command-line entrypoint pull
  Will not be respawned.
  Log file: /var/log/bonfire.log

Recent messages (use '-n' to view more or less):
  2025-06-13 23:45:44     Phoenix.Router.__call__/5 @ deps/phoenix/lib/phoenix/router.ex:475
  2025-06-13 23:45:44     Bonfire.Web.Endpoint.plug_builder_call/2 @ lib/bonfire/web/endpoint.ex:1

On the first run it will take some time as it will need to perform database migrations, in general you can check its output by reading /var/log/bonfire.log.

Running a reverse proxy

The last piece to be able to access your instance from the Internet is a reverse proxy. We'll use NGINX but any one should work. To configure NGINX to forward traffic to Bonfire the following must be added to your configuration:

(use-modules (gnu services web)         ;for 'nginx-service-type'
             ...)

...

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               ...)

       ;; Postgres
      (service postgresql-service-type
               ...)

      (service postgresql-role-service-type
               ...)

      (service iptables-service-type
               ...)

      ;; The DBus clique
      (service elogind-service-type)
      (service dbus-root-service-type)

      (service rootless-podman-service-type)
      
      (service oci-service-type
               ...)
      
      ;; meilisearch
      (service oci-meilisearch-service-type
               ...)

      ;; Bonfire
      (service oci-bonfire-service-type
               ...)

      (service nginx-service-type
               (nginx-configuration
                ;; Wait for bonfire to start
                (shepherd-requirement
                 '(podman-bonfire))
                (server-blocks
                 (list
                   (nginx-server-configuration
                    (server-name (list "bonfire.fishinthecalculator.me"))
                    (listen '("443 ssl" "[::]:443 ssl"))
                    ;; Replace with your domain
                    (ssl-certificate "/etc/certs/bonfire.fishinthecalculator.me/fullchain.pem")
                    ;; Replace with your domain
                    (ssl-certificate-key "/etc/certs/bonfire.fishinthecalculator.me/privkey.pem")
                    (locations
                     (list
                      (nginx-location-configuration
                       (uri "/")
                       (body (list (string-append "proxy_pass http://localhost:4000;")
                       ;; Taken from https://www.nginx.com/resources/wiki/start/topics/examples/full/
                       ;; Those settings are used when proxies are involved
                       "proxy_redirect          off;"
                       "proxy_set_header        Host $host;"
                       "proxy_set_header        X-Real-IP $remote_addr;"
                       "proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;"
                       "proxy_http_version      1.1;"
                       "proxy_cache_bypass      $http_upgrade;"
                       "proxy_set_header        Upgrade $http_upgrade;"
                       "proxy_set_header        Connection \"upgrade\";"
                       "proxy_set_header        X-Forwarded-Proto $scheme;"
                       "proxy_set_header        X-Forwarded-Host  $host;")))
                      ;; Statically serve uploads media.
                      (nginx-location-configuration
                       (uri "/data/uploads/")
                       (body
                        (list "alias /var/lib/bonfire/uploads/;")
                              "index  index.html index.htm;"))))))))))))

Now after reconfiguring you should be able to access your instance and set it up. The first user to register is the admin, they can decide how further user can be added to the instance.

Configuring database backups

If you'd like to have nightly jobs dumping Bonfire's database to disk, to be able to recover from data corruption for example, you can use the postgresql-backup-service-type. There is a PR for sending it upstream, but to use it now you'll have to add the small-guix channel to your .config/guix/channels.scm. After running guix pull, in a new shell, guix describe should mention small-guix.

Now you can add the following to your operating system configuration:

(use-modules (small-guix services databases)   ;for 'postgresql-backup-service-type'
             ...)

...

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               ...)

       ;; Postgres
      (service postgresql-service-type
               ...)

      (service postgresql-role-service-type
               ...)
               
      (service postgresql-backup-service-type
               (postgresql-backup-configuration
                ;; Every day at 5 AM.
                (schedule "0 5 * * *")
                ;; Databases to backup.
                (databases '("bonfire")')
                ;; Which day to take the weekly backup from (1-7 = Monday-Sunday).
                (day-of-week-to-keep 6)
                ;; Number of days to keep daily backups.
                (days-to-keep 7)
                ;; How many weeks to keep weekly backups.
                (weeks-to-keep 5)))

      (service iptables-service-type
               ...)

      ;; The DBus clique
      (service elogind-service-type)
      (service dbus-root-service-type)

      (service rootless-podman-service-type)
      
      (service oci-service-type
               ...)
      
      ;; meilisearch
      (service oci-meilisearch-service-type
               ...)

      ;; Bonfire
      (service oci-bonfire-service-type
               ...)

      (service nginx-service-type
               ...))))

After some time, you'll get something like this at /var/lib/postgresql-backups:

$ sudo  tree -a /var/lib/postgresql-backups/
/var/lib/postgresql-backups/
├── 2025-06-05-daily
│   └── bonfire.custom
├── 2025-06-06-daily
│   └── bonfire.custom
├── 2025-06-07-weekly
│   └── bonfire.custom
├── 2025-06-08-daily
│   └── bonfire.custom
├── 2025-06-09-daily
│   └── bonfire.custom
├── 2025-06-10-daily
│   └── bonfire.custom
├── 2025-06-11-daily
│   └── bonfire.custom
└── 2025-06-12-daily
    └── bonfire.custom

You can then long term encrypt and backup these on S3, OneDrive, Google Drive with rclone and the restic-backup-service-type, check it out on the Guix Manual, search for restic-backup-service-type.

Enabling automatic upgrades

A good practice for a server exposed to the Internet is to continually apply security fixes. On Guix, for now, this means following the default branch. To automatically upgrade your system overnight, you can use the unattended-upgrade-service-type, configuring it to use gocix (and small-guix in case you use PostgreSQL backups) and to run at any given time. I run it once every night at 2AM:

(use-modules (gnu services admin)              ;for 'unattended-upgrade-service-type'
             (guix channels)                   ;for 'channel->code'
             ...)

...

(define %channels
  (cons*
   (channel
    (name 'small-guix)
    (url "https://codeberg.org/fishinthecalculator/small-guix.git")
    (branch "main")
    ;; Enable signature verification:
    (introduction
     (make-channel-introduction
      "f260da13666cd41ae3202270784e61e062a3999c"
      (openpgp-fingerprint
       "8D10 60B9 6BB8 292E 829B  7249 AED4 1CC1 93B7 01E2"))))
   (channel
    (name 'gocix)
    (url "https://github.com/fishinthecalculator/gocix")
    (branch "main")
    ;; Enable signature verification:
    (introduction
    (make-channel-introduction
     "cdb78996334c4f63304ecce224e95bb96bfd4c7d"
     (openpgp-fingerprint
      "8D10 60B9 6BB8 292E 829B  7249 AED4 1CC1 93B7 01E2"))))
   %default-channels))

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               ...)

       ;; Postgres
      (service postgresql-service-type
               ...)

      (service postgresql-role-service-type
               ...)
               
      (service postgresql-backup-service-type
               ...)

      (service iptables-service-type
               ...)

      ;; The DBus clique
      (service elogind-service-type)
      (service dbus-root-service-type)

      (service rootless-podman-service-type)
      
      (service oci-service-type
               ...)
      
      ;; meilisearch
      (service oci-meilisearch-service-type
               ...)

      ;; Bonfire
      (service oci-bonfire-service-type
               ...)

      (service nginx-service-type
               ...)

      (service unattended-upgrade-service-type
               (unattended-upgrade-configuration
                (schedule "0 2 * * *")
                 (system-expiration
                  ;; 30 days
                  (* 30 24 3600))
                 (channels
                  #~(list #$@(map channel->code %channels)))
                 (operating-system-file
                  ;; The path to your configuration file.
                  (file-append %project-root "/config.scm")))))))

After reconfiguring, you should have a new Shepherd timer:


$ sudo herd status unattended-upgrade 
● Status of unattended-upgrade:
  It is running since Fri 13 Jun 2025 11:21:31 PM CEST (87 minutes ago).
  Timed service.
  Periodically running: /gnu/store/67mzs4f35yrbj224y2ygr73vy1s1iqis-unattended-upgrade
  It is enabled.
  Provides: unattended-upgrade
  Requires: user-processes networking
  Custom action: trigger
  Will be respawned.

Upcoming timer alarms:
  11:10:00 PM (in 22 hours)
  Sun 15 Jun 2025 11:10:00 PM CEST (in 46 hours)
  Mon 16 Jun 2025 11:10:00 PM CEST (in 3 days)
  Tue 17 Jun 2025 11:10:00 PM CEST (in 4 days)
  Wed 18 Jun 2025 11:10:00 PM CEST (in 5 days)

Note, this does not mean you should run major upgrades overnight: you should be able to pin every package version either referencing a package variable that has the version in its symbol, or with one of the many Guix package extension and transformation mechanisms. You can pin OCI services by specifying an explicit image, like so:

(operating-system
  ...

  (services
    (list
      ...

      ;; Bonfire
      (service oci-bonfire-service-type
               (oci-bonfire-configuration
                (image
                 "docker.io/bonfirenetworks/bonfire:1.0.0-rc.1.11-social-amd64")
                ...)))))

Enabling automatic reboots

Not all upgrades can be performed online and restarting sevices regularly sometimes helps (it's the classic magic happening inside computers, if you know, you know). In case you want to regularly reboot your system you can use the unattended-reboot-service-type: it will provision a Shepherd timer which will reboot the system once triggered. This allows updates, for example for the Shepherd, the kernel or core packages, to be automatically finalized. In the future this should probably add an health check to see whether the system is functional and roll back otherwise.

To enable regular reboots add the following to your configuration:

(use-modules (small-guix services unattended-reboot)   ;for 'unattended-reboot-service-type'
             ...)

...

(operating-system
  ...

  (services
    (list
      ...
      (service certbot-service-type
               ...)
      
      (service sops-secrets-service-type
               ...)

       ;; Postgres
      (service postgresql-service-type
               ...)

      (service postgresql-role-service-type
               ...)
               
      (service postgresql-backup-service-type
               ...)

      (service iptables-service-type
               ...)

      ;; The DBus clique
      (service elogind-service-type)
      (service dbus-root-service-type)

      (service rootless-podman-service-type)
      
      (service oci-service-type
               ...)
      
      ;; meilisearch
      (service oci-meilisearch-service-type
               ...)

      ;; Bonfire
      (service oci-bonfire-service-type
               ...)

      (service nginx-service-type
               ...)

      (service unattended-upgrade-service-type
               ...)

      (service unattended-reboot-service-type
               (unattended-reboot-configuration
                ;; Every day at 6AM.
                (schedule "0 6 * * *"))))))

Summing up

Let's try to see it all together:

(use-modules (gnu packages geo)                        ;for 'postgis'
             (gnu packages databases)                  ;for 'postgresql'
             (gnu services admin)                      ;for 'unattended-upgrade-service-type'
             (gnu services certbot)                    ;for 'certbot-service-type'
             (gnu services databases)                  ;for 'postgresql-service-type' and 'postgresql-role-service-type'
             (gnu services containers)                 ;for 'rootless-podman-service-type'
             (gnu services dbus)                       ;for 'dbus-service-type'
             (gnu services desktop)                    ;for 'elogind-service-type'
             (gnu services networking)                 ;for 'iptables-service-type'
             (gnu services web)                        ;for 'nginx-service-type'
             (guix channels)                           ;for 'channel->code'
             (guix gexp)                               ;for 'local-file' and 'plain-file'
             (guix utils)                              ;for 'current-source-directory'
             (oci services containers)                 ;for 'oci-service-type'
             (oci services bonfire)                    ;for 'oci-bonfire-service-type'
             (oci services meilisearch)                ;for 'oci-meilisearch-service-type'
             (small-guix services databases)           ;for 'postgresql-backup-service-type'
             (small-guix services unattended-reboot)   ;for 'unattended-reboot-service-type'
             (sops services sops)                      ;for 'sops-secrets-service-type' and 'sops-sops-secret->secret-file'
             (sops secrets)                            ;for 'sops-secret'
             ...)

(define %project-root
 (current-source-directory))

(define sops.yaml
  (local-file (string-append %project-root "/.sops.yaml")
              "sops.yaml"))

(define-public yoursystem.yaml
  (local-file (string-append %project-root "/yoursystem.yaml")
              "yoursystem.yaml"))

;; PostgreSQL

(define-public bonfire-postgres-password-secret
  (sops-secret
   (key '("postgres" "bonfire"))
   (user "oci-container")
   (group "postgres")
   (file yoursystem.yaml)
   (permissions #o440)))

;; Meilisearch

(define-public meilisearch-key-secret
  (sops-secret
   (key '("meilisearch" "master"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))

;; Bonfire

(define-public bonfire-mail-key-secret
  (sops-secret
   (key '("mail" "bonfire" "key"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))
   
(define-public bonfire-mail-private-key-secret
  (sops-secret
   (key '("mail" "bonfire" "private_key"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))

(define-public bonfire-secret-key-base-secret
  (sops-secret
   (key '("bonfire" "secret_key_base"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))

(define-public bonfire-signing-salt-secret
  (sops-secret
   (key '("bonfire" "signing_salt"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))

(define-public bonfire-encryption-salt-secret
  (sops-secret
   (key '("bonfire" "encryption_salt"))
   (user "oci-container")
   (group "users")
   (file yoursystem.yaml)
   (permissions #o400)))

;; Channels
(define %channels
  (cons*
   (channel
    (name 'small-guix)
    (url "https://codeberg.org/fishinthecalculator/small-guix.git")
    (branch "main")
    ;; Enable signature verification:
    (introduction
     (make-channel-introduction
      "f260da13666cd41ae3202270784e61e062a3999c"
      (openpgp-fingerprint
       "8D10 60B9 6BB8 292E 829B  7249 AED4 1CC1 93B7 01E2"))))
   (channel
    (name 'gocix)
    (url "https://github.com/fishinthecalculator/gocix")
    (branch "main")
    ;; Enable signature verification:
    (introduction
    (make-channel-introduction
     "cdb78996334c4f63304ecce224e95bb96bfd4c7d"
     (openpgp-fingerprint
      "8D10 60B9 6BB8 292E 829B  7249 AED4 1CC1 93B7 01E2"))))
   %default-channels))

(operating-system
  ... ;; Bootloader, filesystems, users, packages, sudoers and so on...

  (services
    (list
      ... ;; Here you will most probably have to append to %base-sytem if this
          ;; is a headless machine
      (service certbot-service-type
               (certbot-configuration
                (email "your@email.org")  ;replace with your email
                (certificates
                 (list
                  (certificate-configuration
                   (domains (list "bonfire.fishinthecalculator.me"))))))) ;replace with your domain
      
      (service sops-secrets-service-type
               (sops-service-configuration
                (config sops.yaml)))

      ;; Postgres
      (service postgresql-service-type
               (postgresql-configuration
                (postgresql postgresql)
                ;; Bonfire requires postgis.
                (extension-packages (list postgis))
                (port 5432)))

      (service postgresql-role-service-type
               (postgresql-role-configuration
                (shepherd-requirement
                 ;; Allow database passwords to be provisioned through SOPS secrets 
                 (append
                  %default-postgresql-role-shepherd-requirement
                  '(sops-secrets)))))
               
      (service postgresql-backup-service-type
               (postgresql-backup-configuration
                ;; Every day at 5 AM.
                (schedule "0 5 * * *")
                ;; Databases to backup.
                (databases '("bonfire")')
                ;; Which day to take the weekly backup from (1-7 = Monday-Sunday).
                (day-of-week-to-keep 6)
                ;; Number of days to keep daily backups.
                (days-to-keep 7)
                ;; How many weeks to keep weekly backups.
                (weeks-to-keep 5)))

      (service iptables-service-type
               (iptables-configuration
                (ipv4-rules (plain-file "iptables.rules" "*filter
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT
-A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p tcp --dport 22 -j ACCEPT
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-port-unreachable
COMMIT
"))
                (ipv6-rules (plain-file "ip6tables.rules" "*filter
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT
-A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p tcp --dport 22 -j ACCEPT
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -j REJECT --reject-with icmp6-port-unreachable
COMMIT
"))))

      ;; The DBus clique
      (service elogind-service-type)
      (service dbus-root-service-type)

      (service rootless-podman-service-type)
      
      (service oci-service-type
               (oci-configuration
                (runtime 'podman)))
      
      ;; meilisearch
      (service oci-meilisearch-service-type
               (oci-meilisearch-configuration
                ;; We use the host network since we have a firewall.
                ;; An OCI network could be used between Bonfire and meilisearch.
                (network "host")
                (port "7700")
                ;; Since secrets are provisioned by sops-guix
                ;; sops-secrets must be added as a Shepherd dependency.
                (shepherd-requirement '(user-processes sops-secrets))
                (master-key
                 (sops-secret->secret-file meilisearch-key-secret))))

      ;; Bonfire
      (service oci-bonfire-service-type
               (oci-bonfire-configuration
                (configuration
                 (bonfire-configuration
                  ;; Networking
                  (hostname "bonfire.fishinthecalculator.me")  ;replace with your domain
                  (port "4000")
                  (public-port "443")
                  (network "host")
                  ;; Postgres
                  (postgres-user "bonfire")
                  (postgres-db "bonfire")
                  ;; Mail
                  (mail-domain "bonfire.fishinthecalculator.me")                ;replace with your domain
                  (mail-from "friendlyadmin@bonfire.fishinthecalculator.me")))  ;replace with your domain
                  ;; Shepherd
                  (upload-data-directory "/var/lib/bonfire/uploads")
                  (requirement
                   '(user-processes postgresql postgres-roles sops-secrets podman-meilisearch))
                  (extra-variables
                   `(("MAIL_BACKEND" . "mailjet")
                     ("SERVER_PORT" . "4000")
                     ("SEARCH_MEILI_INSTANCE" . "http://localhost:7700")))
                  ;; Secrets
                  (meili-master-key
                   (sops-secret->secret-file meilisearch-key-secret))
                  (postgres-password
                   (sops-secret->secret-file bonfire-postgres-password-secret))
                  (mail-key
                   (sops-secret->secret-file bonfire-mail-key-secret))
                  (mail-private-key
                   (sops-secret->secret-file bonfire-mail-private-key-secret))
                  (secret-key-base
                   (sops-secret->secret-file bonfire-secret-key-base-secret))
                  (signing-salt
                   (sops-secret->secret-file bonfire-signing-salt-secret))
                  (encryption-salt
                   (sops-secret->secret-file bonfire-encryption-salt-secret))))

      ;; Reverse proxy
      (service nginx-service-type
               (nginx-configuration
                ;; Wait for bonfire to start
                (shepherd-requirement
                 '(podman-bonfire))
                (server-blocks
                 (list
                   (nginx-server-configuration
                    (server-name (list "bonfire.fishinthecalculator.me"))
                    (listen '("443 ssl" "[::]:443 ssl"))
                    ;; Replace with your domain
                    (ssl-certificate "/etc/certs/bonfire.fishinthecalculator.me/fullchain.pem")
                    ;; Replace with your domain
                    (ssl-certificate-key "/etc/certs/bonfire.fishinthecalculator.me/privkey.pem")
                    (locations
                     (list
                      (nginx-location-configuration
                       (uri "/")
                       (body (list (string-append "proxy_pass http://localhost:4000;")
                       ;; Taken from https://www.nginx.com/resources/wiki/start/topics/examples/full/
                       ;; Those settings are used when proxies are involved
                       "proxy_redirect          off;"
                       "proxy_set_header        Host $host;"
                       "proxy_set_header        X-Real-IP $remote_addr;"
                       "proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;"
                       "proxy_http_version      1.1;"
                       "proxy_cache_bypass      $http_upgrade;"
                       "proxy_set_header        Upgrade $http_upgrade;"
                       "proxy_set_header        Connection \"upgrade\";"
                       "proxy_set_header        X-Forwarded-Proto $scheme;"
                       "proxy_set_header        X-Forwarded-Host  $host;")))
                      ;; Statically serve uploads media.
                      (nginx-location-configuration
                       (uri "/data/uploads/")
                       (body
                        (list "alias /var/lib/bonfire/uploads/;")
                              "index  index.html index.htm;"))))))))))

      ;; Automatic upgrades.
      (service unattended-upgrade-service-type
               (unattended-upgrade-configuration
                ;; Every night at 2AM.
                (schedule "0 2 * * *")
                 (system-expiration
                  ;; 30 days
                  (* 30 24 3600))
                 (channels
                  #~(list #$@(map channel->code %channels)))
                 (operating-system-file
                  ;; The path to your configuration file.
                  (file-append %project-root "/config.scm"))))

      ;; Automatic reboots.
      (service unattended-reboot-service-type
               (unattended-reboot-configuration
                ;; Every day at 6AM.
                (schedule "0 6 * * *"))))))

This post has still many assumptions baked in which will require some experimentation on your side, if you have questions let me know. You can also look at the configuration for the machine running bonfire.fishinthecalculator.me.

Future work

Currently bonfire.fishinthecalculator.me runs pretty smoothly but the users are very few and the machine hosts other services. Next steps would probably be setup monitoring with Prometheus and Grafana for the VM and the Bonfire instance.

Declarative, transactional, cloud native OCI provisioning with Guix

2025-05-06T23:33:00Z

Many applications are packaged in OCI images but not in Guix. A good subset of them is written either in NodeJS, Go, Rust or languages that, as a general approach, encourage applications to have huge dependency graphs.

The Guix project accepts package contributions that comply to very strict standards in terms of whether the package and its dependencies can be completely built from source. It's the reason why practically no Javascript application (or even web applications with complex frontends) are in Guix mainline. It is not clear whether they will ever be.

OCI backed services

Yet the Guix System is completely usable for self hosting purposes. If you use docker compose on the Guix System, you end up having two different interfaces to manage your system services: Shepherd and Docker/Podman. The oci-service-type aims at implementing Shepherd Services that look and feel native (so you can configure and manage them with the usual consistent interface that Guix exposes) but under the hood are implemented as docker run or podman run invocations.

    (simple-service 'oci-provisioning
                    oci-service-type
                    (oci-extension
                      (networks
                        (list
                          (oci-network-configuration (name "monitoring"))))
                      (containers
                       (list
                        (oci-container-configuration
                          (image "prom/prometheus")
                          (network "monitoring")
                          (ports
                           '(("9000" . "9000")
                             ("9090" . "9090"))))
                        (oci-container-configuration
                          (image "docker.io/grafana/grafana:latest")
                          (network "monitoring")
                          (ports
                           '("3000:3000"))
                          (volumes
                           '("/var/lib/grafana" . "/var/lib/grafana")))))))

In this example two different Shepherd services are going to be added to the system. Each oci-container-configuration record translates to a docker run or podman run invocation and its fields directly map to options. You can refer to the Docker or Podman upstream documentation for semantics of each value. If the images are not found, they will be pulled. You can refer to the Docker or Podman upstream documentation for semantics.

A backend example

Let's start with a simple example, you can imagine this being the equivalent of your backend service or SQL process. Its behavior is quite simple: when someone sends an HTTP GET request for the /whosin path at the port 7777, the script returns out of office and writes empty into /tmp/office:

    $ curl localhost:7777/whosin
    out of office
    
    $ cat /tmp/office
    empty

In case any other path is queried, it returns unknown path and the requested path over HTTP and writes on fire into /tmp/office. This is what happens when the /lallo path is requested:

    $ curl localhost:7777/lallo && cat /tmp/office
    unknown path /lallo
    on fire
    empty

We can now run the server in background with Shepherd, to check its behavior as a native process inside a Shepherd service:

    $ herd spawn transient -- `pwd`/backend.scm /tmp
    $ herd status | grep transient-
     + transient-198
    $ herd status transient-198
    ● Status of transient-198:
      It is transient, running since 07:24:53 PM (4 minutes ago).
      Main PID: 17653
      Command: backend.scm /tmp
      It is enabled.
      Provides: transient-198
      Requires: transient
      Will not be respawned.

As you can see the behavior is the same as the one the script manifests when run from shell as a regular command:

    $ curl localhost:7777/whosin && cat /tmp/office
    out of office
    empty

The frontend script

You can imagine this being the equivalent of your frontend or some kind of middlweare service. Its behavior is: when someone sends an HTTP GET request for the /doorbell path at the port 7778, the frontend calls the backend at http://localhost:7777/whosin, then reads the /tmp/office file contents and returns everything via HTTP to the client. Let's try:

    $ herd spawn transient -- `pwd`/frontend.scm /tmp http://localhost:7777/whosin

If we check now, we should have two different transient services, one for the backend and one for the frontend:

    $ herd status | grep transient-
     + transient-198
     + transient-199

We can now test whether the code is sound:

    $ curl localhost:7777/lallo && cat /tmp/office
    unknown path /lallo
    on fire
    
    $ curl localhost:7778/doorbell
    backend state: out of office
    office state: empty
    
    $ cat /tmp/office
    empty

Now let's try running these scripts inside containers and see what changes from the native Shepherd services and the containerized ones.

An Home service example

Let's start by defining a Guile OCI image in our Guix Home configuration and gexps for the scripts:

    (define guile-oci-image
      (oci-image
       (repository "guile")
       (value
        (specifications->manifest '("guile")))
       (pack-options
        '(#:symlinks (("/bin" -> "bin"))))))
    
    (define (script-file script-name)
      (local-file (string-append (getenv "HOME")
                                 ;; I happen to store these scripts in my HOME directory,
                                 ;; you should replace this with the directory where you
                                 ;; store your scripts.
                                 "/" script-name ".scm")
                  (string-append script-name ".scm")))

    (define backend-script
     (script-file "backend"))
    
    (define frontend-script
     (script-file "frontend"))

And add the following to your Home services:

    (service home-oci-service-type
             (for-home
              (oci-configuration
               (runtime 'podman))))
    
    (simple-service 'home-oci-provisioning
          home-oci-service-type
          (oci-extension
           (networks
            (list
             (oci-network-configuration
              (name "my-network"))))
           (volumes
            (list
             (oci-volume-configuration
              (name "my-volume"))))
           (containers
            (list
             (oci-container-configuration
              (provision "backend")
              (image guile-oci-image)
              (entrypoint "/bin/guile")
              (network "my-network")
              (command
               '("-s" "/backend.scm" "/my-volume"))
              (volumes
               `(("my-volume" . "/my-volume")
                 (,backend-script . "/backend.scm"))))
             (oci-container-configuration
              (provision "frontend")
              (image guile-oci-image)
              (requirement '(backend))
              (entrypoint "/bin/guile")
              (network "my-network")
              (command
               '("-s" "/frontend.scm" "/my-volume" "http://backend:7777/whosin"))
              (volumes
               `(("my-volume" . "/my-volume")
                 (,frontend-script . "/frontend.scm"))))))))

You can now run guix home reconfigure .... After Guix Home is done, check the services' status:

    $ herd status backend
    ● Status of backend:
      It is running since 11:21:38 PM (9 seconds ago).
      Main PID: 1756
      Command: /home/paul/.guix-home/profile/bin/podman run --rm --replace --name backend --entrypoint /bin/guile --network my-network -v my-volume:/my-volume -v /gnu/store/qiqwy2j7a598wp8v68294fpbjmmahrqc-backend.scm:/backend.scm localhost/guile:latest -s /backend.scm /my-volume
      It is enabled.
      Provides: backend
      Requires: home-podman-networks home-podman-volumes
      Custom action: command-line
      Will not be respawned.
    
    Recent messages (use '-n' to view more or less):
      2025-03-09 23:21:40 Copying config sha256:d111b778901b847eca5043a590e41c432a0471b3d8c3b75fe00fb2ad15088d59
      2025-03-09 23:21:40 Writing manifest to image destination
      2025-03-09 23:21:40 Loading image for backend from /gnu/store/2ndlvrpblk171qixkspywrrm4z5fah5n-backend.tar.gz...
      2025-03-09 23:21:40 Loaded image: localhost/guile.latest:latest
      2025-03-09 23:21:40 Tagged /gnu/store/2ndlvrpblk171qixkspywrrm4z5fah5n-backend.tar.gz with localhost/guile:latest...

and the same for the frontend:

    $ herd status frontend
    ● Status of frontend:
      It is running since 11:21:38 PM (36 seconds ago).
      Main PID: 1757
      Command: /home/paul/.guix-home/profile/bin/podman run --rm --replace --name frontend --entrypoint /bin/guile --network my-network -p 7778:7778 -v my-volume:/my-volume -v /gnu/store/m1avaw2681bbljxcidy7vb4x0i3898db-frontend.scm:/frontend.scm localhost/guile:latest -s /frontend.scm /my-volume http://backend:7777/whosin
      It is enabled.
      Provides: frontend
      Requires: home-podman-networks home-podman-volumes backend
      Custom action: command-line
      Will not be respawned.
    
    Recent messages (use '-n' to view more or less):
      2025-03-09 23:21:40 Writing manifest to image destination
      2025-03-09 23:21:40 Untagged: localhost/guile.latest:latest
      2025-03-09 23:21:40 Loading image for frontend from /gnu/store/ak3pzlin4ay98p14blxaj7zgrv8fh632-frontend.tar.gz...
      2025-03-09 23:21:40 Loaded image: localhost/guile.latest:latest
      2025-03-09 23:21:40 Tagged /gnu/store/ak3pzlin4ay98p14blxaj7zgrv8fh632-frontend.tar.gz with localhost/guile:latest...

Now let's test the functionality. We only exposed the frontend port, so we expect not to be able to connect to 7777:

    $ curl localhost:7777/whosin
    curl: (7) Failed to connect to localhost port 7777 after 0 ms: Couldn't connect to server

only to the frontend port (which is the 7778):

    $ curl localhost:7778/doorbell
    backend state: out of office
    office state: empty

We can check the office file contents with:

    $ podman volume export my-volume | tar xv
    office
    $ cat office
    empty

Now, let's have a closer look at the Shepherd services. Shepherd services provisioned by the oci-service-type support different set of actions.

The network provisioning service provides the following action:

    $ herd doc home-podman-networks list-actions
    command-line:
      Prints home-podman-networks OCI runtime command line invocation.
    
    $ herd command-line home-podman-networks
    /home/paul/.guix-home/profile/bin/podman network create my-network

the same goes for the volumes provisioning service:

    $ herd doc home-podman-networks list-actions
    command-line:
      Prints home-podman-volumes OCI runtime command line invocation.
    
    $ herd command-line home-podman-volumes
    /home/paul/.guix-home/profile/bin/podman volume create my-volume

and for containers which reference a cached image, like in our case:

    $ herd doc backend list-actions
    command-line:
      Prints backend OCI runtime command line invocation.
    
    $ herd command-line backend
    /home/paul/.guix-home/profile/bin/podman run --rm --replace --name backend --entrypoint /bin/guile --network my-network -v my-volume:/my-volume -v /gnu/store/qiqwy2j7a598wp8v68294fpbjmmahrqc-backend.scm:/backend.scm localhost/guile:latest -s /backend.scm /my-volume

OCI containers that have a remote image reference in their image field, additionally support a pull action:

    $ sudo herd doc podman-forgejo list-actions
    command-line:
      Prints podman-forgejo OCI runtime command line invocation.
    
    pull:
      Pull podman-forgejo image (codeberg.org/forgejo/forgejo:10.0.1-rootless).
    
    $ sudo herd command-line podman-forgejo
    /run/current-system/profile/bin/podman run --rm --replace --name podman-forgejo --env USER_UID=34595 --env USER_GID=98715 -p 3000:3000 -p 2202:22 -v forgejo:/var/lib/gitea -v /etc/timezone:/etc/timezone:ro -v /etc/localtime:/etc/localtime:ro codeberg.org/forgejo/forgejo:10.0.1-rootless
    
    $ sudo tail -n 17 /var/log/forgejo.log
    2025-03-09 23:50:11 Trying to pull codeberg.org/forgejo/forgejo:10.0.1-rootless...
    2025-03-09 23:50:14 Getting image source signatures
    2025-03-09 23:50:14 Copying blob sha256:15ab256cd4da0c6bf93a7cfa7e85e16dc9da5020c3415b630e59b09df76d27db
    2025-03-09 23:50:14 Copying blob sha256:71d5a7a4eeb57275b451cfab8e904d9fa727a37f88fa3bc75942e1b4460acd44
    2025-03-09 23:50:14 Copying blob sha256:66a3d608f3fa52124f8463e9467f170c784abd549e8216aa45c6960b00b4b79b
    2025-03-09 23:50:14 Copying blob sha256:662951a9d959644cb6d446eed38ee5a42b231df7100397a93a5c2f22bf68712b
    2025-03-09 23:50:14 Copying blob sha256:9cace756fe1f230966ec1022c17c33168f14c9e067e559c97950fdbfa2bba40b
    2025-03-09 23:50:14 Copying blob sha256:d962a541e1a1144a3c08f44260b04c7e37cdd99db572c8ae6a8f54cd8f9eafbe
    2025-03-09 23:50:14 Copying blob sha256:7dc8ff21196384b18f484da3eb1bde3064c7b506de71fdf45c6a12e7425d3bb9
    2025-03-09 23:50:14 Copying blob sha256:82cdec355329fbe7dbbfeee91deca478d805ea4a20a07bf4f486beeb6ec9c342
    2025-03-09 23:50:14 Copying blob sha256:c768a4796c3cea8130cd679fcdebbe3ec0c2d7399bc5e85758e20efb4eb834b6
    2025-03-09 23:50:14 Copying blob sha256:43402951a99e9088f7eb19a737a806b81906305e15cd5bb0ecf1a1fa816da5f9
    2025-03-09 23:50:14 Copying blob sha256:003e5af9ef5613ad37b723e8bf9fdbf80c2a656c9a61741941797ebcc1891cc3
    2025-03-09 23:50:14 Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
    2025-03-09 23:50:14 Copying config sha256:55f1bcd32c34de7e1544e5dfac45208cce1a8bb298858ff61978ff858c7c5f6b
    2025-03-09 23:50:14 Writing manifest to image destination
    2025-03-09 23:50:14 55f1bcd32c34de7e1544e5dfac45208cce1a8bb298858ff61978ff858c7c5f6b

oci-container-service-type vs oci-service-type

The new oci-service-type deprecates the oci-container-service-type: it is completely backward compatible and now, while deprecated, the oci-container-service-type is actually implemented extending the oci-service-type.

It brings additional features, such as: rootless podman support, the ability to provision networks and volumes, and better image caching.

To make the switch in service code you need to change your extension from

    (service-extension oci-container-service-type
                       oci-bonfire-configuration->oci-container-configuration)

    (service-extension oci-service-type
                       (lambda (config)
                         (oci-extension
                          (containers
                           (list
                            (oci-bonfire-configuration->oci-container-configuration config))))))

To make the switch in operating-system records, you need to change from

    (simple-service 'oci-containers
                    oci-container-service-type
                    (list
                     (oci-container-configuration
                      ...)))

    (simple-service 'oci-containers
                    oci-service-type
                    (oci-extension
                     (containers
                      (list
                       (oci-container-configuration
                        ...)))))

Unprivileged container management on the Guix System

2024-08-23T04:20:00Z

Docker is known to have less than optimal security defaults, hence the hype for Podman. If you want to run rootless containers in your Guix System, it is sufficient to add the following to your operating-system configuration.

(use-service-modules containers networking ...)
(use-modules (gnu system accounts))  ;for 'subid-range'

(operating-system
  ...
  (users (cons (user-account
                (name "alice")
                (comment "Your own user")
                (group "users")

                ;; Adding the account to the "cgroup" group
                ;; makes it possible to run podman commands.
                (supplementary-groups '("cgroup" "wheel"
                                        "audio" "video")))
               %base-user-accounts))
  (services
    (append (list (service iptables-service-type)
                  (service rootless-podman-service-type
                           (rootless-podman-configuration
                             (subgids
                               (list (subid-range (name "alice"))))
                             (subuids
                               (list (subid-range (name "alice")))))))
            %base-services)))

You can now run guix system reconfigure ..., after Guix is done, reboot your machine and you should be able to run Podman containers. This is the hello world suggested by Podman:

$ podman run -it --rm docker.io/alpine cat /etc/*release*
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.20.2
PRETTY_NAME="Alpine Linux v3.20"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

and with guix shell podman-compose, you can run this Podman compose hello world from the root of the repository:

$ mkdir data
$ echo hello world > data/index.html
$ podman compose up -d

...

exit code: 0
$ curl localhost:8080
hello world

Secrets management with SOPS Guix

2024-01-02T16:20:00Z

Dealing with secrets in functional operating systems can range from pretty usable to complete hell. Nix has several answers to this problem, the more integrated of which appears to be sops-nix. After spending some months envying our neighbors grass, I figured it was time for Guix to have its own (attempt at an) answer to the secrets problem.

This is how the SOPS Guix channel was born, the first take I'm aware of at implementing secure deploying of secrets with Guix and SOPS and as the name shows is quite inspired from Nix's sops-nix.

Secure secret provisioning with Guix

This channels exposes the sops-secrets-service-type Guix service and the sops-secret record to safely handle secrets with Guix. It works by putting encrypted secrets in the store and by adding a one-shot Shepherd service that decrypts them at startup in a ramfs/tmpfs filesystem. This means that clear text secrets never hit the disk and that you can (and actually are encouraged to) check in your SOPS secrets in the same version control system you use to track you Guix configurations.

Assuming that the right private keys are also provided, sops-secrets can be included in Guix images, deployed with guix deploy and included in Guix System/Home containers. Before starting, make sure you add it to your .config/guix/channels.scm, run guix pull and make sure sops-guix appears in your guix describe output.

Creating secrets with SOPS

First of all you need to create encrypted secrets with SOPS. To do so I'm assuming you already have a GPG key for yourself and the machines you want to deploy secrets to. You should be able to list the private keys you have in your keyring with

user1@home:~ $ gpg --list-secret-keys
/home/user1/.gnupg/pubring.kbx
------------------------
sec   ed25519 2023-12-01 [SC] [expires: 2907-11-30]
      8D1060B96BB8B7249AED41CC193B701E2SODIJNS
uid           [ultimate] user1@example.org
ssb   cv25519 2023-12-01 [E]

pub   rsa3072 1970-01-01 [SCE]
      8C3E4F6EB38828939029AE7BE9B6AF0CD39DD935
uid           [ unknown] root (Imported from SSH) <root@localhost>

pub   rsa3072 1970-01-01 [SCE]
      ZZ3E4VREB38800039029AE7BE9B6AF0CD39AALH9
uid           [ unknown] root (Imported from SSH) <root@localhost>

If you don't have a suitable set of GPG keys it's pretty simple to find online how generate them. Once you have a suitable set of keys for yourself and your machines you are ready to create the only configuration you need for SOPS: a .sops.yaml file that you will place in your project's root directory, or anyway in the same directory where you keep your system configuration. In this file you define which keys will be able to access your secrets files, it may very well be something like:

keys:
    - &user_user1 8D1060B96BB8B7249AED41CC193B701E2SODIJNS
    - &host_host1 8C3E4F6EB38828939029AE7BE9B6AF0CD39DD935
    - &host_host2 ZZ3E4VREB38800039029AE7BE9B6AF0CD39AALH9

creation_rules:
    - path_regex: .*common\.yaml$
      key_groups:
          - pgp:
                - *user_user1
                - *host_host1
                - *host_host2
    - path_regex: .*host1\.yaml$
      key_groups:
          - pgp:
                - *user_user1
                - *host_host1

In this file we define three keys called user_user1, host_host1 and host_host2 . The prefixes host_ and user_ are just a convention to indicate that some GPG keys belong to users and some belong to machines.

We now have defined two secrets file names patterns and we declared permissions for each key, it should be possible now to run the following in your projects root directory:

sops common.yaml

This will open your default editor with an example content to define your secrets value. You can edit it or delete it and your own content for example:

wireguard:
    private: MYPRIVATEKEY

after saving and closing the file you can see by catting the secret file that sops encrypted it before saving it, so you are free to check it in your VCS.

Making sure the right host keys are in the configured GnuPG keyring

For hosts to be able to decrypt secrets you need to provide in the root user keyring (or anyway the keyring located at the configured gnupg-homedir) the keys you defined in your .sops.yaml. So based on the above example you'd need to provide 8C3E4F6EB38828939029AE7BE9B6AF0CD39DD935's private key on host1 and ZZ3E4VREB38800039029AE7BE9B6AF0CD39AALH9's private key on host2 .

To check that your key is correctly imported into the keyring run:

user1@host1:~ $ sudo gpg --list-secret-keys
/root/.gnupg/pubring.kbx
------------------------
pub   rsa3072 1970-01-01 [SCE]
      8C3E4F6EB38828939029AE7BE9B6AF0CD39DD935
uid           [ unknown] root (Imported from SSH) <root@localhost>

By setting generate-key? to #t in sops-service-configuration a GPG key will be automatically derived for you from your system's /etc/ssh/ssh_host_rsa_key and added to the configured keyring. It is discouraged to do so and you are more than encouraged to autonomally provide a key in your configured keyring.

Adding secrets to your `operating-system` record

Now, supposing you have your operating-system file in the same directory where you have your .sops.yaml and common.yaml files, you can simply add the following to your configuration:

(use-modules (sops secrets)
             (sops services sops)
             (guix utils))

(define project-root
  (current-source-directory))

(define sops.yaml
  (local-file (string-append project-root "/.sops.yaml")
              ;; This is because paths on the store
              ;; can not start with dots.
              "sops.yaml"))

(define common.yaml
  (local-file (string-append project-root "/common.yaml")))

(operating-system
  [...]
  (services
    (list
       [...]
       (service sops-secrets-service-type
                (sops-service-configuration
                  (gnupg-homedir "/mnt/.gnupg")
                  (generate-key? #t)
                  (config sops.yaml)
                  (secrets
                    (list
                      (sops-secret
                        (key '("wireguard" "private"))
                        (file common.yaml)
                        (user "user1")
                        (group "users")
                        (permissions #o400)))))))))

Upon reconfiguration, this will yield the following content at /run/secrets:

user1@host1:~ $ sudo ls -la /run/secrets/
total 12
drwxr-xr-x 1 root root    50 Jan  2 12:44 .
drwxr-xr-x 1 root root   254 Jan  2 12:44 ..
lrwxrwxrwx 1 root root    53 Jan  2 12:44 .sops.yaml -> /gnu/store/lyhyh91jw2n2asa1w0fc0zmv93yxkxip-sops.yaml
-r-------- 1 user1 users  44 Jan  2 12:44 wireguard
user1@host1:~ $ cat /run/secrets/wireguard/private
MYPRIVATEKEY

Adding secrets to your `home-environment` record

sops-guix also provides a Guix Home service that is able to provide most feature of the system service. Most significant limitations are:

AFAIK home-environments can't configure ramfs mount points hence the secrets-directory option is not available. Secrets are stored in /run/user/$UID/secrets which usually is mounted on tmpfs.
sops-secret-user and sops-secrets-group are ignored. All secrets belong to the user running the Guix comman line but you can still set permissions.
There's no option to automatically generate GPG keys since probably users can easily generate one.

Now, supposing you have your home-environment file in the same directory where you have your .sops.yaml and your secrets files, you can simply add the following to your configuration:

(use-modules (sops secrets)
             (sops home services sops)
             (guix utils))

(define project-root
  (current-source-directory))

(define sops.yaml
  (local-file (string-append project-root "/.sops.yaml")
              ;; This is because paths on the store
              ;; can not start with dots.
              "sops.yaml"))

(define user1.yaml
  (local-file (string-append project-root "/user1.yaml")))

(home-environment
  [...]
  (services
    (list
       [...]
       (service home-sops-secrets-service-type
                (home-sops-service-configuration
                  (gnupg-homedir (string-append (getenv "HOME") "/.gnupg"))
                  (config sops.yaml)
                  (secrets
                    (list
                      (sops-secret
                        (key '("wireguard" "private"))
                        (file user1.yaml)
                        (permissions #o400)))))))))

Upon reconfiguration, this will yield the following content at /run/secrets/$YOUR_UID/secrets:

user1@host1:~ $ ls -la /run/user/$(id -u)/secrets
total 12
drwxr-xr-x 1 user1 users    50 Jan  2 12:44 .
drwxr-xr-x 1 user1 users   254 Jan  2 12:44 ..
lrwxrwxrwx 1 user1 users    53 Jan  2 12:44 .sops.yaml -> /gnu/store/lyhyh91jw2n2asa1w0fc0zmv93yxkxip-sops.yaml
-r-------- 1 user1 users    44 Jan  2 12:44 wireguard
user1@host1:~ $ cat /run/user/$(id -u)/secrets/wireguard/private
MYPRIVATEKEY

Monitor your Guix System with Grafana

2023-12-30T16:20:00Z

If you need to run Grafana on the Guix System this post is the right place. In this example we'll setup Grafana to read metrics from the same machine it's run upon, but you can adapt this to use a remote datasource.

First of all, add gocix to your Guix channels and run guix pull.

After making sure you have gocix in your guix describe output, you just need to add the following services to your operating-system record and reconfigure your system.

(use-modules (oci services grafana)
             (oci services prometheus)
             (gnu services monitoring))

(operating-system
  [...]
  (services
    (list
      [...]
      ;; Prometheus node exporter
      (service prometheus-node-exporter-service-type)
      ;; Prometheus OCI backed Shepherd service
      (service oci-prometheus-service-type
               (oci-prometheus-configuration
                (network "host")))
      ;; Grafana OCI backed Shepherd service
      (service oci-grafana-service-type
               (oci-grafana-configuration
                (network "host"))))))

After you can see both Prometheus and Grafana spawned as Docker containers, such as in this example

docker ps
CONTAINER ID   IMAGE                     COMMAND                  CREATED          STATUS          PORTS                                                                                  NAMES
fede69fcf4bf   prom/prometheus:v2.45.0   "/bin/prometheus --w…"   26 seconds ago   Up 25 seconds   0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 0.0.0.0:9090->9090/tcp, :::9090->9090/tcp   docker-prometheus
caa2dc1f50c0   bitnami/grafana:10.1.5    "/opt/bitnami/script…"   26 seconds ago   Up 25 seconds   0.0.0.0:3000->3000/tcp, :::3000->3000/tcp                                              docker-grafana

you should be able to visit localhost:3000 and see the Grafana login page.

The default credentials are admin:admin, log in and configure a new datasource with http://localhost:9090 as URL. In this example we'll avoid creating a new dashboard from scratch and we'll import one from the Grafana's website. Just click on the button to create a new dashboard and choose the import option, paste 1860 as the Dashboard ID, select the Prometheus datasource you just created and import the dashboard.

You should be presented with the full data available from the prometheus-node-exporter-service-type by default.

Tame Docker selfhosting with Guix

2023-12-10T16:20:00Z

Many applications are packaged in OCI/Docker images but not in Guix. A good subset of them is written either in NodeJS, Go, Rust or languages that, as a general approach, encourage applications to have huge dependency graphs.

Let's take Grafana as an example. According to its package.json Grafana's web frontend has more than 180 direct dependencies and more than 170 build time dependencies, for a grand total of ~350 direct dependencies. This is excluding all transitive dependencies. In the same way, Grafana's Go backend has ~440 direct dependencies. Some of them may be optional, but go figure.

This phenomenon is not uncommon in modern software development, it's not that Grafana is doing worse than everyone else. Yet it's quite problematic, it clearly shows that nobody put the effort of auditing or even reviewing the dependency graph of one of the most used dashboarding application in the industry.

Dissecting the interests which prevent the investment in building software products with a sustainable maintenance process is a topic for another post, the point is that the Guix project accepts package contributions that comply to very strict standards in term of licensing and other criteria, such as whether the package and its dependencies can be completely built from source. It's the reason why practically no NodeJS application (or even web applications with complex frontends, such as Grafana) can be upstreamed to Guix. It is not clear whether they will ever be, since especially in the Javascript world packages suffer from cyclical dependency which complicate the packaging process even more.

It is very painful to have to package whole dependency graphs just to use some service on the Guix System. This is especially true when you have a goal that you want to achieve and you could use Guix to achieve it, but it does not necessarily involve sending a bunch of patches upstream. It could be that you want to start some Matrix community around Guix or other people-sized technologies, you may want to self host a personal cloud with Nextcloud or maybe you want to setup an accessible monitoring dashboard for your services with Grafana and Prometheus. These are all softwares that do not follow the best sustainability practices, but they are pretty useful to use while we build the alternative technology (also called alt-tech) that we need to survive sustainably in our digital lives.

Container taming on Guix

If you use docker compose on the Guix System, you end up having two different interfaces to manage your system services: Shepherd and Docker. The oci-container-service-type aims at implementing Shepherd Services that look and feel native (so you can configure and manage them with the usual consistent interface that Guix exposes) but under the hood are implemented as docker run (or docker rm) invokations, so that you don't have to package those huge dependency graphs just to start self hosting on Guix.

Next you can find an example of how run Prometheus on the Guix System through the oci-container-service-type. You just need to add

(use-modules (gnu services docker)
             (gnu services monitoring)
             (guix gexp))

(define prometheus.yml
  (plain-file "prometheus.yml"
              "global:
  scrape_interval: 30s
  scrape_timeout: 12s

scrape_configs:
  - job_name: prometheus
    metrics_path: /metrics
    static_configs:
      - targets: ['localhost:9090','localhost:9100']\n"))

(operating-system
  [...]
  (services
    (list
      [...]
      ;; Prometheus node exporter
      (service prometheus-node-exporter-service-type)
      ;; Prometheus OCI backed Shepherd service
      (simple-service oci-container-service-type
                      (list
                        (oci-container-configuration
                          (command
                            '("--web.enable-lifecycle"
                              "--config.file=/etc/prometheus/prometheus.yml"
                              "--web.enable-admin-api"))
                          (image "prom/prometheus:v2.45.0")
                          (ports
                            `(("9000" . "9000")
                              ("9090" . "9090")))
                          (volumes
                            `(("/var/lib/prometheus" . "/prometheus")
                              (,prometheus.yml . "/etc/prometheus/prometheus.yml:ro")))))))))

to the services field of your operating-system record. In this example it's also shown how to install the Prometheus node exporter to collect metrics from the host.

This approach obviously is lacking in reproducibility and bootstrappability, but in my experience often users need some software that is not yet guixable (i.e. possible to include in Guix upstream) so they choose to use Nix or something else entirely.

What's next?

The oci-container-service-type implements Shepherd services through Docker containers, but a Shepherd service is only a small block in the implementation of a nicely integrated Guix System service. To provide a secure, consistent and integrated experience a Guix System service may declare user accounts, to allow for less then root authority execution, or it may initialize some state upon activation: all things for which an additional service extension is required.

This is why I'm implementing a library of (hopefully) community maintained Guix System services for many common self hostable applications such as Matrix Conduit, Forgejo, Grafana and more. These services try to be as similar as possible to an upstream provided service, in the hope of being upstreamed as soon as the underlying dependency graph is packaged.