Tony Finn

Preserve - Jellyfin Music Client

#Projects
In the last few months, I've moved away from using Spotify and back to using my own music collection, largely driven by tracks which I include in my playlists disappearing off the service due to licensing issues. Moving fully back to local media has left to issues keeping my library in sync across devices, so I've begun using Jellyfin, an open source media server to store my collection and make it available to all my devices. However, the default web interface in Jellyfin did not provide the same efficiency I liked from Clementine, my music player I used locally.

Much like foobar2000, Clementine has a very efficient interface for the way in which I tend to listen to music - i.e. rapidly creating a playlist of disparate tracks via search, listening to it for a few days or weeks, then making a new collection. The same is not true for Jellyfin's web interface (and wasn't true for Spotify either). So I built my own client to replicate that experience.

Preserve

Preserve is a web based media player frontend to a jellyfin server. It aims to replicate the experience of using players like foobar2000 or Clementine with its two panel design allowing you to quickly search tracks and assemble/order a playlist how you like, including full keyboard controls. It's fully open source, under GPLv2+.

Since it is a jellyfin client, you will first need to host your music using a Jellyfin server, but once that is done you can use Preserve in your browser at preserveplayer.com or install a desktop client from the Preserve Gitlab releases.

Preserve v0.5.3 Screenshot
Preserve v0.5.3

Read more about Preserve

Jujutsu (jj), a git compatible VCS

#Tech

It's quite a common opinion that git (while a big improvement on what came before) still has plenty of rough edges, particularly with regards to the user interface. At the same time, there's also a significant barrier to entry for a new version control system with all the tooling that's built up around git over the years, particularly if you want to try it in your workplace without migrating the entire company.

jj is one of the latest round of git-compatible version control systems which allows you to have a better experience locally, without having to abandon everything that depends on git. I've been trying it out now for about two months, and this post shares some of my thoughts.

Read the full jj review

Building Arch Packages With Nix

#Nix

In recent months I've been building more and more of my packages with Nix. However, most of my personal systems are still running Arch. While NixOS is making headways there and is probably the eventual destination for all my systems, I'm not interested in migrating all my personal infrastructure all at once.

Instead, what I really want to do is having my Nix tooling output Arch packages that I can install on those systems in this transitional period. After some experimentation, I got it working for one of my packages, this blog, and this post details the steps it took.

Learn how to build Arch packages with Nix

Runnable Flakes (Nix From First Principles: Flake Edition #9)

#Nix

This is part 9 of the Nix from First Principles: Flake Edition series.

In the last two parts, I covered using flakes for two different types of output: packages and developer environments. This time I'll cover a third type of output - runnable commands. These are what power the nix run command that you've seen a few times throughout the series, and now you will find out how to get this functionality for your own flakes.

Run Flakes without installation

Flakes and Developer Environments (Nix From First Principles: Flake Edition #8)

#Nix

This is part 8 of the Nix from First Principles: Flake Edition series.

While packages are the most common type of item you'll find contained in flakes, another type of item is a development shell. This is an environment that you can enter with the nix develop command. By default, each package will provide a development shell which is the build environment used to build it. By running just nix develop without specifying which shell is wanted, nix will put you in a bash shell with the environment from the default package.

However, sometimes you may wish to set up additional tools in a development environment, for example you may have scripts written in Python and want to have Python available as for a developer to use, yet you do not want to include Python as a dependency in your built package. Or you might want to set some environment variables so that logging is set to debug mode in your development environment.

You can customise the nix develop environment with the devShells attribute set in a flake. The helper function pkgs.mkShell is useful here. This is an extended version of the mkDerivation function that has been used in previous examples, except this version is specialised for shells. The following flake shows an example:

flake.nix

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

  outputs = { self, nixpkgs }:
    let
      pkgs = import nixpkgs { system = "x86_64-linux"; };
    in
    {
      devShells.x86_64-linux.default = pkgs.mkShell {
        packages = [
          # Made available on the CLI
          pkgs.cargo
          pkgs.rustc
          pkgs.python3
        ];
        RUST_LOG = 1; # Set as environment variable
      };

      # You could also define extra shells or packages here
    };
}

Now if you run nix develop in the same directory as this flake, you'll be left at a bash shell with cargo, python3 and rustc available. If you run the command env | grep RUST, you can observe the following output:

$ env | grep RUST
RUST_LOG=1

Or which python3:

/nix/store/xcaaly5shfy227ffs8nipxrd49b56iqq-python3-3.10.8/bin/python3

You could even define multiple shells. For example, lets say you wanted to support Python 3.8 through 3.10 in a single project. nixpkgs provides python artifacts in specific names like python38 if you want specific versions. So you could build a flake which defines your python versions for each one, like so:

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

  outputs = { self, nixpkgs }:
    let
      pkgs = import nixpkgs { system = "x86_64-linux"; };
      # A list of shell names and their Python versions
      pythonVersions = {
        python38 = pkgs.python38;
        python39 = pkgs.python39;
        python310 = pkgs.python310;
        default = pkgs.python310;
      };
      # A function to make a shell with a python version
      makePythonShell = shellName: pythonPackage: pkgs.mkShell {
        # You could add extra packages you need here too
        packages = [ pythonPackage ]; 
        # You can also add commands that run on shell startup with shellHook
        shellHook = ''
          echo "Now entering ${shellName} environment."
        '';
      };
    in
    {
      # mapAttrs runs the given function (makePythonShell) against every value
      # in the attribute set (pythonVersions) and returns a new set
      devShells.x86_64-linux = builtins.mapAttrs makePythonShell pythonVersions;
    };
}

Now if you run the command nix develop .#python38, you will end up in a shell that contains Python 3.8, something you can confirm with python --version.

$ nix develop .#python38
Now entering python38 environment.
$ python --version
Python 3.8.15

You can also inspect all items that are available in the flake with the command nix flake show:

$ nix flake show
git+file:///home/tony/code/nix-guide?dir=8-dev-envs%2fmanypython
└───devShells
    └───x86_64-linux
        ├───default: development environment 'nix-shell'
        ├───python310: development environment 'nix-shell'
        ├───python38: development environment 'nix-shell'
        └───python39: development environment 'nix-shell'

But what if I don't want bash?

Of course, you might prefer other shells than bash - however, Nix seems hardcoded to use bash. So you might wonder, how can you override the shell? For example, you might prefer the zsh or fish shells.

The first method of doing this requires no external tools, just Nix itself. You install your desired shell in the your devShell, and execute it in the shellHook. Here's an example that starts the fish shell.

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

  outputs = { self, nixpkgs }:
    let
      pkgs = import nixpkgs { system = "x86_64-linux"; };
    in
    {
      devShells.x86_64-linux.default = pkgs.mkShell {
        packages = [
          pkgs.fish
        ];
        # Note that `shellHook` still uses bash syntax.
        # This starts fish, then exists the bash shell when fish exits.
        shellHook = ''
          fish && exit
        '';
      };
    };
}

While the big advantage here is that you don't need any tools apart from nix, one big disadvantage is that your shell of choice needs to be included in each development environment, and other users of your flakes are forced to use your preferred shell also.

Another option is to run nix develop -c fish. This overrides the command on shell launch to launch your preferred shell. (Thanks to u/Mysteriox7 on reddit for bringing this to my attention). This is an improvement over forcing all users of your flake to use your preferred shell, but does need to be typed each time.

With both approaches however, one thing that can be a pain is they require typing that command every time you enter a project which you want to use nix develop for.

Luckily, there are third party tools to the rescue.

nix-direnv

nix-direnv is a integration between nix and the direnv tool. direnv is a command to run commands whenever you enter a directory with a .envrc file. nix-direnv builds upon this to load a cached nix environment into your currently open shell. Effectively, the first time you open a project with nix-direnv, it will build your development environment, record the environment changes this process makes then apply those to your current shell. On opening the project again in the future, it will load the cached environment, so it doesn't need to execute your nix scripts again if they are unchanged,

To use this, you'll need to install direnv and nix-direnv. Follow their installation instructions at the links provided. Then to set up your flake project with direnv, you just need to create a .envrc file with one command, use flake

.envrc

use flake

Alternatively, you can pass it a flake reference if you want to use a different devShell. For example, my hosted version of the Python 3.8 development shell can be used with

use flake "gitlab:tonyfinn/nix-guide?dir=8-dev-envs/manypython#python38"

That's it for this part of the Nix guide. Next time I'll cover another type of flake output - runnable commands.

What about flakes then? (Nix From First Principles: Flake Edition #7)

#Nix

This is part 7 of the Nix from First Principles: Flake Edition series.

One of the big new concepts in Nix is the flake, which is a new standard format for nix projects to declare all their outputs. At the beginning of this series, I mentioned them as being one of the major components of modern Nix, and now this series has introduced enough of a foundation it's time to explain flakes themselves.

What is a flake?

A flake is a standard format for describing a collection of Nix resources. These resources can be packages, of the type described in previous posts, intended to be used to install some software to your system. There's also a number of other types of resources that can be exposed by the flake which we'll cover in later posts. Some examples of these are:

  • developer environment descriptions
  • modules for configuring NixOS systems
  • runnable commands

By standardising a format to list all these outputs (compared to the ad-hoc files this series has been using until now), Nix allows a number of higher level tools to operate on these resources, like the nix profile install and nix run commands that have been shown in earlier parts.

Finally, the development of the flake based tooling allowed the Nix team to resolve some pain points with the older generation of tooling, such as making the process of recording versions for reproducibility more ergonomic with lock files, or making better ways to identify newer versions of the same package to allow tools to manage updates.

Start learning about Flakes

Nixpkgs - How Not to Reinvent the Wheel (Nix From First Principles: Flake Edition #6)

#Nix

This is part 6 of the Nix from First Principles: Flake Edition series.

In the last section I discussed creating your first derivation, which allows you to make a first nix package. As you might have noticed, the default execution environment is incredibly barebones, to the point that you needed to include such fundamental tools as chmod and cp. If that process had to be repeated by every Nix user, it would be very inconvenient. Luckily, there is nixpkgs, which provides a number of packages that the community has already built, along with the standard environment which includes a number of tools to use in building your own. You may remember installing some of these packages in part 3.

Read how to reduce boilerplate with Nixpkgs

Overview (Nix From First Principles: Flake Edition #1)

#Nix

This is part 1 of the Nix from First Principles: Flake Edition series.

One technology that I'd been hearing more and more over the last two years has been Nix, in its various forms. The big draws I've heard about include declarative, portable, cross lang dependency specification, and the ability to have a config file declare the entire state of a system without storing huge docker container artifacts for every permutation.

Actually learning Nix though felt much harder than it needed to be. The biggest problem is that the documentation you need is often fragmented among the different subprojects, and also the transition that Nix is in at the moment from the current standard methods of default.nix, configuration.nix and shell.nix to a new world of flakes.

As an outsider looking in, the flake world felt kind of in limbo at the moment where anyone I talked to indicated that flakes fix so many of the problems with current nix, but also they're still officially experimental so the official documentation declines to mention flakes.

After spending a fair amount of time understanding Nix for myself, I've decided to write my own guide which explains it from scratch in a world using only the new Nix features, with the nix CLI and flakes for the use cases I want from Nix. Credit to ianthehenry for his guide on Nix focused on the "old world" approach, and Xe Iaso for explaining flakes. The information included in this guide is largely from synthesising their blogs and the official documentation.

This first post will describe the major parts of the Nix ecosystem.

Dive into Nix

Installation (Nix From First Principles: Flake Edition #2)

#Nix

This is part 2 of the Nix from First Principles: Flake Edition series.

Nix can be installed on Linux or Mac systems. If you're using the Nix based Linux distribution, NixOS, then you'll already have it installed. Otherwise the official nix website provides the most up to date installation steps, which at the time of writing are

Linux:

sh <(curl -L https://nixos.org/nix/install) --daemon

Mac:

sh <(curl -L https://nixos.org/nix/install)

Windows (run in WSL):

sh <(curl -L https://nixos.org/nix/install) --no-daemon

Read detailed installation/configuration information

Nix Package Manager CLI (Nix From First Principles: Flake Edition #3)

#Nix

This is part 3 of the Nix from First Principles: Flake Edition series.

The foundational part of Nix is the Nix package manager. This is a tool that allows you to install packages. The "first layer" of Nix usage is to use this tool to install packages. The Nix package manager, similar to projects like os-rpmtree (used in Fedora Silverblue) or snapper (used in snap based Linux distros), will allow you to easily revert package installs or updates if something goes wrong.

This post covers the basics of using the nix package manager at the command line - while most users use nix through more advanced workflows, the basics explained here will be useful in understanding those workflows, as well as for users who want to dip their toes into Nix by treating it like a more traditional package manager to start out.

Read about installing packages manually

Just Enough Nixlang (Nix From First Principles: Flake Edition #4)

#Nix

This is part 4 of the Nix from First Principles: Flake Edition series.

This is a crash course in Nix's language, which I'll also sometimes call Nixlang from here on, to avoid confusing it with the Nix package manager. For more detailed instructions see the manual. The Nix language is used heavily within Nix, for defining packages, environments, runnable commands and library functions.

Nix's language is expression oriented. This means everything, including up to the level of an entire program, has an output value. It will be most familiar if you have experimented with functional langages such as Haskell, but it is less complicated than those languages, so you don't need to have mastered one to start.

To start experimenting with it, you can run the nix repl command which will let you type in Nix expressions and see the output.

Learn the basics of the Nix language

Derivations (Nix From First Principles: Flake Edition #5)

#Nix

This is part 5 of the Nix from First Principles: Flake Edition series.

So, time to explain the nix derivation. A Nix derivation is a special type of set created from the derivation function which describes how to obtain and build a package. The Nix package manger will then evaluate this derivation, and use the result to copy the built package into the Nix store, which is effectively a folder with all locally Nix built or installed packages in it. The subfolders are hashes derived from the inputs to the expression that built the stored packages.

Build your first derivation

Screenshot testing with Rust

#Gamedev

Intro

This is a follow on from my previous blog post about capturing screenshots with Rust + OpenGL. While the previous article most revolved around the process of actually capturing the rendered pixel data for my OpenGL application, this time around I will focus on the process of actually validating the screenshots.

Read about screenshot testing with Rust

Capturing screenshots with Rust + OpenGL

#Gamedev

Intro

One ongoing area of side projects for me has been game development. It's an area which I've often found interesting, and while the only games I've completed so far have been small games for Ludum Dare with the deadline hanging over, I've often had a few larger side projects running around.

In general for these side projects, I tend to write a lot of the lower level code myself, mostly because it's something that I enjoy learning about. Part of this means that I can't just rely on the framework or engine rendering my game correctly, and instead need some way to test it. Unit tests clearly aren't enough here, while they can verify that I'm calling the functions I expect, they really can't verify that this results in something cohesive being rendered to the screen.

Inspired by tools that exist in my day job of web development like Percy, I decided to take the approach of comparing screenshots taken to reference screenshots.

For some reason, I was under the impression that Factorio had used a similar approach and detailed it in one of their FFF blogs, but when researching for this blog post, I was unable to track this down. They certainly perform integration testing but it appears screenshots are not involved.

Learn how to capture screenshots with Rust + OpenGL