Patrick Van der Spiegel's

Ceramic Glaze Scanography

Thu, 02 Jul 2026 00:00:00 +0000

“Glazes: Cone 6” by Michael Bailey contains recipes for a set of beautiful mid-fire ceramic glazes. One of those recipes is a orange red iron glaze, ranging from yellow to brown and bright orange depending of the percentage of red iron oxide added to the base recipe.

The “Orange red iron glazes” chapter of Michael Bailey’s book, listing the OR1 base recipe.

The book’s test tiles showing the progression from 0% to 23% red iron oxide.

Ingredient	%
Potash feldspar	46.7
China clay	4.0
Bentonite	2.0
Bone ash	15.0
Lithium carbonate	4.0
Talc	16.9
Quartz	11.4

After preparing batches of the base glaze with various percentages of red iron oxide, we noticed that our glaze rather ranged from light to dark green and brown. The obtained colour looked great, but we were wondering why our glaze gave unexpected results.

Tests from a cut up cylinder with the OR1 base recipe at 1240°C, with 1 to 5% red iron oxide.

Glazes are influenced by various factors including the clay body, firing technique, and temperature of the kiln. We wondered what changes we could make to our process as to obtain results that are closer to those of the book. The book itself gave us a hint:

If the glaze fires a rather plain, satin brown […] the maturing temperature can be lowered by cutting out the 4% china clay and increasing the talc […] or the feldspar […].

"Glazes: Cone 6" by Michael Bailey

I decided to base my ceramics graduation project on a study of OR1 glaze variants:

Cutting out china clay and increasing the talc;
Cutting out china clay and increasing the feldspar;
Using synthetic red iron oxide instead of natural red iron oxide.

Other ideas included:

Using natron feldspar instead of potash feldspar;
Using Bentone EW instead of bentonite.

In the end, we saw that increasing the talc made the glaze more matte whilst increasing the feldspar made it more shiny without really having a large impact on colour. What did have an impact on colour was the clay body used and the use of synthetic red iron oxide. By using synthetic instead of natural red iron oxide, the resulting glaze was closer to what we would expect from the book.

Glaze Preparation

We numbered each of the glaze variants and gave a unique identifier based on their percentage of red iron oxide. Glazes 0–19 correspond to the original recipe with natural red iron oxide ranging from 0% to 19%, 20–39 correspond to the altered recipe with increased feldspar, and so on.

Dozens of numbered glaze batches, each stored in their own labelled tub.

For this project, I repeatedly prepared 100g of each base glaze variant and added natural or synthetic red iron oxide. Each of these recipes get their own labelled tub with glaze identifier, the recipe, and a QR code that leads to a page with examples of that glaze.

Glaze #029: tub labelled with the recipe and a QR code to its recipe’s page.

The printed label containing the recipe percentages and a QR code linking to that glaze’s recipe page.

Test Tiles

Using a paint brush, we applied these different glazes on 5x5cm test tiles of various clay bodies: we used reclaimed clay, white clay, and porcelain tiles; all fired at 1230°C or 1240°C.

Audrey Blackman porcelain, one of the clay bodies the glazes were tested on.

A tray of bisque-fired reclaimed and white clay test tiles, ready to be glazed.

There’s a large difference in how these test tiles look before and after firing, going from a soft pink before firing to a range from green to red afterwards.

Glazed test tiles before firing.

Same tiles after firing, ranging from green to speckled red.

A spread of fired test tiles laid out on a table, together with some test cylinders and photo prints of tile scans.

Splatters

During the preparation of one of the test tiles, I accidentally spilled a drop of one glaze over the tile of another glaze, which gave a surprising effect after firing.

Close-up scan from an accidental spill of one glaze over the test tile of another glaze.

I tried to replicate this afterwards with drops of high percentages red iron oxide glaze on a tile with the base recipe without any red iron oxide, but the results were not as expected:

Glaze with red iron oxide splattered onto the base glaze without red iron oxide, before firing.

After firing, the splatter results in a light brown stain with a faint halo.

Because of this result, I abandoned the investigation of splatters and kept focusing on other glaze variants.

Test Spheres

Finally, we also applied these glazes on white clay and porcelain spheres using a spray booth to see what the glaze would look like on curved surfaces.

A finished sphere drying before its bisque firing.

Unglazed test spheres, made by press molding.

Glazed and fired spheres.

Glaze Scanography

A large part of this work was tile scanning on a Canon CanoScan LiDE 210 flatbed scanner to obtain a close-up of each test tile.

Bright red flowers, faintly looks like “mocha diffusion”.

Black speckles across a red background.

Olive surface with tiny speckles and crazing.

Overview of the scanned glaze tiles.

All tiles are visible here. Each scan links to its own recipe page, where you can see the recipe and other scans of the same glaze.

Some tiles are not perfectly flat and thus some spots look blurry when scanned because of a lacking depth of field of my flatbed scanner. This is especially visible at the edges of the tile. I’d love to replicate these scans on a CCD scanner and see if tile edges turn out sharper.

Blurry edge of a test tile on scan, which I assume is due to the lacking depth of field of the flatbed scanner.

Projection

I initially wanted to display the tile scans using a cheap second-hand Elmo Omnigraphic 252, a carrousel slide projector. This specific model sadly does not have a built-in timer. Some shops sell universal interval timers which seem compatible with the projector, but these go for several times the price I paid for the projector itself.

Tile scans mounted as 35mm slides, held up to the light.

A single tile projected using the “Elmo Omnigraphic 252” carrousel slide projector.

I created my own WiFi-enabled timer allowing me to:

Manually, but remotely, cycle through my slides;
Automatically cycle through all the slides of the tray;
Change the time interval of this cycling.

I tackled this project using a Wemos D1 Mini, soldering a 6-pin DIN connector, and struggling to make a basic electronics circuit containing a relay, diode, and transistor on a breadboard. This worked (and I learned a lot in the process), but in the end I had to abandon the idea of using slides for the presentation of my work because of time and budget constraints: the cost of ordering 80 slides with express delivery was not worth the novelty of using this slide projector.

Instead of going for the slide projector, I opted to create a moving image from a handful of scans by zooming in and sliding over the image. This video could then be displayed using a small media player connected to a beamer over HDMI.

The close-up tile scan video projected on a roll up projector screen.

Exposition

View from the exposition space, with cabinets and the tile scan video projected at the back.

The projected close-up video, flanked by display cabinets of the scanned test tiles and glazed spheres and prints from scans on the side wall.

Test tiles of recipe variants in a display cabinet.

Glazed spheres in a display cabinet.

Test tiles of recipe variants in a display cabinet.

Large-format prints of individual tile scans.

A print of the full grid of scanned tiles, arranged from pale to dark.

"Current Trends" guest lecture

Wed, 06 May 2026 00:00:00 +0000

You can find the pbsf-lib Python module on GitHub; follow the installation instructions in the README.md.

In this introduction, we focus on univariate time series.

In the example above, we look at the total load on the Belgian grid over time. You can find this data at opendata.elia.be. We see some seasonality: the load on the grid is generally higher in the colder months, and drops during the warmer months.

Zooming in on just one month, we see a repeating pattern of higher load during workdays and lower load during weekends.

Overlapping all workdays of September, we see the rise in load between 3 and 6 AM, with a small dip in load around noon, and a gradual drop throughout the evening.

See “Time Series Data Mining Using the Matrix Profile”.

The more fine-grained the representation, the more accurate the distance approximation.

Increasing the alphabet_size solves the issue.

See “Sequence Learning: Paradigms, Algorithms, and Applications”.

See “Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress” by Renjie Wu and Eamonn J. Keogh.

Note that the SlopeSignNode is just an example to explain the concept of an approximation. It generally performs poorly in approximating and comparing most time series segments. Other approximations implement a distance that lower bounds the Euclidean distance.

See “Hierarchical pattern matching for anomaly detection in time series” by M. Van Onsem et al.

See the pbsf-lib documentation on how to visualise results.

Maasmarathon

Sun, 03 May 2026 00:00:00 +0000

Distance 42.99 km

Moving time 3:59:09

Elevation 273 m

Spent the weekend at Blegny for the 27th Maasmarathon in Visé. Took the opportunity to walk over the recently opened passerelle de Caster near Kanne, a suspension bridge at 55 metres height over the Albert Canal.

Took the 15-minute trip from Visé to Maastricht in the Netherlands with the Drielandentrein. Visited a bookshop in a church, went to the “Porous Grounds, Sacred Codes” exhibition at Marres, House for Contemporary Culture, and stood at the Drielandenpunt in Vaals where the borders of Belgium, the Netherlands, and Germany intersect.

For the marathon itself, I finally managed to run one under four hours. Couldn’t have done it without the pacer! My previous chip times were:

Race	Year	Chip time
Stockholm Marathon	2024	05:12:03
In Flanders Fields Marathon	2024	04:11:00
European Running Championships	2025	04:12:47

I’ll limit myself to one marathon per year. My next marathon would be the European Running Championships 2027 in Belgrade.

Anagram hikes

Sat, 21 Mar 2026 00:00:00 +0000

bpost has a postal code validation tool, providing a spreadsheet of postal codes with names of Belgian communes and their deelgemeenten. We can convert the spreadsheet to a CSV file using ssconvert, and filter out all special postal codesSinterklaas has his own postal code: 612, for 6th of December. (which are not linked to a commune, but to large public institutions and broadcasters that get their own postal codePostal codes in Belgium – Wikipedia) with xan.

ssconvert zipcodes_num_nl_2025.xls postal.csv
xan head postal.csv | xan view

Displaying 5 cols from 10 rows of <stdin>
┌───┬──────────┬────────────────┬──────────────┬────────────────┬───────────┐
│ - │ Postcode │ Plaatsnaam │ Deelgemeente │ Hoofdgemeente │ Provincie │
├───┼──────────┼────────────────┼──────────────┼────────────────┼───────────┤
│ 0 │ 612 │ Sinterklaas │ <empty> │ Sinterklaas │ <empty> │
│ 1 │ 1000 │ Brussel │ Neen │ BRUSSEL │ BRUSSEL │
│ 2 │ 1005 │ Verenigde Ver… │ <empty> │ Verenigde Ver… │ <empty> │
│ 3 │ 1006 │ Raad van de V… │ <empty> │ Raad van de V… │ <empty> │
│ 4 │ 1007 │ Assemblée de … │ <empty> │ Assemblée de … │ <empty> │
│ 5 │ 1008 │ Kamer van Vol… │ <empty> │ Kamer van Vol… │ <empty> │
│ 6 │ 1009 │ Belgische Sen… │ <empty> │ Belgische Sen… │ <empty> │
│ 7 │ 1011 │ Vlaams Parlem… │ <empty> │ Vlaams Parlem… │ <empty> │
│ 8 │ 1012 │ Parlement de … │ <empty> │ Parlement de … │ <empty> │
│ 9 │ 1020 │ Laken │ Ja │ BRUSSEL │ BRUSSEL │
└───┴──────────┴────────────────┴──────────────┴────────────────┴───────────┘

Filtering out these special cases:

Displaying 5 cols from 10 rows of <stdin>
┌───┬──────────┬─────────────────────┬───────────┬─────────────────────┬───────────┐
│ - │ Postcode │ Plaatsnaam │ Deelgeme… │ Hoofdgemeente │ Provincie │
├───┼──────────┼─────────────────────┼───────────┼─────────────────────┼───────────┤
│ 0 │ 1000 │ Brussel │ Neen │ BRUSSEL │ BRUSSEL │
│ 1 │ 1020 │ Laken │ Ja │ BRUSSEL │ BRUSSEL │
│ 2 │ 1030 │ Schaarbeek │ Neen │ SCHAARBEEK │ BRUSSEL │
│ 3 │ 1040 │ Etterbeek │ Neen │ ETTERBEEK │ BRUSSEL │
│ 4 │ 1050 │ Elsene │ Neen │ ELSENE │ BRUSSEL │
│ 5 │ 1060 │ Sint-Gillis │ Neen │ SINT-GILLIS │ BRUSSEL │
│ 6 │ 1070 │ Anderlecht │ Neen │ ANDERLECHT │ BRUSSEL │
│ 7 │ 1080 │ Sint-Jans-Molenbeek │ Neen │ SINT-JANS-MOLENBEEK │ BRUSSEL │
│ 8 │ 1081 │ Koekelberg │ Neen │ KOEKELBERG │ BRUSSEL │
│ 9 │ 1082 │ Sint-Agatha-Berchem │ Neen │ SINT-AGATHA-BERCHEM │ BRUSSEL │
└───┴──────────┴─────────────────────┴───────────┴─────────────────────┴───────────┘

We are interested in both the names of hoofdgemeenten and deelgemeenten in lowercase. We put both of these in one column using xan unpivot 1,3, which puts all values in a column named value. We now have some duplicates in this column, which is why we also need dedup before writing these to a towns.txt file.

xan filter "!eq(Deelgemeente, '')" postal.csv \
	| xan transform 1,3 'lower(_)' \
	| xan unpivot 1,3 \
	| xan select value \
	| xan dedup > towns.txt

We end up with a list of 2733 names:

xan head towns.txt | xan view

Displaying 1 col from 10 rows of <stdin>
┌───┬─────────────────────┐
│ - │ value │
├───┼─────────────────────┤
│ 0 │ brussel │
│ 1 │ laken │
│ 2 │ schaarbeek │
│ 3 │ etterbeek │
│ 4 │ elsene │
│ 5 │ sint-gillis │
│ 6 │ anderlecht │
│ 7 │ sint-jans-molenbeek │
│ 8 │ koekelberg │
│ 9 │ sint-agatha-berchem │
└───┴─────────────────────┘

With the alphabetic map of these names, we get 18 anagram town pairs for which we plan hiking routes using the openrouteservice API.

rosée ⟷ sorée
idegem ⟷ diegem
olen ⟷ olne
tiegem ⟷ itegem
seraing ⟷ ragnies
herent ⟷ herten
erpe ⟷ peer
laken ⟷ alken
meerbeke ⟷ meerbeek
mere ⟷ meer
mollem ⟷ lommel
herve ⟷ hever
virelles ⟷ serville
veulen ⟷ leuven
borsbeke ⟷ borsbeek
berloz ⟷ borlez
lembeke ⟷ lembeek
edingen ⟷ gedinne

Your work is not just content

Fri, 06 Feb 2026 00:00:00 +0000

Social media platforms provide value to independent creatives by allowing them to fully focus on creating instead of having to figure out how to share their work. Sadly, however, these same platforms reduce creative artists to content creators. When we label artwork — be it short- or long-form writing, illustration, photography, music, or video — as content, we turn unique works into interchangeable slop for consumption by users. I believe that the corporate web’s focus on user engagement plays a large role in this degradation, and that the independent web might offer a solution.

Keep reading at Good Internet!

Visualising Order in Sequences

Tue, 13 Jan 2026 00:00:00 +0000

Sequences

Linear Order

In a sequence, each element directly follows at most one element and precedes at most one other, forming a linear chain. We can visualise this through nodes of directed graphs that are connected using directed edges that represent the linear order between consecutive elements.

The English word “sequence” (which is an ordered sequence of the eight consecutive elements s, e, q, u, e, n, c, e), for example, can be visualised using a directed graph:

Full sentences can also be seen as sequences of ordered symbols. As an example, we have the sentence “This is a visualisation of linear order”. Spaces are also considered elements of the sequence, but we slightly reduce the opacity of their nodes for readability:

We can drop the node labels to visualise the same sentence as:

Order in sequences matters, and these sequences are not restricted to written language. Other examples include sequences of actions (e.g., the sequence of actions that you perform to pour yourself a cup of coffee, as you can’t brew coffee before turning on the machine) or music (e.g., the consecutive notes that make up a specific melody, as swapping the notes around gives us another melody).

Hierarchically Nested Matchings

Many structures also have a natural hierarchical organisation (e.g., we can group words into sentences, paragraphs, sections, and so on). For this reason, we also want a way to describe and visualise hierarchical structure.

This is the visualisation of the linear order in the sequence “nested word”:

We know that the sequence “nested word” can naturally be split into two contiguous subsequences: “nested” and “word”. If we add symbols “⟨” and “⟩” that respectively denote the start and end of a word, we get the following:

We name the “start of a word” a call position and “the end of that same word” its matching return position. All other elements are internal positions. These matches give rise to different nestings.

We represent call and return positions using differently-coloured nodes, and represent matches using dashed directed edges from nodes representing call positions to nodes representing matching return positions:

Nested Words

More formally, a nested wordNested Words - Rajeev Alur is a sequence of elements $w = s_{1} \ldots s_{\ell}$ together with a matching relation, which is a set of matches $i \rightsquigarrow j$ that connect call positions $i$ to their return positions $j$.

For all matches $i \rightsquigarrow j$:

Nestings only go forward: $i < j$

No two nestings share positions: $|\{ i \mid i \rightsquigarrow j \}| \leq 1$ and $|\{ j \mid i \rightsquigarrow j \}| \leq 1$

No two nestings cross: if $i \rightsquigarrow j$ and $i’ \rightsquigarrow j’$, we cannot have $i < i’ < j < j'$

Visualising Nestings

If we visualise “this is a visualisation of linear order”, we get:

Trying out different graph layouts, we get:

Visualising the palindromic phraseList of English palindromic phrases - Wikipedia “⟨⟨⟨Lived⟩ ⟨on⟩ ⟨decaf⟩;⟩ ⟨⟨faced⟩ ⟨no⟩ ⟨devil⟩.⟩⟩”, for example, gives us some nice symmetries (as we have two groups, each with three words, that each respectively have five, two, and five elements):

Nested Substitution Rules

Reading through Stephen Wolfram’s ruliology, I wondered what kind of nested structures we could obtain using simple substitution rules (like repeatedly changing internal positions labelled a with a nesting ⟨aaa⟩).

Example #1

Initial Condition

Replacement Rule

a → ⟨aaa⟩

Looping Visualisation (6 steps)

Example #2

Initial Condition

ab

Replacement Rule

a → bb
b → ⟨aa⟩

Looping Visualisation (9 steps)

Visualising this post

Finally, we can apply nested word visualisation to this post itself by treating the post’s structure as a nested word:

< 1000 kilometres in 2025

Wed, 07 Jan 2026 00:00:00 +0000

See 2024; will post "> 1000 kilometres in 2026" next year.

Scrambled Stations

Fri, 08 Aug 2025 00:00:00 +0000

Scrambled Stations

8 August 2025

Gehusselde stationsnaam (NS)

The Dutch NS railway operator has the “gehusselde stationsnaam” game on their on-board train information displays: a puzzle where you have to figure out the hidden train station through an anagram of that station’s name. These puzzles are also on their Instagram page: the puzzle “Los Pichor Pathir”, for example, matches the Schiphol Airport railway station when unscrambled. As I barely know any Dutch railway station names, I was curious if something similar was feasibile for Belgian railway stations using Flemish dialect words.

Retrieving wordlists

We first retrieve two wordlists: a list of Belgian railway stations, and a wordlist of Flemish dialect words. Afterwards, we use these wordlists to figure out if there are any single- or multi-word anagrams of one wordlist inside the other.

Belgian railway stations

There’s a complete list of Belgian railway stations available on GitHub, maintained by iRail. We can curl the stations.csv file from that repository.

curl --output railway_stations.csv \ 
https://raw.githubusercontent.com/iRail/stations/refs/heads/master/stations.csv

This list of railway stations also contains lots of non-Belgian (often Dutch, German, and French) stations, which I assume are included as they are reachable by train from Belgium. We can filter out these non-Belgian stations using the country-code column.

$ xan filter 'col("country-code") ne "be"' railway_stations.csv | \
 xan select 'name,country-code' | \
 xan view -l 5

┌───┬─────────────────────┬──────────────┐
│ - │ name │ country-code │
├───┼─────────────────────┼──────────────┤
│ 0 │ 's Hertogenbosch │ nl │
│ 1 │ Aachen Hbf │ de │
│ 2 │ Agde │ fr │
│ 3 │ Aime-la-Plagne │ fr │
│ 4 │ Aix-en-Provence TGV │ fr │
│ … │ … │ … │
└───┴─────────────────────┴──────────────┘

Belgian railway stations sometimes have both a Dutch and French name. These names are mostly similar (like “Brussel-Centraal” in Dutch and “Bruxelles-Central” in French), but sometimes completely different (like “Diesdelle” and “Vivier d’Oie”). For railway stations in Brussels, these variants are separated by a slash in the name column of our CSV.

$ xan filter "'/' in col('name')" railway_stations.csv | \ 
 xan select "name" | \
 xan view -l 5

┌───┬────────────────────────────────────┐
│ - │ name │
├───┼────────────────────────────────────┤
│ 0 │ Arcaden/Arcades │
│ 1 │ Boondaal/Boondael │
│ 2 │ Bosvoorde/Boitsfort │
│ 3 │ Brussel-Centraal/Bruxelles-Central │
│ 4 │ Brussel-Congres/Bruxelles-Congrès │
│ … │ … │
└───┴────────────────────────────────────┘

We filter out non-Belgian railway stations, select the names of the remaining stations, split these names on forward slashes so that we get both the Dutch and French name on separate lines where appropriate, and write these lines to a separate text file.

xan filter 'col("country-code") eq "be"' railway_stations.csv | \ 
xan select name | \ 
tr '/' '\n' > railway_stations.txt

Flemish dialect words

As far as I can tell, the “Flemish dictionary” does not provide a downloadable wordlist. They do, however, have a page per letter of the alphabet containing all words starting with that letter. Luckily, these letter pages follow the same structure and contain all words on a single page, so by looping through only 26 pages (one for each letter of the alphabet), we can extract all listed words using grep and sed.

for letter in {a..z}; do
 curl -s https://www.vlaamswoordenboek.be/definities/begintmet/$letter | \
 grep -o '<a href="/definities/term/[^"]*">[^<]*</a> <br />' | \
 sed 's/<a href="[^"]*">\([^<]*\)<\/a> <br \/>/\1/' >> flemish_wordlist.txt
done

Finding anagrams

We first look at how to find single-word anagrams—where single Flemish dialect words are anagrams of certain railway stations—using “alphabetic maps”. We pair sets of railway stations with their anagram set, for example:

{'Ternat'} = {'ratten'}
{'Drongen'} = {'gronden'}
{'Aalst'} = {'slaat', 'staal'}

Ratten 🐀

Afterwards, we look at possible ways of finding multi-word anagrams—where multiple words combine to make up an anagram. Some examples, of which "houten afritser" (“wooden playground slide”) is my favourite:

{'Athus-Frontiere'} = {'houten'} + {'afritser'}
{'Mechelen-Nekkerspoel'} = {'schoempelen'} + {'krekelen'}
{'Dave-Saint-Martin'} = {'mandataris'} + {'in vet'}
{'Mortsel-Deurnesteenweg'} = {'lutter'} + {'messe'} + {'onderwegen'}

Houten Afritser 🛝

Single-word anagrams

When alphabetically ordering all characters within words, two distinct words result in the same ordering only if they are anagrams of each other. In Python, for example:

>>> sorted("undefinability") == sorted("unidentifiably")
True

Thus, to find anagrams of Belgian railway stations in Flemish dialect words, we build two alphabetic maps: dictionaries that map ordered characters to a set of “intra-wordlist anagrams”.

from collections import defaultdict

def get_key(word):
	return "".join(sorted(filter(str.isalpha, word.lower())))

def group(words):
	groups = defaultdict(set)
	for word in words:
		groups[get_key(word)].add(word)
	return groups

with open('railway_stations.txt', 'r') as f:
 stations = set(line.strip() for line in f)
 grouped_stations = group(stations)
 
with open('flemish_wordlist.txt', 'r') as f:
 wordlist = set(line.strip() for line in f)
 grouped_wordlist = group(wordlist)

To obtain the keys of these maps, we turn all characters of the words lowercase and filter out non-alphabetic characters before sorting. We loop through the set of words from each wordlist and build two separate alphabetic maps. Using these alphabetic maps, we can look for anagrams within the wordlists themselves. It turns out, for example, that only two pairs of stations are anagrams of each other:

Mollem = Lommel

Diegem = Idegem

To find the “inter-wordlist anagrams”—which is what we were originally interested in—we look at the intersection of the keys between alphabetic maps. Every common key corresponds to pairs of anagrams:

for key in (grouped_stations.keys() & grouped_wordlist.keys()):
	grouped_stations[key], grouped_wordlist[key]

This results in the following set of single-word anagrams. (As there’s a small overlap between our wordlists, we also get a couple of trivial anagrams in return.)

Details

({'Jette'}, {'tetje'})
({'Temse'}, {'smete'})
({'Lierde'}, {'leider'})
({'Leman'}, {'lamen'})
({'Ternat'}, {'ratten'})
({'Kijkuit'}, {'kijkuit'})
({'Boom'}, {'boom'})
({'Godinne'}, {'doening'})
({'Bleret'}, {'bretel'})
({'Muizen'}, {'muizen'})
({'Asse'}, {'asse'})
({'Landen'}, {'landen'})
({'Weerde'}, {'weerde'})
({'Mol'}, {'mol'})
({'Geel'}, {'leeg', 'Geel'})
({'Sint-Niklaas'}, {'Sint-Niklaas'})
({'Drongen'}, {'gronden'})
({'Welle'}, {'welle'})
({'Ruisbroek'}, {'ruisbroek'})
({'Zele'}, {'zeel'})
({'Meiser'}, {'misere', 'remise', 'eremis'})
({'Spa'}, {'SAP', 'pas'})
({'Menen'}, {'menne', 'nemen'})
({'Lens'}, {'snel'})
({'Tielt'}, {'titel'})
({'Eupen'}, {'peune'})
({'Engis'}, {'signe'})
({'Eichem'}, {'chemie'})
({'Ekeren'}, {'ne keer', 'nekeer'})
({'Niel'}, {'lein'})
({'Manage'}, {'gemaan'})
({'Ronet'}, {'tenor'})
({'Lot'}, {'lot', 'tol'})
({'Essen'}, {'se-n-se', 'sense'})
({'Lobbes'}, {'belbos'})
({'Merode'}, {'moedre'})
({'Blankenberge'}, {'Blankenberge'})
({'Luttre'}, {'lutter'})
({'Lede'}, {'Deel', 'elde', 'leed'})
({'Wavre'}, {'verwa'})
({'Eine'}, {'eine'})
({'Tienen'}, {'ineten', 'tienen'})
({'Aalst'}, {'slaat', 'staal'})
({'Coo'}, {'C.O.O.'})
({'Mollem', 'Lommel'}, {'mollem'})
({'Simonis'}, {'simonis'})
({'Puurs'}, {'Pruus'})

Click to collapse

({'Jette'}, {'tetje'})
({'Temse'}, {'smete'})
({'Lierde'}, {'leider'})
({'Leman'}, {'lamen'})
({'Ternat'}, {'ratten'})
({'Kijkuit'}, {'kijkuit'})
({'Boom'}, {'boom'})
({'Godinne'}, {'doening'})
({'Bleret'}, {'bretel'})
({'Muizen'}, {'muizen'})
({'Asse'}, {'asse'})
({'Landen'}, {'landen'})
({'Weerde'}, {'weerde'})
({'Mol'}, {'mol'})
({'Geel'}, {'leeg', 'Geel'})
({'Sint-Niklaas'}, {'Sint-Niklaas'})
({'Drongen'}, {'gronden'})
({'Welle'}, {'welle'})
({'Ruisbroek'}, {'ruisbroek'})
({'Zele'}, {'zeel'})
({'Meiser'}, {'misere', 'remise', 'eremis'})
({'Spa'}, {'SAP', 'pas'})
({'Menen'}, {'menne', 'nemen'})
({'Lens'}, {'snel'})
({'Tielt'}, {'titel'})
({'Eupen'}, {'peune'})
({'Engis'}, {'signe'})
({'Eichem'}, {'chemie'})
({'Ekeren'}, {'ne keer', 'nekeer'})
({'Niel'}, {'lein'})
({'Manage'}, {'gemaan'})
({'Ronet'}, {'tenor'})
({'Lot'}, {'lot', 'tol'})
({'Essen'}, {'se-n-se', 'sense'})
({'Lobbes'}, {'belbos'})
({'Merode'}, {'moedre'})
({'Blankenberge'}, {'Blankenberge'})
({'Luttre'}, {'lutter'})
({'Lede'}, {'Deel', 'elde', 'leed'})
({'Wavre'}, {'verwa'})
({'Eine'}, {'eine'})
({'Tienen'}, {'ineten', 'tienen'})
({'Aalst'}, {'slaat', 'staal'})
({'Coo'}, {'C.O.O.'})
({'Mollem', 'Lommel'}, {'mollem'})
({'Simonis'}, {'simonis'})
({'Puurs'}, {'Pruus'})

Click to expand

Multi-word anagrams

Above, we only look at one-to-one mappings where a shuffling of one Flemish dialect word gives us a Belgian railway station name. What if we want to have a combination of two (or more) words that, when shuffled together, gives us a station name?

Bruteforce

A bruteforce approach to multi-word anagrams is straightforward: for anagrams of one word list that are made up of $n$ words of the other wordlist, we can take all unordered combinations of $n$ keys of one alphabetic map with replacement, combine them, and check if this combined key is present in the other alphabetic map:

from itertools import combinations_with_replacement as cwr

def combine_keys(*keys):
	return "".join(sorted("".join(keys)))

def n_agrams(n):
	keys = list(grouped_wordlist)
	for key in cwr(keys, n):
		word_sets = (grouped_wordlist[k] for k in key)
		if (combined_key := combine_keys(*key)) in grouped_stations:
			stations = grouped_stations[combined_key]
			print(f"{stations} = {list(word_sets)}")

For $n = 2$, we get 1078 different combinations:

Details

{'Diesdelle'} = {'dees'} + {'dille'}
{'Kapelle-op-den-Bos'} = {'bedelke'} + {'salonpop'}
{'Kapelle-op-den-Bos'} = {'poekele', 'poeleke'} + {'plonsbad'}
{'Kapelle-op-den-Bos'} = {'sloppel'} + {'bakendoe'}
{'Kortemark'} = {'karot'} + {'merk'}
{'Kortemark'} = {'krak'} + {'om ter'}
{'Kortemark'} = {'kram'} + {'kroet', 'krote', 'roket'}
{'Kortemark'} = {'kroam', 'kraom'} + {'trek'}
{'Kortemark'} = {'rakker', 'kraker'} + {'mot'}
{'Kortemark'} = {'ram'} + {'kroket'}
{'Landskouter'} = {'dals'} + {'konteur'}
{'Landskouter'} = {'dokteur'} + {'lans'}
{'Landskouter'} = {'duks'} + {'lantoer'}
{'Landskouter'} = {'dutsen'} + {'akrol'}
{'Landskouter'} = {'kadul'} + {'storen', 'roste(n)', 'rotsen'}
...

Click to collapse

{'Diesdelle'} = {'dees'} + {'dille'}
{'Kapelle-op-den-Bos'} = {'bedelke'} + {'salonpop'}
{'Kapelle-op-den-Bos'} = {'poekele', 'poeleke'} + {'plonsbad'}
{'Kapelle-op-den-Bos'} = {'sloppel'} + {'bakendoe'}
{'Kortemark'} = {'karot'} + {'merk'}
{'Kortemark'} = {'krak'} + {'om ter'}
{'Kortemark'} = {'kram'} + {'kroet', 'krote', 'roket'}
{'Kortemark'} = {'kroam', 'kraom'} + {'trek'}
{'Kortemark'} = {'rakker', 'kraker'} + {'mot'}
{'Kortemark'} = {'ram'} + {'kroket'}
{'Landskouter'} = {'dals'} + {'konteur'}
{'Landskouter'} = {'dokteur'} + {'lans'}
{'Landskouter'} = {'duks'} + {'lantoer'}
{'Landskouter'} = {'dutsen'} + {'akrol'}
{'Landskouter'} = {'kadul'} + {'storen', 'roste(n)', 'rotsen'}
...

Click to expand

For $n \geq 3$, however, things start to slow down drastically. This naive bruteforce approach could further be optimised through some tricks to disregard impossible candidates, but as $n$ increases this would be “mopping with the faucet running”, as we say in Dutch. For $n \geq 3$, we could use a specialised data structure.

Anatrees

An anatreeCharles Reams. 2012. Anatree: A Fast Data Structure for Anagrams. ACM J. Exp. Algorithmics 17, Article 1.1 (2012), 16 pages. https://doi.org/10.1145/2133803.2133804 is a directed edge-labelled tree that describes a set of words. Internal nodes of the anatree represent symbols of an alphabet, and the leaves represent subsets of the set of words. Edges are labelled with positive integers, including zero. Paths of nodes $n_1, \ldots, n_l$ from the root $n_1$ to a leaf $n_l$ along edges labelled with integers $e_1, \ldots, e_l$ arrive at a leaf $n_l$, representing the subset of words that contains exactly $e_i$ times the symbol $n_i$, for all nodes on its path.

One possible anatree for the same set of words {'odd', 'goo', 'dog', 'fog', 'loo'} as used in the original anatree paper, using an own implementation of anatrees and Graphviz

The anatree shown above is just one possible anatree for a certain set of words, as there is not just one anatree for a given set. You could have different strategies to pick which character a certain node represents, each resulting in a different anatree. To generate the figure used above, at every step of construction, the most frequently occurring but not yet considered character of the remaining set of words is used as the symbol for the next node. Creating the anatree for the Flemish wordlist resulted in a tree consisting of 172.210 nodes, if we use the least frequent character first. If we use the most frequent character first, we get 127.534 nodes. Other strategies might lead to further reduction of the size of the anatree.

One approach to find multi-word anagrams of one specific train station is starting with that station’s bag of letters and using it as a “budget” while traversing the anatree of dialect words. For a train station with two occurrences of the letter e, for example, we could explore paths that contain at most two e’s, subtracting “spent” e’s from our available budget. After reaching a leaf node on a partial budget, we could explore other paths with the remaining budget. If we are able to reach another leaf with that remaining budget, we have found a multi-word anagram.

I did not implement this multi-word anagram algorithm in the end, as I’m not sure on performance (you’d have to perform a lot of different anatree traversals to find multi-word anagrams, and there are a lot of different ways in which you could “spend” your budget). Exploring this idea already took a bit more time than expected, and I was happy with my $n = 2$ anagram list.

Side notes

Displaying anagrams

To display these anagrams (and eventually also the title of this post), I wanted to imitate the classic Belgian railway station name signs using CSS. Looking for the typeface, I landed on a 2017 thread on a Belgian train forum where a poster named Nadieeh found out that the Brussels-based graphic design studio Speculoos created Alfphabet, a digitised version of the typeface I was looking for. I ended up downloading the typeface from the osp.kitchen.

The FONTLOG.txt, which is included with a download of Alfphabet, also contains some typical Belgian history:

The Alfphabet family is based on the Belgian road signage called ‘Alphabet’ in French and ‘Alfabet’ in Flemish. It was introduced in 1945 by 3M system working for the Marshall plan after the end of the war. In 1975, it was replaced by the Swiss SNV fonts, but is still in used randomly by the Belgian railroad and Charleroi’s metro. In the early nineties, Pierre Huyghebaert was able to copy the original plates just before the split of the national office of the roads ‘Fond des Routes’ in three regional entities and the burial of the documents deep into regional archives.

Alfphabet FONTLOG.txt

For the background colour, I used a colour picker on a picture of an old railway station sign to end up with rgb(0, 33, 84), and with some trial and error managed to get a white rounded border to simulate old railway signage:

Alfphabet Typeface

Shortest stations

Going through the CSV of train stations with a friend, we were curious about the shortest name for a station in Belgium, which turns out to be Sy (part of Ferrières near the Ourthe river):

$ awk '{ print length, $0 }' railway_stations.txt | sort -n | cut -d" " -f2- | head -n 5

Sy
Ans
Ath
Aye
Coo

The Dutch Wikipedia page claims that Sy is the shortest name for a train station in the whole Benelux.

Bucket lists

I recently read Adam Aaronson’s "I Drank Every Cocktail" about how he managed to drink all “IBA official cocktails” spread over a couple of years; and I loved his story.

Based on his journey, I wonder how long it would take me to visit all Belgian railway stations. Not just passing the station by train, but to actually visit and take a picture with one of the station name signs.

$ xan filter 'col("country-code") eq "be"' railway_stations.csv | \ 
 xan count
 
578

Almost 600 stations spread over a couple of years is certainly not impossible, I guess. Who knows, maybe I’ll be able to post my own “I Visited Every Station” eventually!

Cryptmanteaux

Sun, 01 Jun 2025 00:00:00 +0000

I recently learned through Wikipedia and Wiktionary that the word electrocution is a portmanteau of electro- and execution.The New York Times hating the word electrocution, and Thomas Edison preferring alternative words like dynamort, are also some fun facts found on Wikipedia. This makes sense in hindsight, but it never clicked before that the -cution part of electrocution comes from execution. As I like short word puzzles (like Minute Cryptic and Connections), I wondered if there were any similar word games that use portmanteaux as their main mechanic.

In cryptic crosswords, you find words based on clues which themselves are kind of word puzzles consisting of a definition (typically at the start or end of a clue) and the wordplay (giving instructions on how to get the answer based on the clue). As an easy example clue, which I found online:

Error concealed by city police (4)

decrypting-crosswords

Here, “error” is the definition: we’re looking for an alternative word for “error”. The words “concealed by” indicate that the four-letter word we’re looking for is hidden inside the words “city police”. Other possible types of wordplay include anagrams, double definitions, homophones, and more (as seen on the Minute Cryptic guide.)

Combining portmanteaux, cryptic crosswords, and Connections gives us a word game where you need to find two related portmanteaux through four cryptic clues that describe four base words. By correctly matching pairs of base words, you find the two hidden portmanteaux. Try it out, and check the walkthrough afterwards!

Clue 1

Fast brake ruined morning meal. (9)

Clue 2

Dairy product makes one grin! (6)

Clue 3

Noon activity sounds like lunging. (8)

Clue 4

Sandwiches for German citizens. (10)

Find two portmanteaux: one with (9) letters and one with (13) letters!

Answer 1:

Answer 2:

Coming up with a name for this game, I found the made-up portmanteau of cryptic and portmanteau to be appropriate: “cryptmanteau”. I guess all ideas really are second-hand, as Googling this name returns a PDF file dating back to 2015, containing a game where you are given two cryptic clues for two words that form a portmanteau when combined.

Google search results for cryptmanteau, with one result: a pdf titled GALAXY WORD GAMES from the English Literary Society

The PDF linked above has some fun examples of the concept, like the one quoted below:

cerebrum puzzled Nabir (5)

fan in man I acknowledge (6)

GALAXY WORD GAMES (AV round)

The clues state that the first base word is a five-letter word related to “cerebrum”, and that it is an anagram (hinted at by “puzzled”) of the word “Nabir”, and the second word is a six-letter word related to “fan” that we can find “in” "man I acknowledge". Combining “brain” and “maniac” gives us the portmanteau “brainiac”.

Walkthrough

In the interactive example above, we are looking for two related portmanteaux of length 9 and 13, and we are given four cryptic clues for their base words:

Fast brake ruined morning meal. (9)

Dairy product makes one grin! (6)

Noon activity sounds like lunging. (8)

Sandwiches for German citizens. (10)

Solving these cryptic clues gives us:

Breakfast (morning meal that is an anagram of “fast brake”)

Cheese (exclamation when smiling for picture, also dairy product)

Lunching (noon activity that is a homophone of lunging)

Hamburgers (double definition: citizens of Hamburg, Germany + meat sandwiches)

Connecting the right base words gives us the two portmanteaux:

Breakfast + Lunching → Brunching (9)

Cheese + Hamburgers → Cheeseburgers (13)

Part of the fun is that you might not need to find all four base words from scratch; maybe one from each pair combined with the other clue and figuring out the common theme could be enough to connect the dots.

I’ll certainly be trying to come up with more cryptmanteaux in the future and posting them on my website; send me a message if you come up with some yourself!

1000 kilometres in 2024

Sat, 04 Jan 2025 00:00:00 +0000

intervals.icu API and jq:

BASE_URL="https://intervals.icu/api/v1/athlete"
OLDEST="2024-01-01"
NEWEST="2024-12-31"

# Fetch activities
activities=$(curl -s -u API_KEY:$API_KEY \
 "$BASE_URL/$INTERVALS_UID/activities?oldest=$OLDEST&newest=$NEWEST")

# CSV header
echo "date,cumulative_distance"

# Initialise cumulative distance and iterate through each date
cumulative_distance=0
current_date=$(date -I -d "$OLDEST")
end_date=$(date -I -d "$NEWEST + 1 day")

while [ $current_date != $end_date ]; do
 # Calculate total distance for the current day
 day_distance=$(echo "$activities" | \
 jq --arg date "$current_date" '
 map(select(.type == "Run" and 
 (.start_date_local | startswith($date)))) | 
 map(.distance) | 
 add // 0') # return day distance (or return 0 if false or null)

 # Add day distance to cumulative distance
 cumulative_distance=$(echo "[$cumulative_distance, $day_distance]" | jq 'add')
 
 # Date and cumulative distance up until current date
 echo "$current_date,$cumulative_distance"
 
 # Move to the next day
 current_date=$(date -I -d "$current_date + 1 day")
done

date,cumulative_distance
2024-01-01,0
2024-01-02,6919
2024-01-03,6919
2024-01-04,10034
2024-01-05,10034
...
2024-12-27,958647.54
2024-12-28,973656.54
2024-12-29,973656.54
2024-12-30,980075.54
2024-12-31,1000088.54

gnuplot:

set terminal pngcairo size 1024,768 font "iA Writer Quattro V"
set output 'cumulative_distance_graph.png'

# Set data file separator and time-related settings
set datafile separator ","
set xdata time
set timefmt "%Y-%m-%d"
set format x "%B"
set ylabel "Cumulative Running Distance in 2024"
set grid

# Customize axis and range
set xtics rotate by -45
set yrange [0:1150]
set xrange ['2024-01-01':*]

# Set margins
set lmargin 16
set rmargin 10
set tmargin 5
set bmargin 6

# Mark specific races with arrows and labels
set arrow from '2024-06-01',0 to '2024-06-01',1150 nohead lw 1 dashtype 3 linecolor "gray30" # Stockholm marathon
set label "🇸🇪" at '2024-06-03',950

set arrow from '2024-09-15',0 to '2024-09-15',1150 nohead lw 1 dashtype 3 linecolor "gray30" # In Flanders Fields marathon
set label "🇧🇪" at '2024-09-17',950

# Plot cumulative distance and 1000 km goal with markers and styles
plot "cumulative_distance.csv" using 1:($2/1000) with lines title "Cumulative Distance (km)" lw 2 linecolor rgb "orange-red", \
 0 with lines title "Marathons" lw 1 dashtype 3 linecolor "black", \
 1000 with lines notitle "1000 km" lw 1 dashtype 2 linecolor "black"

Common Favourite Films on Letterboxd

Thu, 26 Dec 2024 00:00:00 +0000

Letterboxd allows users to list up to four favourite films, but there’s no built-in way to find people with similar tastes. To solve this, I developed a script to scrape ‘fans’ of specific films and identify overlapping favourites.

My favourite films on Letterboxd

Scraping Fans

As there is no public Letterboxd APII requested access to the Letterboxd API beta in 2016, but never received a response. but I still want to find people with similar movie tastes, I wrote a script that takes a Letterboxd movie identifier as an argument and scrapes the list of users who have listed this movie as one of their four favourites (also called “fans” on Letterboxd). This script curls every fan page of the given movie, extracting user identifiers through a regex with grep, and appending them to a CSV file.

#!/bin/sh

movie=$1
page=1

while :; do
 fans=$(curl --silent "https://letterboxd.com/film/$movie/fans/page/$page/" | \
 grep -Po '(?<=a href="/)[^"]+(?=/\" class=\"name\")')
 [ -z "$fans" ] && break
 echo "$fans" >> "${movie}_fans.csv"
 page=$((page + 1))
done

As a test, we can look at the “The Innocents”, which currently has 88 fans. "./letterboxdFans.sh the-innocents-2021" produces a CSV named "the-innocents-2021_fans.csv" in the same folder. Using the fish shell, we check the number of entries by counting newlines in the CSV with cat and count.

$ cat the-innocents-2021_fans.csv | count
88

Looking at some popular films (like Parasite, for example) shows that we can see at most 6.4k fans even though there are 102k of them in this specific case, however. As there are 25 fans displayed per page, and we can only scrape 6.4k of them, this means that the fan pages are capped at page 256. Manually confirming this indeed shows that there are at most 256 pages of alphabetically ordered fans per film.

Finding Similar Profiles

After scraping ~~all~~ at most 6.4k fans of each of my four favourite films and putting the resulting CSVs in a separate folder, we can find profiles that have at least two favourite films in common by looking for duplicate entries:

$ ls letterboxd_fans/
good-time_fans.csv 
paterson_fans.csv 
perfect-days-2023_fans.csv 
uncut-gems_fans.csv

$ cat letterboxd_fans/* | sort > merged.csv
$ uniq -c merged.csv | sort -nr | head -n 5
 2 mandypixel
 2 malcolmconger
 2 louistoth
 2 loshmy
 2 ln42

Automatically Following Profiles

I attempted to automatically follow profiles that share at least two favourite films using Selenium, but only got as far as automatically visiting every profile with shared favourite films.

Current issues include not being able to use an existing Firefox profile that is logged into my Letterboxd account to Selenium, and not being able to automatically click on the Follow button.

As this is just a quick one-time project, I manually logged in using the browser that pops up after starting my script before automatically cycling through profiles from the merged.csv list and manually clicking the follow button before the next profile page loads. This allowed me to quickly follow a couple of profiles with common favourite movies.

"Using AI to Detect the Unusual"

Mon, 25 Nov 2024 00:00:00 +0000

This is an abbreviated transcript of my FARI “Happy Hour” talk titled “Using AI to Detect the Unusual”, held at BeCentral on the 25th of November, 2024. The slides I used during my talk can be found here. This transcript was created using OpenAI’s Whisper model and edited using GPT-4o.

Introduction

Tonight, I’m going to talk about how we can use artificial intelligence to detect the unusual. If I knew nothing about artificial intelligence, I would have three main questions regarding this title:

What is artificial intelligence and how do we use it in general?
What is the unusual, and how would we traditionally detect this without using artificial intelligence?
How would we then use AI to detect the unusual?

What is Artificial Intelligence?

Artificial intelligence (AI) is a field of research. It is a subset of mathematics and computer science, and it overlaps with philosophy and linguistics, depending on the specific subset of AI being studied. This interdisciplinary nature makes it a fascinating field because you often collaborate with a diverse group of people across various domains.

AI has been researched since the 1950s, and although we’ve heard a lot about it in media recently, the increase in computing power over the last few decades has enabled us to perform remarkable tasks based on the research conducted during this time.

Examples of AI Applications

I have some examples of how we use artificial intelligence:

Search:
- In 1996, an AI algorithm named Deep Blue managed to beat the world chess champion, Garry Kasparov.
- More recently, AlphaGo competed against the second-best player in Go, a game with greater complexity than chess. For those who haven’t seen it, I recommend the excellent documentary about it.
- Day-to-day applications like Google Maps and Waze help you navigate from point A to point B in the most efficient way possible. You might not think of this as AI, but it results from extensive AI research.
Classification: If you have a large dataset containing labeled images of cats and dogs, for instance, you can use AI to assign the correct label to new images as they come in. Google offers “teachable machines” that allows users to train classification models using their webcam, requiring no programming experience.
Generative AI: Tools like ChatGPT for text generation and DALL-E for image generation have gained popularity recently. While impressive, it’s crucial to recognize that generative AI is not the only area of AI research. There are also significant studies in classification, search, and reinforcement learning, for example.

What AI Isn’t

Now that we’ve discussed what artificial intelligence is, it’s also important to clarify what AI isn’t.

First, AI isn’t about creating conscious machines, a notion often propagated in science fiction. While artificial general intelligence—AI with capabilities akin to human intelligence—is a goal for some researchers, most focus on narrow AI tasks. When companies vaguely claim they “use an AI” for significant goals, it raises red flags.

For example, it would be like a toothpaste manufacturer saying they used “a chemistry” to create better toothpaste—it doesn’t quite make sense. It’s important not to fall for the hype surrounding AI in the media. AI has nothing to do with Terminators or humanoid robots; it’s grounded in real-world applications.

Examples of AI Misuse

I’ve put together some examples of how AI can be misused:

The Smartschool Case: Smartschool is a platform where parents can communicate with their child’s teachers and access their test results. Recently, the company behind the platform announced plans to use AI to predict student dropout risks. This admirable goal (trying to reduce student dropout) was poorly executed, essentially reducing these students to test results and vaguely implementing AI in their product without public dialogue, leading to significant backlash.
Social Media Companies: Social media platforms are designed to maximize user engagement for profit through targeted advertisement. However, this focus can pose risks to younger users: vulnerable youth are presented with content that romanticizes self-harm, for example. Internal Facebook reports revealed that toxic content on Instagram negatively affects teenagers, but the company plays this down in public.
Dutch Child Care Benefit Scandal: Thousands of families in the Netherlands were wrongly accused of fraud when applying for child care benefits, based on algorithmic predictions. This led to severe financial hardships for these families, illustrating how false positives can have disastrous impacts.

Detecting the Unusual

Let’s now discuss the unusual and how we detect it.

Gathering Data

Everything around us—every process, system, or activity—produces data, which we can collect through observation and measurement. Here are some examples:

Health Monitoring: Tracking heart rate and blood pressure.
Financial Activity: Every transaction you make with a credit card—transaction date, time, location, and amount.
Social Network Activity: Actions like sending messages, making friend requests, and liking posts.

Understanding Anomalies

Anomalies are deviations from what we consider to be normal behavior, and these deviations leave traces in our observations. We are interested in understanding the underlying causes of these deviations. For example:

Health Monitoring: An ECG can show if you have arrhythmia, indicating an underlying health issue.
Financial Activity: If you generally conduct transactions in Belgium, an unexpected transaction from Nigeria would likely be labeled as anomalous.
Social Network Activity: You can have spam messages or friend requests, also classifiable as anomalies.

Anomalies vs. Outliers

It’s essential to distinguish between anomalies and outliers. Some sources claim they are the same, but this is misleading.

Traditional examples of anomalies often show high peaks and obvious outliers in time series data. However, context matters significantly in determining whether a data point is an anomaly. For instance, a large peak in time series data could be normal if it occurs at regular intervals, suggesting that context is critical.

Using AI to Detect the Unusual

Now that we’ve reached the point of understanding what anomalies are, we can discuss using AI for detecting these unusual patterns.

Anomaly Detection

Anomaly detection refers to finding patterns in data that do not conform to expected behavior. The goal of identifying anomalies is to alert domain experts that something deviates from the norm and to provide actionable information to address the situation.

For instance, in the medical field, detecting arrhythmia would alert healthcare professionals to investigate further.

The Role of Context

Detecting anomalies is highly context-dependent. While classification involves labeling items, anomaly detection focuses on recognizing deviations from expected patterns—often rare occurrences.

Traditional approaches to identifying anomalies are rule-based, which can be cumbersome and require constant updating. Relying solely on these rules may lead to missed anomalies or false triggers.

A Rule-Based Approach

If we would have a manual rule-based configuration for anomaly detection:

Some anomalies would be missed if predefined rules do not trigger: no rule for certain anomalies means that these anomalies go undetected
Rule-based trigger check on a “per event” basis, no “bigger picture”: context matters, and anomalies can involve multiple events
Expert time would be wasted on hard-coded rule design instead of tackling the root cause of anomalies

Challenges in Anomaly Detection

In my research, I previously focused on developing a pattern-based series framework. Instead of relying on hard-coded rules, my framework allows us to apply various AI algorithms to learn a model of normal behavior and test new data against it. However, challenges exist, including:

Complexity of Normal Behavior: Context and variation can make defining ’normal’ difficult.
Domain-Specific Definitions of Anomalies: Definitions of anomalies can vary significantly depending on the domain (e.g., cybersecurity vs. healthcare).
Ambiguity of Artifacts: Different events might present similarly in data, making it challenging to differentiate between normal and anomalous behavior.
Impact of Mistakes: Algorithms can produce both false positives and false negatives, leading to significant consequences.

Conclusion

In summary:

AI is a research field: It focuses on narrow tasks and is not about conscious machines.
Anomaly detection aims to identify unusual patterns in various domains. These patterns are often context-dependent deviations from what is deemed normal.
Algorithms can make mistakes, resulting in false positives and negatives that can have disastrous consequences.

Thank you all for your attention. Slides and transcripts are available on my website. If you have any questions, please feel free to ask. Thank you!

"Hell's Hex Station" Write-Up

Fri, 22 Nov 2024 00:00:00 +0000

In this write-up, I go over the step-by-step solution of “Hell’s Hex Station”, a challenge I created for the 2024 edition of Infogroep’s Capture the Flag.

Description

Mein Fraulein,
I haven't heard from you in a while. Won't you write me?

Solution

The title of the challenge is a reference to Rudolf Hell (inventor of the “Hellschreiber”), the hexadecimal system, and numbers stations. The description is a reference to a 2006 talk at DEF CON and also references the recurring German theme of this challengeThe “Mein Fraulein” description, Rudolf Hell being a German engineer, and the correct flag of this challenge containing a ß..

The provided .wav file starts with the tune of the “Lincolnshire Poacher” numbers station followed by a voice repeating "CP437" and ends with beeping. Using exiftool to look at the metadata of the audio file, we get the following output:

$ exiftool recording.wav

⋯
Comment : FELDHELL freq=14071.500
Software : fldigi-4.1.06 (libsndfile-1.0.28)
⋯

The Comment and Software values bring us to the solution of the challenge: the beeping that makes up the bulk of the .wav file is “Hellschreiber” in Feld Hell mode, a technique developed in 1927 by Rudolf Hell and currently in use by Ham radio hobbyists. Hellschreiber modes are based on character scanningFeld Mode: we transmit characters as patterns of beeps, resembling a dot-matrix printer.

As fldigi contains Feld Hell as one of their supported digital modes, we can use it to decode the beeping in our file. After installing and opening fldigi, we select the correct operational mode via "Op Mode → Hell → Feld Hell" and load our .wav file via "File → Audio → Playback".

In the waterfall view on the bottom of the fldigi window, we select the area that lights up when the beeping begins. Below, you can see a screenshot of the fldigi program, around a minute after loading the .wav file. A list of numbers in hexadecimal representation appear on screen:

A screenshot of the fldigi program, a minute after loading the provided .wav file. A list of numbers appeared on the screen.

49 47 43 54 46 7b 48 33 4c 4c e1 43 48 52 33 31 42 33 52 21 7d

In general, we could use a range of different tools to transform this list of hexadecimal numbers to a string of characters. The simplest way of doing this would be through CyberChef. After giving our list of numbers as input, we can pick the "From Hex" recipe (or choose “Magic” and provide IGCTF as the crib) to obtain the following output:

IGCTF{H3LLáCHR31B3R!}

This is not the correct flag, however! The provided .wav file specifically mentioned "CP437", which is a specific character encoding. Using Python (or any other tool that allows us to use CP437), we obtain IGCTF{H3LLßCHR31B3R!}, which is the correct flag:

>>> bytes.fromhex(hex_string).decode("cp437")
'IGCTF{H3LLßCHR31B3R!}'

Mindful use of LLMs

Tue, 24 Sep 2024 00:00:00 +0000

I believe large language models can bring value in use cases where they assist with tasks like summarising, translating, and reformulating text. I have previously used these models for some quick translation work, and I also started using the OpenAI API to help me summarise the contents of my clipboard through an Espanso shortcut.

Outside of summarising and translating texts, there are advantages for software developers as well. GitHub Copilot drastically reduces the time needed to figure out popular APIs and successfully assists in writing proper documentation, for example. These tools could increase both the speed of development, codebase quality, and consistency as long as developers understand, reason about, and ultimately agree with every generated line of code and comment before committing their changes.

I am also surprised about the quick adoption by a non-technical audience. LLMs would have seemed like magic to me only a couple of years ago, even though they are now commonplace—with friends and family members asking me to teach them “how to use ChatGPT” or enthusiastically telling me they already use it regularly. During a lightning talk on the risks of these models, for example, I asked the audience to raise their hands if they had already used ChatGPT in a professional setting. Almost everyone in the audience had done so recently.

Flip side of the coin

Assistance with writing and coding aside, I believe there is also a serious flip side of the coin. Overreliance and excessive use of this technology could degrade writing proficiency and programming skills. As Belgian universities allow LLM usage by their students, we urgently need to reform how we evaluate the quality of a student’s work now that we can’t rely on traditional assignments. I’ve learned a lot by struggling through my university classes’ programming projects, and I’m afraid I would have learned a lot less if I had easy access to software that completes my work for me.

Next, it is easy to misuse these models (not always out of malice, but also out of misunderstanding how the technology works) — applying them to use cases that range from wasteful but innocent to downright unethical and harmful. Consequences of this misuse include the pollution of the internet with slop and spreading misinformation (as some users do not realise that tools like ChatGPT are not a replacement for search engines and should not be trusted to return reliable information). Sadly, because of these consequences, the whole field of AI is getting backlash—even though the field itself is so much more than just generative AI.

As an example of unethical and harmful misuse, we have ZetaForge, a no-code tool to “rapidly build and deploy advanced AI pipelines”. The video linked above demonstrates how they use their tool to generate a textual description of war scenario images, on which they use a large language model to suggest a course of action to military personnel. In their own words, their demonstration pipeline “can be deployed as is, without any modification.” I’m not looking forward to a future where we rely on statistical models outputting As a large language model, I recommend opening fire. 🔫🤠 as a military strategy to ensure civilian safety.

Finally, the resources necessary to keep these models operational: a summary of a scientific paper — which I can perfectly write myself, albeit slower — is not worth the computing time, energy, and the bottle of water necessary to generate that summary. Combined with the fact that these models are trained on data without permission from artists and writers made me question my own use of these models in their current form.

Personal ethics

I have decided to stop using LLMs as a writing tool, as — personally — their downsides outweigh the benefits. Summarising text was my initial use case for LLMs, but I summarise text to internalise and learn from the material I am working with. In my case, summarising should take time. LLMs promise speed, but speed isn’t always desired.

I also previously used an LLM to help write a blog post for a small half-day project. I still believe I used the tool responsibly, in this case, as I only used it to rephrase or expand certain lines of text, using the output as an initial draft and not copying generated text verbatim. I also clearly indicated which text was written with assistance, even though most was still my own. However, I plan to never use LLMs for writing blog posts again: I strongly believe that if I don’t take the time to write a text, no one should waste their time reading it either.

I plan to put my money where my mouth is by taking the amount that I usually pay for API costs (which is only pennies, so I’m throwing in the price of a subscription to ChatGPT Plus) and donating it to a charity standing up for nature and biodiversity in my region. I know I am not making a dent by donating a small amount of money to a charity and mindful use of LLMs, but I am doing my best to align my actions to my values.

llm + Espanso: Clipboard Summaries

Mon, 09 Sep 2024 00:00:00 +0000

I recently wrote an Espanso trigger named :llm(⋯) that pipes the contents of my clipboard to Simon Willison’s llm CLI tool and expands the trigger to llm’s output in a custom Obsidian callout. This regex trigger expects a keyword (like summarise, simplify, or translate) between its parenthesis, and transforms this keyword to a prompt that summarises, simplifies, or translates whatever text I current have within my clipboard.

After frequent use of this trigger, I noticed that I mostly use :llm(summarise). To simplify my workflow, I decided to transfer summarisation to a dedicated :summary trigger so that I don’t have to rely on providing and capturing keywords using regex:

- trigger: ":summary"
 replace: "> [!llm] _\"{{title}}\"_\n> {{summary}}"
 vars:
 - name: clipboard
 type: clipboard
 - name: summary
 type: shell
 params:
 cmd: "echo $ESPANSO_CLIPBOARD | llm -m 'gpt-4o' -s
 \ 'Summarise the provided text into a concise UK English paragraph. 
 \ Ensure the summary captures essential information.'"
 - name: title
 type: shell
 params:
 cmd: "echo $ESPANSO_SUMMARY | llm -m 'gpt-4o-mini' -s
 \ 'Generate a short one-sentence title that accurately reflects
 \ the main idea of the summary. Use UK English.'"

Notes from August

Thu, 05 Sep 2024 00:00:00 +0000

In August, after learning about the IndieWeb, I updated my website and turned it into this blog. It has been years since I did something web-related, and I always enjoyed tinkering on the web, so I am glad I took the time to try and rebuild my website. This month was my first time using Hugo, customising an existing theme to fit my needs. I also:

Automated the creation of daily notes on this blog using GitHub Actions
Created category filters for this blog using the :has selector in CSS
Pulled up-to-date statistics, like distance ran in a year — 470.5 kmData pulled from intervals.icu, and my current Tetra League ranking — 10716.96 TR (), using various APIs

In terms of other small projects,

I found new breakfast spots using Overture Places and duckdb,
Created a Pomodoro timer script that keeps track of my work in Obsidian,
Started learning the Rust programming language,
Looked into visualising Belgian roads,
Learned more about uv for Python

Writing daily notes

Initially, I aimed to write a daily note on this blog, sharing small personal projects or interesting things I encountered that day. After twenty consecutive notes, however, I started putting quantity before quality on days I did not feel like writing. If I did not enjoy writing a note, I am sure it isn’t pleasant for others to read that note either. I am still looking for how and when to best write and share short notes for “blogumentation”.

Blogs I enjoyed reading this August

Fabien Sanglard’s 0X10 RULES are great rules to follow when creating (small) websites. On that note, I also appreciated Justin Wong’s collection of “monospace webpages”, The Brutalist Report, and computer.rip.

Other blogs that I added to my RSS feed this month include Simon Willison’s Weblog, Sebastian De Deyne, Matt Hodges, Robin Rendle, Brad Barrish, anhvn.com, and James G.

Can GPT Boggle?

Tue, 27 Aug 2024 00:00:00 +0000

During the programming and writing of “Automating Boggle”, I remember being surprised that popular AI models (like GPT-4) — capable of many remarkable tasks — couldn’t do something so “trivial” as reading rotated dice. I regularly retry using the OpenAI API to recognise the labels on these dice, but the models always fail to do so (and they often fail spectacularly). The prompt I use is the following:

"Given an image of a 4x4 grid of Boggle dice, extract and return the 16 letters from the grid, in a continuous string from top left to bottom right, left-to-right across each row. Account for any rotations of the letters. The output should be the correct sequence of letters as a continuous string, without any additional text, newlines, and formatting."

It often happens that I get more than 16 letters in return, most of them not even present on the board. If we look at the OpenAI API documentation, under the section of Limitations, we see:

Rotation: The model may misinterpret rotated / upside-down text or images.

Instead of manually calling the API once in a while to see if an updated version of a vision model can complete the task, I plan to write a GitHub Actions workflow that regularly calls the API and compares the output to the expected solution, printing the results on this website. Guess this will be my AI model benchmark: how long before these models can read rotated dice from an image and the instructions from the prompt above?

git mailmap

Mon, 26 Aug 2024 00:00:00 +0000

I work on my Git repositories from different machines, which separates my contributions over different Git identities. Looking at all developers that contributed to one of my project’s repository, we get the following:

(base) pvdsp@laptop:~/git/⋯$ git shortlog -sne
 71 Patrick Van der Spiegel <pvdsp@⋯.be>
 58 Patrick Van der Spiegel <patrick@⋯.com>
 ⋯

Adding a .mailmap file to the root directory of the repository — following the Proper Name <proper@email.xx> <commit@email.xx> structure — fixes this issue:

Patrick Van der Spiegel <pvdsp@⋯.be> <patrick@⋯.com>

(base) pvdsp@laptop:~/git/⋯$ git shortlog -sne
 129 Patrick Van der Spiegel <pvdsp@⋯.be>

Joining Mastodon

Sun, 25 Aug 2024 00:00:00 +0000

After years of lurking on Twitter — barely contributing to conversations on the platform, and only using it to read tweets from creators I follow — I am trying out Mastodon for a while. I planned to do this a while ago: most people I follow on Twitter also use or cross-post to Mastodon, and it makes sense to use non-commercial alternatives where possible.

Reading blog posts through my RSS reader is excellent for keeping up with news related to my interests, but I sometimes miss some interaction and discussion on these posts. I hope to find this interaction through Mastodon, as not every blog supports Webmentions (and I do not yet have it fully implemented either).

For now, I have:

Manually searched and followed people of which I read their blogs
Added a link to my Mastodon profile on my main page (including ⁂)
Verified my domain by adding <link href="https://c.im/@pvdsp" rel="me"> to the <head> of my pages

I looked into using my custom domain for Mastodon so that I could use @patrick@vanderspie.gl as my username, but I will postpone this as it involves setting up your own Mastodon server or using a managed host (and I feel like using a custom domain is not worth yet another subscription).

I’ve heard good things about Mastodon so far, so I look forward to seeing if it is a better Twitter alternative!

CSS filter posts by category

Sat, 24 Aug 2024 00:00:00 +0000

As I also plan to upload non-technical posts and notes on my website, I added the ability to filter content based on categories. Trying to avoid another Javascript solution, I remembered the CSS :has selector (which I read about in a blog post from Ryan Mulligan’s website).

I based my solution on his examples, tailoring it to the classes available in my Hugo theme. First, I added a partial with a fieldset and included it in the HTML template used for lists. For now, the categories are hard-coded, as I don’t plan on adding more in the future:

<fieldset>
 <legend>Filter by category:</legend>

 <div>
 <input type="radio" id="all" name="filter" value="all" checked />
 <label for="all">Show all</label>
 </div>

 <div>
 <input type="radio" id="tech" name="filter" value="tech" />
 <label for="tech">Only technical</label>
 </div>

 <div>
 <input type="radio" id="nontech" name="filter" value="nontech" />
 <label for="nontech">Only non-technical</label>
 </div>
</fieldset>

Next, I added the following to my css:

fieldset {
 opacity: 75%; 
 font-size: 0.75em;
 border: dotted;
 border-width: 1px;
}

body:has([name="filter"][value="tech"]:checked)
 .post-line:not([category="tech"]) {
 display: none;
}

body:has([name="filter"][value="nontech"]:checked)
 .post-line:not([category="nontech"]) {
 display: none;
}

Finally, to make the filters work, I added a category to every post or note in Hugo:

---
title: "CSS filter posts by category"
date: "2024-08-24"
⋯
category: "tech"
---

and included this category in the partial for post entries:

<div class="post-line" category="{{ .Params.category }}">
	⋯
</div>

Python packaging in Rust

Fri, 23 Aug 2024 00:00:00 +0000

Thanks to a blog post by Simon Willison, I learned about the open-source Python tooling by Astral. Coincidentally, I started looking into the Rust programming language a few days ago and was pleasantly surprised by Cargo.

uv — which is a Python package and project manager written in Rust — is a kind of “Cargo, for Python”. Looking at the highlights in uv’s README.md file, the tool sounds promising, and I fully intend to try it out soon. The demonstrations on their blog post are convincing, especially uvx and uv run:

With uv run, you don’t have to think about activating virtual environments, managing dependencies, or keeping your project up-to-date. It just works.

Astral - uv: Unified Python packaging

and, more importantly:

(⋯) it also adds features that are essential to local development but not covered by the standards, like relative paths and editable dependencies.

Astral - uv: Unified Python packaging

Astral also provides a Python linter and code formatter written in Rust, named ruff. I’m currently not using a dedicated linter; I’m just relying on my IDE and its built-in checks, but I might try adding ruff to my workflow.

Tracking tasks in Obsidian

Thu, 22 Aug 2024 00:00:00 +0000

A couple of days ago, I made a simple addition to my Pomodoro timer. To better keep track of work done in a given day, I edited my daily notes template of my Obsidian journal to include a table with a row for each pomodoro interval, explaining what I did during that interval:

---
type: research journal
pomodori: 
---
# Tasks of the day
| Hours | Task |
| ----- | ----- |
# Topics
- 

# Ideas and Insights
-

I now keep track of when a new interval starts, and write a new row after each interval ends. This row includes:

The start and end time of the interval
Space for a one-sentence summary of what I did during that interval

START_TIME=$(date +%H:%M)
ENTRY="$JOURNAL/$(date +%Y-%m-%d) Research Journal.md"
⋯

NEW_ROW="| $START_TIME — $(date +%H:%M) | ⋯ |"
sed -i "s/# Topics/$NEW_ROW\n# Topics/" "$ENTRY"

A short and quick addition, but I’ve noticed that it helps me keep focused on one specific task during each 25-minute interval, for now. As an example:

Hours	Task
09:02 — 09:27	Literature review on `[[Complex Systems\|DES]]` `#phd`
09:34 — 09:48	Reading `[[🔴 (Zimmermann, 2008)]]` `#phd`
10:48 — 11:16	Reading paper draft `#paper`
⋯	⋯

Learning Rust

Wed, 21 Aug 2024 00:00:00 +0000

I started experimenting with the Rust programming language by reading “the book”, trying some basic things and loosely following along with the examples. Although I’ve only gone through the first three chapters — stopping after the common programming concepts and leaving the idea of ownership for later — I’m still planning to write down what I found interesting about the language compared to the ones I use more often (mainly Python and Julia).

Nothing interesting will be written here for anyone familiar with Rust, but I’m growing fond of the idea behind “blogumentation” for self-documentation. In this case, writing things down that I found interesting will help me remember them:

Terminating statements with `;`

Like multiple other languages, Rust uses the semicolon after statements. I wondered why this is so often the convention, and found a blog post by Nicole Tietz speculating why that is the case.

Cargo and initialising Git repositories

Using Cargo — which also initialises a new Git repository by default and creates the project structure when you begin a new project — seems like a great workflow.

Cargo expects your source files to live inside the src directory. The top-level project directory is just for README files, license information, configuration files, and anything else not related to your code. Using Cargo helps you organise your projects. There’s a place for everything, and everything is in its place.

The Rust Programming Language

In general, I think it’s helpful that it initialises and enforces this organisation. I tend to ignore conventions when quickly whipping something up in Python, which gets messy quickly.

Shadowing of immutable variables

Variables are immutable by default, but I was initially confused as to why you would allow shadowing for immutable variables. I did not immediately see any scenario where it would make sense to shadow an immutable variable. This will probably make more sense once I see some examples in practice. Another thing about shadowing that I found a bit weird is the example below:

let spaces = " ";
let spaces = spaces.len();

The first spaces variable is a string type and the second spaces variable is a number type. Shadowing thus spares us from having to come up with different names, such as spaces_str and spaces_num; instead, we can reuse the simpler spaces name.

The Rust Programming Language

This feels like a fast way for things to get messy, shadowing the same variable in different scopes but changing the type (and thus the meaning) of the variable, but keeping the name. spaces_str and spaces_num don’t sound so bad to me, as the names at least convey meaning.

Visual separator for integer literals

Number literals can also use _ as a visual separator to make the number easier to read, such as 1_000, which will have the same value as if you had specified 1000.

The Rust Programming Language

Through looking up other languages that provide a visual separator for integer literals, I stumbled upon this GitHub issue, which contains some interesting discussion on separators.

Other things that I like so far, at first glance

A lot of these might seem very standard, but as Python is my daily driver, it’s nice to try a language that has:

Integer division truncating toward zero to the nearest integer:

let truncated = -5 / 3; // Results in -1

The terminology “destructuring” for breaking up a single tuple into parts (which Python also has, but first time that I encounter this terminology):

let tup = (500, 6.4, 1);
let (x, y, z) = tup;

The distinction between arrays and vectors, and the initialisation of an array using this notation:

let a = [3; 5]; // same as writing let a = [3, 3, 3, 3, 3];

The fact that you must declare the type of each parameter in function signatures:

fn main() {
 print_labeled_measurement(5, 'h');
}

fn print_labeled_measurement(value: i32, unit_label: char) {
 println!("The measurement is: {value}{unit_label}");
}

The distinction between acting and returning through statements and expressions (and that expressions don’t have ending semicolons). I already see myself making mistakes by turning expressions into statements — not letting it return a value — by accidentally adding a semicolon. Luckily, the thrown errors are very readable, so far:

$ cargo run
 Compiling functions v0.1.0 (file:///projects/functions)
error: expected expression, found `let` statement
 --> src/main.rs:2:14
 |
2 | let x = (let y = 6);
 | ^^^
 |
 = note: only supported directly in conditions of `if` and `while` expressions

The fact that conditions must explicitly be booleans:

$ cargo run
 Compiling branches v0.1.0 (file:///projects/branches)
error[E0308]: mismatched types
 --> src/main.rs:4:8
 |
4 | if number {
 | ^^^^^^ expected `bool`, found integer

For more information about this error, try `rustc --explain E0308`.
error: could not compile `branches` (bin "branches") due to 1 previous error

One-line assignment using if, but enforcing that the result of each arm must be of the same type:

let number = if condition { 5 } else { 6 };

The loop keyword and loop labels (probably used sparingly as there are while and for keywords as well):

fn main() {
 loop {
 println!("again!");
 }
}

Looking forward to learning more about Rust!

Visualising Belgian roads

Tue, 20 Aug 2024 00:00:00 +0000

When exploring Overture data earlier this month, I stumbled upon “All Streets” by Ben Fry. I loved the visualisation and wondered how I could reproduce it for other countries. Looking at Ben Fry’s writing archive, he briefly discussed his work in a blog post from 2008:

Nothing particularly genius about this piece—it’s mostly just a matter of collecting the data and creating the image. But it’s one of those cases where even in a (relatively) raw format, the data itself is quite striking.

Ben Fry

“Collecting the data and creating the image”: as easy as drawing an owl. Luckily, the rest of the blog post reveals a bit more of the process — but sadly doesn’t really help me in reproducing it for other countries.

After a bit of Googling, I found great work by Andrei Kashcha: city-roads, which allows you to render roads in any area using data from OpenStreetMap, which I used for the image at the top of this note. I plan to dive into the open-sourced code of city-roads, and I might try to make something similar from scratch using Overture (just to learn how it works, as inspired by a comment on lobste.rs):

You should make a toy version of every major tool you use, if you can find the time and energy to do so. It’s a great way to understand the constraints that go into the tools you use and the limits of their capabilities.

lobste.rs comment

I want to do something similar as city-roads, but expand it by combining it with Overture Places data. Possibilities for visualisation include heatmaps for distance from road segments to specific types of places (like the nearest train station or bus stop, for example), or create a Chronotrains-style alternative on a Belgian map, using open data from NMBS, De Lijn, and TEC.

Displaying tags in Hugo

Mon, 19 Aug 2024 00:00:00 +0000

As I plan to write about a variety of different topics on this website, I decided to add tags to the properties of all my posts and notes so far:

---
title: "Displaying tags in Hugo"
date: "2024-08-19"
tags: ["hugo"]
⋯
---

Tags were already available in the Hugo theme that I am currently using, but I did not like the way they were displayed. I edited partials/post-entry.html so there’s a loop over every tag of a post, displaying them as a comma-delimited list next to the post or note’s title:

<div class="post-line">
 {{ $dateFormat := "2 Jan 2006"}}
 {{ with .Site.Params.listDateFormat }}
 {{ $dateFormat = .}}
 {{ end }}

 <p class="line-date">{{ .Date.Format $dateFormat }} </p>

 <span>
 <a href="{{ .RelPermalink }}">
 {{- .Title -}}
 </a>

 {{ if .Site.Params.listSummaries }}
 <span class="post-tags">
 {{- range $index, $tag := .Params.tags -}}
 {{ if $index }}, {{ end }}<a href="{{ printf "/tags/%s/" $tag | urlize }}">#{{ $tag }}</a>
 {{- end -}}
 </span>

 <div class="line-summary"> {{ .Description | markdownify }} </div>
 {{ end }}
 </span>
</div>

.post-tags {
 font-size: 0.7em;
 opacity: 50%;
 vertical-align: baseline;
}

Clicking on the link of the tag brings you to a list of all posts or note’s with the same tag.

Using data sources in Hugo

Sun, 18 Aug 2024 00:00:00 +0000

Until now, I have been storing different variables as site parameters in my Hugo website’s hugo.toml file. A regularly triggered GitHub Actions workflow updates these variables by using various APIs to check for any changes. If changes are detected, the workflow updates the hugo.toml file with the new values (replacing the old ones using sed) and pushes the changes to my website’s GitHub repository. There, another GitHub Workflow builds my Hugo website and deploys it to GitHub Pages.

As I start tracking more variables than originally planned, it makes sense to store them the proper way: using data sources in Hugo. My plan is to keep a yaml file in my Hugo website’s data directory, which will store all variables that I want to display on my website. The rest of the workflow should remain unchanged, but this approach should separate these variables from actual site parameters.

running_distance: 720851.2999999999
running_cp: 3.5694862545565007
writing_streak: 12
best_writing_streak: 12
best_writing_streak_date: "2024-08-19"
tr_value: 6150.468645723648
tr_rank: b

I’m planning to write a decent and complete post explaining which APIs I use, the variables that I store (and plan to store in the future), and how these GitHub Workflows work. I currently keep track of the following variables:

Variable	Value
Longest writing streak	20 days \| Achieved on 27 Aug 2024
Current writing streak	0 days \| Updated on 12 Aug 2026
Running Critical Power	3.490 W/kg
Distance ran this year	470.5 km
TETR.IO’s TETRA LEAGUE	10716.96 TR ()

Displaying TETR.IO TR on website

Sat, 17 Aug 2024 00:00:00 +0000

TETR.IO’s started its new season yesterday. A couple of days ago, I planned to scrape my TETR.IO profile to display the change in my TR over time. The game’s patch notes include the feature that I was missing in the game’s previous season, however:

You can now see your TR change over time on the TETRA CHANNEL site.

TETR.IO Patch Notes

Additionally, if I still want to include my TR over time on my website, scraping isn’t necessary at all: the patch notes mention that the game provides an API. Simply using curl and jq gives me all the data I need:

curl https://ch.tetr.io/api/users/geuze/summaries/league | jq '.data | .tr'

I store this data as a site parameter in my hugo.toml file:

TETRIO_TR = 3090.2731291544947
TETRIO_RANK = "c+"

Now, I can use it in a Hugo shortcode:

<span>
	{{ printf "%.2f" .Site.Params.TETRIO_TR }} TR
	(<img class="tr_rank" 
		 src="https://tetr.io/res/league-ranks/{{ .Site.Params.TETRIO_RANK }}.png" />)
</span>

Which looks like this: 10716.96 TR (). To keep this value up-to-date, I add it to a GitHub Actions workflow that uses the game’s API to retrieve my TR and rank, and replaces the current value in the hugo.toml file using sed.

Visualising multi-sport events

Fri, 16 Aug 2024 00:00:00 +0000

Today, I participated in my first duathlon — 5 kilometres of running, followed by 32 kilometres of cycling, and finishing with another 5 kilometres of running:


Run #1	00:24:54
⋮ Transition #1	00:01:06
Bike	01:21:49
⋮ Transition #2	00:01:53
Run #2	00:23:44
⋮ Total	02:13:28

As I do with my separate running or cycling sessions, I record my performance (HR, power output, pace) and upload it to Strava and intervals.icu. It surprises me, however, that there is no clear way of combining these separate activities into one multi-sport event. I assume there’s much to gain when looking at these activities in their context, rather than separately.

As a small side project, I could look into the best way of combining the different .fit-files into one well-visualised multi-sport event. It would surprise me if no one else has done this before, but this idea gives me the best opportunity to dive into how these files are structured (as I am totally unfamiliar with how they work) and how I could combine them in one multi-sport visualisation.

I could use the data from different .fit-files to display some maps of the course and compare the first and last running segments of a duathlon, for example. I could show a well-visualised race timeline with interesting annotations like weather conditions and elevation profiles, and the location of the highest achieved heart rate and power output. For friend groups participating in the same event, I could show different participants’ data on the same visualisation, to easily let them compare their performance.

Target power to predicted race time

Thu, 15 Aug 2024 00:00:00 +0000

For both training and racing, I use a Stryd pod: a running power meter that allows me to base my running on my power output, instead of looking at my pace or heart rate (which are both influenced by elevation). The Stryd running pod calculates my Critical Power (CP), which is similar to Functional Threshold Power (FTP). My CP is currently 3.490 W/kgData from intervals.icu (and was 3.75 W/kg at the time of writing) — something that I want to keep improving as I’m planning to run my second marathon mid-September.

Predicting finish time with Stryd

The Stryd app has two handy features: one predicts race calculations for a set of fixed distances (1 mile, 5k, 10k, HM, and full marathon) and the other returns a target power + predicted race time based on CP for a given GPX. I was wondering how they calculate these predicted race times, and stumbled upon a blog post by Stryd themselves.

Their simplified formula — specifically meant for a flat course with no air resistance — is rather straightforward: $T = \frac{1.04 * d}{TP / m}$, where $T$ is the predicted race time in seconds, $TP$ is the target power for the race in watts, $m$ is the runner’s body weight in kilograms, and $d$ is the distance in meters:

from datetime import timedelta

def predicted_time(d, TP, m):
	seconds = 1.04 * d / (TP / m)
	return str(timedelta(seconds=seconds))

“In Flanders Fields” marathon prediction

For my next marathon, I am trying to go for a time under 4 hours. As my target power is ~250 W, I would finish my next marathon in 3 hours and 51 minutes, in the best case scenarioFlat course, but weather can have a big impact.

d = 42195 # marathon distance in m
TP = 250 # target power in W
m = 79 # weight in kg

>>> predicted_time(d, TP, m)
'3:51:06.964800'

This is pretty similar to the Stryd app’s estimated time based on the provided GPX of the marathon course, which predicts I will run the marathon in around 3 hours and 50 minutes — if I can maintain an average power output of ~252 W. Using the simplified formula directly, however, gives me a bit more flexibility to see what my predicted time would be if I increase my target power or lose weight, but retain my CP:

weight = range(80, 74, -1)
time = [predicted_time(d, TP, w) for w in weight]

>>> for w, t in zip(weight, time):
... print(f"{w} kg — {t}")

'80 kg — 3:54:02.496000'
'79 kg — 3:51:06.964800'
'78 kg — 3:48:11.433600'
'77 kg — 3:45:15.902400'
'76 kg — 3:42:20.371200'
'75 kg — 3:39:24.840000'

Stryd also just released a blog post about Clayton Young and his preparation for the Olympic Marathon in Paris, in which he finished 9th: Stryd predicted his finish time at 2:09:54, and he ultimately ran 2:08:44.

Pomodoro timer in a terminal

Wed, 14 Aug 2024 00:00:00 +0000

I usually use the Pomodoro Technique to schedule my work: intervals of 25 minutes of work, followed by a 5 minute break. Traditionally, I used pomofocus.io as a timer in my browser that alerts me when an interval is over. As I often have a terminal emulator opened somewhere — and as I wanted to keep track of the number of finished pomodori in Obsidian, it made more sense to create my own shell script that:

Starts a 25 minute timer, alerting me when time runs out
Gives me the option to take a break, starting a 5 minute timer
Keeps alternating between work and break intervals until I stop working
Keeps track of finished pomodori for the day in my Obsidian work vault

Displaying a timer

Below, we define a function that takes a label (e.g., working or taking a break) and the amount of seconds the countdown should take. We start by moving the text cursor to the upper left corner using echo -ne '\033[2A'StackOverflow, and then, as long as our variable time_left is larger than -1:

Show the label in bold, using \033[1m• $label\033[0m
Format remaining time using $(date -d@$time_left -u +%M:%S), so that it shows as remaining time in minutes and seconds, instead of just seconds
Use \033[0K to delete text to the end of the line, and \r to move to the beginning of the current lineStackOverflow
Wait one second using sleep, and substract 1 from the time_left variable

display_timer() {
 local label=$1
 local time_left=$2
 echo -ne '\033[2A'
 while [ $time_left -gt -1 ]; do
 echo -ne "\033[1m• $label\033[0m — $(date -d@$time_left -u +%M:%S)\033[0K\r"
 sleep 1
 ((time_left--))
 done
}

We can start and display our work timer and break timer as shown below. We start a 25 minute timer (displayed as • Work left — MM:SS), immediately followed by a 5 minute timer (displayed as • Break left — MM:SS):

WORK_TIME=$((60 * 25)) # Work time (25 minutes)
BREAK_TIME=$((60 * 5)) # Break time (5 minutes)

display_timer "Work left" $WORK_TIME
display_timer "Break left" $BREAK_TIME

Keeping track of 🍅’s

In my Obsidian journal, I have daily notes where I keep track of work done, to-dos, and ideas. I recently added a pomodori field to the properties of my Markdown files, where I keep track of how many work intervals I have done that day:

---
date: 2024-08-14
type: research journal
pomodori: 🍅🍅🍅🍅
---

I want this script to automatically add a new interval to this property, by first defining the path to today’s note and then appending a 🍅 to the end of the line of that property using sed. As the property will always be on the fourth line of my notes, we can write:

WORK_TIME=$((60 * 25)) # Work time (25 minutes)
BREAK_TIME=$((60 * 5)) # Break time (5 minutes)
JOURNAL="(⋯)/work/Research Journal"
ENTRY="$JOURNAL/$(date +%Y-%m-%d) Research Journal.md"

display_timer "Work left" $WORK_TIME
sed -i '4s/$/🍅/' "$ENTRY"
display_timer "Break left" $BREAK_TIME

User input to keep going

Using zenity, we can request user input to check if we want to switch from working to taking a break, and vice versa. We use exec "$(readlink -f "$0")"AskUbuntu to restart the script when switching from taking a break back to working:

if zenity --question --title="Pomodoro" --text="Time for a break?"; then
 display_timer "Break left" $BREAK_TIME
 if zenity --question --title="Pomodoro" --text="Time to work?"; then
 exec "$(readlink -f "$0")" # Restart the script for another Pomodoro cycle
 fi
fi

Putting it all together

#!/usr/bin/env bash
clear

# Paths and file names
JOURNAL="(⋯)/work/Research Journal"
ENTRY="$JOURNAL/$(date +%Y-%m-%d) Research Journal.md"

# Pomodoro timers (in seconds)
WORK_TIME=$((60 * 25)) # Work time (25 minutes)
BREAK_TIME=$((60 * 5)) # Break time (5 minutes)

# Function to display the timer
display_timer() {
 local label=$1
 local time_left=$2
 echo -ne '\033[2A'
 while [ $time_left -gt -1 ]; do
 echo -ne "\033[1m• $label\033[0m — $(date -d@$time_left -u +%M:%S)\033[0K\r"
 sleep 1
 ((time_left--))
 done
}

# Start the work session
display_timer "Work left" $WORK_TIME

# Add a 🍅 emoji to the fourth line of the journal entry
sed -i '4s/$/🍅/' "$ENTRY"

# Prompt to take a break
if zenity --question --title="Pomodoro" --text="Time for a break?"; then
 display_timer "Break left" $BREAK_TIME
 if zenity --question --title="Pomodoro" --text="Time to work?"; then
 exec "$(readlink -f "$0")" # Restart the script for another Pomodoro cycle
 fi
fi

I have added clear to the beginning of the script to clear the terminal screen before the script runs, just to ensure a clean display of the timer:

Demonstration of Pomodoro timer

Scraping TETR.IO Tetra League standings

Tue, 13 Aug 2024 00:00:00 +0000

Since 2022, I’ve regularly enjoyed playing a competitive Tetris variant named TETR.IO. It’s been a while since I have played the game, but when visiting its website I read that the game just moved from it’s Alpha stage to Beta, and that the second season of TETR.IO’s TETRA LEAGUE (the game’s competitive matchmaking mode) starts on the 16th of August.

Something that I missed in the Alpha version of the game was the history of progression: I would have loved to see a plot that shows my rank and the game’s equivalent of my ELO score over time. With the second season of the game fast approaching, it is the perfect time to create a cron job for a script that scrapes my TETR.IO profile and writes the current score to a file. The data itself could then be used to create my plot, and maybe showcase it on my website.

Inspecting the HTML of the TETR.IO profile pages, the score is normally displayed in the following div. Because it is the off-season, however, there is no score to display:

<div class="card categorical" id="usercard_league" style="--color: #375433">
	<h1><img src="/res/league.svg">TETRA LEAGUE</h1>
	<h6 id="user_league_np">Off-season</h6>
	<div id="user_leagueset">
		<div id="user_leaguestateset"></div>
		<div id="user_leaguestandingset" class="standingset">abc</div>
	</div>
	⋯
</div>

Initially, I planned to simply scrape this using:

curl -s "https://ch.tetr.io/u/geuze" | grep -oP '(?<=<h6 id="user_league_np">).*?(?=</h6>)'

A simple curl and grep of the webpage won’t do, however, as the information seems to be loaded using Javascript. Before the beginning of this new season, I’ll look into scraping Javascript-rendered web pages using Python.

Partially ordered sets with cycles

Mon, 12 Aug 2024 00:00:00 +0000

A partially ordered set — or poset — $P = (A, \preceq)$ is a pair consisting of a set $A$ and a partial order $\preceq$ on $A$. This partial order is a binary relation on a set $A$ such that — for all $x, y, z \in A$ — it is:

reflexive ($x \preceq x$),
antisymmetric ($x \preceq y$ and $y \preceq x$ imply $x = y$),
and transitive ($x \preceq y$ and $y \preceq z$ imply $x \preceq z$).

For two elements $x, y \in P$, we say that $y$ covers $x$ if $x \prec y$ and there is no element $z \in P$ such that $x \prec z \prec y$. We write this as $x \lessdot y$.

I assumed that every poset could be described as a directed acylic graph through the cover relation between its elements. This intuitively makes sense to me, as cycles (e.g., $x \lessdot y$ and $y \lessdot x$) would be inconsistent with posets due to its antisymmetry, but $x \neq y$ given the definition of the cover relation. Thinking about cycles in orders reminded me of this great example on the link between directed acyclic graphs and genealogy, from the Handbook of Graph Theory by Jonathan L. Gross:

A “family tree” is a digraph, where the orientation is traditionally given not by arrows but by the direction down for later generations. Despite the name, a family tree is usually not a tree, since people commonly marry distant cousins, knowingly or unknowingly. However, it is always a DAG, because if there were a cycle, everyone on it would be older than everyone else on the cycle.

Handbook of Graph Theory by Jonathan L. Gross

I did, however, stumble upon a paper titled “Cyclic Ordering through Partial Orders” by Stefan Haar. Briefly scanning through the paper shows some interesting concepts, so I am certainly curious to learn more about this!

Describing activities with sparklines

Sun, 11 Aug 2024 00:00:00 +0000

Yesterday, I learned about sparklines — “data-dense graphics with typographic resolution that is shown inline and in-context with relevant text.”

When thinking about how I could use sparklines in an original way within my personal website, I thought they would be interesting to convey properties of sports activities. One could add the elevation profile of a course 🏔️ {22,17,27,37,32,27,37,32,27,22,32,27,37,32,27,37,47,42,52,47} , or a heart rate curve ❤ {22,17,27,37,32,27,37,32,27,22,32,27,37,32,27,37,47,42,52,47} and power output ⚡︎ {22,17,27,37,32,27,37,32,27,22,32,27,37,32,27,37,47,42,52,47} over time, as a sparkline within a blog post about a race.

With the help of Sparks, which is a typeface for creating sparklines in text, this was implemented quickly:

{{< sparkline class="bar-medium" color="darkgreen">}} 🏔️ {22,17,27,37,32,27,37,32,27,22,32,27,37,32,27,37,47,42,52,47} {{</ sparkline >}}

{{< sparkline class="dotline-medium" color="crimson">}} ❤ {22,17,27,37,32,27,37,32,27,22,32,27,37,32,27,37,47,42,52,47} {{</ sparkline >}}

{{< sparkline class="dotline-medium" color="purple" >}} ⚡︎ {22,17,27,37,32,27,37,32,27,22,32,27,37,32,27,37,47,42,52,47} {{</ sparkline >}}

In the future, I’d like to use actual data from intervals.icu for these sparklines.

Exploring breakfast spots with Overture Places

Sat, 10 Aug 2024 00:00:00 +0000

I recently learned about Overture places data through a blog post by Drew Breunig. This dataset “contains more than 53 million point representations of real-world entities: schools, businesses, (⋯), mountain peaks, and much more"Places - Overture Maps Documentation. You can explore this data through the Overture Maps Explorer.

I regularly have breakfast with a friend, and we like exploring new breakfast places. The Overture dataset could provide us with new breakfast places in Brussels. For this, we need the bounding box around the area of Brussels in which we’re interested in. We can find this using bboxfinder.com. For this example, we take the bounding box around the Vijfhoek of Brussels.

Screenshot of the bounding box around the Vijfhoek of Brussels

Checking out some of the regular places we visit through the Maps Explorer shows that they have the "breakfast_and_brunch_restaurant" category in common. Using duckdb, we can download all places within our defined bounding box of 50.832939, 4.337368, 50.858410, 4.369812. The code below is based on an example from the Overture Maps website:

-- Load necessary modules
LOAD spatial;
LOAD httpfs;

-- Set the AWS S3 region
SET s3_region = 'us-west-2';

-- Copy the filtered data from S3 to a local Parquet file
COPY (
 SELECT
 id,
 names,
 categories,
 websites,
 addresses,
 ST_GeomFromWKB(geometry)
 FROM 
 read_parquet('s3://overturemaps-us-west-2/release/2024-07-22.0/theme=places/*/*')
 WHERE
 -- Filter by bounding box (xmin and ymin)
 bbox.xmin BETWEEN 4.337368 AND 4.369812 AND
 bbox.ymin BETWEEN 50.832939 AND 50.858410
) TO 'vijfhoek.parquet';

Next, we can query all the places where the primary category is "breakfast_and_brunch_restaurant":

SELECT 
 names.primary AS name, 
 addresses[1].freeform AS address, 
 websites[1] AS website
 FROM 
 'vijfhoek.parquet'
 WHERE 
 categories.primary = 'breakfast_and_brunch_restaurant';

┌──────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ name │ address │ website │
│ varchar │ varchar │ varchar │
├──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Flower food │ Rue Blaes 264 │ │
│ Le Père Tranquille │ Rue des Renards 4 │ │
│ Brussels Waffle Workshop │ Rue des Foulons 30 │ http://www.waffleworkshop.com │
│ Woodpecker 20 │ Rue Jourdan 20 │ │
│ EatClub Brussels │ Rue de Stassart 18 │ │
│ Café Boudin │ Rue Ravenstein 20 │ https://cafeboudin.be │
│ bloom_and_brunch_by_sabri │ Rue de la Paix 7 │ │
│ Lastra │ Rue du Conseil 24 │ │
│ Le Coin Des Saveurs │ Boulevard de l'Abattoir 19 │ │
│ Kandinsky │ Chaussée de Gand 37 │ https://kandinsky-family.be/ │
│ Restaurant Baurade │ Boulevard Anspach 153 │ │
│ Kafei Dansaert │ Rue Antoine Dansaert 57 │ https://kafei.be │
│ Le Odin Royal │ Oude Graanmarkt 2 │ │
│ Movenpick │ │ │
│ Woodpecker St. Cath. │ Quai au Bois à Brûler 27 │ │
│ Restaurant Al Andalus │ Rue de Ribaucourt 4 │ │
│ Gaufres and Waffles │ Chemin du Croquet 1 │ http://www.gaufresandwaffles.com │
│ Oats Day Long │ Europakruispunt 3 │ https://www.oatsdaylong.com │
│ Belrose cake │ Boulevard Émile Jacqmain 3 │ https://www.belrosecake.com/ │
│ Spread the moon │ Rue du Gentilhomme 13 │ https://www.instagram.com/spreadthemoon/ │
│ Longitude Nord-Sud │ Quai aux Pierres de Taille 14 │ http://longitude.blog4ever.com/ │
├──────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ 21 rows 3 columns │
└──────────────────────────────────────────────────────────────────────────────────────────────────────┘

This is a good start, but lots of places we already visited are still missing from this list. We can query for all places where 'breakfast_and_brunch_restaurant' is listed under alternate categories using OR 'breakfast_and_brunch_restaurant' = ANY (categories.alternate);.

I’m planning to pick this idea up later, and check if I can find a way to include more breakfast and brunch spots. In a next step, I could attempt to scrape the provided websites to check menus for our usual order.

intervals.icu, GitHub Actions, and Hugo shortcodes

Fri, 09 Aug 2024 00:00:00 +0000

Intervals.icu API

I sync all my Strava activities to intervals.icu, which calculates my training load, fitness score, and my weekly ramp rate. To test out this platform’s API, I wanted to calculate the distance that I ran or cycled given a custom date range, as this is something that I don’t immediately see inside intervals.icu itself. GET /api/v1/athlete/{id}/activities returns a list of activities between the parameters oldest and newest:

BASE_URL="https://intervals.icu/api/v1/athlete"
OLDEST="2024-01-01"
NEWEST="2024-12-31"
curl -s -u API_KEY:$API_KEY \
 "$BASE_URL/$INTERVALS_UID/activities?oldest=$OLDEST&newest=$NEWEST"

As we can’t select specific activity types through the API (we’re only interested in runs for this example, but this list contains all types of activities), I made use of jq — “a lightweight and flexible command-line JSON processor” — to select the name, start time, and distance of running activities only:

BASE_URL="https://intervals.icu/api/v1/athlete"
OLDEST="2024-01-01"
NEWEST="2024-12-31"
curl -s -u API_KEY:$API_KEY \
 "$BASE_URL/$INTERVALS_UID/activities?oldest=$OLDEST&newest=$NEWEST" | \
jq '[.[] | select(.type == "Run") | {name, start_date_local, distance}]'

[
 ⋯,
 {
 "name": "Night Run",
 "start_date_local": "2024-08-06T21:44:54",
 "distance": 6699
 },
 {
 "name": "Afternoon Run",
 "start_date_local": "2024-08-06T16:06:24",
 "distance": 3400.5
 },
 {
 "name": "Evening Run",
 "start_date_local": "2024-08-04T20:58:21",
 "distance": 4928
 },
 ⋯
]

This shows that our filter for running activities works. To now sum the distance of all our activities, we just need to select the distance of every activity and map add over it. This returns the total distance in meters.

BASE_URL="https://intervals.icu/api/v1/athlete"
OLDEST="2024-01-01"
NEWEST="2024-12-31"
curl -s -u API_KEY:$API_KEY \
 "$BASE_URL/$INTERVALS_UID/activities?oldest=$OLDEST&newest=$NEWEST" | \
jq 'map(select(.type == "Run") | .distance) | add'

701168.5

GitHub Actions for Hugo shortcode

As I wanted to do something with this distance, I made a GitHub Actions workflow that calculates the distance every day at midnight and uses sed to replace the Hugo variable .Site.Params.distanceRun. Using this variable, I created a Hugo shortcode {{< distance_run >}} that returns the distance run in kilometers:

{{- $distanceMeters := .Site.Params.distanceRun -}}
{{- $distanceKilometers := div $distanceMeters 1000 -}}
{{ printf "%.1f" $distanceKilometers }} km

Using {{< distance_run >}} in a Hugo content file returns: 470.5 km. I can put this value in a tooltip with an explanation of the value, which is displayed as 470.5 kmTotal distance since 1st of Jan.:

{{< tooltip >}}{{< distance_run >}} | Total distance since 1st of Jan.{{</ tooltip >}}

Tooltip on mouse hover with Hugo

Thu, 08 Aug 2024 00:00:00 +0000

I added custom tooltipsJust like this one! to this website, as I want to provide extra information to specific text in my posts and notes. This can be done using regular shortcodes, which are simple snippets inside a content file that Hugo will render using a HTML template:

{{< tooltip outer_text="tooltips" tooltip_text="Just like this one!" >}}

This works fine if we just provide simple, static text. One of the criteria for my tooltip shortcode, however, is that I should be able to put other shortcodes inside them, as I have a couple of shortcodes that make use of variables that are updated automatically using GitHub Actions. One example of this is the number of kilometers I ran this year (470.5 km, currently)Updated using GitHub actions + intervals.icu , which I calculate using the intervals.icu API.

As far as I know, there is no standard way of adding these tooltips in Hugo. I created them myself by using paired shortcodes and using .Inner to return the content between the opening and closing shortcode tags, which allows us to do the following:

{{< tooltip >}} {{< inner_shortcode >}} {{</ tooltip >}}

Now, we can get the contents of the inner shortcode through .Inner:

<span class="tooltip">
 {{- .Inner | safeHTML -}}
 <span class="tooltiptext">{{- .Inner | safeHTML -}}</span>
</span>

We want to put some text as the outer text of the tooltip, and other text in the tooltip box itself, however. Through a hack provided by someone on the Hugo forum, we can split up .Inner by providing a custom delimiter. For my tooltips, I divide the contents of the paired shortcodes in the outer text and the tooltip text using |:

{{- $children := split .Inner " | " -}}
<span class="tooltip">
 {{- index $children 0 | safeHTML -}}
 <span class="tooltiptext">{{- index $children 1 | safeHTML -}}</span>
</span>

This way, I can define tooltips like this, for example: 470.5 kmUpdated using GitHub actions + intervals.icu

{{< tooltip >}} {{< distance_run >}} km | See intervals.icu {{</ tooltip >}}

Daily notes with GitHub Actions

Wed, 07 Aug 2024 00:00:00 +0000

I am using GitHub Actions to automatically create new daily notes on this website. Every day at midnight, a GitHub Action checks the content/notes folder for a file titled %Y-%m-%d.md (matching the current date). If this file does not yet exist, a new %Y-%m-%d.md file with a standard Hugo front matter — filled in with the current date and a placeholder title — is created.

To keep track of my daily writing streak, I also need to check if the previous day’s note was modified, and update my streak accordingly. I do this using a combination of git, grep, and sed.

`git log`

The git log command shows the commit log, which is used to check if there have been any other commits involving the previous day’s note, outside of its creation by GitHub Actions. If this is the case, there should be more than one commit involving this file, and I can thus assume that I have written something. Based on this output, the existing streak — which is stored in the hugo.toml configuration file — is updated.

PREV_DATE=$(date -d "${{ env.DATE }} -1 day" +'%Y-%m-%d')
if [[ $(git log --name-only --since="24 hours ago" \
	 --pretty=format: -- \
	 content/notes/$PREV_DATE.md | \
	 grep -c content/notes/$PREV_DATE.md) -gt 1 ]]; then
 echo "MODIFIED=true" >> $GITHUB_ENV
else 
 echo "MODIFIED=false" >> $GITHUB_ENV
fi

For this to work, however, we need to pass the HEAD’s SHA as a ref to the checkout:

- name: Checkout repository
 uses: actions/checkout@v3
 with:
 ref: ${{ github.event.pull_request.head.sha }}
 fetch-depth: 0

`grep`

To find the current value of the writing streak in the configuration file — and the corresponding date that matches this streak — I use grep. These outputs are stored in environment variables.

CURRENT_STREAK=$(grep -Po '(?<=CURRENT_WRITING_STREAK = )\d+' $TOML_FILE)
CURRENT_STREAK_DATE=$(grep -Po '(?<=CURRENT_WRITING_STREAK_DATE = )\d{4}-\d{2}-\d{2}' $TOML_FILE)

`sed`

The streak value in the configuration file is updated using sed, based on our earlier result from the git log command. We replace the current streak value with $((CURRENT_STREAK + 1)) if there is more than one commit involving the previous day’s note, or replace the streak with 0 otherwise. This new streak is stored in the $NEW_STREAK environment variable. We also do the same to keep track of the best writing streak to date.

sed -i "s/CURRENT_WRITING_STREAK = $CURRENT_STREAK/CURRENT_WRITING_STREAK = $NEW_STREAK/" $TOML_FILE
sed -i "s/CURRENT_WRITING_STREAK_DATE = $CURRENT_STREAK_DATE/CURRENT_WRITING_STREAK_DATE = ${{ env.DATE }}/" $TOML_FILE

The above sed command kept throwing an error, however: sed: -e (...) unterminated 's' command. I tried everything from attempting to escape underscores, to adding -- to sed (as maybe the hyphens in the dates could be confused with a sed option that made something break), as I first thought that these characters were possibly causing these issues.

In the end, I realised that the issue was caused due to grep matching multiple lines in the hugo.toml — which also included newlines and spaces. I did not realise that newlines would be included in the environment variables, but making sure the regex match was only limited to the CURRENT_WRITING_STREAK variable in the configuration file and trimming any newlines, resolved the issue:

CURRENT_STREAK=$(echo "$CURRENT_STREAK" | tr -d '\n' | tr -d ' ')
CURRENT_STREAK_DATE=$(echo "$CURRENT_STREAK_DATE" | tr -d '\n' | tr -d ' ')

By double-checking the used regex and removing unwanted characters, the sed command could successfully update the values in the hugo.toml file without errors:

Writing streak of 0 daysUpdated on 12 Aug 2026
Best writing streak to date is 20 daysAchieved on 27 Aug 2024

Sharing daily notes

Wed, 07 Aug 2024 00:00:00 +0000

For my work, I picked up the habit of transferring my daily notes and insights from my physical notebooks to an Obsidian vault. Thanks to using Obsidian and git, I have a locally-stored, version-controlled, and easily searchable overview of my work that I can reference when preparing reports and presentations.

Equivalently, I miss a digital space where I can transfer personal insights, store interesting links and quotes, and write down notes unrelated to my work. Topics of these notes could include everything from programming, research, and literature, to physical exercise and ceramics.

Why make these notes public?

I could start a new Obsidian vault for these notes, but I think they should be someplace public as I want to “work with my garage door up.” I also want to keep up a daily writing streak: if I would just create another private Obsidian vault, it’ll be too easy to convince myself to skip writing to do other things instead. Public challenges like #100DaysToOffload seem appealing to participate in, as well.

We can write whatever we want on personal websites, without being vulnerable to tech companies and their social media algorithms + keeping ownership of content. I am currently hosting this website through GitHub Pages, but I can pack up my content and move it elsewhere whenever I want. Read more about this in this great article by Matthias Ott:

If you decide to publish your work on a platform like Medium, you’re giving away control over it. What if Medium suddenly decided to extend the already existing paywall to all articles? There’s not much you could do about it. Simply because you don’t own your content anymore.

Matthias Ott

But maybe the most compelling reason why a personal website (⋯) is incredibly valuable is this: community. Since the days of “guest books”, personal websites have been a place to receive feedback and discuss ideas and concepts with others. (⋯) Now imagine, for a moment, an environment where a decentralized fabric of connected personal sites allows everyone to publish their own content but also enables each individual to engage in an open discussion – answering, challenging, and acknowledging the ideas of others through this universe of personal sites.

Matthias Ott

To allow for this engagement, I’m planning to add Webmentions to this website soon.

By trying to write and publish a quick note every day, I’ll hopefully improving my writing. Not all of these notes will be polished, which is why I plan to keep them separate from my regular posts. It is okay to create unpolished things — a quote from Robin Rendle:

I feel like I’m free now to write absolute junk. And this is vital for any creative thing! You have to feel like your mistakes won’t be judged, that there’s no one watching you write shameful, embarrassing things. Because if you have all that pressure of trying to write the next great American novel then I reckon you’re destined to fail.

Robin Rendle

Hopefully, these notes give me inspiration to write some longer posts, where I can combine ideas and flesh out concepts of multiple related notes.

Writing streak

I currently have a writing streak of 0 daysUpdated on 12 Aug 2026, and my largest streak to date is 20 daysAchieved on 27 Aug 2024. You can find these daily notes here. To find the note for a corresponding date, go to /notes/YYYY-MM-DD. Each date since 2024-08-07 should have an existing entry, even if I did not write anything that day.

Obsidian.md, Espanso, and llm

Tue, 23 Jul 2024 00:00:00 +0000

I switched from Notion to Obsidian.md to keep track of meeting notes, to store highlights from scientific papers (using a Zotero plugin), and to maintain a knowledge base for my PhD research.

Obsidian and `llm`

To summarise or rephrase parts of notes and paper highlights, I often use OpenAI models through llm, a CLI utility that provides an easy way to interact with the OpenAI API. Traditionally, my workflow consisted of piping Markdown files from my Obsidian vault to llm, combining them with a system prompt that described the requested task.

cat papers/paper.md | llm -m 'gpt-4o' 
-s "Summarise the methodology described in the provided paper notes"

I then copied the model output from the terminal and pasted it in a custom callout that I specifically defined for LLM-generated content (as to keep it separate from my own writing), giving me a first rough draft that I can edit later.

.callout[data-callout="llm"] {
 --callout-color: 0, 100, 100;
 --callout-icon: bot;
}

A main disadvantage, however, is that I kept switching between Obsidian and the terminal to copy and paste model output. Another disadvantage was that I had no control over which lines from the relevant file to send to OpenAI when piping the full contents of a file to llmI am sure that there are plenty of ways to cat specific lines only given a file, but I assume for this you need to provide specific line numbers, making the process a bit more complex..

Obsidian and Espanso

Outside of tagging LLM output, I also use custom Obsidian callouts to highlight definitions, examples, and quotes from papers and textbooks. For this, I started using Espanso — a text expander that automatically replaces specific keywords with predefined text. This helps me avoid repeatedly typing callout boilerplate:

- trigger: ":def"
 replace: "> [!Definition] ⋯\n> ⋯"

- trigger: ":ex"
 replace: "> [!Example]- ⋯ \n> ⋯"

Every time I type :def — inside or outside of Obsidian — it replaces this keyword with the boilerplate that is needed to define a Definition callout in Obsidian. Other custom triggers include arrows (←, →), dots (⋯, ⋮), and dashes (—) that I use in notes, and abbreviations for commonly used words (all preceded by the : character, which I have reserved for Espanso keywords only).

Regex Triggers, Arguments, and Shell Commands

Regex triggers are an advanced alternative to regular triggers in Espanso. Instead of the static triggers shown above, these dynamic triggers provide flexibility through regex syntax and the use of named groups to pass arguments to expansions. The following two examples are taken from the Espanso documentation:

 - regex: ":greet\\((?P<person>.*)\\)"
 replace: "Hi {{person}}!"

We can also execute scripts and shell commands through Espanso:

 - regex: "=sum\\((?P<num1>.*?),(?P<num2>.*?)\\)"
 replace: "{{result}}"
 vars:
 - name: result
 type: shell
 params:
 cmd: "expr $ESPANSO_NUM1 + $ESPANSO_NUM2"

Expanding LLM Output

My initial goal was to create a new trigger — :llm(⋯) — that allows for the following workflow: copy the text that we want the LLM to work with to the system’s clipboard (so that we don’t have to pipe a full file), type the trigger with the system prompt in the parenthesis, and expand this trigger to the custom LLM callout in Obsidian (with the system prompt as title, and the output of the LLM as the body of the callout). The trigger :llm("Summarise the provided text.") would expand to:

> [!llm]- "Summarise the provided text." 💬
⋯ (→ Summary of text currently in clipboard)

Limitation of Regex Triggers

Regex triggers have a limitation where the maximum length of a regex match, including captured named groups, are restricted to 30 charactersRegex triggers - Espanso Documentation. This constraint is intended to enhance performance, but makes it harder to write an expressive system prompt.

To circumvent this issue, we can use an LLM to first write us a full-sized prompt based on one or two provided keywords (as to keep our trigger under 30 characters) that we can then feed to our main LLM.

We could do this in one call — asking to expand the prompt and then use this prompt to perform the task at hand — but this would make it harder to extract the expanded prompt for the title for our custom callout. Another approach would be to provide the callout boilerplate to llm, asking it to return the filled-in callout; but as API calls are cheap, we’ll go with the most straightforward approach:

- regex: ":llm\\((?P<userprompt>.*?)\\)"
 replace: "> [!llm]- \"{{prompt}}\" 💬\n⋯\n\n"
 vars:
 - name: prompt
 type: shell
 params:
 cmd: "echo $ESPANSO_USERPROMPT | llm -s
 \ 'Expand provided keywords to a short but effective, 
 \ imperative one-sentence prompt in UK English.
 \ The final output when using this prompt should be 
 \ short and to the point.'

For now, if we type :llm(explain to 5yo), we get the following in return:

> [!llm]- "Explain the provided text in simple words 
that a 5-year-old can easily understand." 💬
⋯

Passing Full-Sized Prompt

Now that we can circumvent our character limit and have access to our full-sized prompt, we just have to provide it to another call of llm. We can access the full-sized prompt by using the variable name in all caps, preceded by $ESPANSO_. I pass the output through sed — replacing newlines with the > character — as to make sure that everything sticks together in the final callout.

- regex: ":llm\\((?P<userprompt>.*?)\\)"
 replace: "> [!llm]- \"{{prompt}}\" 💬\n{{output}}\n\n"
 vars:
 - name: prompt
 type: shell
 params:
 cmd: "echo $ESPANSO_USERPROMPT | llm -m 'gpt-4o' -s
 \ 'Expand provided keywords to a short but effective, 
 \ imperative one-sentence prompt in UK English.
 \ The final output when using this prompt should be 
 \ short and to the point.'
 - name: output
 type: shell
 params:
 cmd: "llm -m 'gpt-4o' -s
 \ \"$ESPANSO_PROMPT.\" | sed -z \"s/\\n/\\n> /g\""

Final Espanso Trigger

The last thing we have to do is accessing our clipboard. We do this by creating a new variable inside our trigger,

- name: clipboard
 type: clipboard

and piping it to our final call to llm. The final Espanso trigger looks as follows:

- regex: ":llm\\((?P<userprompt>.*?)\\)"
 replace: "> [!llm]- \"{{prompt}}\" 💬\n{{output}}\n\n"
 vars:
 - name: clipboard
 type: clipboard
 - name: prompt
 type: shell
 params:
 cmd: "echo $ESPANSO_USERPROMPT | llm -m 'gpt-4o' -s
 \ 'Expand provided keywords to a short but effective, 
 \ imperative one-sentence prompt in UK English.
 \ The final output when using this prompt should be 
 \ short and to the point.'
 - name: output
 type: shell
 params:
 cmd: "echo $ESPANSO_CLIPBOARD | llm -m 'gpt-4o' -s
 \ \"$ESPANSO_PROMPT.\" | sed -z \"s/\\n/\\n> /g\""

Demonstration

Demonstration of LLM expansion

Automating Boggle

Sat, 06 Apr 2024 00:00:00 +0000

Last week, I had a conversation with a good friend in the spirit of the “Cult of Done” (valuing action over perfection, boldly experimenting and learning through doing) and sharing our work. Based on this conversation, I decided to get my hands dirty and attempt to create a quick and fun project, constraining myself to a couple hours of work on a Saturday morning. Key here is not worrying about perfection, but getting something out there quicklyAs the goal of this experiment was to move fast, this writing is assisted using GPT-4 through the OpenAI API. Everything with a highlighted background was first written with the help of an LLM using my instructions and provided information, and rewritten and tweaked afterwards to fit my writing style..

Thrifted Boggle

My girlfriend and I enjoy playing board and card games. Recently, we’ve picked up a copy of Boggle in a thrift shop. Boggle is a word game where you try to find as many words as you can from 16 dice, each with a different letter, that are shaken in a covered tray and then set in a $4\times4$ grid. Players get three minutes to find words that are at least three letters long, and each letter in the word has to touch the letter before it in any direction. Players are not allowed to use the same letter dice more than once in a word. Once time’s up, players compare their lists and any word that appears on more than one list gets cancelled out, with players scoring points based on the length of their unique words.

Boggle board

Our second-hand copy of Boggle has a couple of problems, however (excluding the fact that one of the dice was replaced by a Scrabble block by the previous owner). Our main problem is that the hourglass provided with our copy is broken, so we have to set up a smartphone timer. Other problems include:

Counting the points afterwards takes up a lot of time, especially if we want to confirm that our words are actually on the board
Confirming if a found word is an actual word takes time as well (as I love making up some plausible sounding Dutch words, looking them up in the dictionary, and claiming that I was certain that it exists if it’s not there.)

Project Goals

As I want to create something that both helps solve the problems above, but I also want to get something done quickly, the goal for this project is threefold:

First, create a Telegram bot that can:
- Receive a picture of the shaken tray of dice
- Start a built-in timer to know when the round starts and ends
- Count our points by confirming that our words are on the board and that the words themselves exist based on a (Dutch) dictionary
Next, I want to share my work using this blog post (in the spirit of the Cult of Done)
Do all this in a couple of hours on a Saturday morning (instead of enjoying the nice Spring weather like a normal person)

Implementation

I started with a planning, trying to come up with the necessary sub-tasks:

Recognising labelled dice from a picture, which we need to reconstruct the grid
Create a data structure that contains the allowed list of words
Algorithm that traverses the grid, finding all valid words
Create a Telegram bot that uses the above to act as a “referee”
Write the text (that you’re currently reading) whilst doing all of the above

Recognising labelled dice

As I am not that familiar with computer vision techniques, and I wanted to move quickly, my first attempt was to use GPT-4V, or gpt-4-vision-preview, to take a picture of the Boggle grid as input and retrieve the letters from that picture as a continuous string. If this would work out, I could quickly move to the other steps of the project. I used the following prompt:

Given the image of a 4x4 grid of Boggle dice, 
convert the letters visible on the dice into a 
continuous 16-character string by reading from left to right,
top to bottom. 

For example, if the top row contains the letters 'A', 'B', 'C', 'D', 
and the second row contains letters 'E', 'F', 'G', and 'H', etc., 
your output should be 'ABCDEFGH...etc', up to the 16th letter.

Keep in mind that letters could be rotated!

This failed miserably, however. The received output based on the image above (cropped to only display the grid of dice) was:

The continuous 16-character string from the Boggle dice 
is "INRAWMNBUDOVSOLO".

Afterwards, I lost an hour trying out different things with OpenCV and tesseract, but this did not seem to work out. As I still wanted to move quickly, I skipped this step and will rely on user input for the grid.

Recognising valid words

The trie is an efficient data structure for storing large word lists. The data structure itself can be seen as a finite state machine that only accepts words used for its creation. Each node of the trie represents a prefix, with the root as the empty string. Transitions between nodes adds characters to the prefixes. This offers quick search capabilities, therefore making it suitable for our project.

As I’ve recently had to create an implementation of tries for other work, I simply reused this for this project. The idea behind the construction algorithm is very simple: starting from the initial state, we check every symbol in each word. We follow transitions to states if they already exist, and create new states and transitions if they don’t. At the end of each word, we mark the last state as the final state.

def trie(sequences): 
 """Constructs a trie from given collection of sequences.""" 
 fsm = Automaton() 
 for sequence in sequences: 
 current = fsm.initial 
 for symbol in sequence: 
 if symbol not in fsm.alphabet: 
 fsm.add_symbol(symbol) 
 if (next_state := fsm.follow(current, symbol)) is None: 
 next_state = fsm.add_state() 
 fsm.set_transition(current, next_state, symbol) 
 current = next_state 
 fsm.accepting.add(current) 
 return fsm

words = {"tree", "three", "trie", "tried"}
automaton = trie(words)

automaton.accept("tree") # True
automaton.accept("tried") # True
automton.accept("thr") # False

show(automaton)

Example of the trie for set (tree, trie, three, tried)

For the dictionary of valid words, I relied on the OpenTaal wordlist, containing over 400k Dutch words. We can shorten this list significantly by filtering out the ones that are impossible to find in Boggle:

Words that contain something other than the letters from a to z
Words shorter than 3 letters or longer than 10 letters (as we’re usually never finding any words longer than 6, 10 is very generous)

In the end, we end up with a large trie that accepts all words from our Dutch wordlist.

Searching for words in a grid

If we have a string representing the grid as input (e.g., "NTAWIRIRNBDAOASO"), we can find all possible words by converting our grid into a graph, allowing us to utilize graph traversal algorithms to systematically search for potential word formations. This graph representation simplifies the problem into a more navigable structure where each dice is a vertex, denoted by its character. Edges between vertices are formed based on a dice’s spatial relation to its neighbours, indicating possible transitions or movements in our word formation process.

To find all words in the grid, we can perform a depth-first traversal starting from each dice, exploring as far as possible along each branch before backtracking. It is an ideal choice for our case here since we are interested in finding all potential paths that form words. Starting from each dice (or vertex), our DFT traces all unique paths devoid of cycles, while continuously inspecting the trie to confirm whether the traced path is leading to a valid word. We stop traversing further in a given path if the prefix is not present in the trie. This way, we retrieve all acyclic paths in the graph that have a path in the trie, and we don’t go deeper than necessary into the current path.

Telegram bot

Using the above, we can create a Telegram bot that uses the above to confirm our words and to keep track of our scores:

Screenshot of the Telegram chat with bogglebot

Users input their Boggle game board as a grid by invoking the /new command in a chat with the bot, and the bot generates a list of all possible words by simulating all possible paths in the grid. The bot will then confirm the start and the counter of the game. During the game, the users type in the words they find, and the bot checks if the word is valid (i.e., is in the list of possible words). If the word is valid, the user’s score is incremented according to the length of the word. The score and game updates are sent back to the user in the chat. The bot is built using the python-telegram-bot library, which allows easy interaction with the Telegram Bot API.

Possible Extensions

Given the flexibility of the implementation, there are various ways we could extend this project in the future:

Wordlist Variants: Depending on our mood or the challenge we desire, we could spice up our games by introducing different wordlists for specific themes. For example, a game could be based solely on palindromes, or we could introduce a Flemish dialect-based wordlist. The possibilities and variations are endless, making each game a fresh and exciting challenge.
Custom Time Counter: A part of the thrill of Boggle comes from the time pressure. We could experiment with this element by introducing a varying time or perhaps even a countdown whereby the time decreases each round, raising the stakes progressively.
Automatic Scoring: One of the main appeals of this project is that it removes the cumbersome task of manual point counting. This could be further extended to automatically keep track of our scores across multiple games or even different days. Thus, we could concentrate on the game at hand, while the bot takes care of the statistics and progress tracking.
Speed Chess Inspired Mode: Perhaps one of the most exciting possibilities would be to create a ‘speed chess’ inspired mode. Each player could have an individual clock, and you’d need to quickly spot and share a unique word to pause your countdown and switch turns. The game would then continue until someone’s time reaches zero. This offers the perfect blend of frantic quick-thinking, strategy, and suspense — a true test of one’s Boggle skills!

In the spirit of the “Cult of Done”, I have focused on quickly getting this project up and running first in a short amount of allotted time, before worrying about enhancements or delivering the perfect project.

Inferno CTF — Write-Up

Sat, 28 Dec 2019 00:00:00 +0000

In this short write-up, we will go over two challenges in the Misc category of Inferno CTF, which was hosted by Dc1ph3R. The challenges that we will discuss are “Color Blind” (which can be solved with a simple one-liner) and “Registering X” (which was a bit more complex). I completed these challenges with our CTF team, U+1F966.

In this challenge, we are provided with a PNG image with a resolution of $64 \times 1$. Running strings, binwalk, and foremost on the given file does not give us any useful information.

Looking closer at this image, we can see that each pixel has a specific colour. A teammate mentioned that the actual colours of the pixels could be the key to our flag. Each pixel has a colour code that can be expressed in RGB or in HEX. Stringing together all the HEX values is the key to obtaining our flag.

Opening an image, retrieving all the HEX values of the pixels, and stringing them together is a trivial task that can be done with a simple one-liner in the Julia programming language:

using FileIO, Colors
join(map(hex, load("colorblind.png")))

This works because images in Julia are simple arrays containing the RGB value of each pixel. We map the hex function (provided to us by Colors) over the color values, which returns an array with HEX values. We then join all the values together, which gives us the following string:

626C6168626C6168626C61685F68656C6C6F5F686F775F6172655F796F755F746F6461795F695F686F70655F796F755F6172655F6E6F745F646F696E675F746869735F6D616E75616C6C795F696E6665726E6F4354467B6833795F3130306B5F7930755F3472335F6E30375F6833785F626C316E445F3A4F7D5F646F696E675F746869735F6D616E75616C6C795F776F756C645F62655F615F6261645F696465615F796F755F73686F756C646E745F646F5F69745F6D616E75616C6C795F6F6B

This string can probably be translated to readable text using Julia, but it was quicker to copy this output and paste it in an online hex to string converter:

blahblahblah_hello_how_are_you_today_i_hope_you_are_not_doing_this_manually_infernoCTF{h3y_100k_y0u_4r3_n07_h3x_bl1nD_:O}_doing_this_manually_would_be_a_bad_idea_you_shouldnt_do_it_manually_ok

The output above contains the flag:

infernoCTF{h3y_100k_y0u_4r3_n07_h3x_bl1nd_:O}

Registering X

The goal of this challenge was to construct a string that matches the regex given to us in the challenge attachment:

infernoCTF{.(?<=H){21}.[a-z](?<=\+a){1024}[a-z][a-j](?<!([a-u]|[w-z])(j|[a-h])).{2,64}(?<!\S){255}n.{2}(?<=\s)g_fUn\W(?<=[A-z])}(?<=\..{14})(?<=^.{33})

The given regex is complex at first glance, but the challenge itself is very straightforward. We did not find a way to automate the solution of this challenge, so we solved it by hand. First, we simplified the regex by removing the quantifiers after the lookbehinds, because they do not serve a purpose. Removing these quantifiers will give us an equivalent, but shorter, regex:

infernoCTF{.(?<=H).[a-z](?<=\+a)[a-z][a-j](?<!([a-u]|[w-z])(j|[a-h])).{2,64}(?<!\S)n.{2}(?<=\s)g_fUn\W(?<=[A-z])}(?<=\..{14})(?<=^.{33})

We start by learning about lookbehinds. Positive lookbehinds (e.g, "(?<=a)b" ) match the character after the parenthesis, but only if the character is followed by the character in the parenthesis, after the =. Negative lookbehinds (e.g, "(?<!a)b" ) match the character after the parenthesis, but only if they are not followed by the character in the parenthesis, after the !. Knowing this, we can start constructing our flag.

Our flag starts with infernoCTF{, because the given regex states that we have to match these characters literally. . in our regex matches any character in our flag, so we will put an X as a placeholder. We now have to match the positive lookbehind (?<=H)., which means that our string will only match the regex when we have a character following the character H. We could try adding H+ to our flag (making our flag infernoCTF{XH+), but this will not match the given regex. The cause of this mismatch is the fact that lookbehinds only match the character after the parenthesis. It will not match the character in the parenthesis.

The solution for this problem is replacing our previous X with an H. We now have infernoCTF{H+, which matches the first part of our regex. The rest of the regex follows the same principle: we have a bunch of lookbehinds that force us to precede a character with another character. Around those lookbehinds we have more common regex symbols that make it possible to match the character in the parenthesis. Now that we got the gist of the challenge, we will quickly go over the solution:

infernoCTF{ forces us to start our flag with infernoCTF{
.(?<=H).[a-z](?<=\+a) forces us to add H+a
[a-z][a-j](?<!([a-u]|[w-z)(j|[a-h])).{2,64} makes us add vi to our flag, followed by an unspecified amount (between two and sixty-four, which will be constrained later in the regex) of placeholder characters
(?<!\S)n forces us to add an n after a whitespace character
.{2}(?<=\s)g forces us to add a g after a whitespace character
_fUn forces us to add _fUn to our flag
\W(?<=[A-z])} forces us to add a } after a non-word character between A and z in the ASCII table (e.g., ])

(?<=\..{14}) forces us to count fourteen characters back and add a .. We can safely put our . in the collection of unspecified characters mentioned in the third step above.

The last part of our regex forces us to make the flag exactly thirty-three characters long. This can be achieved by adding or removing a bunch of unspecified characters (which were mentioned in the same third step). One of the possible solutions is infernoCTF{H+avi . ng no g_fUn]}.

Because of a lack of knowledge about tools that generate strings based on given regex (that also support lookbehinds!), we decided to solve this challenge by hand. Although it is a bit of tedious work, it certainly is possible to construct the flag manually.

About