Lucas Wilson-Richter

Making Beautiful Things

Sun, 29 Mar 2026 18:03:00 +1100

Today I had a realisation: I feel as though it has been a very long time since I made something beautiful.

This is just a feeling, and it is objectively untrue. I sent a silly, funny, pointless, beautiful text to a friend yesterday. But the feeling is there anyway. I guess I am having trouble remembering these things when I need to.

Thinking about the stuff I have been focussed on for the last 6-12 months, my main foci have been rather ruthlessly functional:

Employment – like many businesses, my employer is not against beauty as long as it serves a functional need. But the functional need will always come first.
Caring for friends & family who are going through a variety of life changes and health events.
Adjusting my own ways of living life, since it insists on changing all the time.

I have spent a lot of time and attention on surviving, and on fighting ugliness: holding out against certain trends of the technology industry; adjusting my daily routines to avoid exhaustion; maintaining my body; arguing against things at work that I think will go badly.

A lot of this work has been creative, in that it has required creative problem solving and communication. And I get some abstract satisfaction from knowing I have made a difference in people’s lives, or from solving problems for my employer and my colleagues. This work is (or can be) important.

But it is not satisfying me. There is something missing. I think that thing is satisfaction from creating beauty.

I also notice that I have not been doing so much of the things that give me a sense of beauty, fulfilment, joy, wonder or expression:

Music, especially singing
Writing
Hiking, trail running, or otherwise getting out into green places
Playing with tech for joy and creativity¹

All in all, my current activities are not giving me as much joyful fulfillment as I need. Some, for sure! But not enough. And the cracks have been showing, at least to me. Lately I have generally been less patient and kind than I want to be, especially at work. I am sure that has dented at least a few relationships, and I suspect it’s cost me some opportunities.

Obviously, I need to change something. At the moment, the set of things I feel ready to change right away is limited to the incremental and relatively undemanding. Even if I made a big, dramatic commitment to turn my life around, quit my job and focus on Making Beauty:tm:, I would expect that resolution to fail within 24 hours.

So what can I change? And how can I make it stick?

Here are my 2 my best ideas so far.

1. Make tiny, beautiful things daily

Every day for 1 month, I will do or make a tiny thing, just for the sake of making something beautiful.

The things themselves do not need to be beautiful! Failure (or pre-success) is an option. I just need to do or make with the intent of making something beautiful.
I will keep a record of the tiny things I have made and done, to look back on when I need to. I may or may not share any of them with anybody.
Things I do for work do not count. They might be beautiful, but I am not doing them for their beauty. I am doing them for money, which is a useful but very different motivation! Since motivation is an important part of this, work things are out.
Beautiful does not mean pretty, though they might coincide and I hope they do. Beautiful means … well, I know it when I see it. My beautiful things might be pretty, or heartwarming, or elegant, or tragic, or harmonious, or funny, or just the right kind of terrible. Or something else entirely.
Right now, I intend to continue after that month. But right now I’m inspired and optimistic, and that will not last. I will need help to keep this up.

2. Stop fighting the ugly with more ugly

(…and undermine the ugly with beauty instead.)

I will try to avoid fighting the ugliness I see (or expect, or fear) with more ugliness – unconstructive argument, futile resistance, passive aggression. It has not been helping me, and I doubt that it helps those who receive it. It lends strength to what I want to oppose.

I am getting a better idea of the places within myself where those responses are coming from. I hope that means I can do better at recognising them when they happen, and find a more beautiful way to respond. Or if I cannot create beauty there, at least let that thing go and use my energy elsewhere.

In the meantime, I know I mostly only recognise my uglier creations after it’s too late. I want to avoid making those messes, and for that I will need earlier, stronger signals that I am about to create one.

Externally, this might not look like much. It will probably be more observable as a lack of crappy behaviour than the presence of anything better.

Internally, I think it will probably mean extending the benefit of the doubt more of the time, and trying to remember that when I feel resistance to an idea, that is the time to listen harder.

But… how?

This will be hard. Like, hard hard.

I do not have a great track record of keeping “every day” resolutions. I will forgive myself for the days I miss, and then keep going.
I know that high-minded resolutions don’t mean squat when someone is wrong on the internet. I need to think about how I can catch myself before I say things I wind up regretting.
My ego and I get so damn tired of realising we know nothing. But that is the time when I need to extend the benefit of the doubt, and listen harder. Learning opportunities are not threats, so I guess I have some reframing work to do.

pursues knowledge

considers

Persisting post-novelty

I have been using Finch for a while, to help me check in on the self care things I intend to do each day. “Make a tiny beautiful thing” would fit in nicely there, to help me remember.

I am also posting this where people might see it. My name is pretty distinctive, so there’s not a lot of deniability here.

Keeping stuff to look back on

I think a folder on my NextCloud server is probably the simplest thing I could do here. Just about anything I am likely to make for beauty’s sake can go in, or be described in, a file: photos, writing, videos, links. If I make something that does not fit, I can solve that when it happens. And a private server means that I can save this stuff there comfortably, even if it winds up being deeply personal or otherwise not for sharing.

Preventing regrettable outbursts

For me, the impulsive stuff I regret has taken the form of written messages online: Slack replies, code review comments, etc. So a tool that checks my writing could be really helpful, as long as the false positive rate is low (I might start ignoring it otherwise). But sending every word I write to a remote server is unacceptable! Respect for privacy is super important, for me and my employer both.

Convosify’s Message Guardian seems too good to be true. A quick search for scammy feedback turned up nothing notable, which I guess is a good thing. I might give it a go, and use Little Snitch to make sure it stays off the internet.

Extending the benefit of the doubt

I have only given this a couple of hours of serious thought, but so far I am coming up short on any ideas for supporting myself to do this in the moment.

Maybe I am not ready yet. Maybe those impulsive responses are getting in my way, removing the option of listening or being curious. So I will try to work on that and see if this gets any easier. If not, maybe I will have some ideas about it in the meantime, or I can phone a friend.

Okie dokie. Time to give it a go.

As opposed to working on some of my values-driven tech projects. Those are deserving and worthwhile, but fundamentally driven by anger and opposition. That work is motivated by my desire to fight ugly things, not make beautiful things. Even though both might well result, and I hope they do. ↩

Looks like I write about music on the internet now

Sat, 25 Oct 2025 20:08:00 +1100

I went to a concert last Thursday, and then I wrote about it.

I have a few things to learn about writing reviews 😂 but I had a great time with it.

#fyp @ the Edge

Sun, 12 Oct 2025 14:33:00 +1100

A big part of the Social Media Algorithm ProblemYou know, the one with the addictive dark patterns designed to drive “engagement” at the expense of our wellbeing. - at least as I see it - is that we mortals are not in control of the algorithms’ execution.

Instead, the service operator (Facebook, Tiktok, YouTube, whoever) keeps all the data about what we view, don’t share it with us (or at least not easily), and instead use it to hook us into MOAR ENGAGEMENT! MOAR EYEBALLZ! MOAR ADDZZZZZ!!!

All of which leads to the synthesis of “my” feed, FYP, etc - an infinite list of stuff chosen for me to look at and engage with, but not necessarily for my benefit.

If it’s not for my benefit, then whose benefit is it for? Who wins from it being the way it is?

Well, the service operator, who sells my “engagement time” to advertisers. So, Google. Apple. Netflix. Amazon.

I’m pretty depressed and fed up with social media feeds, app & product recommendations, search results, all of it, all being so untrustworthy. And I have been for a while.

And I’ve started thinking: what would have to be true for me to get social media feeds and product recommendations that I have more control over, and bring control over my FYP out to the edge of the network, closer to me?

What pieces would we need to make that happen?

Speculation mode: ENGAGE!

Data about me

First, obviously, there’s data. Probably quite a bit about me.

Browsing history
Media viewing history
Purchase history
Some data about what kinds of things I’m interested in. I might supply this explicitly, and/or there might be some inference based on my behaviour. Critically, I get to choose.
Some social graph data - who do I know? Which YouTubers’ stuff do I watch to the end? Whose blogs am I subscribed to?
Anything else I’m comfortable tracking, that might supply context for recommendations or feed content choices.

Creepy, huh? Certainly when a faceless corporation does it.

But maybe I’m a bit more comfortable storing this information on systems that I personally trust. These could be my own machines, which I can turn off, or whose storage I can erase, or whose analysis I can actually learn and benefit from. Or they could be machines run by someone I know, and who I trust not to screw me over.

Data about others

Now here comes the really tricky part: how do I turn this data into recommendations for things to read, watch or buy?

Suppose I shared some of this data. Voluntarily, and only the parts I’m comfortable making public.

Likes, ratings and comments
Subscriptions
Maybe some reading / viewing info
Content I’ve made myself
Purchases, or summaries of purchases
A bit of demographic info - some stated interests, connections, rough age, nationality, occupation. Nothing I haven’t already put on Facebook or LinkedIn.

And suppose lots of other people did that, too. Collect for themselves, and selectively and voluntarily share, a portion of the data that the Big Tech Villains already harvest and sell to advertisers.

Suddenly I can find out quite a bit about what my connections, and other folks similar to me, are engaging with. But only as they have chosen to share it.

And assuming they don’t lie, I guess. But even that can be handled if I control the algorithms behind my feed.

Ignoring important obstacles? Hell yes I am.

Obviously this is nowhere near as simple as I’m making it out here. I know there are huge obstacles to all of this. Just for starters:

Getting all this data out of the walled gardens where it lives.
Persuading enough people to gather their data (e.g. via Google Takeout), sift through it and share it publicly - without resorting to unethical manipulation!
Protecting people’s privacy as they do that - all of this stuff is potential fuel for scams, blackmail and identity theftNot that most of the really juicy information isn’t around already, but why make it worse? .

Algorithms

With that sweet sweet data, I can start to do a little bit of the recommending stuff that drives all those feeds & FYPs. And I can do that using systems and algorithms that I control – not in the sense that I’d necessarily understand them, but at least in the sense that I could turn one off, tune it a little, or swap it out for another algorithm that matches my priorities better.

All of which seems quite nice, but aren’t we now just implementing all those dark patterns on our own machines instead of Google’s?

Well, maybe. But I hope not, or at least I think it creates room for an alternative.

Because we own the machine that runs the algorithms, we can (at least in theory) choose our own algorithms. For example, say you have a bit of a problem with videos about cake decorations going horribly wrong. They’re your kryptonite - you see one, you have to click, you just can’t look away, and when you surface an hour later you’re invariably filled with regret.

So you decide you don’t want recommendations for any of those “baking disaster” videos (at least not during the week - maybe a little guilty pleasure on weekends is not too bad).

Rather than trying and failing to manipulate someone else’s algorithms into doing what you want, you tell your algorithm “don’t recommend videos about baking disasters during the week - and keep it to just a few on the weekends.”

Other examples:

Avoiding triggering material - mentions of violence or abuse, discrimination, hate speech, etc.
Promoting viewpoints that challenge your own, in order to avoid echo chambers and filter bubbles.
Changing your algorithm to suit what you’re doing (or supposed to be doing).
- News in the morning, sitcoms in the evening.
- Holding off on the more grown-up content until after the kids’ bedtime.

You’re in control, at least more so than before. Or, a little more hyperbolically: you’ve siezed the means of recommendation.

Hard problems I am totally glossing over right now

There are quite a few of these, each with its own unique difficulties:

Coming up with alternative algorithms:
- They need to be efficient enough to actually run on consumer hardware.
- They need to provide a better experience than the status quo. Otherwise, who would switch?
- Finding a way to decide what to recommend, that isn’t dark-pattern-driven engagement, is probably a huge challenge.
- How will all the TikTok-ers and YouTubers get their social validation, if not for Likes?
Making the tech accessible to people who are not self-hosting enthusiasts, programmers and accessibility nerds (i.e. almost everyone).
Nasty people selling algorithms that are - overtly or subtly - not serving the end user’s interestsNot that selling algorithms is necessarily bad! But the product has to be worth the money, respect your privacy, serve your best interests, and just generally not be a scam. And making a profit within those constraints may be literally impossible, especially in the face of probably-unhealthy competition from the incumbents. .
- Honestly, I reckon this would be a bloody amazing public good for an open source project, non-profit or charity to champion.

Filters

On the sharing side, I need a way to choose what I publish about myself and my activities. And it will need to not involve checking back over my reading and viewing every day. That will get old super quick.

Ideas:

Allow- and deny-lists of what to share, and what not. Defaulting to private, of course. Easily understood, but prone to falling out of date without regular maintenance.
Subscribing to lists published by knowledgeable parties. This works decently in the IT security space, e.g. blocking known spammers, or domains that host malware.
Training a machine learning model to predict what I will and won’t choose to share. Possibly a bit more chaotic, but could be capable of adapting as your preferences change, or new cases arise.

Ignored issues

Privacy and oversharing
- This is an area where people may sometimes need protection from their own impulsivity.
- I don’t know of any really good systems for reviewing privacy settings. Largely because the kinds of places where they’re required (cough Facebook cough) also have strong incentives to make them overwhelming and difficult to use.
Protecting children and vulnerable people
Ethics of building machine learning models, especially those trained on copyrighted and unpaid-for source material.

Feeds

The means by which we share, and which our crawlers (see below) crawl. Probably similar to RSS, ActivityPub, or other existing protocols for sharing events like posts, likes, updates and so on.

Ignored issues

https://xkcd.com/927

Crawlers

These are our hunters and gatherers, bringing us data for our Crunchers.

They might:

Crawl the social graph that we and our connections share (e.g. via HTML microformats), gathering shared data.
Go and find updates from creators that we have subscribed to. (Yes, just like RSS. This isn’t new.)
Periodically check webpages for updates (“Is this item back in stock yet?”)
Assemble your daily agenda based on your email or calendar data - if you decide to trust it with that!

Issues

Crawlers need to be respectful clients - especially if we’re aiming to put one (or 1000) in every home.
I don’t imagine that writing a crawler capable of withstanding life on the open web is an easy feat.
Consumer hardware constraints again.
Validating & sanitising incoming data. These little critters are meant to introduce arbitrary data into your systems. Literally anything could be in there! So we need malware scanning, sandboxing, protection against injection attacks that could exfiltrate, delete, or change your stored data.

Crunchers

The decision making parts - these are components that look at your gathered data and decide what’s likely to appeal to you at the moment.

Crunchers would decide what goes in your FYP, but also potentially what you get realtime notifications about, what goes in your daily summary email, the order of podcasts in your queue, or which TV shows to suggest tonight (Full calendar today? Let’s suggest cartoons or reality TV, rather than that highly cerebral Scandinavian Noir thriller).

These might be things you want to tune in the moment, e.g. whether you’re looking for something familiar to watch, or something a little bit new or different. Or how much time you have available.

Issues

Transparency of decision making
- Making explainable decisions
- Presenting explanations that we users can make sense of, and that help us figure out what (if any) changes we want to make.
This is where our software is most likely to contain bias. It might be introduced by the author of the software, by the methods & data used in training if ML models are involved.

Presenters

These are different methods of showing things to you, e.g.: instant notifications, a daily or weekly bulletin, a scrollable feed, a search index, an infographic, your podcast queue. A context layer on your web browser or online map. Probably a few million other things, too!

Issues

It’s getting late! I don’t want to think about the issues anymore.

Summary

This is all wild speculation. But given the availability of small, cheap, silent computers; more and more (and faster, and cheaper!) ubiquitous network access; and growing distrust of tech companies, maybe it’s time to work a little harder on some decent alternatives.

Writing tools for software developers

Fri, 06 Aug 2021 15:45:00 +1000

I’m hardly the first to observe that the further a software developer gets into their career, the more their job requires communicating, as opposed to simply writing excellent code. And as workplaces are starting to get more remote- and async-friendly, that communication tends to become more written than spoken. So developing your writing practice is a critical career skill.

But why give up your excellent coding toolset just because you’re writing in a different language, right? You worked hard to gain your current level of editor wizardry, and writing can benefit from version control just as much as code can.

Here’s a quick list of coder-oriented tools I use for writing.

Markdown (markup language)

It’s the fashionable programmer’s markup language. I’m fluent with Markdown, so I tend to prefer tools that support it.

I’ve been looking into reStructuredText, and I could be convinced that its technical merits exceed those of Markdown, especially in the area of extensibility (e.g. name-based linking within a site, such as a wiki or docsite). But nothing beats Markdown for sheer popularity. And, well, I’m invested now.

Joplin (note taking app)

I use Joplin for drafting and note-taking. It’s how I keep track of a lot of the stuff I do: ideas for writing, projects at work, toys or gear that I’m keen to try, triathlon training plans, all of that.

I started using it because I was sick of Evernote. There were a few reasons:

Its formatting was clunky, especially for code samples.
Having my personal notes stored by a corporation was not my favourite.
They constantly nagged me to upgrade, even though I was paying them money.

So I looked for something free, Markdown-y and with support for self-hosted syncing. Et voila!

I also like Joplin’s inbuilt support for Mermaid diagrams – we’ll get to that in a moment.

Just for giggles, I would love to have Evernote’s OCR capabilities in Joplin, though I’m not quite ready to go the machine learning route to get them. I’ve read that plugins do exist for that. I’ll have to try them out some day.

Mermaid.js (diagrams)

Mermaid is a plain-text language for making diagrams. I love it because it solves several problems I’ve had with so many diagramming tools:

They’re usually graphical, which means I need to use a mouse a lot. If there’s a thing I hate, it’s too much clicking.
They usually require a separate program (or webapp) to edit. This disrupts the flow of writing, and makes the written product and diagram harder to maintain.
The diagrams usually can’t be inlined into the article. Instead of one plain-text file for my article, I now have one text file, plus one or more PNGs or something, which can be hard to associate back to the articles that include them. That can be solved with discipline, but discipline like that is hard to maintain for me, let alone my entire team.
Their output is usually binary, making them work poorly with the version control tools I like to use when I can.

Mermaid is a plain-text format that seems to do for GraphViz what Markdown does for HTML. It supports a few different types of diagrams, though I mostly use the graph type to draw boxes and lines. Mermaid source is easily inlined, and a bunch of Markdown-y tools have plugins or extensions for supporting Mermaid too.

Excalidraw (whiteboard, diagrams)

As great as plain text is, you do sometimes need a graphical diagramming tool. Excalidraw is the best one I have come across so far.

It has a bunch of keyboard shortcuts, meaning I only need to mouse for drawing.
It can produce “drawn look” diagrams, which help to suggest the intended level of precision for a lot of stuff I make ;)
It’s self hostable.
The hosted version supports collaboration – I’ve used this for pairing, remote interviews, and a few other things.
It’s not hard to pick up and use for the first time, like your Visio or whatever.

I actually just found out about a cool Excalidraw feature today. Apparently, if you copy-paste some 2-column CSV data (or data from Excel or a HTML table) into Excalidraw, it will automagically chart it for you!

Reveal.js and `reveal-md` (slides)

Reveal.js is a framework for writing slide decks in HTML and presenting them in the browser. It also does a nifty thing where you can navigate in 2 dimensions; slides can be grouped into columns, instead of running from start to finish in a single line.

Making slides using only a text editor and a browser? You better believe I love that. In fact, there’s only one thing that could make it better – Markdown!

Enter reveal-md. This takes the features of Reveal.js and makes them available in (slightly extended) Markdown. Instant happiness!

This pairing is my go-to for slide making. Especially since I added some special sauce for embedding Mermaid diagrams in my slides :grinning:

As an added bonus, reveal-md slide documents will work with an ordinary Markdown renderer. So with some well-written speaker notes, you can also use your Markdown to produce an acceptable (though not amazing) print document.

Honorable mention: MkDocs (documentation)

I only started playing with MkDocs recently, so I haven’t had time to run into many of its quirks. But I have enjoyed a reasonably smooth experience while putting together a doc site for my team.

MkDocs is a static site generator. In its default configuration, it produces sites that look a lot like readthedocs.ioYes, those are the docs for ReadTheDocs . As you might imagine, it’s Markdown-y. It also supports plugins for extending the Markdown syntax in all kinds of interesting ways (if you need that).

So there you have it! Hopefully one of these little gems brightens your day :slightly_smiling_face:

Switching to Tufte CSS

Wed, 14 Jul 2021 23:31:00 +1000

Tufte CSS seems pretty popular, and I like a lot of what Tufte advocates about communicating visual information clearly, succinctly and (though this is possibly something of a side benefit) beautifully.

So I’ve switched this thing over to a Tufte-inspired layout. Tada! :tada:

You might have noticed some differences from the usual “Tufte” look. I’ve been learning a bit about dyslexia recently, and decided to make a few adjustments to the style of the site in the name of accessibility.

I changed the default serifed fonts to non-serifed. This was simply a matter of taking the non-serifed fonts that were already part of Tufte CSS and making them the default.
I also removed italics from the stylesheet entirely. Italics are known to be confusing to many dyslexic readers, because they change the shape of letters, which makes letters harder to recognise.

I have to confess, though, I found the lack of bullet points rather painful, particularly when it came to posts from times long past. I get the reasoning, I just didn’t have it in mind when I wrote those posts long ago, and I’m not entirely certain that the writing I do here is the kind on which ET focussed his work.

So I changed the rules and put them back.

Running a Docker container as a non root user

Sun, 26 Apr 2020 11:54:00 +1000

⊕“Containerbow” by Michael Phillips Photography

The Problem: Docker writes files as `root`

Sometimes, when we run builds in Docker containers, the build creates files in a folder that’s mounted into the container from the host (e.g. the source code directory). This can cause us pain, because those files will be owned by the root user. When an ordinary user tries to clean those files up when preparing for the next build (for example by using git clean), they get an error and our build fails.

There are a few ways we could deal with this problem:

We could try to prevent the build from creating any files, but that’s very limiting – we lose the ability to generate assets, or write any data to the disk. This is definitely too restrictive to solve the problem in a way that I could use with any build.
We could tell Git to ignore the affected files, but that carries the risk that they’ll hang around in the file system and have an effect on future builds. We’ve encountered that problem in the past at Redbubble, so we are wary about letting that happen again.
We could clean up the files at the end of the build, while we’re still running our Dockerised process. But that would require us to implement lots of error trapping logic to ensure the cleanup happens, but still exit the build with the correct result.

It would be more elegant if we could simply create files in a way that allows ordinary users to delete them. For example, we could tell Docker to run as an ordinary user instead of root.

Time to be someone else

Fortunately, docker run gives us a way to do this: the --user parameter. We’re going to use it to specify the user ID (UID) and group ID (GID) that Docker should use. This works because Docker containers all share the same kernel, and therefore the same list of UIDs and GIDs, even if the associated usernames are not known to the containers (more on that later).

To run our asset build, we could use a command something like this:

⊕This will tell Docker to run its processes with user ID 1000 and group ID 1000. That will mean that any files created by that process also belong to the user with ID 1000.

docker container run --rm -it \
  -v $(app):/app \                  # Mount the source code
  --workdir /app \                  # Set the working dir
  --user 1000:1000 \                # Run as the given user
  my-docker/my-build-env:latest \   # Our build env image
  make assets                       # ... and the command!

But I just want to be me!

But what if we don’t know the current user’s ID? Is there some way to automatically discover that?

There is!

id is a program for finding out exactly this information. We can use it with the -u switch to get the UID, and the -g switch to get the GID. So instead of setting --user 1000:1000, we could use subshells to set --user $(id -u):$(id -g). That way, we can always use the current user’s UID and GID.

docker-compose

We often like to run our tests and things using docker-compose, so that we can spin up any required services as needed – databases and so on. So wouldn’t it be nice if we could do this with docker-compose as well?

Unfortunately, we can’t use subshells in a compose file – it’s not a supported part of the format. Lucky for us, we can insert environment variables. So if we have a docker-compose.yml like this:

# This is an abbreviated example docker-compose.yml
version: '3.3'
services:
  rspec:
    image: my-docker/my-build-environment:latest
    environment:
      - RAILS_ENV=test
    command: ["make", "assets"]
    # THIS BIT!!!1!
    user: ${CURRENT_UID}
    volumes:
      - .:/app

We could use a little bash to set that variable and start docker-compose:

CURRENT_UID=$(id -u):$(id -g) docker-compose up

Et voila! Our Dockerised script will create files as if it were the host user!

Gotchas

Your user will be $HOME-less

What we’re actually doing here is asking our Docker container to do things using the ID of a user it knows nothing about, and that creates some complications. Namely, it means that the user is missing some of the things we’ve learned to simply expect users to have – things like a home directory. This can be troublesome, because it means that all the things that live in $HOME – temporary files, application settings, package caches – now have nowhere to live. The containerised process just has no way to know where to put them.

This can impact us when we’re trying to do user-specific things. We found that it caused problems using gem install (though using Bundler is OK), or running code that relies on ENV['HOME']. So it may mean that you need to make some adjustments if you do either of those things.

Your user will be nameless, too

It also turns out that we can’t easily share usernames between a Docker host and its containers. That’s why we can’t just use docker run --user=$(whoami) – the container doesn’t know about your username. It can only find out about your user by its UID.

That means that when you run whoami inside your container, you’ll get a result like I have no name!. That’s entertaining, but if your code relies on knowing your username, you might get some confusing results.

Wrapping up

We now have a way to use docker run and docker-compose to create files, without having to use sudo to clean them up!

Happy building!

I originally published this post on Medium. I think the ability to publish and keep control of one’s own work is one of the Internet’s most powerful benefits, so I’ve also posted it here, just on general principle.

Making practise tools for amateur musicals, Linux edition

Wed, 01 Aug 2018 08:16:39 +1000

A couple of years ago, I was Musical Director for a local musical theatre production. This year, I got the opportunity to do something similar for a production of Rodgers & Hammerstein’s Cinderella, and I spent quite a bit of time making practise materials for the cast. In the intervening time, I switched from Mac to Linux for my personal machine, so I thought an update on the tools I used might be in order.

There were a couple of differences in my objectives for this project. For example, rather than producing both scores and sing-along practice tracks as with the last project, I only needed to make the tracks in this case (what a relief!).

That had an effect on the methods I chose to use when recording. When I was making scores, getting note values exactly right was a high priority, so I used step recording for note input. That gave me the precision I needed for a score, but was quite time consuming. Since I was only making the sing-along tracks for this latest project, I was able to play the score directly into the recording (albeit in short sections, one hand at a time). You’ll also find that I don’t mention any score engraving software this time around.

Hardware

System76 Galago laptop

System76 specialise in Linux laptops, customisable in almost every respect. Their 13-inch Galago model features a HiDPI display, which I’m quite enjoying. The onboard audio, however, leaves a bit to be desired when it comes to recording. The built-in headphone and mic channels are both exposed to quite a bit of noise, which results in substandard recordings, especially when you’re recording 4 vocal parts and layering them together!

Hence, my next investment.

Focusrite Scarlett 2i2 USB audio interface

This little fellow cleared my line noise problems right up, and works with Linux (or at least Ubuntu) out of the box.

Shure SM58 microphone

A standard choice for an omnidirectional dynamic mic. No regrets here.

M-Audio Keystation 49

This one hasn’t changed from the previous edition. I’m still pretty pleased with it, and I’ve learned a little more about how to use it as well. Fortunately for me, it also works out of the box with both Mac and Linux.

Software

DAW: Ardour

The Digital Audio Workstation (DAW) I chose is called Ardour. I chose it mostly because it (a) connects with JACK easily, (b) is libre and free, and (c) seemed popular, and hence would have some degree of community support if I got into trouble.

It served me well, though I was on a continuous learning curve through the entire process of recording about 20 tracks!

JACK & Qjackctl

JACK (JACK Audio Connection Kit) does the job of connecting a system’s many audio inputs and outputs together. It can be a bit daunting to get started, but I found that this guide really helped me understand what I was doing.

FluidSynth & Qsynth

FluidSynth is a MIDI software synthesiser, which uses SoundFonts to turn MIDI signals into sounds.

Qsynth is a GUI for FluidSynth, which makes it easier to manage.

I found these useful for when I wasn’t recording with Ardour (which manages its own FluidSynth), but still wanted to use the MIDI keyboard and hear what I was playing.

a2jmidid

So Ardour, FluidSynth and friends were all using JACK to communicate with one another, and that was great. I faced a little challenge when I realised that my MIDI keyboard was sending signals on the older ALSA bus. A quick search revealed a2jmidid, or the ALSA to JACK MIDI Daemon, which basically does what it says on the tin: it takes MIDI signals from the ALSA bus and retransmits them over a connection to JACK, and vice versa. Very handy.

Foreman

After a while, I got tired of starting all these different tools in sequence. Occasionally I’d start them up in the wrong order, or I’d be running command number 5 and suddenly realise that I’d made a mistake in command number 2. Super frustrating.

Fortunately for me, I’d encountered a tool for a purpose very like this, in my work as a software engineer. Foreman is a handy little utility for running a number of programs at the same time. To use it, you specify all the processes you want to run in a file named Procfile. Foreman reads that file and starts the processes you ask for. It’s quite a simple tool, so it lacks (as far as I know) any features about getting those processes to communicate with each other. Fortunately, that’s what Jack is made for.

Controlling Ardour from a tablet using TouchDAW and QMidiNet

Once I’d recorded all my piano and vocal tracks (with some help from a couple of very generous friends), it was time to mix. It turns out that controlling a mixer using a mouse or trackpad can be a tedious experience. To help with this, I started looking into other ways of controlling Ardour, preferably using touch.

I tried out a few different tools, and none of them really satisfied. I got the best results using TouchDAW and this guide to setting it up. I can’t say I was overjoyed with the results - I eventually went back to using a mouse to control the mixer. That may have been partly due to the fact that I was working on the opposite side of the house from the WiFi access point. All the same, I like the promise of the Open Stage Control protocols, but I don’t think the current libre technology fulfills that promise just yet.

Publishing

I published all the finished tracks to a bucket in Amazon S3, which I made public so all the singers could download what they needed. Maintaining the index pages for all the parts got tedious as I uploaded new tracks, so I ended up maintaining those in Markdown, and using a little script to render them to HTML and sync a folder on my laptop to the S3 bucket. From there, it was easy to send links via email or other means.

One little gotcha with using S3, though: be sure to set the metadata on each file with Content-Disposition: attachment, so that web browsers know to download the file to disk instead of playing it in the browser.

So there it is, an updated round-up of this project’s technology choices. I hope it’s been helpful!

Making practise tools for amateur musicals

Sun, 23 Apr 2017 11:40:00 +1000

Last year I served as the Musical Director for a local musical theatre production. My duties involved teaching the music to the cast so that they could perform from memory on stage, including harmony singing in up to 4 parts.

Because the community group I work with (the Diamond Valley Singers) has a policy of not auditioning the chorus for our shows, we get people of all skill levels joining the cast. Some people have already developed the skills they need to learn the music and sing in parts from memory, while acting and dancing at the same time. Others haven’t, and need as much help as the creative team can provide to get them ready for performance. So I spent quite a lot of time preparing materials to help people practise, including more readable scores and practise backings for all the vocal parts.

Here’s a little run-down of the tools I used to put these materials together. All the software is free, or comes packaged with Mac OSX. Hardware is another story, but investing in even a modest MIDI keyboard made a huge difference to the amount of effort required to get everything done.

MuseScore

MuseScore is an open source music editor, which has all the same basic features as Sibelius. More advanced features such as music scanning are not included, but the price is right, particularly for an amateur like me. The vital features for me were:

Input from a MIDI keyboard. This speeds up input in a huge way, especially as scanning wasn’t an option. I used an M-Audio KeyStation 49, which was exactly the right size for my desk, and connects easily via USB.
WYSIWYG (What You See Is What You Get) score editing. Believe it or not, not every music editing program has this feature (I’m looking at you, Lilypond).
Plain-text output in an open format (MusicXML). This was an important feature for me because I wanted to use the programming tools I know and love, to help me with things like version control (which I’ll go into more later). MusicXML is not MuseScore’s default format, but it beats the pants off any binary format for my purposes.
Pretty good help from the community. After consulting Google, I was able to get solutions to quite a few problems from the MuseScore user forums.
Finally, PDF and MIDI exports. This allowed me to turn the scores I was working on into backings for the practise recordings. I could also be certain that those backings would match the score exactly. Again, a pretty standard feature for this kind of software, but I mention it because it was especially important for this project.

Git

I’m a software engineer by profession, and I like to apply programming tools to my other pursuits because I know how to use them, and they help me approach problems in ways I understand. I used Git to version control my scores, so I knew that if I completely messed things up, I could always revert to a previously saved version. At the end of the process, I also got a nice little story created by the sequence of commit messages I’d written. Some day, a musical archaeologist will be able to follow my creative process step by step, because of those records.

I should point out that I only used Git’s most basic features. I did most of my work on a single branch, and didn’t do any merging at all. Change tracking and save points were all I needed in this case. Git has some great features to help with integrating changes amongst a team, but I had no need for them this time, as I was the only one contributing.

Using Git was only practical because I could get MuseScore to output my work in a plain text format, in this case MusicXML. It’s very hard to do version control in a space-efficient way for binary files, as it’s a lot easier to compress plain-text files. Even if plain-text files are less space efficient individually, they compress much better in large numbers.

Tascam stereo recorder

Once I had my scores and backings done, I was ready to record voice parts. I dubbed these over the top of the backings to help my singers practise, since as I mentioned, I was working with singers of all skill levels. I used a Tascam portable stereo recorder mounted on a tripod to record the parts, singing the bass and tenor parts myself and calling in a couple of friends to help me with the soprano and alto parts.

Audacity

The one thing that wasn’t ideal for me about the Tascam recorder was that I couldn’t figure out how to record at a decent volume. Even though my singing voice is quite loud and I was standing right in front of the recorder, the resulting file was very quiet. I suspect it’s because the microphones can handle very loud sounds (e.g. a symphony orchestra playing fortissimo), and even my loud voice is quiet by comparison. Anyway, I used Audacity to amplify the raw audio files before I combined them with other sounds.

GarageBand

Once all the parts were recorded and amplified, I used Apple’s GarageBand to mix everything together. With GarageBand I was able to prepare instruments-only tracks for more confident singers, tracks with individual parts for the less confident, and also tracks with all the parts included, which turned out to be very helpful during choreography rehearsals.

GarageBand is an entry level tool as far as recording goes, but I didn’t need anything more sophisticated for this project. Just decent MIDI sounds, some chopping and swapping of recorded samples, and basic mixing tools.

Adjusting tempos

As we approached the end of the rehearsal process, I found I needed to adjust some tempos to be more suitable for the venue and the choreography. Rather than going back and re-recording everything at a different tempo, I was able to use Audacity’s “Change Tempo” feature (under the Effects menu) to speed up and slow down songs (or parts of songs) as required. You do need to be pretty accurate with your selection if you only want to adjust part of the song, but the UI for this feature is surprisingly functional considering Audacity’s otherwise unattractive interface. I say this because it does the sums for you; you tell it what speed it is now and what speed you want it to be, and it figures out the percentage to adjust by. Exactly the kind of thing that computers do more easily than humans.

So those are the tools in my practise materials toolbox. I hope you’ve come across something new here, or been reminded of something you already knew.

Happy arranging!

Welcome to Jekyll!

Sat, 27 Aug 2016 18:45:00 +1000

I haven’t done any blogging in quite a while. This is partly because I’ve been running the tech blog at work, and partly because my blogging framework (that one I wrote for fun, but ended up looking a lot like Jekyll) depended on a machine that I had to decommission, and also started looking a lot like Jekyll, only not as reliable or feature-rich.

So I’ve switched to Jekyll. I was actually pretty surprised by the degree of similarity between my format and Jekyll’s. It tells me I didn’t exactly have the wrong idea about that bit.

Out of this whole thing, I’ve learned a few lessons.

Text-only, version controlled blogging is a good idea.Or at least, it sucks a lot less than WYSIWYG without change control!

Building your blogging framework around Rake is not such a good idea.Especially when you don’t really know Rake that well

It would have been great to learn more about Ruby gems when I was building, so I could have used them better. That said, Git submodules were not a complete waste of time.[Sure, but there’s usually a better tool for the job – Ed.]

The “view history” aspect of the whole idea is not as useful for me as it might be for some others. Nevertheless, it’s useful to think about.

I used to be pretty dumb about some of my predictions, and I was right on the money about some others. Jury’s still out on the rest.

I’m hoping that I’ll publish more if I’m using tools where I don’t feel the burden of maintaining them to quite the same degree. We’ll have to see.

Using SublimeText with Lilypond

Fri, 02 Jan 2015 13:23:55 +1100

I’ve spent a little time recently working on my Lilypond workflow, trying to improve on the process I used when I was arranging a moderately long and complex work last year. I used a tedious process involving hand coding (in Vim, until I decided learning both Vim and Lilypond at the same time was a bad plan), running Lilypond in a terminal, then checking the output in Preview (OSX’s built in PDF viewer).

Once I started using Sublime Text, I got much faster, but the feedback cycle (edit, switch to Terminal, run Lilypond, switch to Preview, check result) was still a killer. So I started using a Makefile that a mate had put together, along with Sublime’s build functionality. So now I create a PDF by hitting ⌘-B (or Ctrl-B in Windows) or F7. That saves a little time.

Little did I realise that the SubLilyPond plugin contains a build system for creating Lilypond PDFs. SubLilyPond is pretty great even without that; it does syntax highlighting for Lilypond files. There’s some talk in the README about snippets coming up in future, and I’m looking forward to that.

Meanwhile, though, it’s not a bad idea to create your own Lilypond snippets, especially for layouts or instrumentations that you use frequently. Creating Sublime snippets is pretty easy - I learned how to do it the other day. The next day, it made the difference between writing down a little musical idea I had, and forgetting it forever.

Friends have recommended Frescobaldi as a graphical alternative to all that hand coding. I haven’t been able to get it working after installing it with Homebrew, and if it doesn’t work when I install it with a package manager, I’m not going to sink much time into fixing it by hand. And anyway, one of the things I like most about Lilypond is the fact that the file format is plain text. Programmers have developed some great tools for working with plain text over the years (version control, anyone?), and given that I know how to use a few of them, it seems silly not to. Plus, I just love crafting stuff by hand like this, at least as long as it’s a hobby with few deadlines.

Next, I’d like to get the PDF side by side with the code. Preview updates automatically when you change the file, so that’s taken care of. Mostly I think I just need a bigger (another?) screen for this whole thing to work best.

Lucas Wilson-Richter

Making Beautiful Things

1. Make tiny, beautiful things daily

2. Stop fighting the ugly with more ugly

But… how?

Persisting post-novelty

Keeping stuff to look back on

Preventing regrettable outbursts

Extending the benefit of the doubt

Looks like I write about music on the internet now

#fyp @ the Edge

Data about me

Data about others

Ignoring important obstacles? Hell yes I am.

Algorithms

Hard problems I am totally glossing over right now

Filters

Ignored issues

Feeds

Ignored issues

Crawlers

Issues

Crunchers

Issues

Presenters

Issues

Summary

Writing tools for software developers

Markdown (markup language)

Joplin (note taking app)

Mermaid.js (diagrams)

Excalidraw (whiteboard, diagrams)

Reveal.js and reveal-md (slides)

Honorable mention: MkDocs (documentation)

Switching to Tufte CSS

Running a Docker container as a non root user

The Problem: Docker writes files as root

Time to be someone else

But I just want to be me!

docker-compose

Gotchas

Your user will be $HOME-less

Your user will be nameless, too

Wrapping up

Making practise tools for amateur musicals, Linux edition

Hardware

System76 Galago laptop

Focusrite Scarlett 2i2 USB audio interface

Shure SM58 microphone

M-Audio Keystation 49

Software

DAW: Ardour

JACK & Qjackctl

FluidSynth & Qsynth

a2jmidid

Foreman

Controlling Ardour from a tablet using TouchDAW and QMidiNet

Publishing

Making practise tools for amateur musicals

MuseScore

Git

Tascam stereo recorder

Audacity

GarageBand

Adjusting tempos

Welcome to Jekyll!

Using SublimeText with Lilypond

Reveal.js and `reveal-md` (slides)

The Problem: Docker writes files as `root`