Marble music machine: how we make one marble play Für Elise

How our marble music machine videos are made: one marble falls through glass, kalimba tines, bells and tiles, and every hit is a note of Für Elise, on its frame.

A phone showing Steve, Plumb's teal marble, on a wooden rail, beside a music staff whose notes are coloured marbles with dotted hops from one to the next

Our newest reels for Plumb have no puzzle in them. One marble rolls off a rail and falls through a machine of glass tubes, kalimba tines, hand bells and xylophone tiles. Every bounce plays the next note of Beethoven’s Für Elise. On the last note it drops into its hole on a real Plumb board.

Two frames from the reel side by side: Steve, a teal marble with googly eyes, sitting on a wooden rail in a pastel room, and Steve inside the teal ring of his hole on a cream Plumb board

The marble is Steve, Plumb’s teal marble with the googly eyes. It started on 14 November with a reel I came across: a 66-second marble music machine where an orange marble falls through bells, cymbals and keys, and people guess the songs in the comments. We wanted our own, with Plumb’s marbles and a Plumb board at the end. Here is how a song turns into a machine like that.

Not our idea

Marble music machines are older than us. Martin Molin of the band Wintergatan built a real one, a hand-cranked music box that drops 2,000 steel marbles on a vibraphone, a bass guitar and drum sounds. Its video came out in 2016. Long before that, Wayne Lytle’s Animusic made computer animations where the music’s own notes drive what you see. In “Pipe Dream” (2001), balls fly out of pipes and land on instruments, each one on its note.

Ours sits between the two. Like Animusic’s, our machine lives in a computer and is built from the music. Like Wintergatan’s, it is meant to look like something you could build, in wood, glass and brass.

The music comes first

We start with the notes, not the machine. Für Elise goes in as notes and lengths, at a speed where every short note lasts exactly 7 frames of the video, at 30 frames a second. So every note lands on a whole frame, and we know the exact frame where the marble must hit something.

Between two hits the marble only falls. If it leaves one hit at the point p₀ and must reach the next one at p₁ exactly Δt seconds later, it has to leave with the velocity

v₀ = (p₁ − p₀) / Δt − g Δt / 2

where g is gravity, pointing down. It is the usual formula for a thrown ball, solved backwards: we know where it must land and when, and it tells us how to throw.

Then each instrument is turned to send the marble that way. A bounce only pushes straight out of a surface, so the instrument has to face along the difference between the marble’s velocity going out and coming in. Last, we check how bouncy each hit would have to be, so that no glass tube throws the marble back harder than it arrived.

A chart titled Every note is a hit, on a whole frame. The first seventeen notes of Für Elise sit in a row along the top, E, D sharp, E, D sharp, E, B, D, C, A, then C, E, A, B, then E, G sharp, B, C. Under each note is a coloured dot, and a teal line hops from dot to dot, with two dashed gaps marked rest: a chute, over a time axis from 1 to 5 seconds

Where the music rests

A long note followed by a short one cannot be two plain bounces. The second hit would have to give the marble more energy than it got. So where the music rests, the marble lobs into a short wooden chute that catches it softly, with no sound, rolls along it and drops onto the next instrument right on time. The machine breathes where the music does.

When the machine gets stuck

For each note our planner tries a few places for the next instrument. For each one it follows the marble’s whole path and checks that it clears every bell, tube, rail and post by at least 12 mm, and it takes the first one that fits. When nothing fits, the trouble is usually where the last hop left the marble. So the planner goes back one note, tries that one somewhere else, and carries on.

Fast notes need small instruments in a row. At 5 or 6 frames a note the marble moves only 15 to 25 cm between hits, so tines, tiles and tubes fit, but a steel pan big enough to see does not.

Für Elise is 120 hits on four instruments in turn, and one last note as the marble drops home.

Four tall frames from the reel: Steve passing a spiral of pastel glass tubes, Steve striking a lit kalimba tine, Steve on a rail beside hand bells, and Steve on a row of glowing xylophone tiles

Every note is made, not recorded

There are no recordings in the reels. Every note is made from numbers, on the frame it is hit. A bell’s sound is a handful of pure tones, its modes, each ringing at its own pitch and dying away at its own rate. This is called modal synthesis, and Julius O. Smith’s book Physical Audio Signal Processing explains it well. Glass, tines, bells and tiles each get their own set of modes, so they sound like different things.

Then a check listens to the finished soundtrack, finds every strike and compares it with the frame of its note. In Für Elise the strikes land a median of 1.3 ms from their frames. The one it could not find is note 58, inside a trill on the tines, which rings into the note before it and rises only 5 dB.

Drawing it

The first machine, a 19-second Ode to Joy test on 15 November, was rendered in Blender at 8.5 seconds a frame. That was lovely and slow, so the same machine moved to three.js, a free 3D library for the web, drawn in Chrome on our laptop’s graphics chip. The first three.js test drew a frame in 1.19 seconds, about five times faster, and looked close enough on a phone.

Each frame of the video is really 96 pictures averaged together, each a little apart in time and in the lens, which gives the motion blur, the soft background and the soft shadows. Für Elise took 80 minutes to draw on the laptop, about 3.6 seconds a frame. The newer reels are drawn on rented graphics cards in the cloud, in a few minutes each. Its room is a pastel studio in the morning, with a low sun laying the window’s light across the floor. The reflections in the glass and metal come from a 360 degree photo of a real studio, Brown Photostudio 02 by Sergej Majboroda, on Poly Haven.

Two things we learned on the way:

  • Never open dark. The first frame decides whether people stay, and Shorts and Reels play on black. So even a night room opens with its lights on the marble.
  • Blurred lights need more pictures. At 32 pictures a frame, a room full of out-of-focus bulbs looks speckled. At 96 it is close to smooth.

There are no words over the picture. Our name and “follow” stand in the room as 3D letters instead, placed where the camera passes them.

Five tall frames from other music machine reels: a log cabin full of fairy lights, an old toy shop with shelves of wooden toys, a floor of star-shaped tiles seen from above, a pink and cream sweet shop, and a frozen lake at dusk with pine trees

How they are doing

The first one, Ode to Joy, went up on 15 November. On YouTube Shorts it had 1,282 views in its first day, and 52.6% of the people it was shown to stayed to watch. Our best reel before it kept 35.5%.

So there are more on the way, each with its own song and its own room. Steve has now played more Beethoven than I have. Every tune is in the public domain, every sound is our own, and every one ends with a marble dropping home on a Plumb board. Which song should the marble play next? Tell me at hello@chaosmos.studio, and if you want to roll Steve home yourself, he is in Plumb.

Related: Backing up our videos to Telegram, eight connections at a time, where these reels are kept.

Sources

  1. Wintergatan. Marble Machine (music instrument using 2000 marbles), 2016.
  2. Wikipedia. Animusic.
  3. Ludwig van Beethoven. Für Elise, WoO 59, on IMSLP.
  4. Julius O. Smith III. Modal Representation, in Physical Audio Signal Processing.
  5. The three.js authors. three.js.
  6. Sergej Majboroda. Brown Photostudio 02, on Poly Haven.

On this page