Intuitive music creation tool made for the Designing User Interfaces using Emerging Technologies course (DUIET) at TU/e.
This device uses an overhead camera to track the position of ArUco tagged cubes. Their locations are mirrored on the screen and colored lines are shown. Rotating the cubes lengthens the line logarithmically.
As a line sweeps from left to right, every time it passes over the colored trails, it plays a sound whose pitch is decided by the vertical location of the cube, the relative time based on the horizontal location and the duration based on the length of the line (rotation).
Different colored cubes represent different instruments. The three colors here play square, sine and sawtooth waves, with limitless expansion to natural instruments possible.
The four buttons are used to change the BPM (±), to pause the music and, the most interesting one, to “hold” the notes: until the button is released, the color of the trails gets lighter and they are stuck in place, no longer controlled by the cubes, allowing users to still have the notes playing while they create a new arrangement, for seamless transitions and interesting effects.
Due to the volatility of life, not a lot of artifacts remain. I was unable to snap enough pictures before disassembly, so my non-designer friend had to do his best while I was on my plane.
Hardware is simple. A generic 7″ IPS LCD, a STM32 board in HID mode with 4 connected switches and a no-name USB camera are connected to a USB-C dock, all hidden within the case, allowing a computer to interface via a single USB-C cable.
The case is fully 3D printed and doesn’t serve a designerly purpose, other than holding all the parts together in an aesthetically pleasing acceptable manner.
Read a short paper about the project here. Code and 3D models are available on GitHub.
