Final Project

Spinphony

Spinphony turns computer audio into live motor music.

FRDM-KL46ZPlatform
4× NEMA17Motors
Live STFTAnalysis
230,400Baud
20 kHzControl Rate

What is Spinphony?

Spinphony is an embedded audio-to-motor system built on the FRDM-KL46Z microcontroller. Instead of recording through an analog microphone, the current system listens to computer audio routed through a virtual audio cable, analyzes it live on the host, and streams motor commands to the board as the song plays.

On the laptop, system audio is captured at 48 kHz, mixed to mono, normalized, resampled to 8.4 kHz, and analyzed with a Short-Time Fourier Transform. The Python code turns the notes it finds into motor frequencies and sends compact serial packets to the board.

The FRDM receives those frames over USB-UART, buffers them, and drives four NEMA17 stepper motors with a 20 kHz phase-accumulator loop. The live visualizer helped us see the spectrum and the motor notes while a song was playing.

Audio Input
Computer Audio Stream
Songs are routed through VB-CABLE so the Python pipeline can process audio directly from the player.
Signal Processing
STFT Frequency Detection
Hann-windowed STFT frames, note scoring, harmonic rejection, and short-note filtering select up to four pitches for the motors in real time.
Actuation
4× NEMA17 via DRV8825
Each selected note drives one stepper motor at a proportional step rate via a 32-bit phase accumulator, vibrating the motors.

Pipeline Architecture

Clean computer audio is analyzed on the host, then streamed as motor frames to the embedded controller.

HOST AUDIO ANALYZE + STREAM ACTUATE Music Player Browser / app output Virtual Cable Loopback recording 48 kHz Blocks Stereo to mono Volume Normalizer Rolling peak level Resampler 48 kHz → 8.4 kHz Note Tracks → Motor Frequencies up to four detected pitches per frame STFT Engine 2048-sample Hann window Serial Packetizer 20-byte frames @ 230,400 baud Motor Controller Phase accumulator ×4 DRV8825 Drivers ×4 GPIO STEP / PORT E NEMA17 Motors ×4 Audio via vibration Python visualizer monitors spectrum USB-UART carries motor frames

Demonstration & Samples

Samples
Demon Slayer - Op 1 · 3:58
Open full playlist
  1. 01 Spinphony - A speaker made of motors Demo4:07
  2. 02 Demon Slayer - Op 1 3:58
  3. 03 Naruto - Silhouette 1:33
  4. 04 One Piece - "We Are" 1:40
  5. 05 Pirates of the Caribbean 1:25
  6. 06 Gravity Falls 0:45
  7. 07 Naruto - Blue Bird 1:41
  8. 08 Your Lie In April 1:36
  9. 09 Neon Genesis Evangelion - Op 1 1:26
  10. 10 K 0:15
  11. 11 Super Mario Main Theme 1:18
  12. 12 Star Wars - Main Theme 1:28
  13. 13 Mario Kart 2:53
  14. 14 Avicii - Waiting For Love 1:21
  15. 15 Geometry Dash - Back on Track 1:55
  16. 16 Fall Out Boy - Centuries 3:39
  17. 17 Death Note - L's Theme 2:57
  18. 18 Bad Apple 2:57

How It Works

Hardware

The embedded side is built around the FRDM-KL46Z (ARM Cortex-M0+ at up to 48 MHz). The board receives motor frames from the host computer over USB-UART, and four DRV8825 stepper motor drivers receive STEP signals via GPIO port E pins 16–19.

Power is distributed from a 24V supply. An LM2596 buck converter steps 24V down to 5V for the FRDM's VIN rail; the FRDM then supplies 3.3V logic to the DRV8825 inputs. Each DRV8825 is decoupled with a 100 µF capacitor across VMOT and GND. Motor direction is fixed; only step frequency varies.

Hardware assembly

Startup & Timing Constants

The firmware starts through App_init(), which calls the generated board setup, initializes the motor and serial modules, then starts the motor timer. The main loop stays small: it services serial input while the PIT interrupt handles motor stepping.

void App_init(void) {
    // first init board at custom rate and motor and serial
    BOARD_InitBootPins();
    BOARD_InitBootClocks();
    BOARD_InitDebugConsole();
    Motor_Init();
    SerialStream_Init();

    // Start stepping after the hardware and serial input are ready
    Motor_StartTimer();
}

void App_run(void) {
    // Main loop keeps checking serial, motors interrupt this
    while (1) {
        SerialStream_Service();
    }
}

The shared constants define the timing used by both serial and motor code:

// using higher than 15MHz for both timer and serial math
#define RUN_HZ           20971520

// Motor interrupt rate and other info
#define MOTOR_CONTROL_RATE          20000u
#define MOTOR_CONTROL_RATE_HZ       MOTOR_CONTROL_RATE
#define NUM_MOTORS                  4u
#define MOTOR_STREAM_BUFFER_FRAMES  128u
#define MOTOR_STREAM_START_FRAMES   32u

Motor Control - Phase Accumulator

Each motor is driven with a 32-bit phase accumulator at 20 kHz. The current speed[motor] increment is added to phase[motor] every timer tick; overflow emits a STEP pulse.

for (motor = 0; motor < NUM_MOTORS; motor++) {
    uint32_t oldPhase;

    // Step pulses only set high for one timer tick
    if (stepHigh[motor]) {
        step_set((uint8_t)motor, 0u);
        stepHigh[motor] = 0u;
    }

    if (speed[motor] == 0u) {
        continue;
    }

    // phase overflow means step
    oldPhase = phase[motor];
    phase[motor] += speed[motor];
    if (phase[motor] < oldPhase) {
        step_set((uint8_t)motor, 1u);
        stepHigh[motor] = 1u;
    }
}

Step frequency follows the streamed phase increment. With 800 microsteps/rev the acoustic fundamental matches the step frequency. Pre-computed frames stream into a 128-slot circular buffer; the ISR pops frames without blocking the main loop.

Constant Value
MOTOR_CONTROL_RATE_HZ 20,000 Hz
MOTOR_STREAM_BUFFER_FRAMES 128
MOTOR_STREAM_START_FRAMES 32 (pre-roll before playback)

Serial Stream & DMA

Motor frames are received over UART0 at 230,400 baud. DMA fills a 20-byte buffer, and SerialStream_Service() copies the finished chunk, restarts DMA, and feeds each byte into the packet parser.

void SerialStream_Service(void) {
    uint32_t status = DMA0->DMA[UART_RX_DMA_CHANNEL].DSR_BCR;
    uint8_t i;

    // Error, restart DMA
    if ((status & (DMA_DSR_BCR_BES_MASK | DMA_DSR_BCR_BED_MASK | DMA_DSR_BCR_CE_MASK)) != 0u) {
        uart_dma_start();
        return;
    }

    // DMA is done collecting packets
    if ((status & DMA_DSR_BCR_DONE_MASK) == 0u) {
        return;
    }

    for (i = 0u; i < UART_RX_DMA_CHUNK_SIZE; i++) {
        parseBuffer[i] = dmaBuffer[i];
    }

    // restart DMA
    uart_dma_start();

    for (i = 0u; i < UART_RX_DMA_CHUNK_SIZE; i++) {
        parse_byte(parseBuffer[i]);
    }
}

The packet parser waits for sync byte 0xA5, checks the XOR checksum, converts four little-endian increments, and pushes valid frames into the motor stream queue.

STFT Frequency Analysis

Audio is analyzed live in Python using a Short-Time Fourier Transform. The input is captured from a loopback device at 48 kHz, converted to mono, normalized, and resampled to the 8.4 kHz analysis rate before note detection.

source_block = recorder.record(numframes=CAPTURE_BLOCK_SIZE)
source_block = mono_float32(source_block)
rms = float(np.sqrt(np.mean(source_block * source_block))) if len(source_block) else 0.0

if not started:
    if rms < CAPTURE_START_RMS:
        continue

    started = True
    audio_status = "Audio started"
    if ui_state is not None:
        publish_status(ui_state, audio_status, serial_connected)

process_block = volume.process(source_block)
target_samples = resampler.process(process_block)
found_frames = notes.push_samples(target_samples)

for found_freqs in found_frames:
    motor_freqs = short_note_filter.push_and_get_output(found_freqs)
    if motor_freqs is None:
        continue

    ticks, new_serial_connected = motor_serial.send_frame(motor_clock, motor_freqs)

The analyzer scores MIDI-note candidates across Hann-windowed frames, filters harmonics and very short notes, then emits up to four motor frequencies per frame. The live UI displays the spectrum and the selected motor frequencies as they are sent.

Frequency → Phase Increment Mapping

Converting a target motor frequency (Hz) to a 32-bit phase increment is computed on the host before transmission:

MOTOR_CONTROL_RATE = 20000
PHASE_SCALE = 1 << 32

STREAM_SYNC = 0xA5
STREAM_PACKET_SIZE = 20


def freq_tracks_to_phase_increments(freq_tracks):
    increments = np.rint(freq_tracks * PHASE_SCALE / MOTOR_CONTROL_RATE)
    return increments.astype(np.uint32)

Detected frequencies are tracked across frames and assigned to the four motors. Each serial packet includes a duration in motor ticks plus four phase increments, so the firmware can keep the timing steady even while new frames arrive.

packet = bytearray()
packet.append(STREAM_SYNC)
packet += struct.pack("<H", duration_ticks)

for inc in phase_increments:
    packet += struct.pack("<I", int(inc))

checksum = 0
for value in packet:
    checksum ^= value

packet.append(checksum)

Verification & Validation

Verification focused on the live host-audio pipeline, serial transport, and motor playback behavior.

Completed

Hardware

  • USB-UART link verified between the host computer and FRDM-KL46Z at 230,400 baud
  • All connections soldered and continuity-tested; common ground verified across FRDM, drivers, and 24V supply
  • LM2596 outputs stable 5V under motor load
  • Each DRV8825 independently verified to drive motor at the correct pitch for known test frequencies
Completed

Software

  • Loopback audio capture confirmed with virtual-cable devices and common desktop players
  • 48 kHz input blocks normalize and resample to the 8.4 kHz STFT analysis rate without dropping frames
  • STFT detects single and dual tones within ±2 Hz across 2–3 octaves on known WAV files
  • 20-byte serial framing and XOR checksum validated via loopback
  • STFT output matches actual piano notes across the full tested range
  • Phase increment accuracy confirmed with a frequency counter on motor output

References & Datasheets

The following resources were referenced during design and implementation.

FRDM-KL46Z Reference Manual (KL46P121M48SF4RM)
MCG clock modes, TPM timer, UART0, eDMA, and GPIO registers
NXP ↗
DRV8825 Stepper Motor Driver Datasheet
STEP/DIR timing, microstepping configuration, current-limit adjustment
TI ↗
NEMA17 Stepper Motor Datasheet
Step angle, holding torque, coil resistance, rated current
PBC ↗
Python SoundCard Documentation
Loopback recording API used to capture computer audio for the live pipeline
SoundCard ↗
NumPy FFT Documentation
Windowing, overlap-add, and spectral leakage for the Python STFT analyzer
NumPy ↗
LM2596 Buck Converter Datasheet
Output voltage setting, inductor/capacitor selection for 24V → 5V
TI ↗

The Team

Host audio, signal processing, firmware, motor hardware, and web development.

IA
Ibrahim Ahmed
DSP & Audio Pipeline
  • Loopback audio capture and normalization
  • DRV8825 stepper driver setup and configuration
  • STFT note detection and frequency extraction algorithm
GW
Geneustace Wicaksono
Firmware & CAD
  • UART/DMA serial frame receiver
  • 20 kHz motor control loop integration
  • CAD design for the motor assembly
IA
Ibrahim Alyamani
Systems & Integration
  • System-level design and hardware integration
  • 3D printing and assembly
  • Website development and documentation

Use of Generative AI

AI was used for system validation, consulted on very specific design choices that were made and ensuring we weren't missing anything key to our project. Additionally, AI was used in styling this website and verifying documentation.