The Teesside Photographer

Synaesthesia

Photographs scored as music. Each folder of images from a walk through Middlesbrough becomes one movement of a piece — the camera settings and the light in every frame decide the notes. Nothing is composed by hand: the photographs play themselves.

A deterministic image-to-music pipeline: EXIF metadata and pixel statistics are extracted per photograph, mapped through an editable table onto musical parameters, and rendered as MIDI plus a rough WAV preview. One image is one bar; one folder is one movement. The whole system is a 300-line Python script and a Markdown file of numbers.

How it works

Every digital photograph secretly carries its own making: which lens was on, how wide the aperture was, how fast the shutter, how sensitive the sensor, and what time of day it was. The picture itself carries more — how dark or bright it is overall.

Synaesthesia reads all of that and treats it as a score. A wide lens plays busier bars; a dim frame chooses a darker scale; an evening shot sits lower. The photographer's decisions at the moment of capture become the music's decisions, one image at a time, in the order they were taken.

Per image the script extracts six readings: focal length, f-stop, shutter speed, ISO and capture hour from EXIF, and mean luminance from the pixels. Focal, f-stop, shutter and ISO are scaled logarithmically — in stops, the way photography actually works — while hour and luminance map linearly.

Each reading is clamped into an input range and mapped to an output range for one musical parameter. The mapping lives in mapping.md, a plain Markdown table the script parses at run time — the file is the instrument's front panel, editable without touching code.

Six readings, six drivers

Focal length → density A wide lens takes in more of the world, so it plays more notes per bar. 16–200 mm → 9–2 notes per bar, inverted and log-scaled.

F-stop → dynamics Wide open and intimate plays quietly; stopped down plays at full strength. f/1.4–f/16 → MIDI velocity 52–112, log-scaled in stops.

Hour → register Morning light sits high; night sinks low. 04:00–23:00 → +12 to −12 semitones on the movement's root.

Shutter → note length A fast shutter is staccato; a long exposure is a held note. 1/4000 s–1/2 s → 0.3–3.0 beats, log-scaled.

ISO → jitter Grain is human mess — high ISO makes the timing and volume wobble. ISO 100–12800 → 0–1 timing/velocity wobble plus extra detune.

Luminance → mode The darker the frame, the darker the scale it chooses to play in. Mean luma 20–235 → position 0–1 along the movement's mode list, dark→bright.

Every mapping is one row in mapping.md — in-range, out-range, nothing else.

The movements

The photographs were sorted into folders by feeling, not by place: the architecture of power, authority and bureaucracy; the flat evidence of the documentary frames; the streets of neglect — forgotten, powerless, hopeless. Each folder becomes a movement with its own character: its own home note, its own scales, its own tempo.

Power marches on a rigid grid over a low drone. Evidence is sparse and neutral — nearly a third of its notes are silence. Neglect barely keeps time: slow, detuned, notes left ringing until they die.

Per movement the palette sets root note, an ordered dark→bright mode list (phrygian, aeolian, dorian, mixolydian…), tempo, quantise (1.0 = machine grid, 0.0 = human drift), silence probability, oscillator detune in cents, decay tail and an optional root drone.

The tuning is the meaning: power runs at 92 BPM, quantise 1.0, drone on — bureaucracy keeps perfect time. Evidence drops to 60 BPM with 30% silence and no drone. Neglect falls to 54 BPM, quantise 0.35, 40% silence, 18 cents of detune and a 3.5-second decay — the movement is audibly falling apart, because that is what the streets in the frames are doing.

One photograph, one bar

Power, authority, bureaucracy — the frame used as the worked example

Power-authority-bureaucracy · 24 mm, f/4.5, 1/40 s, ISO 125, 17:34

This frame was shot wide, at quarter-to-six in the evening, on a slowish shutter at base ISO, and it is a dark picture. So its bar is busy (wide lens), pitched slightly low (evening), made of longer notes (slow shutter), played precisely with no wobble (clean ISO), in a dark scale (dark frame). The next photograph plays the next bar differently.

Readings → parameters: focal 24 mm → ~8 notes in the bar; f/4.5 → velocity ~70, on the quiet side; hour 17.6 → a shade below the root register; shutter 1/40 s → mid-length notes around a beat; ISO 125 → jitter ≈ 0; mean luminance 66/255 → low on the mode list, so phrygian. All of it lands on the power movement's C2 root, over the drone, at 92 BPM.

The found images

Two of the movements — the Gresham demolitions and the derelict Crown cinema — are built from found and archive images. Those files carry no camera settings at all: no lens, no shutter, no hour.

They are given a photographer's neutral defaults for everything the file cannot say — but their brightness is real, read from the pixels. Their music is driven by the light alone: the darker the ruin, the darker the scale.

No-EXIF images take f/8, 1/125 s, ISO 400, 35 mm, midday — deliberately inert values that flatten five of the six drivers to their midpoints. Only luminance→mode stays live, so found imagery produces music of pure tonality: no density or dynamic shape, just brightness choosing the mode bar by bar. Ordering falls back to filename where there is no capture time.

Hearing neglect

Neglect, forgotten, powerless, hopeless — terraced street frame from the movement

Neglect-forgotten-powerless-hopeless · the longest movement

The neglect movement is the heart of the piece, and the system's tuning does the emotional work. Its clock drifts. Almost half its notes never sound. What does sound is slightly out of tune with itself, and each note is left to decay for seconds before the next arrives. You are not told the streets are falling apart — you hear it.

Silence is treated as material: at probability 0.40 a scheduled note becomes a rest, so the bar structure survives while its content erodes. Detune (18 cents) beats against the fundamental; quantise 0.35 lets onsets drift off-grid; the 3.5 s decay overlaps successive notes into a smear. The same parameters at the other extreme produce the power movement — one mapping table, opposite meanings, numbers only.

Notes

The whole instrument is a small Python script and one editable text file of numbers. Change a number, run it again, and the same photographs play a different piece. The images in this showcase are the frames used to tune each movement's sound; the recordings alongside them are what those frames play.

Python, standard library plus Pillow only: EXIF via Pillow, WAV synthesis with wave and struct, MIDI written byte by byte. The MIDI files are the deliverable — voice them in any DAW; the WAVs are previews from a deliberately bare sine synth, marked in the source as the honest shortcut it is. Judge the notes, not the sound.

See more work →