Generative AI declaration: The making of this work involved sustained use of generative AI tools, specifically Claude (Anthropic) and Gemini (Google), as CLI-based development agents. In writing about the work, AI assisted with structural feedback, grammar, and clarity. All creative direction, intellectual content, and the final work remain my own.
The Mangrove MIDI Platform began as an attempt to imagine the hidden flow patterns of mangrove ecosystems, down to the activity of sulphate-reducing bacteria in the mud. It is grounded in field research at the Minnamurra River estuary (Dharawal Country, NSW South Coast) and Ash Island in the Hunter River (Worimi and Awabakal Country), where bioelectric sensors attached to mangrove leaves and roots captured signals that were converted into MIDI data for visualisation.
At its core the platform is a real-time particle synthesiser. Incoming MIDI drives a 3D visual system in which every note spawns particles whose physics respond to velocity, pitch, and duration, so the data is felt as movement rather than read as a graph. The aim is a kind of micro-choreography: making ecosystem processes that usually sit beyond perception both visible and audible. It is built on Three.js for WebGL rendering and Tone.js for polyphonic synthesis, using the Web MIDI and Web Audio APIs, and it accepts both live MIDI input and file playback.
Imaging the invisible
The platform is a technical proof-of-concept, and an extensible browser-based foundation for audiovisual performance instruments and MIDI data visualisation. Part of what it tests is methodological: how CLI-based, AI-assisted co-coding inside a command-line development environment can accelerate the prototyping of web-based creative tools, and how that working process can itself be captured as part of practice-led research.
The deeper experiment is one of translation. The piece asks how bioelectric data from living mangrove systems can be turned into computational sound and image, and how sensor readings can be mapped to particle physics in a way that meaningfully suggests ecological flow rather than arbitrary motion. Working through that produced repeatable methods and shared code for browser-based MIDI-to-visual translation, a viable architecture for real-time biodata visualisation using accessible web technologies, and a contribution to bio-sonification, the practice of giving ecological signals an audible form.
Connections
The platform is part of preliminary research toward Remediation in the Rhizosphere, an ongoing inquiry into how invisible ecological processes might be made perceptible through digital audiovisual means, work that the Cymatic Swell Visualiser carries from the estuary out to the open swell. As groundwork, it also serves as a foundation for future audiovisual performance instruments and installation works, and as a teaching example for creative coding that uses contemporary, AI-assisted development tools.
How it was made
The platform was developed through CLI-based, AI-assisted co-coding, an iterative process using Claude and Gemini as development agents inside VS Code's integrated terminal. Working conversationally, sharing code examples, linking to precedents from JavaScript framework galleries, and feeding back browser console errors, the build progressed across more than seventy HTML iterations of tests, aesthetic trials, and bug fixes between July and August 2025, peaking at more than thirty versions in a single day. The final rendering and behaviour of the particles, balanced against the browser's memory economy, became the key points of refinement.
Project links
- Live platform: midimangroves.pages.dev
- Data deposit (Zenodo): doi.org/10.5281/zenodo.19331714
Created 2025.