A note on the making. This page was written for, with, and by AI: drafted by a language model working inside Walter OS, the system it describes, and shaped at every step by a deliberately designed human-in-the-loop. Nothing here reached you without my reading, correcting and approving it — the system proposes, I dispose. The human presence is architectural, not incidental.
The question I couldn't put down
This one started, the way a lot of my projects do, from a low-grade dread I couldn't quite reason away. I teach motion design, interactive media and creative-coding systems, and the honest version of the worry is this: what exactly is my product now that anyone can teach themselves almost anything instantly, now that a model will summarise my field in a paragraph and generate a passable outcome before I've finished setting the brief? The energy I pour into slides and lectures and lesson plans can be bypassed, compressed, served up on demand. So what is left? What is the actual product in the transfer of knowledge once the transfer itself is subsumed?
The answer I keep arriving at, and I'll grant it might be a little self-serving, is that the product was never really the slides. It is me, the way it has more or less always been. Not the information but the judgement: how would Greg tackle this brief, what would Greg do when the obvious move fails, where would Greg look when there is nothing left to Google. The interface has gone, or is going. What remains is the harder, slower thing underneath it.
Benjamin gives me a word for the distinction, and finding it is what turned a vague anxiety into a project. In "The Storyteller" (1936) he separates Information from Erfahrung (a German word for lived, integrated experience — the deep kind, as against mere data). Information is instant, verifiable and perishable; it arrives already explained and is spent the moment it is consumed. Erfahrung, by contrast, is the kind of experience that ripens into counsel and can only be passed down in the telling. The storyteller deals in Erfahrung, the modern news-machine deals in Information, and Benjamin's quiet alarm is that the cheap reproducible form drives out the other. That is more or less exactly the substitution I am watching happen to teaching in real time. An AI dispenses Information beautifully. What it cannot dispense, what walks out of the room when I do, is the Erfahrung. Walter OS is my attempt to build a vessel for that: to archive and engineer my own context so that what survives the disappearance of the interface is not a worse copy of the knowledge, but the judgement that knew what to do with it.
There is a second idea sitting right behind the first, and in Benjamin's vocabulary it is a claim about aura: the non-reproducible presence of the originator, the here-and-now that no copy carries with it. The famous argument in the artwork essay is that mechanical reproduction withers the aura. The wager of this project is the inverse, or at least the stubborn hope of one. That aura can be made to persist inside the reproducible. That you might reach for an AI and, through a deep enough substrate of context and correction, get something closer to Greg than to the generic. I am fairly sure a real Benjamin scholar would wince at me pressing the concept into service this way, but I am pressing anyway, because it names the thing I am reaching for better than anything else I have found.
A fitting return
Walter Benjamin.
Paul Klee, Angelus Novus (1920) — the work Benjamin owned, and read as his 'angel of history'.
I should admit the personal angle, because it is part of why this stuck. Benjamin turned up in my PhD; I wrote about him back then, when the questions were about media and materiality. Coming back to him now, with a system named after him and built on his theory of memory, feels less like borrowing a fashionable reference and more like a return to something unfinished. The same thinker who helped me make sense of media as a researcher is the one I am leaning on to make sense of what AI is doing to the practicing educator. That symmetry is, frankly, a large part of the pleasure of building it.
An archive that waits
The system is named after Walter Benjamin, and the name is doing real work rather than decorating the front of it. From 1927 until his death in 1940, Benjamin worked on the Arcades Project: thousands of citations and fragments he never finished organising, gathered into lettered bundles he called Konvoluts, held together by shared concern rather than by chronology or argument. Walter OS borrows that logic almost directly. Material enters through a single folder, unsorted and untriaged, whatever I happened to capture and in whatever state, and it is allowed to sit there. The system does not nag. It waits. A fragment finds a home in a bundle or a project when I decide it has one, and not before.
That patience is a design principle, not an accident. The thing is built to survive my neglect, to be returned to after months away and still make sense, to accumulate value precisely in the gaps where I am not looking. It is, by some distance, the least finished thing I have ever made, and the one I expect to be living inside the longest.
Traces, aura and the dialectical image
Underneath the folders is a small piece of Benjamin's theory of memory, taken more or less literally. The whole system reduces to two things and the relation between them, a dyad I lifted almost word for word from the Arcades:
The trace is appearance of a nearness, however far removed the thing that left it behind may be. The aura is appearance of a distance, however close the thing that calls it forth. In the trace, we gain possession of the thing; in the aura, it takes possession of us.
— Walter Benjamin, The Arcades Project (Passagen-Werk), Konvolut M
A trace is the indexical mark left by contact: the footnote, the highlight, the worn step, something you go and grab. Aura is the opposite movement, the pull a thing exerts on you, accreted from use, never something you can simply demand. This dyad (§II in the system's own glossary, a partial copy of which is appended below) is the thing the whole architecture turns on. Walter treats the edges between notes as traces, and the slow gravity that certain notes develop as aura, and most of the interesting behaviour of the system lives in honouring the difference between what you can fetch and what has to come to you.
Retrieval, when it actually works, is what Benjamin called a dialectical image: not a summary but a collision. The system's own instruction to itself puts it more bluntly than I would.
Do not harmonise these. Do not summarise. Synthesise the shock where these conflicting traces collide to produce a definitive next move.
Standard retrieval summarises; this one (the Dialectical Image, §VI) is meant to collide. I was always less interested in a tool that could answer questions, there is no shortage of those now, than in one that could occasionally produce the other thing: the shock of recognition, a present preoccupation striking an older fragment so the two become suddenly, uncomfortably legible together. The connection I didn't know I was making. The system has names for that jolt too. It calls the offered connection a Surfacing, and the faculty that throws it up the Optical Unconscious (§VI), after Benjamin's idea that a medium can reveal what the naked eye slides straight past.
How it actually works
None of this stays at the level of metaphor, and that is the part I am quietly proud of. It is a real system, and it has been months of building. The architecture is three concentric rings. At the centre is the memory itself: plain Markdown notes, vector embeddings kept in disposable sidecar files, and a graph of edges between them. Around that sits an access layer, a protocol and a schema and folder-level memory files that let any agent pick up the state cold, months later, with no warm context to lose. In the outer ring is whichever AI happens to be reading at the time, and that ring is deliberately interchangeable. The model is a guest; the memory and the protocol are the house. When the model changes, and it will, repeatedly and faster than anyone is comfortable with, nothing essential is lost. Model-agnosticism is not a feature here so much as the whole point. The intelligence is rented; the context is owned.
Underneath the dashboard above there is more plumbing than the calm surface lets on. A file-watcher sits over the vault and notices when anything lands. A scheduler wakes every few minutes and works through whatever is waiting. A handful of quiet background agents tidy and audit the memory while I am away, proposing corrections rather than making them. It runs whether I am paying attention or not, which is the only way a personal archive ever survives a busy semester.
What surprised me while building it was how quickly one model became a small fleet of them, each with a different job and a different sense. Plain text is the easy case. The harder and more interesting material is everything else I capture, and each kind needs its own reader. A vision-language model looks at images and screenshots and writes out what is actually in them. A speech-to-text model transcribes voice memos and recordings into searchable text. For sound itself, where a transcript would miss the point, an audio model and a contrastive audio-text model between them tag and describe what a clip sounds like: that this is music, that it is an acoustic guitar over a field recording, that the mood sits closer to drone than to song. A separate text-embedding model turns all of it, the words and the generated descriptions alike, into vectors, so the archive becomes searchable by meaning and not only by keyword. A local reasoning model does the distilling on top. And underneath all of them, the old workhorse ffmpeg does the unglamorous media wrangling: pulling the audio track out of a video, normalising formats, trimming a file down to size before a model ever sees it.
This sensing layer runs locally, on my own Apple Silicon machine, through Apple's MLX framework, with a second local engine alongside it for the models MLX does not carry. The distilling sits here too. That part is deliberate: the continuous, high-volume, faintly intrusive work of reading my own captures happens on my own hardware and stays there, which matters to me more, the more personal the archive becomes.
It is not all local, though, and I would rather not dress it up that way. The system runs on a kind of intelligence gradient. The cheap, repetitive, always-on work sits at the bottom, on the local fleet just described. The rarer and heavier work climbs higher: the between-session agent that tidies and audits the memory while I am away (a flâneur figure I call the Arcade Walker, §XVII, after Benjamin's stroller of the covered arcades), the slow self-improvement loop named Excavation and run by a figure called the Digger (§XXII), and the writing of outward-facing pages like this one. The very session I am drafting in now sits in that upper tier as well. That work routes out through a single budget-guarded engine that reaches first for a free cloud model and falls back to a metered one only when it has to, under a hard monthly ceiling that simply refuses to spend past its cap. So the model doing the thinking is sometimes on my desk and sometimes in someone else's data centre, and the whole thing is arranged so I can swap either one without the memory underneath it noticing. The cloud is somewhere I reach deliberately, and on a budget, not the default that everything quietly leaks into.
The journey of a note
The clearest way to explain the engineering is to follow one note through it. Say I am at my desk and something lands: a voice memo, a screenshot, a half-formed paragraph, a photograph from the field. It begins life in the Now, the working surface, the desk rather than the filing cabinet. When I am done with it there it drops into a single inbox folder, unsorted, alongside everything else I have captured.
From that point the system takes over, but only as far as reading and proposing. The watcher notices the new file. The scheduler picks it up, tidies its formatting, reads it properly, and writes a distillation: a short summary, the concepts and entities it touches, the questions it leaves open. It computes an embedding and tucks it into a throwaway sidecar so the note becomes findable by meaning and not just by keyword. Then it does the one genuinely opinionated thing it is allowed to do: it proposes a home, a destination somewhere in the projects or the bundles, with a confidence score and a one-line reason for the suggestion. And then it stops. The note sits in the inbox until I triage it, at which point I accept the suggested home, send it somewhere else, or leave it to settle. Nothing moves without me.
Raw capture is where the Benjamin frame turns practical. The Media page above gathers everything the system makes from audio, image and video across the whole vault, and its quiet subtitle is the whole thesis in four words: Erlebnis converted to Erfahrung. A raw recording is Erlebnis, mere lived-through stuff. Transcribed, captioned, its music and mood described, distilled and connected to the rest of the work, it starts to become Erfahrung, experience that can actually be drawn on later. That conversion, done patiently and in the background, is most of what the machine is for.
Human in the loop, on purpose
The part I am most insistent about is the part that is easiest to get wrong in 2026. Walter OS is not an autonomous agent, and I have no interest in it becoming one. There is no standing promise that it goes off and runs my life while I sleep. It is human-in-the-loop by design, structurally, not as a safety notice bolted on afterwards. Walter proposes: a distillation, a destination, a connection it noticed. I dispose. Nothing is asserted into the graph without me. The correction is always mine to make, and the system's job is to surface, never to decide.
That constraint is, again, downstream of Benjamin, this time of "The Author as Producer" (1934), where the task is to refunction the apparatus of production rather than be replaced by it, to take up a position within the means rather than merely supplying them. I take the political point and shrink it to my own scale: I want to operate the machine, not be operated by it. It is also why I am wary of the autonomous-everything pitch, the systems that promise to think for you and quietly hand the judgement back to the vendor. The grounding in media and production matters here. Benjamin was writing in the teeth of oppression and war about exactly this question, what the maker owes to the apparatus that reproduces the work, and that pressure is part of why I keep returning to him rather than to more comfortable recent AI hype on "knowledge management". The stakes felt real to him in a way they are starting to feel real to me.
Casting it outward
There is a closing loop I find quietly pleasing. The page you are reading is itself the system in one of its modes: the archive turned outward, the interior work cast into its exhibited, audience-facing form, what Benjamin called the work's exhibition value (Ausstellungswert) as against the cult value of the thing made in private. The studio is the cult value. This is the exhibition. The system has a name for the gesture, of course. It calls this casting-outward Exhibition-cast (§XXIV), the productive twin of an inward-looking Exhibition-watch (§XXIII) that keeps an eye on the public face of the work and reports back. The page you are reading is the cast; the watch will, in time, notice it went live. Walter holds the thinking, the site holds the result, and a small part of the system's job is to carry the one carefully into the other without flattening it on the way, to let the Erfahrung — the lived, handed-down experience — survive its own reproduction.
Not a product, not yet
I should be plain about what this is and is not. Walter OS is not something you can download or sign into. For now this writeup is the only part of it that faces outward at all; the system itself runs on my own machines and nowhere else. None of that is for secrecy's sake. The framework underneath, the protocol and the schema and the skills and the patient scaffolding, is general enough that I may well put a version of it on a public git repository one day, so that anyone who wanted to build their own archive along these lines could take the bones of it.
But the bones are not the thing. What makes Walter worth building is that it is slowly becoming a version of me: the accumulated context, the corrections, the judgement, the Erfahrung. The long aim is to use all of that to fine-tune a local model into something that genuinely answers the way I would, rather than the way a general model guesses I might. Until that exists, the framework on its own is an empty house, and the house is not the person. So the project is not deployable as the thing it is actually for, the version of me you might ask when I am not in the room, until there is enough of me gathered inside it to fine-tune on. That threshold is the real Phase X of the title: an unknown, a date I cannot yet put down, the point at which the archive has gathered enough to become the educator and not merely the filing cabinet.
Benjamin never finished the Arcades. I have come to suspect the incompleteness was not a failure of the project so much as it was the project, the form the thinking had to take. Walter OS is built the same way, on purpose: unfinished, patient, accumulating in the gaps. If it works, the thing that is left when the interface finally goes will not be a thinner copy of what I know. It will be closer to how I would have thought it through myself.
Appendix: a partial glossary
Walter keeps its own internal glossary, a small constitution of terms it has to agree with itself about. Most of it is plumbing and would bore you. What follows is a stripped-down reading copy of the part that is not: the Benjamin-anchored vocabulary the system actually thinks in, the terms that turn up throughout this page. The § numbers point back into that internal document.
- Trace (§II) — The mark left by contact: a footnote, a highlight, a worn step. Something you can go and fetch. In Walter, every link between notes is a species of trace.
- Aura (§III) — The opposite pull. Not the mark you reach for but the gravity a thing exerts on you, built up slowly from use, that you cannot simply demand. Benjamin's "appearance of a distance, however near the thing that calls it forth." Certain notes quietly accrue it.
- Strike — A mark for something that came in hot: the quote that struck, the fragment that would not settle. The one signal you set by hand at capture, which is why it lives under aura. After Benjamin's Chockerlebnis, the jolt that breaks habitual attention.
- The Dialectical Image (§VI) — Benjamin's name for meaning that arrives as a collision rather than a summary: a present concern striking an old fragment so that both light up at once. Walter's retrieval is built to collide, where most tools are built to summarise.
- The Optical Unconscious (§VI) — Benjamin's idea that a medium reveals what the naked eye glides past, the way a camera catches what we never consciously saw. Here, the faculty that notices structural rhymes between distant notes and surfaces them unbidden.
- Surfacing (§IV) — An involuntary recall the system offers up: you have not touched this in months, and it just became relevant to what you are doing now. You accept it or wave it away. It is never asserted on your behalf.
- Capture Modes (§VIII) — the register a note is captured in, running from raw, unworked scrap to finished, considered thought.
- The Storyteller — The finished, narrative end of that range, from Benjamin's 1936 essay. The teller passes down Erfahrung, lived and integrated experience, where the modern news-machine passes down Information. The distinction this whole project hangs on.
- The Flâneur, or the Arcade Walker (§XVII) — Benjamin's stroller of the covered arcades, wandering without a fixed destination and noticing what calls to him. In Walter, the agent that walks the memory between sessions and keeps it honest.
- Excavation, and the Digger (§XXII) — After Benjamin's instruction that one "must not be afraid to return again and again to the same matter." The slow loop that digs back through the archive to improve the system's own capacities.
- Profane Illumination (§XIX) — Benjamin's secular, materialist version of mystical insight: the worldly yes, that's it when something clicks into place. In Walter it is always the human's moment, never the machine's.
- Exhibition value (Ausstellungswert, §XXIII) — From the artwork essay: the work in its displayed, distributable, audience-facing form, as against the cult value of the thing made in private. A website is almost pure exhibition value.
- Exhibition-watch and Exhibition-cast (§§XXIII–XXIV) — The paired figures at the threshold of the public. One watches the work's exhibited face and reports back inward; the other casts the archive's own material outward into pages like this one.