Every weight started as random numbers on one desktop machine and was trained on public-domain books, a school curriculum, and its own conversations. No GPT. No Claude. No pretrained anything. The tokenizer is ours, the weights are ours, the memory is a file you could read.
For scale: GPT-2 small, the smallest model OpenAI ever released publicly, was 124M parameters — Ember is about a quarter of that. GPT-3 was 175 billion, roughly 5,900× Ember. The sizes of GPT-4 and everything after are not published, but they are far larger again.
Trained on a single consumer graphics card — one desktop machine, minutes to tens of minutes per run, not a datacentre. Answering needs no graphics card at all: it runs on the processor, so it will serve from a cheap VPS or an old laptop. That second number is the one that decides whether a thing like this can actually belong to somebody.
It didn't learn English from scraped internet. It learned from a purpose-built school curriculum written in-house — kindergarten through AP, already graded, so the easy language comes first the way it would for a child — plus 59 public-domain classics in full, and conversations manufactured on this machine. The reading order is part of the design, not an accident of what was available.
Most language models have never looked at anything. They learn that "apple" is applish because of the words that surround it. Ember learned the word and the picture together, from emoji — because a committee already distilled "appleness" into one canonical drawing, and a thousand photographs of apples just look like balls and peaches.
Shown an apple drawn in an art style it had never trained on, it names it correctly. And its word for "apple" sits +0.772 closer to what its eyes do than chance allows. The seeing and the saying are the same space.
The name unpacks to what it's made of. Nothing here is a black box you have to take on faith — every layer is a file on a disk you can open, read, and argue with.
A dual encoder trained on the same concept drawn by two independent art teams, so it has to learn the idea rather than the picture. It names an apple drawn in a style it never trained on.
A drawer of chips with typed relations between them. Facts carry a source and a confidence; a "never" can outvote a "yes" without erasing it. It grows heavier where it's used, like a path worn into grass.
A walker crosses the typed edges and composes answers nobody handed it. Told a fuji is an apple, an apple is a fruit, and a pie is made of fruit, it can tell you a fuji belongs in a pie — and show the chain.
A from-scratch transformer with its own byte-level tokenizer, whose word embeddings are anchored to what the eyes do. When memory holds the answer it is assembled from the stored facts instead of generated. Text only — there is no voice here, and no plans to pretend otherwise.
Ember's memory is not a pile of numbers. It's a drawer of chips: a six-sided coin with the word and its picture on the face, and the six sides carrying the word itself once — that's the bright one — plus five things it is bound to. Every side is a row you can read, correct, or tear up by hand.
Chips stack. A concept reached for often grows a taller stack, each chip turned one slot from the one beneath it, so how much something matters is a physical height rather than a number in a column.
This is a live chip out of the running drawer, not a diagram of one. Click a name to pull a different one.
When a big model is wrong about you, the mistake is smeared across billions of weights and nobody can point at it. When Ember is wrong, the mistake is one line in a table with a source and a confidence attached, and deleting it fixes the problem for good.
Facts carved by a person outrank facts a machine guessed. A "never" can outvote a "yes" without erasing it — the losing side stays in the drawer as history. That is what a memory you own actually looks like.
Ember is a work in progress and says so. She is a fraction of a percent of the size of the models you are used to, and it shows. What she has instead is that every part of her is accounted for.
| Can | |
|---|---|
| Write fluent English sentences | yes |
| Recognise 1,884 concepts by sight, across art styles it never trained on | yes |
| Remember facts across restarts, and show its work | yes |
| Answer a question by reasoning over what it was told | yes |
| Stay reliably on the topic you raised | not yet — in training |
| Write code, do maths, reason about the world | no, and it is not trying to |
| Replace a large model | no |