Documentation

Two minutes to a memory.

One install, one setup command, every surface below. Everything marked ships now runs locally and free. Everything marked cloud needs the hosted connector, which is in waitlist.

Install

Requirements: Node 18+, plus either a Claude subscription, where the assistant writes each memory itself and no key is needed, or an Anthropic, OpenAI or Gemini key. Recall never needs a key.

terminal
$ git clone https://github.com/Kerneta/daidocs daidocs-app
$ cd daidocs-app
$ node setup.js
Installing dependencies (first run only)…
 Claude Desktop: daidocs-mcp registered. Restart Desktop to load
 Claude Code: .mcp.json written in this folder
 SessionStart hook: your memory index loads at the start of every session
 Stop hook: sessions save themselves as you work, with no API key
 SessionEnd hook: every Claude Code session saves itself
Not installed here: cursor, windsurf, codex, cline, continue, zed.
If you add one later: node setup.js --client <name>
 reading protocol installed for 4 targets: Claude Code, Codex, Gemini CLI, Cursor
 .dai file type registered, for your user account only
 Observer: anthropic:claude-opus-5 (free on your subscription; change it with --observer)
Your existing chat history is not converted yet. When you want it: node daidocs.js convert

Done. Your memory lives in plain files at: ~/DaiDocs

The clone is named daidocs-app on purpose: plain git clone makes a folder called daidocs, and the default memory store is DaiDocs, which on Windows and macOS is the same folder. A clone made from your home directory would land on your own memory. Setup refuses to run from inside the store if it happens anyway. Each line is its own command. setup.js installs the dependencies on its first run, so there is nothing to install first and nothing to join: Windows PowerShell 5.1, still the default shell on Windows, treats && as a parse error. npm run setup does the same thing, but node is the one to reach for on Windows, where PowerShell refuses to run npm at all until its execution policy is changed. From the folder above the clone, npm --prefix daidocs run setup does the last two lines in one.

Setup asks nothing: it finds every surface you have and configures all of it. node setup.js --status is the settings view afterwards, listing each switch with the command that changes it, and node setup.js --ask brings back a question per surface for anyone who would rather choose. The one thing it never starts on its own is converting your existing history, which can run for a while and, with an API key, costs money: node daidocs.js convert when you want it, and it is worth wanting: it reads your Claude Code sessions, anything the hook captured but has not converted, or a folder of exports you point it at, shows what it found, asks which ones, and quotes the worst-case cost before any paid call, which on a Claude subscription is nothing. Each session lands in the folder it came from. For an MCP client other than Claude, one command each, node setup.js --client codex and the same for cursor, windsurf, cline, continue and zed, with --client generic --config <file> for anything else: nothing has to be pasted into a config by hand. Note the fourth step: by default the installer registers the .dai file type with your desktop, which writes user-scope keys under HKCU\Software\Classes on Windows and MIME and icon files under ~/.local/share on Linux. Say n, or pass --unregister later, and nothing is left behind.

setup.js is idempotent, so run it as often as you like. Every config it touches is backed up first as *.daidocs-bak, and your API key is stored only in your OS user environment, never written to a file. The flags, in full:

FlagWhat it automates
--desktopregisters the server in Claude Desktop's config
--codewrites .mcp.json for Claude Code in your project
--hookthe end-of-session hook: every Claude Code session saves itself when it ends
--autosavethe in-session hook: saves every 4,000 new tokens as you work, written by the assistant you are talking to, so no API key and nothing billed
--contextthe start-of-session hook: your memory index loads at the top of every session
--instructionsteaches Claude Code how to read a store, via ~/.claude/CLAUDE.md
--project <dir>which folder --code writes .mcp.json into
--iconregisters the .dai file type and icon with your desktop (user scope)
--unregisterremoves that file-type registration again
--restoreputs every file setup touched back exactly as it was, and deletes the ones it created
--key sk-ant-…stores the indexer key in the user environment
--observer …pins the observer model the key is stored for
--alleverything detectable. A bare node setup.js already does this, so the flag is only useful in a script that wants to be explicit
--askthe opposite: a question per surface, for choosing them one at a time
--statuswhat is switched on right now, and the command that changes each one. Also --settings
--client <name>configure one MCP client: cursor, windsurf, codex, cline, continue, zed, all, list, or generic --config <file> for anything else
--no-installskip the dependency install, which otherwise happens once on the first run

Where memory lives

By default everything sits in ~/DaiDocs (override it with the DAIDOCS_STORE environment variable), and any project folder can be declared to keep its own store inside it, at <project>/.daidocs/store, so the memory travels with the code: one readable .dai document per conversation, the _index/ retrieval layer, byte-exact originals in _raw/, and _unconverted/ holding what is not converted yet, which the next session reads and which empties as conversion catches up. The first time you open a session in a folder with no store of its own you are asked once, here or general, and the answer is remembered. Open the folder, because reading your own memory is the product. More detail on the how-it-works page and in the format spec.

Saving uses one small extraction call: about $0.0009 per 1,000 tokens with GPT-4.1-mini, the observer that produced every published number through an API key, or free on a Claude subscription, where the assistant already in the conversation writes the extraction itself and no key is used.

Every folder keeps its own memory, and you do not have to ask for it. The first session you open in a folder that has no memory of its own gives it one, in .daidocs/ right there, and says so in a line. Your home folder and a drive root are never claimed, and a folder inside a project carries on with that project's memory rather than starting a third store. To send a folder's memory to the general store instead, say keep this folder's memory in the general store: what was already saved stays exactly where it is. A project with several parts, a website, an app, is several projects: at the top say make this folder a project that reads its parts, in each part say make this folder a project, so a session loads only its part's memory; say look in the main project's memory too when a part needs it, and it asks once. Give a folder a type if you want one: normal, locked (existing memory stays, nothing new is written), frozen (read-only for good), connected (reads a sibling folder too), shared, confidential (never included in any wider read: a client's folder, anything under NDA) or temporary (deleting the folder deletes its memory). A parent project reads its parts; a confidential part is the one exception. Each store has a permanent id, so a folder that moves is found again with node daidocs.js stores --scan.

See what you have. In the install folder, npm run dashboard-live keeps the memory map current while you work: it rebuilds whenever a store changes, and the page reloads itself only once you have stopped touching it, landing back on the store, the memory and the scroll position you were on. npm run dashboard builds the same page as a one-off snapshot, writes daidocs-dashboard.html beside package.json and opens it in your default browser, or prints the path if you pass --no-open. One self-contained page that works offline, rebuilt to refresh: every store and what it costs on disk, what is still waiting to be converted grouped by project, each memory as its own page, the file in its three zones beside the verbatim original, search, and the folder settings that otherwise mean a terminal. It contains your memories in full, so it is built by you and never shipped.

Folders and projects

A folder you work in gets its own memory, without being asked. The first session you open in a folder that has none creates .daidocs/ inside it and mentions it in a line. Your home folder and a drive root are never claimed, and neither is a folder whose memory you have already sent somewhere else.

A subfolder of a project is the exception, and it is deliberate. Open a session in website/landing-pages and it carries on writing into the website's memory rather than starting a third store. That is what stops a repository sprouting a store in every directory. When you do want one to stand on its own, say so in a session there: make this folder its own project. From that point its sessions are its own, and what was already saved stays where it is.

So a brand with two sides is two projects. One folder for the site, one for the product, each with a memory that fills up only with its own work:

brand/
├── website/              ← its own memory, made on the first session
│   ├── .daidocs/            every session about the site lands here
│   └── landing-pages/       a subfolder: uses the website's memory
└── product/              ← its own memory, separate from the site's
    └── .daidocs/            nothing from the site is in here

Sessions spent on pricing pages do not fill the product's memory with copywriting, and a week of firmware work does not surface when you ask the site folder what you decided about the hero section. That separation is the default, and if it is all you want there is nothing further to do.

When you do want them to talk, you say which way it goes. Each of these is one sentence, said in a session in the folder it is about:

What you sayWhat it does
let this folder read the main project toothe part can draw on the folder above it, which is the shape for answer from here, and fall back to the main project
let this folder read its partsthe top of the project reads every declared folder one level down, and a part added later is picked up without changing anything
let this folder read ../websiteone side reads the other by name, which is how the product folder reaches the site's decisions without the reverse being true
make this folder confidentialnever included in any wider read, by a parent or by anything else, whoever asks

Reach is one-way unless you set it at both ends: letting the product folder read the site does not let the site read the product.

Inside a git repository, the folder keeps itself out of git. Working in a repository gives it a .daidocs/ like anywhere else, and what goes in there is verbatim transcripts of what you said. So the folder ignores everything inside itself, the store and its own config alike: git status never lists it, and an ordinary git add -A cannot sweep it into a commit. Your repository's own .gitignore is left alone, because that file is yours and not ours to edit. If you ever do want to share a store, that stays possible and stays deliberate: copy the folder, or git add -f. A folder made by an older version, which ignored only the store, is brought in line the next time DaiDocs runs in it.

Nothing widens on its own. A recall reads the folder you are in and stops there. If that folder has nothing saved yet, it tells you which other stores it could reach and asks rather than quietly answering from one of them. The first time it does reach another store it asks your permission once and remembers the answer, and every answer names the stores it read.

Two model decisions, not one

You bring your own key, or your own subscription. Nothing ships with a key; if you set one it lives in your OS user environment rather than in any file, and on a Claude subscription you need none at all. Setup asks once which model converts, and that choice then applies to conversion, the hooks, the server and the CLI until you change it. Converting and answering are separate jobs, and they deserve separate choices.

RoleWhat it doesWe recommend
Observerreads each document once and writes the .dai fileanthropic:claude-opus-5 on a Claude subscription, openai:gpt-4.1-mini with an API key
Actoranswers your questions from the storeopenai:gpt-4o, or anything you like

Spend the quality on conversion, not on answering. The observer runs once per document and its output is baked into that file permanently, so a better observer improves every future answer drawn from it. The actor is swappable per question at no cost to the store. That is why the same store read by five different answering models spans 78.00% to 92.00% with none failing to read it: convert once with a capable model, then answer with whatever is cheapest or fastest, without re-converting anything.

Which observer to pick, and why the floor matters more than the ceiling. Every published number here used GPT-4.1-mini as the observer, so with an API key that is the recommendation, and the one the benchmark figures belong to. On a Claude subscription in Claude Code, use Claude Opus 5: the assistant already in the conversation writes the extraction itself, so it costs no key and nothing extra, and you get the stronger model for free. Gemini 3.1 Pro is the other good choice. Converting a session costs roughly what the session already occupies plus a question or two at that size: it reads what is new, once. Claude Sonnet 5 will work, and we would expect it below both. Claude Haiku 4.5 is where it breaks down. There we have seen extraction come back malformed, and a weak observer does not cost you a little accuracy at the end, it costs you the whole answer: the # Content zone is verbatim whatever converts it, but the summary, the # Understanding block and the facts, events and profile indexes are all written by the observer, and those are exactly what retrieval ranks on. Extract the wrong facts and recall hands the reader the wrong files, at which point the answering model never sees the evidence at all. Stated plainly: we have not published a measured observer comparison. The ranking above is mechanism and our own experience, not a benchmark, and it stays labelled that way until the comparison runs.

on a Claude subscription
$ node setup.js --observer anthropic:claude-opus-5   # free in-session: the assistant writes each memory itself, no API key anywhere

Activate reading

Connecting the server tells your assistant the store exists. It does not tell it how to read one, and that gap is expensive. Reading whole .dai files instead of the three zooms costs roughly 15x the tokens for no accuracy gain, and answering every question in one uniform style was the largest single source of wrong answers we measured during development.

So there is one more step, and it takes seconds. Pick the row that matches where you work.

Where you workWhat to doTime
Claude CodeRun node setup.js --instructions. Writes the protocol into ~/.claude/CLAUDE.md, so every session on the machine reads stores correctly. Nothing to remember afterwards.5 sec
Claude DesktopSettings, then Profile, then paste the one-liner below.15 sec
claude.ai, ChatGPT, GeminiPaste the activation prompt below at the top of the chat, then paste your _index/manifest.jsonl. Nothing to install.30 sec
A Project, Custom GPT or GemSame prompt, but in the persistent instructions instead of pasted each time. Attach the manifest as knowledge.2 min
Your own API codeSame prompt as the system message.1 min

Claude Desktop, one line

Settings → Profile
"When I reference past work or preferences not in this conversation,
call recall_memory before answering, and read stores in three zooms:
manifest first, then a file's Understanding block, then specific
segments only if needed."

Any chat window, copy this in

This is the whole activation prompt. Paste it once at the top of a chat with Claude, ChatGPT, Gemini or a local model, then paste your _index/manifest.jsonl underneath it. It ships in the repo as prompts/READER-PROMPT.txt.

prompts/READER-PROMPT.txt
You are reading a DaiDocs (.dai) knowledge store. It is plain text: one file
per conversation or document, each with three zones, plus a shared _index/.

Read it in three zooms and STOP at the shallowest one that answers the question.

  ZOOM 1  _index/manifest.jsonl, one JSON line per file (id, path, title, date,
          summary, topics, entities, tags). Always start here. Pick the 1 to 3
          files that match the question.
  ZOOM 2  That file's YAML frontmatter plus its "# Understanding" fenced JSON
          block. Most questions are fully answered at this zoom.
  ZOOM 3  Only if you still need exact wording: ask me for specific
          "## [seg n/N]" blocks from that file's "# Content" zone, by number.
          Never ask for a whole file.

Supporting indexes, load only when the question needs them:
  _index/facts.jsonl    every dated fact; kind = event | attribute | preference | plan
  _index/events.jsonl   one row per countable occurrence (date, category, what, src)
  _index/profile.jsonl  the user's stated and implied preferences

Classify the question first, then answer in that type's style.

  LOOKUP (a single fact)  Only the short answer, no preamble. If it asks what was
    SAID, quote the exact wording. The only type allowed to answer "There is no
    information about that."
  TEMPORAL (first, last, before, after, how long, in what order)  Load facts.jsonl.
    Show the dates and the day arithmetic, then a final line "ANSWER: ...".
  AGGREGATION (how many, total, list all)  Count from events.jsonl, one row per
    real-world event, deduplicated. Exclude "asked about X" rows: those are
    conversation topics, not occurrences. Then a final line "ANSWER: ...".
  PREFERENCE (recommend, should I, any tips)  Load profile.jsonl. Two to four
    sentences, explicitly naming the past item you are building on. Suggest NEW
    things, never repeat back what they told you. Never decline this type.

Rules for every type:
  - Never invent. Every claim must trace to something you actually read.
  - When several dated versions of the same information exist, the most recent is
    current, unless the question asks about an earlier time.
  - A fact with kind "plan" is an intention, not a record that it happened.
  - "Previous" means one step before the current value, not any older value.
  - When asked for "the results" or "the data", extract and tabulate the values
    themselves. A file listing is not an answer unless the question was about files.
  - Cite the file id you used.

If I ask you to write code that reads a .dai store through a model API, rather than
answering from one yourself, set up prompt caching for me as part of that code. I
will probably not know to ask, and the engine does not configure it. Put the cache
breakpoint on the store content you assemble, not on the extraction prompt. The
minimum cacheable prefix is model-dependent and not monotonic (512 tokens on Opus 5,
1,024 on Opus 4.8 and Sonnet 5, and higher on the small models), and below it caching silently
does nothing rather than erroring, so check usage.cache_read_input_tokens. Keep the
cached prefix byte-stable. Tell me what you set up.

Confirm you understand, then wait for my manifest.

Then paste your manifest, ask, and hand over segments only when the model asks for them by number. That handover discipline is where the saving comes from, so resist pasting whole files to save a round trip.

Every rule above is a measured fix, not a style preference. Type-routed answering, counting from the event table rather than prose, showing competing dated values instead of guessing between them, and narrow abstention are four of the six changes that took our score from 61.0 to 83.00. The long-form version, with what each rule is worth in points, is docs/READ-DAIDOCS.md in the repo. The numbers


1 · Claude Desktop ships now

After node setup.js --desktop and a restart of Claude Desktop, chat normally.

Save: say "save this chat to memory" and Claude calls save_memory with the transcript.

Recall: in any later chat, ask about past topics. Claude calls recall_memory, using about 10k tokens of context (a measured mean of 10,065 per question on LongMemEval) and answers on your subscription.

Make recall automatic with one line in Settings → Profile:

Settings → Profile
"When I reference past work or preferences not in this
conversation, call recall_memory before answering."

2 · Claude Code ships now

Terminal, IDE extension or desktop app, all the same setup: node setup.js --code --hook --instructions. Approve the daidocs-mcp server once when a session starts. The --instructions flag is the one people miss: without it Claude can reach your store but reads it the expensive way. See Activate reading.

With the hooks installed, every session saves itself: every 4,000 new tokens as you work, and again at the end, written by the assistant already in the conversation so it costs no key and nothing extra. Close the terminal earlier and nothing is lost: the unconverted part waits in _unconverted/, the next session in that folder starts with it in front of the assistant, and recall_memory reads it too, so it is usable before it is converted. A backlog converts whenever you choose, a project at a time, from the memory map or with npm run catch-up, and each session lands in the folder it was recorded in. This is the strongest demo of the whole system. Work all day, and tomorrow's session remembers.

Recall works exactly as in Desktop. Just ask about past work.

3 · claude.ai web cloud · waitlist

The browser cannot reach a local process, so claude.ai needs the hosted connector, a remote MCP server over HTTPS: Settings → Connectors → Add custom connector → enter your Kerneta Cloud URL.

Custom connectors work on every Claude plan, including free, where free plans get one custom connector. The hosted connector is the paid tier, and the local server stays free forever. Create a free Kerneta ID for first access

4 · Claude mobile cloud · waitlist

iOS and Android only load connectors from the official directory, so mobile memory arrives with Kerneta Cloud's directory listing. Submission requires a Team or Enterprise org, a privacy policy and Anthropic's review, all in progress as part of the Cloud launch. Once listed it is one tap to install, and your store follows you to the phone.

5 · Cowork cloud · waitlist

Cowork uses the same remote connector as claude.ai web. Add the Kerneta Cloud URL once and every Cowork session can save and recall, with nothing surface-specific to configure.

6 · Anthropic API ships now

Two integrations, both free and open source:

  • MCP: run daidocs-mcp (mcp_server.mjs, or npm run server) and point any MCP-capable agent at it over stdio, or at a hosted URL over HTTP.
  • Native memory tool: back Anthropic's memory_20250818 tool with the .dai store, so Claude autonomously reads and writes memory while Kerneta does the storage, indexing and retrieval. The handler class is scaffolded in the repo.

Set up prompt caching yourself on this surface. The engine does not do it for you: lib/providers/anthropic.js sends a plain request with no cache_control. In Claude Desktop and Claude Code the host handles caching, which is why surfaces 1 and 2 never need to think about it. Calling the API directly you get none, and every token is billed at full input price on every call.

The two effects are independent and they multiply: caching makes re-reading cheap, roughly 10x off input, while reading a store in three zooms makes there be less to re-read. Measured on one real session of ours, caching alone was worth 7.8x, the three zooms alone 2.9x, and both together 16.7x. Skip either and most of the saving is left on the table.

Cache the store content you assemble into the prompt, not the extraction prompt. Every document you ingest is different, so there is no shared prefix worth caching on the write side, and the minimum cacheable prefix is model-dependent and not monotonic: 512 tokens on Opus 5, 1,024 on Opus 4.8 and Sonnet 5, and higher on the small models. An extraction prompt sits under 4,096, so a breakpoint there caches nothing on a model with a high minimum and raises no error to tell you.

7 · Claude Agent SDK ships now

Same two code paths as the API: register the stdio MCP server in your agent definition, or wire the memory-tool handler class. Agents built on the SDK get persistent, inspectable memory with no extra infrastructure, and you can read what your agent remembers in a text editor.

8 · Enterprise cloud · enterprise tier

Org admins provision the Kerneta connector centrally with enterprise-managed auth, for example Okta through your identity provider. One admin action, and every seat has memory, with no individual setup. Team vaults add per-file permissions and an audit log of what the AI read and wrote. Talk to us

9 · Other MCP clients ships now

Cursor, Windsurf, ChatGPT connectors and apps, the OpenAI Agents SDK: anything that speaks MCP can mount the same server and the same store. Locally this ships today, so register daidocs-mcp as a stdio server in your client's MCP config. ChatGPT's hosted surfaces need the Cloud connector, same as claude.ai.

This is the portability story in practice, one memory, every assistant: in a cross-actor test over one shared set of prompts the same store answered through all five models we tried, every one of them over the full 500 questions. The accuracy does not travel with the format, though: it ran from 78.00% on Claude Haiku 4.5, our weakest actor, to 92.00% on Claude Fable 5, a spread of 14.0 points. The format is portable; the score is not. The numbers


Self-host it, free and permanently Apache 2.0

Everything here runs on your own machine with no account, no key and no payment. The format and the engine are Apache 2.0: clone the repository, run setup.js, choose Opus on your Claude subscription if you want zero API cost, and you are done. There is no reduced "community edition" of the file format: a .dai file written by the free engine is the same file the paid one writes.

The paid plans exist for people who would rather not run it themselves: hosted conversion, the browser console, team vaults and support. If you are happy with a terminal, the free path is the whole product.


What is in the repository Apache 2.0

Memory you can read deserves code you can read. One repository holds all of it: the format spec, the engine that produced every number, and the entire local product (server, archiver, setup, retrieval). A memory format you cannot inspect is a lock-in format, and this one is auditable down to the retrieval loop.

github.com/Kerneta/daidocs is the whole thing, not a demo build and not a crippled community edition. It is the same code these docs describe, end to end, and the same engine behind the numbers. It was published only after the key rotation and history scrub described under security posture, so nothing in its history exposes a credential.

PathWhat it is
spec/the .dai format specification, the thing any other tool needs in order to read your files
lib/methods/daidocs-v44n/the v4.4n engine, the one that scored 83.00%, not a lagging edition
lib/the .dai store format, indexing, retrieval and the decision layer
mcp_server.mjsdaidocs-mcp, the MCP server: save_memory, recall_memory, list_memories, read_memory, declare_project, brief_parent
session_*.mjsthe three Claude Code hooks: memory loads at session start, the session saves every 4,000 tokens, and archives at the end
daidocs.jsthe CLI: convert, pending, stores, backup, scrub, ingest, ask
tools/dashboard/npm run dashboard, the memory map. Ships as a builder; a built page holds your memories and is never shipped
setup.jsone-command configuration of every detected surface, idempotent, with backups
mcp_selftest.mjsend-to-end self-test of the server
docs/REPLICATION.mdthe exact actor, observer and judge settings behind every published number

Licence: Apache 2.0 for all of it, so commercial use, modification and redistribution are welcome, with a patent grant included. The MCP server you approve in your client is still called daidocs-mcp.


Security posture: no keys, anywhere

  • API keys live only in your OS user environment, never in files, configs, logs or the repo. setup.js --key writes to the environment and nothing else.
  • Every config setup.js touches is backed up first as *.daidocs-bak, and every step is idempotent.
  • The repository went public only after a full key rotation and history scrub, so nothing in its history exposes a credential.
  • On the free tier your memory never leaves your machine unless you point an indexer key at a provider you chose. On a Claude subscription nothing is sent to an indexer at all: the assistant in the session writes the memory itself.

Why we give the real product away

Because the moat is not the code, it is trust in a format. A memory standard only matters if you can leave any vendor, including us, and take your files along. So:

  • The local product will never be crippled to upsell Cloud. Cloud sells servers (hosting, sync, teams), not ransomed features.
  • The store format is plain text and documented, so any tool can read it, today and in twenty years.
  • The engine is public, so anyone can run our exact configuration and check us, competitors included. Mastra's Observational Memory already beats us on the same protocol and the same GPT-4o actor, 84.80 to our 83.00 (both micro-averaged; they headline 84.23 task-averaged, against our 84.02 averaged the same way), which puts us second among memory systems whose configuration is reproducible. Where we do come out ahead is cost: their published run averages about 30k tokens of context per question against our 10,065, 3.0× cheaper per answer at GPT-4o input pricing, $0.025 against $0.075. Good. A closed configuration nobody outside its vendor can re-run scores above both of us; it sits on the results page outside the ranking, stated rather than deleted. The protocol is right there.

Check our headline number yourself

You reproduce it by running it, not by downloading our run: we publish nothing out of the working machine, so there is no answer file of ours to re-score. A reproduction pays for ingest, 500 answers on the actor of your choice, and the 500 official judge calls over the hypothesis file you produce. Budget roughly $60 to $75 for a cold full-500 run. Most of that is ingest: the LongMemEval-S haystack is about 64 million tokens of source (50 million after de-duplicating repeated sessions), and ingest costs $0.92 per million, so it works out at $46 to $59 depending on whether you de-duplicate. The 500 answers plus the 500 official judge calls come to about $17 on top. Ingest is cached, so a second run against the same store costs only that $17, and a batch API halves it again.

reproduce the official run · clone → drive the engine → official eval
$ git clone https://github.com/Kerneta/daidocs
$ cd daidocs
$ npm install
…then about fifty lines of your own around two engine calls…
  engine.ingest(session, observer)
      once per session, observer openai:gpt-4.1-mini
  engine.answerMulti(question, store, actor)
      once per question, actor openai:gpt-4o at temperature 0
…500 answers, one {"question_id", "hypothesis"} line each…
$ python -X utf8 evaluate_qa.py gpt-4o hyp.jsonl longmemeval_s.json
…500 verdicts…
we scored 415/500 = 83.00%

All three moving parts are public. The dataset is LongMemEval-S, a public academic benchmark, all 500 questions with the abstention items left in. The engine is daidocs-v44n in the daidocs repository, Apache 2.0, the edition that scored 83.00% (415/500, GPT-4o answering), so you are running the thing we published the number for. The scorer is the benchmark authors' own evaluate_qa.py against judge snapshot gpt-4o-2024-08-06, which is neither our script nor our judge.

We ship the engine, not a runner: the loop in the middle is fifty lines you write around ingest and answerMulti, which is the cost of us not publishing our internal harness. Score all 500 questions with the 30 abstention items left in, because that is what our 415/500 is over. One thing to watch: the official script does not enforce the denominator. It scores the rows you hand it and silently skips the rest, so "scored with the official script" is not by itself a specification. Report how many rows you scored. Ours is 500 of 500. The exact actor, observer and judge settings are in docs/REPLICATION.md in the repository. What we do not publish is our internal benchmark harness: it carried sixty-two method versions, adapters for other benchmarks and run artifacts tangled with private material. It gave us convenience, never verifiability, and nothing in the claim depends on it.

Expect some movement against our figure, because GPT-4o's API is not fully deterministic at temperature 0. The spreads we have measured directly are ±3 points at n=100 and ±7 points at n=20, and we will not quote a tighter full-500 band than the data supports. If your run comes out below ours, that is the number we would want to hear about.


Something not covered here? The README in the repo carries the canonical instructions, and issues can be filed there. The community page explains how that works.