# Effective Context Capacity for OERSI: External Memory and Bounded Evidence Packs

**Date:** 29 October 2026  
**Evidence status:** Reproducible synthetic engineering benchmark; research prototype  
**Scope:** Standalone reference implementation. Not deployed into the live OERSI inference path.

## Executive conclusion

A model's hard context-window limit cannot be made literally infinite by a prompt-management layer. The practical engineering target is different: let an agent retain and search a growing archive while sending only a small, relevant, source-linked evidence pack to each model call. That makes the *effective working history* much larger than one prompt, while the model's per-call limit, latency, storage capacity and retrieval error remain real constraints.

This release implements and tests the first bounded prototype: persistent SQLite storage, overlapping text chunks, BM25-style lexical retrieval, document/chunk SHA-256 metadata, and context packing under an explicitly approximate token budget. Stored excerpts are marked as untrusted data; packing skips whole excerpts rather than silently truncating them.

**This is an integration of established engineering ideas, not a claim of a new scientific discovery.**

## Why this direction is scientifically grounded

- **Titans: Learning to Memorize at Test Time** combines a neural long-term memory module with attention to recent context and reports experiments with context beyond two million tokens. It is an architectural research direction, not evidence that all information can be retained without loss. [NeurIPS 2025 paper](https://papers.neurips.cc/paper_files/paper/2025/hash/a4ca07aa108036f80cbb5b82285fd4b1-Abstract-Conference.html) · [arXiv](https://arxiv.org/abs/2501.00663)
- **InftyThink** turns long reasoning into shorter segments connected by progress summaries, aiming to increase reasoning depth without growing a single prompt indefinitely. [ICLR 2026 paper](https://proceedings.iclr.cc/paper_files/paper/2026/hash/af1fd844255b6bc846efa1bec0eb6411-Abstract-Conference.html)
- **MemGPT** describes virtual context management: an agent moves information between limited in-context space and external memory tiers. [Paper](https://arxiv.org/abs/2310.08560)
- **FlashMemory-DeepSeek-V4** proposes looking ahead to preserve query-critical KV-cache chunks for ultra-long-context serving, reporting substantially smaller physical KV-cache footprints in its own evaluation. It is a model-serving technique, more invasive than this prototype's external-memory layer. [Technical report](https://arxiv.org/abs/2606.09079)
- **Stanford HAI's 2026 AI Index** notes that increasing the accepted context length does not guarantee reliable use of all that context; the gap between context capacity and usable context remains important. [Technical performance chapter](https://hai.stanford.edu/ai-index/2026-ai-index-report/technical-performance)

The proposed OERSI direction combines these ideas at the system level: persistent external memory first, retrieved evidence second, bounded model context third, and explicit verification of answer support after generation.

## Prototype design

1. **Ingest:** accept one explicitly selected UTF-8 file or text record, with a 32 MiB per-document ceiling. No network fetch is performed.
2. **Preserve:** store the original document, its SHA-256 digest, source label, title, priority and ingestion timestamp in SQLite.
3. **Chunk:** derive overlapping chunks and index their terms. The original full document remains stored; the chunk offsets and digests are retained.
4. **Retrieve:** rank matching chunks with BM25-style lexical scoring and a mild explicit-priority factor. No embeddings or external model are used.
5. **Pack:** construct a bounded prompt with whole evidence chunks, source labels and both document/chunk hashes. The pack tells the downstream model that excerpts are untrusted data, not instructions, and to cite evidence IDs.
6. **Audit:** recompute hashes to detect content inconsistency. Empty searches abstain; context packing fails closed when even the query and safety header cannot fit the requested proxy budget.

The token estimator uses ceil(UTF-8 bytes / 4). It is only a reproducible proxy—not an exact tokenizer and not a provider-side token guarantee. A production adapter must use the exact tokenizer for each target model, then reserve separate space for system prompts, tool schemas, output tokens and provider formatting.

## Reproducible synthetic experiment

The benchmark creates **1,000 synthetic records** with unique record IDs and validation codes, then asks **200 deterministic-seed queries** sampled across that archive. Every query includes its target record identifier. No private corpus, model inference or network call is used.

| Measure | Recency-only baseline | External-memory candidate |
|---|---:|---:|
| Records retained/available | Last 24 records | 1,000 indexed records |
| Target fact retrieved | 6 / 200 (3.0%) | 200 / 200 (100.0%) at top 5 |
| Target fact present in packed context | Not evaluated | 200 / 200 (100.0%) |
| Maximum estimated context pack | Not evaluated | 173 / 256 budget units |
| Median lexical search latency | Not applicable | 23.43–23.93 ms across two runs |
| P95 lexical search latency | Not applicable | 25.35–34.53 ms across two runs |

Two executions using the same fixed seed produced the same baseline/candidate hit counts, context-budget measurements, and hash-integrity results. Across those two runs, median search latency ranged from 23.43 to 23.93 ms and P95 from 25.35 to 34.53 ms on this one server with 1,000 synthetic chunks. This small repeatability check is not a production SLO or comparison against another production search engine. Context-pack sizes use the byte-based proxy described above.

### Verification

- Python syntax compilation: passed.
- Unit tests: **8 / 8 passed**.
- Source integrity: **1,000 / 1,000 document hashes and 1,000 / 1,000 chunk hashes checked; zero mismatches**.
- Benchmark exit status: zero in both deterministic executions; hit counts and integrity results matched.
- Model calls: 0. Network calls: 0. Private data used: no.

### What the results do and do not show

The experiment demonstrates a narrow implementation property: in a synthetic archive where each query includes a unique identifier, source-indexed retrieval can recover old facts that a recency-only window does not contain, and the selected evidence can fit inside the prototype's bounded pack. It does **not** test semantic paraphrase, LLM-generated answer accuracy, deep reasoning, long-horizon task completion, adversarial corpus behavior, retrieval quality on real user documents, multi-user isolation, storage failure recovery, or production-scale latency. Because the benchmark queries reveal target identifiers, its 100% result must not be presented as a general memory or reasoning score.

## Proposed OERSI architecture

- **Tier 0 — active window:** current user request, task state, short-term conversation and execution constraints.
- **Tier 1 — working memory:** retrieved chunks relevant to the current subtask, exact source references, unresolved questions and recent verified progress.
- **Tier 2 — persistent episodic memory:** original conversations, tool results and task events stored with timestamps, access control, provenance and integrity digests.
- **Tier 3 — semantic memory:** verified facts and compact summaries, each linked to source evidence and expiry/confidence metadata. Summaries must never silently replace the authoritative source.
- **Tier 4 — procedural memory:** reusable workflows with entry conditions, expected outputs, failure modes and independently checked postconditions.
- **Retrieval controller:** query rewriting, lexical + vector retrieval, reranking, diversity control, freshness/priority handling and conflict detection.
- **Context compiler:** exact model tokenization, prompt budget allocation, evidence labels, truncation prevention and explicit reporting of omitted candidates.
- **Answer verifier:** checks whether each material claim is supported by retrieved source IDs; abstains or requests more evidence when support is inadequate.
- **Compaction and lifecycle controller:** consolidates duplicate records, creates hierarchical summaries with back-links, expires only records covered by an explicit retention policy, and preserves a recoverable raw archive when retention requires it.

The prototype released here implements only the basic external store, lexical retrieval, prompt packing and hash audit. It does not yet implement embeddings, a neural long-term-memory model, automatic summaries, model-specific tokenizers, answer verification or production multi-user controls.

## Next experiments and promotion gates

1. **Real long-memory QA:** use public benchmark data where license terms permit, including LongMemEval and LoCoMo, with an explicit train/development/test split and answer-level evaluation. Compare recency truncation, naive retrieval, hybrid retrieval and verified hierarchical memory under the same model, token budget and call budget.
2. **Evidence fidelity:** measure exact fact recall, answer accuracy, citation precision, unsupported-claim rate, conflicting-memory handling and abstention accuracy. Blindly reviewed examples should be included.
3. **Cost and performance:** track input/output tokens using the actual tokenizer and provider telemetry; measure p50/p95 retrieval and end-to-end latency, storage growth and compaction cost at 1k, 10k, 100k and larger corpus sizes.
4. **Robustness and privacy:** test prompt injection embedded in stored documents, stale or contradictory facts, deletion requests, per-user access boundaries, backup/restore, tampered hashes, and retrieval failure. Stored text must never grant new tool permissions.
5. **Production gate:** do not connect the candidate to the public OERSI route until it demonstrates at least a 10-percentage-point held-out answer-accuracy improvement over a predeclared baseline at the same context budget, zero unauthorized consequential actions, working data isolation, and no material increase in unsupported claims. These are proposed engineering gates, not achieved results.

## Security and privacy

This public package contains only synthetic benchmark data. Do not publish private conversation histories, API keys, server paths, customer documents or internal telemetry in a corpus or benchmark result. A production deployment needs authentication and per-tenant access control, encryption and backup policy, a deletion/retention contract, prompt-injection resistance, and tests that prevent cross-user retrieval. SHA-256 detects accidental or unauthorized changes only when compared with a trusted digest; it is not a substitute for access control or signed releases.

## How to reproduce

Python 3.10+; standard library only.

    python3 -m py_compile context_memory.py benchmark.py test_context_memory.py
    python3 -m unittest -v test_context_memory.py
    python3 benchmark.py

Example use:

    python3 context_memory.py init --db ./oersi-memory.sqlite3
    python3 context_memory.py ingest --db ./oersi-memory.sqlite3 --file ./notes.txt --priority 3
    python3 context_memory.py search --db ./oersi-memory.sqlite3 --query "release decision evidence"
    python3 context_memory.py pack --db ./oersi-memory.sqlite3 --query "what decision was verified?" --max-tokens 512
    python3 context_memory.py audit --db ./oersi-memory.sqlite3

Source, automated tests, and raw output are linked from the public research report. No standalone reuse license is declared by this report; follow any applicable repository or file-level license.

## Public artifacts

- report.html — formatted public report
- report.md — complete technical report
- results.json and results-repeat.json — raw outputs from both deterministic benchmark runs
- context_memory.py — SQLite-backed memory store, retrieval and context compiler
- benchmark.py — synthetic benchmark source
- test_context_memory.py — automated unit tests
- effective-context-memory.json — machine-readable result record

## References

1. Ali Behrouz, Peilin Zhong, Vahab Mirrokni. “Titans: Learning to Memorize at Test Time.” NeurIPS 2025. https://papers.neurips.cc/paper_files/paper/2025/hash/a4ca07aa108036f80cbb5b82285fd4b1-Abstract-Conference.html
2. Yuchen Yan et al. “InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models.” ICLR 2026. https://proceedings.iclr.cc/paper_files/paper/2026/hash/af1fd844255b6bc846efa1bec0eb6411-Abstract-Conference.html
3. Charles Packer et al. “MemGPT: Towards LLMs as Operating Systems.” https://arxiv.org/abs/2310.08560
4. Yan Wang et al. “FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention.” https://arxiv.org/abs/2606.09079
5. Stanford Institute for Human-Centered Artificial Intelligence. “AI Index Report 2026 — Technical Performance.” https://hai.stanford.edu/ai-index/2026-ai-index-report/technical-performance