Technology / World-State

Keep understanding.
Release the history.

A model should carry what it has learned about the world, without carrying every token it has ever seen.

01 / The core idea

Memory is a state.
Not a transcript.

World-State is our approach to representing an evolving history inside a fixed-capacity model state.

The state is updated as new information arrives. Recent details stay directly accessible; a recurrent summary and learned anchors carry selected information forward. Capacity remains fixed, while the contents keep changing.

Three complementary ways to carry history

Recurrent state

Compress the stream into a continuously updated representation.

Carry forward

Local attention

Keep precise token-level access to a bounded recent window.

Look nearby

Memory anchors

Select useful information from a reusable bank of historical representations.

Recall selectively

Conceptual view of the EdgeRock history system. Fixed capacity describes storage growth, rather than unlimited information retention.

A working memory, continuously renewed

The question changes from “How much history can we store?” to “What should the model carry forward?” That makes writing, admission and retrieval part of the architecture itself.

World-State does not require an external vector database. An application can still add document retrieval when its task calls for access to a larger source of information.

02 / EdgeRock

Read. Select.
Update. Continue.

Open model

Rock Memory Anchors (RMA) gives older information a bounded place to live.

EdgeRock-4B replaces the full-attention layers of its source model with RMA: local attention plus a fixed bank of memory anchors. Gated DeltaNet layers retain recurrent computation.

A stream of tokens. A fixed memory budget.

New tokens
Read the stream

Local window
1,024 tokens

Anchor bank
1,024 / KV group

Next state
Same capacity

Schematic RMA path. A commit occurs every 256 tokens. Recurrent computation runs alongside the local-and-anchor attention system.

Write

Form a candidate

Use the incoming history to propose information for longer retention.

Admit

Renew the bank

Update a fixed pool rather than appending new memory indefinitely.

Retrieve

Read what matters

Select anchors relevant to the current computation alongside recent tokens.

Does the memory matter?

The report’s controlled memory experiment compares the same system with RMA enabled and disabled.

Study the experiment

Controlled memory test / Query accuracy
RMA enabled	RMA disabled
94.1%	0.4%

1,024 queries across 256 contexts in the report’s controlled task. This ablation is evidence for memory use in that task, rather than a general recall guarantee.

03 / DiffusionRock

One shared state.
Many steps forward.

Research direction

Explore generation that does more useful work in parallel.

DiffusionRock studies how a compact World-State can condition block-parallel generation. A shared representation of context could let a model refine multiple positions together instead of advancing through every position sequentially.

State-conditioned block generation

World-State / Shared context

↓ Condition parallel blocks ↓

Block 01
Refine

Block 02
Refine

Block 03
Refine

Research schematic. Parallelism, quality and state consistency must be evaluated together.

Key questions include how to update state across blocks, preserve consistency, and trade refinement steps against latency. DiffusionRock is a research direction; a public model checkpoint is not listed yet.

04 / SiliconRock

Design the model.
Design its machine.

Architecture research

Fixed weights. Fixed history state. A specialized path from one token to the next.

SiliconRock explores an execution path where fixed weights become part of the computation itself, while the bounded history state stays on chip. The goal is to remove repeated weight movement and keep each token’s work close to its data.

Weights encoded
in computation

SiliconRockSpecialized execution

History state
on chip

Architecture concept, rather than a photograph or specification of a fabricated device.

Design target / Not measured hardware

1 million tokens per second, per user.

The whitepaper explores this target with an early approximately 2.1B-parameter, 28-layer design. Its modeled 884-cycle token path at an assumed 1 GHz implies about 1.13 million tokens/s. This is a cycle-and-frequency estimate, not measured silicon or EdgeRock-4B inference performance.

The work ahead includes physical implementation, memory organization, power and area, numerical behavior, and end-to-end model validation. We welcome research and engineering conversations around those questions.

Explore a hardware collaboration

Take a closer look.

Read the architecture, inspect the model, or bring your own research question.

Open the research