name: matryoshka description: Analyze large documents (100x larger than LLM context) using recursive language model with Nucleus DSL. Use when searching, filtering, aggregating text logs, reports, or structured text data without loading everything into context.

Matryoshka - Recursive Language Model

Analyze large documents (> 300 lines). Use when searching, filtering, aggregating text logs, reports, or structured text data without loading everything into context.

Workflow

1. Start server - Launch lattice-http in background (auto-starts if needed)
2. Load document - Load your file for analysis
3. Query progressively - Refine with grep → filter → aggregate
4. Close session - Free memory when done

Query results return compact handle stubs like $res1: Array(1000) [preview...] instead of full data. Use expand to inspect only what you need.

Quick Start

# Start server
scripts/start-server.sh

# Load a document
scripts/load.sh ./logs.txt

# Search for patterns
scripts/query.sh '(grep "ERROR")'

# Count results
scripts/query.sh '(count RESULTS)'

# Sum numeric values
scripts/query.sh '(sum RESULTS)'

# Close session
scripts/close.sh

Nucleus Query Examples

Search Commands

(grep "pattern")              ; Regex search
(fuzzy_search "query" 10)     ; Fuzzy search, top N results
(text_stats)                  ; Document metadata
(lines 1 100)                 ; Get line range

Collection Operations

(filter RESULTS (lambda x (match x "pattern" 0)))  ; Filter by regex
(map RESULTS (lambda x (match x "(\\d+)" 1)))      ; Extract from each
(sum RESULTS)                                       ; Sum numbers in results
(count RESULTS)                                     ; Count items

String Operations

(match str "pattern" 0)       ; Regex match, return group N
(replace str "from" "to")     ; String replacement
(split str "," 0)             ; Split and get index
(parseInt str)                ; Parse integer
(parseFloat str)              ; Parse float

Type Coercion

(parseDate "Jan 15, 2024")           ; -> "2024-01-15"
(parseCurrency "$1,234.56")          ; -> 1234.56
(parseNumber "1,234,567")            ; -> 1234567
(coerce value "date")                ; Coerce to date
(extract str "\\$[\\d,]+" 0 "currency")  ; Extract and parse

Variables

• RESULTS - Latest array result (auto-bound by grep, filter, etc.)
• _0, _1, _2, ... - Results from each command in sequence
• context - Raw document content

Common Patterns

Find and count error entries

(grep "ERROR")
(count RESULTS)

Extract and sum sales from specific region

(grep "SALES.*NORTH")
(map RESULTS (lambda x (parseCurrency (match x "\\$[\\d,]+" 0))))
(sum RESULTS)

Find recent entries by date

(grep "2024-01-1")
(filter RESULTS (lambda x (> (parseDate (match x "\\d{4}-\\d{2}-\\d{2}" 0)) "2024-01-15")))

Extract emails from text

(grep "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}")

Script Reference

All scripts read default port from LATTICE_PORT env var (default: 3456).

start-server.sh

Starts lattice-http server in background.

scripts/start-server.sh [port]   # Default: 3456

load.sh

Loads a document for analysis.

scripts/load.sh <file-path>
scripts/load.sh ./logs.txt

# Or pass content directly
scripts/load.sh - < file.txt  # Read from stdin

query.sh

Executes a Nucleus query.

scripts/query.sh '(grep "ERROR")'
scripts/query.sh '(count RESULTS)'

status.sh

Shows session status (timeout, queries, document info).

scripts/status.sh

bindings.sh

Shows current variable bindings.

scripts/bindings.sh

expand.sh

Expands a handle to see full data (optional limit/offset).

scripts/expand.sh RESULTS           # Show full RESULTS
scripts/expand.sh RESULTS 10        # First 10 items
scripts/expand.sh RESULTS 10 20     # Offset 10, limit 10

close.sh

Closes the current session and frees memory.

scripts/close.sh

stats.sh

Gets document statistics (length, line count).

scripts/stats.sh

health.sh

Health check with session info.

scripts/health.sh

Troubleshooting

Server not running

# Check if server is running
scripts/health.sh

# Restart
scripts/close.sh
scripts/start-server.sh

Session expired

Sessions auto-expire after 10 minutes of inactivity. Simply load the document again:

scripts/load.sh ./file.txt

Query returns errors

• Check syntax - Nucleus uses S-expressions with parentheses
• Use single quotes around queries to avoid shell expansion
• Reference scripts/help.sh for command reference

Environment Variables