context-optimizer

# Context Pruner

Advanced context management optimized for DeepSeek's 64k context window. Provides intelligent pruning, compression, and token optimization to prevent context overflow while preserving important information.

## Key Features

- **DeepSeek-optimized**: Specifically tuned for 64k context window
- **Adaptive pruning**: Multiple strategies based on context usage
- **Semantic deduplication**: Removes redundant information
- **Priority-aware**: Preserves high-value messages
- **Token-efficient**: Minimizes token overhead
- **Real-time monitoring**: Continuous context health tracking

## Quick Start

### Auto-compaction with dynamic context:
```javascript
import { createContextPruner } from './lib/index.js';

const pruner = createContextPruner({
  contextLimit: 64000, // DeepSeek's limit
  autoCompact: true,    // Enable automatic compaction
  dynamicContext: true, // Enable dynamic relevance-based context
  strategies: ['semantic', 'temporal', 'extractive', 'adaptive'],
  queryAwareCompaction: true, // Compact based on current query relevance
});

await pruner.initialize();

// Process messages with auto-compaction and dynamic context
const processed = await pruner.processMessages(messages, currentQuery);

// Get context health status
const status = pruner.getStatus();
console.log(`Context health: ${status.health}, Relevance scores: ${status.relevanceScores}`);

// Manual compaction when needed
const compacted = await pruner.autoCompact(messages, currentQuery);
```

### Archive Retrieval (Hierarchical Memory):
```javascript
// When something isn't in current context, search archive
const archiveResult = await pruner.retrieveFromArchive('query about previous conversation', {
  maxContextTokens: 1000,
  minRelevance: 0.4,
});

if (archiveResult.found) {
  // Add relevant snippets to current context
  const archiveContext = archiveResult.snippets.join('\n\n');
  // Use archiveContext in your prompt
  console.log(`Found ${archiveResult.sources.length} relevan...

# Context Pruner

Advanced context management optimized for DeepSeek's 64k context window. Provides intelligent pruning, compression, and token optimization to prevent context overflow while preserving important information.

## Features

- **DeepSeek-optimized**: Specifically tuned for 64k context window
- **Multiple pruning strategies**: Semantic, temporal, and extractive compression
- **Adaptive pruning**: Different strategies based on context usage levels
- **Priority-aware**: Preserves high-priority and system messages
- **Real-time monitoring**: Continuous context health tracking
- **Token-efficient**: Minimizes token overhead from pruning operations

## Installation

```bash
# Install dependencies
npm install

# Or install globally for CLI use
npm install -g .
```

## Quick Start

```javascript
import { createContextPruner } from './lib/index.js';

const pruner = createContextPruner({
  contextLimit: 64000, // DeepSeek's limit
  autoPrune: true,
  strategies: ['semantic', 'temporal', 'extractive'],
});

await pruner.initialize();

// Process messages with automatic pruning
const messages = [
  { role: 'user', content: 'Hello!', priority: 5 },
  { role: 'assistant', content: 'Hi there!', priority: 5 },
  // ... more messages
];

const processed = await pruner.processMessages(messages);

// Get status
const status = pruner.getStatus();
console.log(`Health: ${status.health}`);
console.log(`Tokens: ${status.tokens.used}/${status.tokens.limit}`);
```

## CLI Usage

```bash
# Run tests
node scripts/cli.js test

# Show status
node scripts/cli.js status

# Prune a JSON file
node scripts/cli.js prune input.json output.json

# Show statistics
node scripts/cli.js stats
```

## Pruning Strategies

### 1. Semantic Pruning
Removes semantically similar messages using embeddings. Useful for eliminating redundant information.

### 2. Temporal Pruning
Removes older messages first, preserving recent conversation. Configurable preservation of recent messages.

### 3. Extractive...