name: domain-advisor description: > Plan experiments or development tasks using past knowledge. Adapts behavior based on project domain (research, unsloth, cuda) by reading registry.json. Triggers on or questions like "What should we try next?" metadata: short-description: "Plan next steps using past knowledge" tags: - planning - experiments - advice

Skill: domain-advisor

When to use

Use this skill when:

• The user message starts with <advise>, or
• The user asks "What should we try next?" or similar planning questions.

Initialization

1. Read .codex/skills/registry.json to determine:

   • domain: research | unsloth | cuda
   • paths.reports: where to find experiment/benchmark reports
   • paths.experiment_log: path to experiment log
   • paths.troubleshooting: path to troubleshooting guide

2. Adapt behavior based on domain (see Domain-Specific Behavior below).

Inputs

• User goal (including constraints like run budget, hardware, time).
• Optional references to files (configs, reports, logs) mentioned in the goal.

Behavior

1. Understand the goal

   • Parse the research/development goal and constraints.
   • If unclear, ask a concise clarifying question.

2. Gather context

   • Scan .codex/skills/ for relevant result skills matching the task.
   • Read relevant reports from the paths.reports directory.
   • Skim recent entries in paths.experiment_log.
   • If errors are mentioned, check paths.troubleshooting for known patterns.

3. Propose a plan

   • Design 2–5 concrete experiments/tasks, each with:
     • Clear objective
     • Key configuration/parameters
     • Expected outcome or hypothesis
     • Any relevant variation to test
   • Respect constraints (budget, hardware, time).
   • Reuse defaults from existing skills instead of inventing new ones.

4. Output format

   • Start with a short natural-language summary.

   • Provide a markdown table:

     id	description	key_differences	notes

   • Optionally propose file paths for configs or reports to create.

   • If errors were mentioned, explain how this plan avoids known failure patterns.

5. Logging

   • When useful, append a short entry to paths.experiment_log summarizing the proposed plan (only with user approval).

Domain-Specific Behavior

Research Domain

When domain: research:

Focus areas:

• Hyperparameter sweeps (learning rate, batch size, epochs)
• Model architecture variations (layers, dimensions, attention heads)
• Dataset mixtures and sampling strategies
• Training dynamics (warmup, schedulers, checkpointing)

Context to gather:

• training_reports/*.md - past training runs
• Model configs and their performance
• Loss curves and convergence patterns

Output emphasis:

• Parameter sweep tables with specific values
• Ablation study designs
• Baseline comparisons

Unsloth Domain

When domain: unsloth:

Focus areas:

• LoRA rank and alpha selection
• Quantization settings (4-bit, 8-bit, nf4)
• Gradient checkpointing configuration
• Fine-tuning hyperparameters for specific model families
• Memory optimization strategies

Context to gather:

• training_reports/*.md - past fine-tuning runs
• Model-specific Unsloth configurations
• Memory usage patterns

Output emphasis:

• LoRA configuration recommendations
• Memory/speed tradeoffs
• Model-specific settings (Llama, Mistral, Qwen, etc.)

CUDA Domain

When domain: cuda:

Focus areas:

• Tiling strategies and block sizes
• Shared memory usage patterns
• Warp-level primitives
• Memory coalescing optimization
• Triton autotuning configurations

Context to gather:

• benchmark_results/*.md - past kernel benchmarks
• Profiling data (nsight, ncu reports)
• Bandwidth and occupancy metrics

Output emphasis:

• Kernel configuration parameters
• Expected speedup estimates
• Memory access pattern recommendations
• Profiling metrics to track

Example Output

Research Example

## Experiment Plan: Attention Head Ablation

Based on previous runs in `training_reports/baseline-2025-01.md`, the 8-head
configuration achieved 92% accuracy. Testing whether fewer heads can match
this with lower compute.

| id | description | key_differences | notes |
|----|-------------|-----------------|-------|
| A1 | 4-head attention | heads=4 vs baseline 8 | Test if 4 heads sufficient |
| A2 | 6-head attention | heads=6 | Middle ground |
| A3 | 4-head + wider FFN | heads=4, ffn_dim=4096 | Compensate with FFN |

All runs use: lr=1e-4, batch_size=32, epochs=10 (from baseline config).

Unsloth Example

## Fine-tuning Plan: Llama-3 8B with Unsloth

Based on `training_reports/llama3-lora-v1.md`, rank=16 showed good results
but OOM'd at batch_size=4. Testing memory-efficient configurations.

| id | description | key_differences | notes |
|----|-------------|-----------------|-------|
| U1 | rank=8 + grad_ckpt | Lower rank, enable checkpointing | Memory baseline |
| U2 | rank=16 + 4bit | Full rank with 4-bit quantization | Quality vs memory |
| U3 | rank=32 + offload | Higher rank with CPU offload | Max quality attempt |

All runs use: alpha=32, dropout=0.05, target_modules=["q_proj", "v_proj"]

CUDA Example

## Kernel Optimization Plan: Softmax

Based on `benchmark_results/softmax-v1.md`, current implementation achieves
80% of theoretical bandwidth. Testing tiling strategies.

| id | description | key_differences | notes |
|----|-------------|-----------------|-------|
| K1 | 2D tiling | BLOCK_M=64, BLOCK_N=64 | Better L2 reuse |
| K2 | Warp reduction | Use warp shuffles | Reduce shared mem |
| K3 | Online softmax | Single-pass algorithm | Fused with attention |

Profile with: `ncu --set full` to capture memory metrics.