rxnim

# RxnIM Skill

Extract structured reaction data (SMILES, conditions) from chemical reaction images using the RxnIM model. RxnIM is a multimodal large language model designed for chemical reaction image parsing, achieving 84%-92% soft match F1 score on various benchmarks. It performs three tasks: reaction extraction, condition OCR, and role identification.

## Features

- **Image input**: PNG, JPG, etc.
- **Output**: JSON with reactants, products, conditions (reagents, solvents, temperature, yield).
- **Two modes**:
  1. **Web API**: Calls the Hugging Face Spaces demo (no local model needed).
  2. **Local inference**: Runs the full model locally (requires GPU and ~14 GB disk space).
- **Tasks supported**:
  - Reaction extraction (SMILES of reactants and products)
  - Condition OCR (text extraction and role labeling)
  - Role identification (reagent, solvent, temperature, yield)

## Background

RxnIM (Reaction Image Multimodal large language model) is the first multimodal LLM specifically designed to parse chemical reaction images into machine‑readable reaction data. It aligns task instructions with image features and uses an LLM‑based decoder to predict reaction components and conditions. The model is trained on a large‑scale synthetic dataset (Pistachio) and real ACS publications.

**Key capabilities**:
- Extracts SMILES of reactants and products with high accuracy.
- Interprets textual conditions (reagents, solvents, temperature, yield) and assigns roles.
- Outputs structured JSON or formatted reaction strings.
Performance: 84%–92% soft match F1 score on multiple test sets, outperforming previous methods.



##Quick Start
**Web API Mode (default)**
node scripts/rxnim.js --image /path/to/reaction.png

**Local Mode**
First, download the model checkpoint (see RxnIM repository(https://github.com/CYF2000127/RxnIM)) and set the environment variable RXNIM_MODEL_PATH.
export RXNIM_MODEL_PATH=/path/to/RxnIM-7b
node scripts/rxnim.js --image...