AI Engineering

RAG Chunking Playground

Compare recursive, estimated-token, sentence, paragraph, and Markdown-aware document chunking locally with configurable overlap, exact source offsets, metrics, and JSON or JSONL export.

Processed locally in your browser

Source document

570 characters200,000 maximum

StrategyTarget size (estimated tokens)Overlap (estimated tokens)

Semantic strategies prefer natural boundaries, then split oversized units safely. Exact model tokenizers differ by provider and model.

RAG chunks

Chunking strategy comparison
Strategy	Chunks	Average tokens	Maximum tokens	Overlap characters
Recursive	2	82	111	80
Estimated token	2	81.5	120	80
Sentence-aware	2	81.5	117	80
Paragraph-aware	2	82	111	80
Markdown-aware	2	82	111	80

Note: Token counts are provider-neutral estimates, not exact model tokenizer output.

Chunk 1chars 0–441 · 441 characters · ~111 tokens

# Production retrieval guide

Retrieval quality starts with documents that have clear structure. Preserve headings because they carry useful context into each chunk.

## Chunk sizing

Smaller chunks can improve precision, while larger chunks retain more surrounding meaning. Measure the tradeoff with your own queries and relevance labels.

## Example configuration

```json
{
  "strategy": "markdown",
  "size": 120,
  "overlap": 20
}
```

Chunk 2chars 361–570 · 209 characters · ~53 tokens

tion

```json
{
  "strategy": "markdown",
  "size": 120,
  "overlap": 20
}
```

Use overlap deliberately. Too little can separate related ideas; too much increases index size and can return redundant passages.

Source document

570 characters200,000 maximum

StrategyTarget size (estimated tokens)Overlap (estimated tokens)

Semantic strategies prefer natural boundaries, then split oversized units safely. Exact model tokenizers differ by provider and model.

How to use this tool

Paste a representative source document and choose a target estimated-token budget.Compare strategies and inspect exact offsets, semantic boundaries, overlap, and chunk-size variance.Export JSON or JSONL, then evaluate retrieval with your real embedding model, queries, and relevance labels.

Frequently asked questions

Does this create embeddings?

No. It prepares and compares chunks locally without calling a model or sending document text anywhere.

Are token counts exact?

No. They are provider-neutral planning estimates. Validate final limits with the tokenizer for your embedding or generation model.

Which strategy is best?

There is no universal winner. Markdown-aware chunking often helps structured docs, while sentence or recursive strategies can suit prose. Measure retrieval quality on representative queries.

Strategy

Chunks

Average tokens

Maximum tokens

Overlap characters

Recursive

111

Estimated token

81.5

120

Sentence-aware

81.5

117

Paragraph-aware

111

Markdown-aware

111

# Production retrieval guide Retrieval quality starts with documents that have clear structure. Preserve headings because they carry useful context into each chunk. ## Chunk sizing Smaller chunks can improve precision, while larger chunks retain more surrounding meaning. Measure the tradeoff with your own queries and relevance labels. ## Example configuration ```json { "strategy": "markdown", "size": 120, "overlap": 20 } ```

tion ```json { "strategy": "markdown", "size": 120, "overlap": 20 } ``` Use overlap deliberately. Too little can separate related ideas; too much increases index size and can return redundant passages.