# Yaroslav Markin

> Entrepreneur, fractional CTO, consultant

### Yaroslav Markin

# Yaroslav Markin

## Entrepreneur, fractional CTO, consultant

<span class="tag">[Available for projects](#contact)</span>

<address class="global-location">Worldwide remote from Dubai, UAE</address>

**Previously**: founded <span class="evil_martians">[Evil Martians](https://evilmartians.com)</span> in 2006 and served as CTO.

> Evil Martians transform growth-stage startups into unicorns, build developer tools, and create open source products.
>
> Over the last 18 years, we've helped startups solve their technical challenges, ship faster and more reliably, build their teams, and raise new rounds of funding. 8 of them became unicorns.
>
> Across more than 100 open source projects, we have 25B+ total downloads.


## Recent posts

- [Inkmark: a very fast, feature-packed, AI-first Markdown gem for Ruby](/posts/inkmark-fast-ai-first-markdown.md) — Markdown is the lingua franca of the AI internet. Inkmark is a very fast Rust-backed Ruby Markdown gem, has features galore, and comes with first-class support for LLM/RAG pipelines.
- [Bundlebun bundles Bun, and a tour of asset pipelines and JavaScript runtimes Rails has been through](/posts/bundlebun-bundles-bun.md) — Bundlebun is a Bun bundler for Rails. The why, the how, and a tour of every asset pipeline and JavaScript runtime Rails has been through to get here.

## Open Source

- [Kino](/opensource/kino.md) — A high-performance Ractor web server for Ruby 4.0+: Rack 3-based, with a Rust Tokio/Hyper front end, Ractor-parallel Ruby workers, and a threaded fallback mode.
- [Bundlebun](/opensource/bundlebun.md) — Bundles Bun—a fast JavaScript runtime, package manager, and builder—with Ruby and Rails applications.
- [Carbon Fiber](/opensource/carbon_fiber.md) — A high-performance Ruby Fiber scheduler powered by Zig and libxev.

## Press

- [Evil Martians: How developers help startups to take off (and how much they earn doing that)](https://web.archive.org/web/20200318152645/https://incrussia.ru/fly/evil-martians/) (Inc. Russia)
- [IT specialist on how the Internet will survive the Crisis: Remote work, skills development, and automation of everything](http://www.lookatme.ru/mag/how-to/tops/211105-it-expert) (Look At Me magazine)
- [Evil Martians: the journey from two people in a cafe to offices in NYC, SF, and Osaka](https://vc.ru/life/147200-zlye-marsiane-put-ot-dvuh-chelovek-v-kafe-do-ofisov-v-nyu-iorke-san-francisko-moskve-i-osake) (VC.ru)

### Open Source

# Open Source

- [Kino](/opensource/kino.md) — A high-performance Ractor web server for Ruby 4.0+: Rack 3-based, with a Rust Tokio/Hyper front end, Ractor-parallel Ruby workers, and a threaded fallback mode.
- [Bundlebun](/opensource/bundlebun.md) — Bundles Bun—a fast JavaScript runtime, package manager, and builder—with Ruby and Rails applications.
- [Carbon Fiber](/opensource/carbon_fiber.md) — A high-performance Ruby Fiber scheduler powered by Zig and libxev.
- [Inkmark](/opensource/inkmark.md) — A very fast, feature-packed, AI-first Markdown (CommonMark/GFM) gem for Ruby, built on pulldown-cmark.
- [nosj](/opensource/nosj.md) — Pull, push, and fused-cursor JSON parsers plus a streaming writer, SIMD-accelerated (NEON, SSE2, runtime-detected AVX2), for hosts that build native values during the parse.
- [gem nosj](/opensource/nosj-ruby.md) — A fast, json-gem-compatible JSON parser and generator for Ruby—Rust and SIMD via the first-party nosj crate, precompiled PGO platform gems, lazy documents and partial parsing, and a one-line JSON drop-in.
- [Audition](/opensource/audition.md) — Auditions Ruby scripts, gems, Rack apps, and Rails applications for Ractor-readiness—static analysis of isolation violations plus dynamic in-Ractor probing, with explanations and fixes.
- [Callback Hell](/opensource/callback_hell.md) — Analyzes Rails application models and provides actionable insights on callbacks and validations.
- [astro-slop](/opensource/astro-slop.md) — Astro integration for LLMs: markdown versions of every page, llms.txt, and alternate links.
- [astro-nice-asset-filenames](/opensource/astro-nice-asset-filenames.md) — Astro integration for short, readable asset URLs.

### astro-nice-asset-filenames

Astro integration for short, readable asset URLs.

**Before:**

By default, Astro's static build emits client-script chunks named after the synthetic module ID, with all the URL boilerplate replaced by underscores:

```
_astro/ThemeToggle.astro_astro_type_script_index_0_lang.BhEhlh7m.js
_astro/BaseLayout.Pt3ADYuj.css
```

**After:**

This integration shortens the script-block boilerplate and converts `PascalCase` to `snake_case` (or `kebab-case`):

```
_astro/theme_toggle.BhEhlh7m.js
_astro/base_layout.Pt3ADYuj.css
```

## Install

```sh
npm install astro-nice-asset-filenames
# or: bun add, pnpm add, yarn add
```

## Usage

```js
// astro.config.mjs
import { defineConfig } from "astro/config";
import niceAssetFilenames from "astro-nice-asset-filenames";

export default defineConfig({
  integrations: [
    niceAssetFilenames()
  ],
});
```

## Options

| Option      | Type      | Default | Description                                                                                            |
| ----------- | --------- | ------- | ------------------------------------------------------------------------------------------------------ |
| `enabled`   | `boolean` | `true`  | When `false`, the integration is a no-op. Handy for env-flag toggling while debugging.                 |
| `separator` | `string`  | `"_"`   | Separator passed to [`decamelize`](https://github.com/sindresorhus/decamelize). Use `"-"` for kebab.   |

```js
niceAssetFilenames({
  enabled: process.env.NICE_FILENAMES !== "0",
  separator: "-",
});
```

## What it does

Three transforms, layered:

1. **Strips Astro's script-block URL boilerplate.** `<File>.astro?astro&type=script&index=N&lang.ts` becomes just `<File>`—no more `_astro_type_script_index_0_lang` in the filename.
2. **Snake_cases (or kebab-cases) the basename.** `ThemeToggle` → `theme_toggle` via [`decamelize`](https://github.com/sindresorhus/decamelize), which handles acronym-heavy names correctly: `OAuth2Client` → `o_auth_2_client`, `HTMLParser` → `html_parser`.
3. **Preserves the script-block index when it matters.** A `.astro` file with multiple `<script>` blocks emits separate chunks. `index=0` is dropped (the common case); `index >= 1` is appended as `-N` so the filenames remain distinguishable: `foo.HASH.js`, `foo-1.HASH.js`, `foo-2.HASH.js`.

## Compatibility

- Astro 6+ (the integration relies on Vite's Environment API, which Astro adopted in 6.0)
- Node ≥ 22.12.0 (matches Astro 6's minimum)

Tested against static builds on rather simple websites; be wary.

Server-output (SSR) and prerender-only builds should work the same way; if you run into edge cases, you can open an issue.

## Development

```sh
npm test
```

Tests run against `src/` directly via Node's built-in test runner.

## License

MIT.

### astro-slop

Content slop for LLMs to slurp. The most complete and flexible Astro integration for LLMs and generating markdown versions of pages. 

- `.md` siblings for every page. Flexible, bottom-up, fully customizable approach: you create simple endpoints with the integration's helpers by example.
- HTML → Markdown conversion for pages that don't have a markdown source from collections. MDX cleaning for MDX sources.
- `<link rel="alternate">` injection on every HTML page.
- Auto-generated—and customizable!—`llms.txt` and `llms-full.txt`.
- HTTP content negotiation for `text/markdown` in dev and SSR.

## Rationale

There are a number of LLM/`llms.txt`/Markdown-exporting integrations for Astro. Most promise a simple config or even zero-configuration, but are no good for real-world cases.

Some only generate `llms.txt`; not super useful. Some generate markdown, but by converting all HTML automatically—can result in a mess. Some try to use collections for export, but are not flexible enough. 

The top-down approach does not work. We need to go for the bottom-up one. The "correct" approach already exists in Astro: create an endpoint; the same approach used in [`@astrojs/rss`](https://docs.astro.build/en/recipes/rss/). The integration, or Astro's role, is to help you do that by giving examples and helper functions, and automating as much as possible.

astro-slop does exactly that: it gives you enough tooling to create your own simple, short endpoints (`posts.md.ts` next to `posts.astro`) where you can export markdown by yourself; next, it collects that markdown for `llms.txt`, auto-injects `alternate` links and does a ton of other stuff to reduce manual changes.

## Features

- **The `md` tagged template** for writing `.md.ts` endpoints. Returns a `Response` with `text/markdown` content-type. Real newlines, no `\n` escapes; arrays auto-join with newlines so a `posts.map(...)` interpolation becomes a bullet list directly; template indentation dedented based on template strings only, so interpolated values aren't affected. Plus inline helpers—`md.link`, `md.heading`, `md.section`—and a `md.string` variant for sub-composition or custom `Response` shapes.
- **Tiny user-facing surface**—one integration call (`slop()`), one tagged template (`md`), one HTML-fallback helper (`fromHtml`), one MDX cleaner (`cleanMdx`), and two "default content" hooks (`defaultLlmsTxt`, `defaultLlmsFullTxt`) for users who want to extend the auto-generated indexes. Six exports, all documented below.
- **`<link rel="alternate" type="text/markdown">`** auto-injected into every HTML page that has a `.md` sibling. Two mutually-exclusive paths so it works across Astro output modes: runtime middleware handles dev / SSR responses, the `astro:build:done` HTML rewriter handles static prerender output. No double-injection, no version-dependent fallback.
- **`cleanMdx(body)`**—strips MDX-specific syntax (top-level `import`/`export` statements, JSX components) from raw collection entry bodies so the result is plain Markdown. HTML is preserved as-is (it's valid Markdown, LLMs parse it fine, and user-written HTML often carries semantic intent). Use it whenever you interpolate `entry.body` into a markdown response — per-entry endpoints, composed pages, anywhere `.mdx` content might leak JSX chrome.
- **`fromHtml(url, options)`**—for static `.astro` pages without a markdown source. Fetches the rendered HTML and runs it through [Defuddle](https://github.com/kepano/defuddle) for main-content extraction, chrome stripping (nav, footer, sidebars, share buttons), and Markdown conversion in a single pass. Lazy-loaded with `@vite-ignore` dynamic imports so Defuddle and its DOM dependency don't appear in your bundle if `fromHtml` is never called.
- **`/llms.txt`** auto-assembled from your `.md` routes per the [llmstxt.org spec](https://llmstxt.org). H1 site name, blockquote summary, then a flat bullet list of every route with title + description—title and description extracted automatically from each endpoint's emitted frontmatter or first H1. Override by writing your own `src/pages/llms.txt.ts` and calling `defaultLlmsTxt(context)` to splice the auto-gen body into your own structure (sections, recommended reading, references, etc.).
- **`/llms-full.txt`**—same site header plus each route's full body content inlined as `### Title\n\n${body}` sections. One document an LLM can ingest end-to-end. Same override pattern via `defaultLlmsFullTxt(context)`.
- **HTTP content negotiation**—when a request's `Accept` header explicitly ranks `text/markdown` above `text/html`, the middleware redirects to the `.md` sibling. RFC 9110-compliant, integration-internal—users add no config. LLM-aware clients (curl scripts, RSS readers, well-behaved crawlers) get markdown automatically; browsers—whose default Accept never lists markdown—keep getting HTML.

## Install

~~~sh
npm install astro-slop
# or: bun add astro-slop
# or: pnpm add astro-slop
~~~

## Configure

~~~js
// astro.config.mjs
import { defineConfig } from "astro/config";
import slop from "astro-slop";

export default defineConfig({
  integrations: [
    // ...
    slop({
      siteName: "Your Site",
      siteDescription: "One-sentence summary used in /llms.txt blockquote.",
    }),
  ],
});
~~~

| Option | Type | Default | Effect |
|---|---|---|---|
| `siteName` | `string` | `"Site"` | H1 of `/llms.txt` and `/llms-full.txt` |
| `siteDescription` | `string` | `undefined` | Blockquote line under the H1 |
| `injectAlternateLink` | `boolean` | `true` | Inject `<link rel="alternate">` into HTML head |
| `llmsFullTxt` | `boolean` | `true` | Generate `/llms-full.txt` |
| `contentNegotiation` | `boolean` | `true` | Redirect HTML→`.md` when `Accept` prefers markdown (request-time only—dev / SSR) |

## Authoring `.md` endpoints

The integration doesn't "auto-generate" markdown content. Instead, it goes for flexibility and custom-fitted approach. 

You write a small `.md.ts` endpoint per page that should have a markdown view, the same way you'd write `rss.xml.ts` for an RSS feed. The integration handles the response wrapper, alt-link injection, and `/llms.txt` indexing.

### Per-entry route (collection-driven)

For dynamic routes backed by a content collection (`[slug].astro`), the markdown sibling re-exports `getStaticPaths` from the HTML page and emits the entry's frontmatter + body. Wrap the body in `cleanMdx()` to strip JSX/imports for `.mdx` collections:

~~~ts
// src/pages/posts/[slug].md.ts
import type { APIRoute } from "astro";
import type { CollectionEntry } from "astro:content";
import { cleanMdx, md } from "astro-slop";

export { getStaticPaths } from "./[slug].astro";

interface Props {
  post: CollectionEntry<"posts">;
}

export const GET: APIRoute<Props> = ({ props }) => md`---
title: ${props.post.data.title}
description: ${props.post.data.description}
---

${cleanMdx(props.post.body ?? "")}`;
~~~

The `getStaticPaths` re-export means the `.md.ts` shares Astro's built path resolution with the HTML page—no duplicate filtering or sorting. Astro's module cache means the underlying `getCollection` query runs once per build.

### Listing route

For listing pages backed by a collection, the `.astro` exports its data; the `.md.ts` imports it:

~~~astro
---
// src/pages/posts/index.astro
import { getCollection } from "astro:content";

export const posts = (await getCollection("posts", ({ data }) => !data.draft))
  .sort((a, b) => /* however you sort */);
---

<Layout>
  {posts.map((p) => <article>{p.data.title}</article>)}
</Layout>
~~~

~~~ts
// src/pages/posts.md.ts
import type { APIRoute } from "astro";
import { md } from "astro-slop";
import { posts } from "./posts/index.astro";

export const GET: APIRoute = () => md`
# Posts

${posts.map((p) =>
  `- ${md.link(p.data.title, `/posts/${p.id}.md`)} — ${p.data.description}`
)}
`;
~~~

The `export` keyword is the only refactor needed—your filter/sort logic stays in the `.astro`, the HTML render reads the same `posts` constant via closure.

### Composed page (homepage with intro + dynamic content, for instance)

~~~astro
---
// src/pages/index.astro
import { getEntry, getCollection } from "astro:content";

export const home = await getEntry("pages", "home");
export const recent = (await getCollection("posts")).slice(0, 5);
---

<Layout>
  <Content>{home!.body}</Content>
  <PostCardList posts={recent} />
</Layout>
~~~

~~~ts
// src/pages/index.md.ts
import type { APIRoute } from "astro";
import { cleanMdx, md } from "astro-slop";
import { home, recent } from "./index.astro";

export const GET: APIRoute = () => md`
${cleanMdx(home!.body)}

## Recent posts

${recent.map((p) =>
  `- ${md.link(p.data.title, `/posts/${p.id}.md`)} — ${p.data.description}`
)}
`;
~~~

`home!.body` is the raw markdown from your collection entry—interpolated as-is, no HTML round-trip. Wrap it in `cleanMdx()` if your homepage is a `.mdx` file with `import` statements or JSX components you don't want in the markdown output (HTML is left alone—valid Markdown, semantic intent preserved). `recent.map(...)` becomes a bullet list because the `md` tag joins arrays with newlines automatically.

### HTML → Markdown for static `.astro` pages without a markdown source

For pages whose content lives in HTML (an `about.astro` with prose mixed into JSX, for example), use `fromHtml`:

~~~ts
// src/pages/about.md.ts
import { fromHtml } from "astro-slop";

export const GET = fromHtml("/about");
~~~

`fromHtml` fetches the HTML at the given URL and runs it through [Defuddle](https://github.com/kepano/defuddle), which detects the main content (stripping nav, footer, sidebars), cleans it up, and emits Markdown. No selector or exclude config needed—Defuddle's heuristics handle the common cases. Pass `defuddleOptions` to forward extra Defuddle settings if you need them.

Works in `astro dev` and SSR modes. For pure static prerender, prefer the source-driven patterns above—HTML → MD round-trip is lossy for code blocks (Shiki) and embedded components.

## Customizing `/llms.txt`

The integration auto-generates `/llms.txt` as a flat bullet list of every `.md` route. To group, reorder, add sections, or replace entirely, write your own endpoint at `src/pages/llms.txt.ts`—when present, the plugin's auto-injection skips and yours wins.

`defaultLlmsTxt(context)` splices the auto-generated body into your wrapping content. Pass the endpoint's `APIContext` so it works in dev and SSR (where it fetches each `.md` route at request time); in static builds it returns a sentinel that the integration's `astro:build:done` hook replaces with the disk-derived body. Simplest case—append a static "About" section:

~~~ts
// src/pages/llms.txt.ts
import type { APIRoute } from "astro";
import { md, defaultLlmsTxt } from "astro-slop";

export const GET: APIRoute = async (context) => {
  const body = md.string`
${await defaultLlmsTxt(context)}

## About this site

Custom site description, hours, contact info, or anything else you want under a top-level heading.
`;

  return new Response(body, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
};
~~~

`md.string` is the variant of the `md` tag that returns a string instead of a `Response`—useful when you need to construct the `Response` yourself (here, with `text/plain` for the `.txt` extension).

If you want the extra section sourced from a content collection (e.g., a list of recommended external resources), the same pattern works—load the collection inside the handler:

~~~ts
// src/pages/llms.txt.ts
import type { APIRoute } from "astro";
import { md, defaultLlmsTxt } from "astro-slop";
import { getCollection } from "astro:content";

export const GET: APIRoute = async (context) => {
  const links = await getCollection("links");

  const body = md.string`
${await defaultLlmsTxt(context)}

## Recommended reading

${links.map((l) => `- ${md.link(l.data.title, l.data.url)} — ${l.data.summary}`)}
`;

  return new Response(body, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
};
~~~

## Customizing `/llms-full.txt`

`/llms-full.txt` reuses `/llms.txt`'s site header (`# Site` plus the `> description` blockquote) and then **inlines every route's full body content** as a flat list of `### Title` sections instead of bullet links. Same source data, different density.

~~~
# My Site

> Site description

### Welcome to My Site

[full body of /index.md, frontmatter stripped]

### About

[full body of /about.md]

### My First Entry

[full body of /entries/first-entry.md]

### Hello, World

[full body of /entries/hello-world.md]
~~~

Override by writing `src/pages/llms-full.txt.ts`:

~~~ts
// src/pages/llms-full.txt.ts
import type { APIRoute } from "astro";
import { defaultLlmsFullTxt } from "astro-slop";

export const GET: APIRoute = async (context) => {
  const body = await defaultLlmsFullTxt(context);
  return new Response(`${body}\n## Notes\n\nExtra context here.\n`, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
};
~~~

Each section uses an `###` heading with the route's title (frontmatter `title:` or first H1) followed by the body with leading frontmatter stripped—so titles aren't repeated and the document reads top-to-bottom.

## API reference

~~~ts
import {
  default as slop,        // the integration
  md,                     // tagged template, returns Response
  fromHtml,               // HTML → Markdown helper, returns GET handler
  cleanMdx,               // strip MDX-specific syntax from a body string
  defaultLlmsTxt,         // returns auto-gen llms.txt body as string
  defaultLlmsFullTxt,     // returns auto-gen llms-full.txt body as string
} from "astro-slop";
~~~

### The `md` tag

Tagged template that returns a `Response` with `Content-Type: text/markdown; charset=utf-8`.

~~~ts
md`# Hello`
// → Response { body: "# Hello\n", headers: { "Content-Type": "text/markdown; charset=utf-8" } }
~~~

**Behavior:**
- Real newlines work—no `\n` escapes needed
- Arrays in interpolation are joined with `\n` automatically
- Leading whitespace from template indentation is dedented (computed from template strings only—interpolated values aren't affected)
- Output has `\n`-terminated lines and a single trailing newline

~~~ts
const items = ["- a", "- b", "- c"];

md`
  # Heading

  ${items}
`
// → Response with body:
// "# Heading\n\n- a\n- b\n- c\n"
~~~

### `md.string`

Same template behavior as `md`, returns a `string` instead of a `Response`. Use for sub-composition or when you need a custom `Response` shape (e.g., `text/plain` for `.txt` URLs):

~~~ts
const intro = md.string`
  # Intro

  Welcome.
`;

const fullDoc = md.string`
  ${intro}

  ## Section 2

  More content.
`;

return new Response(fullDoc, { headers: { "Content-Type": "text/plain" } });
~~~

### Inline helpers

These return `string`s—call them inside template interpolations:

| Helper | Returns | Example |
|---|---|---|
| `md.link(text, url)` | `[text](https://github.com/yaroslav/astro-slop/blob/main/url)` with `[`/`]` in `text` escaped | `md.link("Hello [world]", "/foo")` → `"[Hello \\[world\\]](/foo)"` |
| `md.heading(depth, text)` | Markdown heading | `md.heading(2, "Posts")` → `"## Posts"` |
| `md.section(heading, body)` | `${heading}\n\n${body}` | `md.section("## Section", "- a")` → `"## Section\n\n- a"` |

### `fromHtml(url, options)`

Returns an Astro `GET` handler that fetches HTML from `url` and runs it through [Defuddle](https://github.com/kepano/defuddle) for content extraction + Markdown conversion in one pass.

~~~ts
fromHtml("/about");

// Optionally forward extra options to Defuddle:
fromHtml("/about", {
  defuddleOptions: { /* ... */ },
});
~~~

| Option | Type | Default | Effect |
|---|---|---|---|
| `defuddleOptions` | object | `undefined` | Pass-through to Defuddle. Whatever you set is merged with a forced `markdown: true` (the integration always emits markdown — your `markdown: false` is ignored). See [Defuddle's options](https://github.com/kepano/defuddle) for the full set. |

Defuddle handles main-content detection (`<main>` / `<article>` / smart fallbacks) and chrome removal automatically—no `selector` or `exclude` knobs to tune. The integration lazy-loads Defuddle and `linkedom` (its DOM dependency) on first request so they don't appear in the bundle if `fromHtml` is never called.

### `cleanMdx(body)`

Strips MDX-specific syntax from a content collection entry's raw body, so the result is plain Markdown suitable for `.md` output.

What gets stripped:
- Top-level `import` and `export` statements
- Self-closing JSX components: `<Component prop="..." />`
- Paired JSX components: `<Component>...content...</Component>`—content kept

What's preserved:
- All HTML—valid in Markdown, LLMs handle it, often carries semantic intent (`<address>`, `<time>`, `<cite>`, etc.). If you want HTML scrubbed, run a separate sanitizer pass.
- All actual prose, lists, code blocks, links, emphasis

~~~ts
import { cleanMdx } from "astro-slop";

const cleaned = cleanMdx(`
import Note from '../components/Note.astro';

# Title

<Note type="tip">Just a tip.</Note>

<span class="badge">Featured</span> introduction here.
`);
// → "# Title\n\nJust a tip.\n\n<span class=\"badge\">Featured</span> introduction here.\n"
~~~

The `<span>` is preserved (HTML is valid Markdown). The MDX `import` is stripped, and the `<Note>` JSX component is unwrapped to keep its content.

Use it on `entry.body` for collection entries that may use MDX:

~~~ts
md`...${cleanMdx(entry.body ?? "")}`
~~~

### `defaultLlmsTxt(context?: APIContext): Promise<string>`

Returns the auto-generated `/llms.txt` body. Use inside a custom `src/pages/llms.txt.ts` to extend.

- **Live request (dev / SSR):** fetches each `.md` route over HTTP using `context.request.url` as the base, crawls response bodies for links to per-entry siblings, and returns the assembled body. Pass `context` from your endpoint to get this path.
- **Static prerender, or no `context`:** returns a sentinel string. The integration's `astro:build:done` hook reads `dist/llms.txt`, `replaceAll`s the sentinel with the disk-walked auto-gen body, and writes it back. Your wrapping content (custom sections, etc.) is preserved; the auto-gen content lands in place of the sentinel.

### `defaultLlmsFullTxt(context?: APIContext): Promise<string>`

Returns the auto-generated `/llms-full.txt` body—site header followed by each route's full content inlined as `### Title\n\n${body}` sections. Same prerender / runtime split as `defaultLlmsTxt`; pass `context` from your endpoint.

## Examples

### Collection-backed routes

A common case: a content collection of `.md`/`.mdx` files (the example uses `posts`, but any name works), plus a listing page and per-entry detail pages. Two endpoint files cover both.

**`src/pages/posts/index.astro`**—the listing. Hoist the `posts` array to `export const` so the `.md.ts` sibling can import it:

~~~astro
---
import { getCollection } from "astro:content";

export const posts = (await getCollection("posts"))
  .sort((a, b) => b.data.date.valueOf() - a.data.date.valueOf());
---

<html>
  <body>
    <h1>Posts</h1>
    <ul>
      {posts.map((p) => <li><a href={`/posts/${p.id}`}>{p.data.title}</a></li>)}
    </ul>
  </body>
</html>
~~~

**`src/pages/posts/index.md.ts`**—the listing's markdown sibling:

~~~ts
import type { APIRoute } from "astro";
import { md } from "astro-slop";
import { posts } from "./index.astro";

export const GET: APIRoute = () => md`
# Posts

${posts.map((p) => `- ${md.link(p.data.title, `/posts/${p.id}.md`)} — ${p.data.description}`)}
`;
~~~

**`src/pages/posts/[slug].md.ts`**—per-post markdown. Re-export `getStaticPaths` from the HTML page so both routes resolve from the same source:

~~~ts
import type { APIRoute } from "astro";
import type { CollectionEntry } from "astro:content";
import { cleanMdx, md } from "astro-slop";

export { getStaticPaths } from "./[slug].astro";

export const GET: APIRoute<{ post: CollectionEntry<"posts"> }> = ({ props }) =>
  md`---
title: ${props.post.data.title}
description: ${props.post.data.description}
---

${cleanMdx(props.post.body ?? "")}`;
~~~

`/llms.txt` and `/llms-full.txt` are auto-generated from these endpoints. `<link rel="alternate" type="text/markdown">` is auto-injected into `/posts` and every `/posts/<slug>` HTML page.

### Single page from a markdown collection

A single page sourced from one entry in a content collection (e.g., a `pages` collection containing `home.md`, `about.md`, etc.). The `.astro` page renders the entry's content; the `.md.ts` sibling emits the raw body.

**`src/pages/about.astro`**—exports the entry so the sibling can read its body:

~~~astro
---
import { getEntry, render } from "astro:content";

export const aboutEntry = await getEntry("pages", "about");
const { Content } = await render(aboutEntry!);
---

<html>
  <body>
    <Content />
  </body>
</html>
~~~

**`src/pages/about.md.ts`**—interpolates the raw body, runs through `cleanMdx` to strip MDX/styling tags:

~~~ts
import type { APIRoute } from "astro";
import { cleanMdx, md } from "astro-slop";
import { aboutEntry } from "./about.astro";

export const GET: APIRoute = () => md`${cleanMdx(aboutEntry!.body ?? "")}`;
~~~

That's a single `.md` route at `/about.md`. The integration auto-injects the alt-link and includes the page in `/llms.txt`.

### Single HTML page that needs conversion

For pages whose content is hard-coded into JSX in an `.astro` file (no markdown source available), use `fromHtml` to fetch the rendered HTML and convert via Defuddle:

**`src/pages/contact.astro`** (illustrative—typical handwritten page with prose mixed into JSX):

~~~astro
---
const email = "hello@example.com";
---

<html>
  <body>
    <main>
      <h1>Contact</h1>
      <p>Reach out at <a href={`mailto:${email}`}>{email}</a>.</p>
      <p>Office hours: Mon-Fri 9-5 PT.</p>
    </main>
  </body>
</html>
~~~

**`src/pages/contact.md.ts`**—one line:

~~~ts
import { fromHtml } from "astro-slop";

export const GET = fromHtml("/contact");
~~~

`fromHtml` fetches `/contact`, runs Defuddle (main-content extraction + chrome stripping + Markdown conversion), and returns the result. The integration handles routing, alt-link injection, and `/llms.txt` inclusion automatically.

Mix and match freely: use the `md` tag for collection-backed routes (lossless, source-driven) and `fromHtml` for static `.astro` content (HTML→MD via Defuddle). The integration's manifest treats them uniformly—alt-links and `/llms.txt` entries work the same regardless of which shape produced the `.md` URL.

### Customizing `/llms.txt`

The auto-generated `/llms.txt` is a good default. To extend it (or replace it entirely), write your own endpoint at `src/pages/llms.txt.ts`—Astro's routing prefers user-defined files over integration-injected ones, so yours wins.

Append a static "About this site" section to the auto-gen:

~~~ts
// src/pages/llms.txt.ts
import type { APIRoute } from "astro";
import { md, defaultLlmsTxt } from "astro-slop";

export const GET: APIRoute = async (context) => {
  const body = md.string`
${await defaultLlmsTxt(context)}

## About this site

Custom site description, hours, contact info, or whatever else you want under a top-level heading.
`;

  return new Response(body, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
};
~~~

Or pull the extra section from a content collection—useful for adding curated external links the integration can't auto-discover:

~~~ts
// src/pages/llms.txt.ts
import type { APIRoute } from "astro";
import { md, defaultLlmsTxt } from "astro-slop";
import { getCollection } from "astro:content";

export const GET: APIRoute = async (context) => {
  const links = await getCollection("links");

  const body = md.string`
${await defaultLlmsTxt(context)}

## Recommended reading

${links.map((l) => `- ${md.link(l.data.title, l.data.url)} — ${l.data.summary}`)}
`;

  return new Response(body, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
};
~~~

### Customizing `/llms-full.txt`

Same pattern as `/llms.txt`. Write `src/pages/llms-full.txt.ts` and call `defaultLlmsFullTxt(context)` to wrap or extend.

Append a static "Notes for AI agents" section:

~~~ts
// src/pages/llms-full.txt.ts
import type { APIRoute } from "astro";
import { defaultLlmsFullTxt } from "astro-slop";

export const GET: APIRoute = async (context) => {
  const body = await defaultLlmsFullTxt(context);
  return new Response(
    `${body}\n## Notes for agents\n\nThis content is licensed CC BY 4.0. Attribution: https://example.com.\n`,
    { headers: { "Content-Type": "text/plain; charset=utf-8" } },
  );
};
~~~

Or pre-process the auto-gen—e.g., add a license preamble at the top:

~~~ts
// src/pages/llms-full.txt.ts
import type { APIRoute } from "astro";
import { defaultLlmsFullTxt } from "astro-slop";

export const GET: APIRoute = async (context) => {
  const preamble = `<!--\nThis document is licensed CC BY 4.0.\nGenerated ${new Date().toISOString()}.\n-->\n\n`;
  const body = await defaultLlmsFullTxt(context);

  return new Response(preamble + body, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
};
~~~

## How content negotiation works

When a request comes in to a route that has a `.md` sibling, the middleware checks the request's `Accept` header. If `text/markdown` (or `text/x-markdown`) is ranked strictly higher than `text/html`, the middleware returns a `302` redirect to the `.md` sibling.

~~~
GET /posts/hello-world
Accept: text/markdown

→ 302 Location: /posts/hello-world.md
~~~

Browsers never trigger this—their default `Accept` is `text/html, application/xhtml+xml, ..., */*` with no markdown listed, so HTML always wins. Clients that explicitly request markdown (curl scripts, RSS readers, well-behaved LLM crawlers) get redirected automatically.

The redirect carries a `Vary: Accept` header so downstream caches (Cloudflare, Vercel edge, nginx, etc.) treat it as Accept-conditional. Without `Vary`, a CDN could serve a redirect cached for an HTML-preferring client to a markdown-preferring one (or vice versa).

Test it locally with `bun run dev`:

~~~sh
curl -L -H "Accept: text/markdown" http://localhost:4321/posts/hello-world
# ↑ -L follows the 302; you get the .md content
~~~

~~~sh
curl http://localhost:4321/posts/hello-world
# ↑ default Accept; you get HTML as before
~~~

**Important limitation:** content negotiation only works at request time—`astro dev` and SSR mode. Static prerender bakes one file per URL into `dist/`, and the deployed file server has no awareness of `Accept`. Static-mode users would configure server-level negotiation (Apache `mod_negotiation`, nginx `try_files` + `Accept` matching, Cloudflare Workers, etc.) themselves; the integration can't substitute for that.

To disable: `slop({ contentNegotiation: false })`.

## How alt-link injection works

The integration handles `<link rel="alternate" type="text/markdown">` injection through two complementary paths, one per execution context:

1. **Runtime middleware**—registered via `addMiddleware` in `astro:config:setup`, fires on every dev/SSR response. The middleware skips prerendered routes entirely (those have no live request and would just trigger a `headers` access warning). For HTML responses with a registered `.md` sibling, it injects the `<link>` tag before `</head>`.

2. **Build-time HTML rewriting**—runs in `astro:build:done`, walks the emitted `dist/**/*.html` files, derives each one's URL pathname back from its file path, looks up the manifest, and injects the link tag inline. Handles all static prerender output.

The two paths are mutually exclusive: middleware handles request-time output, build:done handles prerender output. There's no overlap, no double-injection.

Both paths share the same insertion logic (in `src/alt-link.ts`)—same regex for finding the `</head>` tag, same already-injected detection, same tag format. The duplication of *execution context* (request-time vs. build-time) doesn't translate into duplicated *logic*.

The route manifest is built in `astro:routes:resolved` from Astro's resolved route table—every project route ending in `.md` becomes a manifest entry paired with its HTML sibling URL pattern. Static and dynamic patterns are handled identically.

To disable: `slop({ injectAlternateLink: false })`.

## How `/llms.txt` and `/llms-full.txt` are assembled

Two paths produce the same logical output from different sources:

**Static build** — at `astro:build:done`, the integration walks `dist/*.md`, matches each file to a route in the manifest, parses title (frontmatter `title:` or first H1) and description (frontmatter `description:`), strips frontmatter from the body, and writes `dist/llms.txt` and `dist/llms-full.txt`. If the user's `src/pages/llms.txt.ts` produced a sentinel-bearing file at prerender, the build:done hook splices the auto-gen body in place of the sentinel — preserving the user's wrapping content.

**Dev / SSR** — at request time, the injected `/llms.txt` and `/llms-full.txt` route handlers fetch every static `.md` route over HTTP using the incoming request's URL as the base, then crawl response bodies for markdown links (`[…](https://github.com/yaroslav/astro-slop/blob/main/foo.md)`) that match per-entry patterns from the manifest, recursively fetching those. Per-entry routes that aren't linked from any static parent listing won't be enumerated at request time (they still appear in static builds, since the disk walk doesn't depend on link structure).

`/llms.txt` formats each record as a `- [title](https://github.com/yaroslav/astro-slop/blob/main/mdUrl) — description` bullet, in URL-sorted order.
`/llms-full.txt` formats each record as a `### title\n\n${body}` section, in URL-sorted order, with frontmatter stripped from the body.

Auto-gen output is a single flat list — no `## Pages` / `## Posts` bucketing. To group, reorder, or add custom sections, write your own `src/pages/llms.txt.ts` and arrange your output around `defaultLlmsTxt(context)`.

## Verify your setup

Two third-party tools are useful for sanity-checking that your site exposes markdown the way LLM-aware clients expect:

- **[acceptmarkdown.com](https://acceptmarkdown.com/)**—sends a request with `Accept: text/markdown` and checks that your site responds with markdown (via content negotiation or the `.md` sibling).
- **[isitagentready.com](https://isitagentready.com/)**—broader audit: checks for `/llms.txt`, `<link rel="alternate" type="text/markdown">` injection, content negotiation, and other agent-readiness signals.
- **[contentsignals.org](https://contentsignals.org/)**—content-signals standard for declaring how your content can be used by AI agents and crawlers; useful complement to the structural checks above.

Run your site through both after deploying. With `astro-slop` configured per this README, you should pass cleanly: `/llms.txt` is auto-generated, `<link rel="alternate">` is injected on every HTML page that has a `.md` sibling, and content negotiation redirects markdown-preferring clients automatically.

## Caveats

- **`fromHtml` in static prerender**: Pre-rendering may run before the corresponding HTML file exists, so `fromHtml` can fail for purely static builds. Use `astro dev` to verify, or rely on the `md` tag's source-driven patterns above.
- **`.mdx` raw bodies**: `entry.body` for `.mdx` collection entries is the raw source—including `import` statements and JSX components like `<Note>` or `<Hero />`. Wrap it in `cleanMdx()` whenever you interpolate `entry.body` into a markdown response (per-entry endpoints AND composed pages like a homepage). Without it, the LLM sees raw MDX syntax alongside the content.
- **HTML is preserved**: `cleanMdx` deliberately leaves HTML alone. HTML is valid Markdown, LLMs handle it, and user-written HTML often carries semantic intent (`<address>`, `<time>`, `<cite>`). If you want HTML scrubbed, run a separate sanitizer pass after `cleanMdx`.
- **MDX cleanup is regex-based**: `cleanMdx` handles common cases (top-level imports/exports, simple JSX components) but isn't a full MDX parser. For complex MDX (nested JSX expressions, fragments, conditional rendering), inspect the output and extend if needed.
- **Override detection extensions**: the integration checks `.ts`, `.js`, `.mts`, `.mjs`, and `.astro` for user override files at `src/pages/llms.txt.*` and `src/pages/llms-full.txt.*`. Other extensions aren't checked—the integration's auto-gen will collide and Astro will emit a warning.
- **`md.link` escape coverage**: only escapes `[` and `]` in link text. Most other markdown-special chars don't break links—but if your link text contains parentheses, ampersands, or HTML entities and you need strict correctness, post-process or write the link by hand.
- **State sharing**: the integration shares state across module realms via `globalThis[Symbol.for("astro-slop:state")]`. This works for single-process Astro builds (the standard case). Multi-process setups (workers, custom isolation) may need a different mechanism—file an issue if you hit this.
- **Per-entry route enumeration in dev / SSR**: at request time, the injected `/llms.txt` and `/llms-full.txt` handlers fetch every static `.md` route and crawl response bodies for `[…](https://github.com/yaroslav/astro-slop/blob/main/foo.md)` links to discover dynamic per-entry routes. A per-entry route that isn't linked from any static parent listing won't be enumerated in dev / SSR responses (it still appears in static builds, since the disk walk doesn't depend on link structure). The README's listing-route pattern (`/posts.md` linking each `/posts/<slug>.md`) is the canonical way to make per-entry routes discoverable.

## Acknowledgements

Thanks to [Steph Ango](https://github.com/kepano) for [Defuddle](https://github.com/kepano/defuddle), the library `fromHtml` uses for content extraction and HTML → Markdown conversion.

Shout out to Pavel Grinchenko and Margarita Klubochkina.

## License

MIT

### Audition

Point it at a Ruby script, a gem, a Rack app, or a Rails root and it
tells you whether that code can run inside Ractors, why it cannot,
and how to fix it. Unlike a linter, audition does not stop at
pattern-matching your source: whole-program analysis is powered by
[rubydex](https://github.com/Shopify/rubydex), Shopify's Ruby
indexer, and the target is also loaded in a sandboxed subprocess
to observe real `Ractor::IsolationError`s on the live object
graph. Some of the checks and fixes were trained on how Rails
core itself is being ractorized; see the
[pattern study](https://github.com/yaroslav/audition/blob/main/docs/rails_core_best_practices.md).

- **Three probes, one verdict.** Per-file Prism AST checks,
  whole-program semantic analysis powered by
  [rubydex](https://github.com/Shopify/rubydex) (Shopify's code
  graph; class-level state is resolved across files and
  reopenings), and dynamic in-Ractor execution of the actual
  target.
- **Explains, not just flags.** Every finding carries a `why`
  (which rule of the Ractor model it violates) and a `fix`
  (what to write instead).
- **`--fix` like RuboCop, in two tiers.** Safe corrections:
  `.freeze` on string constants, `Ractor.make_shareable(...)` for
  mutable and shallow-frozen containers and Proc constants, and
  boot-time hoisting of method-body requires. `--fix-unsafe` adds
  semantics-affecting rewrites: magic-comment insertion,
  freeze-on-memoize for class-level memoization (both `@x ||=`
  and `return @x if defined?(@x)` idioms keep their caching, the
  memoized value becomes shareable, Rails-core style;
  `Ractor.store_if_absent` only for block initializers and
  invalidated caches), `autoload` to `require`, and write-once
  globals/class variables to frozen constants. `--dry-run`
  previews everything as a diff.
- **Dependency-aware.** Runtime findings are attributed to their
  source via `const_source_location`; when your own code is clean
  but a dependency is not, the verdict is a distinct `blocked`
  state, so `globalid` is not blamed for ActiveSupport's state.
- **Dogfooding.** The scanner for Ractor compatibility is built
  using Ractors: static analysis fans out across CPU cores on
  Ractor workers, and audition passes its own audit.
- **Trained on Rails core.** Several checks and fix suggestions
  come straight from studying the Rails ractorization effort
  (about 75 substantive commits): `Hash.new` default procs,
  in-place mutation of registry constants, closure-carrying
  `define_method`, copy-on-write rewrites. The findings are
  documented in
  [docs/rails_core_best_practices.md](https://github.com/yaroslav/audition/blob/main/docs/rails_core_best_practices.md).
- **Terminal-native output.** Colors, glyphs, and OSC 8 hyperlinks;
  `path:line` is clickable in supporting terminals. JSON output for
  CI.

```console
$ audition worker.rb
* audition 0.1.0 ruby 4.0.6 · script at .

  worker.rb
    x raises inside a Ractor: Ractor::IsolationError: can not
      access global variable $jobs from non-main Ractor
      why: The script ran fine on the main Ractor but failed under
      Ractor.new; the static findings usually pinpoint the line.
    x worker.rb:1  write to global variable $jobs
      why: Non-main Ractors cannot access global variables; this
      raises Ractor::IsolationError the moment the line executes
      in a Ractor (verified on Ruby 4.0).
      fix: Pass the value into the Ractor explicitly
      (Ractor.new(value) { |v| ... }) or over a Ractor::Port; for
      per-Ractor state use Ractor.current[:key].
    x worker.rb:4  read of global variable $jobs
      ...

  dynamic probes
    x script probe failed (details above)

  summary: 3 errors
  verdict: x not ractor-ready
$ echo $?
1
```

And the whole-bundle view:

```console
$ audition Gemfile.lock --static-only
╭───────────────┬─────────┬───────────┬────────┬──────────┬─────────╮
│ gem           │ version │ verdict   │ errors │ warnings │ fixable │
├───────────────┼─────────┼───────────┼────────┼──────────┼─────────┤
│ activesupport │ 8.1.0   │ not ready │    157 │       97 │      77 │
│ i18n          │ 1.14.7  │ not ready │     48 │       40 │      45 │
│ mail          │ 2.9.1   │ not ready │     27 │        4 │      13 │
│ rack          │ 3.2.6   │ not ready │     23 │       45 │      60 │
│ ...           │         │           │        │          │         │
╰───────────────┴─────────┴───────────┴────────┴──────────┴─────────╯
0 of 11 gems ractor-ready
```

**Requires Ruby 4.0 or newer**, strictly: the tool targets the modern
Ractor API (`Ractor::Port`, `Ractor#value`, main-Ractor `require`
proxying) and its verified semantics.

> **Warning:** The entire codebase was written by Claude Fable 5 (Anthropic).
> It has a thorough spec suite and was validated against real
> gems, but no human has reviewed every line. Be wary; read before
> you trust, especially `--fix` rewrites.

## Installation

```console
gem install audition
```

Or in a Gemfile:

```ruby
gem "audition", require: false
```

## Usage

```console
audition worker.rb            # a script: static + run inside Ractor
audition my_gem               # an installed gem, by name
audition path/to/gem-checkout # a gem working copy (*.gemspec)
audition path/to/rack-app     # a config.ru directory
audition path/to/rails-root   # a Rails application
audition lib                  # any directory, static-only
audition Gemfile.lock         # sweep every gem in the bundle
audition path/to/app --deps   # same, from the app root
```

Useful flags:

| Flag | Effect |
| --- | --- |
| `--deps` | sweep the target's Gemfile.lock gem by gem |
| `--write-baseline` / `--no-baseline` | record / ignore known findings |
| `--fix` | apply safe corrections, then re-check |
| `--fix-unsafe` | also apply semantics-affecting corrections |
| `--dry-run` | with a fix flag: preview edits, change nothing |
| `--format json` | machine-readable report for CI |
| `--format github` | GitHub Actions annotations on PR diffs |
| `--compare old.json` | delta vs a previous report: fixed/introduced |
| `--static-only` / `--dynamic-only` | pick one probe layer |
| `--fail-on warning` | stricter CI gate (default: error) |
| `--capabilities` | table of what this Ruby allows in Ractors |
| `--timeout 60` | dynamic probe budget in seconds |
| `--plain` | no colors or hyperlinks (also via NO_COLOR, pipes) |

Exit codes: `0` clean, `1` findings at or above the `--fail-on`
threshold (or a failed dynamic probe), `2` usage error.

## Adopting incrementally

Nobody goes from 150 findings to zero in one commit. Three tools
keep the gate useful from day one:

**Baseline.** Record today's findings, then fail CI only on new
ones:

```console
audition . --write-baseline    # writes .audition-baseline.json
audition .                     # exit 0; summary shows "N baselined"
```

The ledger stores per-check-per-file counts, so line drift never
invalidates it. `--no-baseline` shows everything again.

**Inline pragmas.** Silence a single line, rubocop-style:

```ruby
$legacy_flag = true # audition:disable global-variables
risky_call          # audition:disable
```

**Project config.** `.audition.yml` at the target root
(CLI flags always win):

```yaml
fail_on: warning
timeout: 60
exclude:
  - legacy/**
  - db/schema.rb
checks:
  disable:
    - at-exit
```

## What it catches

Static, with file:line precision:

- **Global variables**, with a verified allowlist: `$stdout`, `$~`,
  `$!`, `$VERBOSE` writes and friends stay legal.
- **Class variables**, resolved on the rubydex graph.
- **Class-level instance variables**, unified across the class
  body, `def self.`, and `class << self`, across files; the classic
  `@cache ||= {}` and `return @x if defined?(@x)` memoizations.
- **Constants that are not deeply shareable**: bare mutable
  literals, interpolated strings, and the subtle shallow freeze
  (`[[1], [2]].freeze` still raises; audition explains why).
  Honors `# frozen_string_literal:` and
  `# shareable_constant_value:` magic comments.
- **Sync primitives and Procs in constants** (Mutex, Queue,
  lambdas), including `Hash.new { }` default procs, which stay
  unshareable even after `.freeze`.
- **Registry-style constant mutation** (`RENDERERS << key`,
  `LOOKUP[k] = v`) and **`define_method` with a literal block**
  (the method carries an unshareable Proc); both patterns and
  their fixes come from the Rails core ractorization study.
- **Runtime require and autoload** (serializes all Ractors through
  the main-Ractor proxy).
- **`Ractor.new` blocks capturing outer locals** (the ArgumentError
  at creation time), resolved through Prism's exact scope depths.
- **Hostile or removed APIs**: `Ractor.yield`/`take` (gone in 4.0),
  ActiveSupport `class_attribute`/`cattr_*`/`mattr_*`,
  `include Singleton`, `fork`, `ObjectSpace._id2ref`, ENV mutation.

Dynamic, on the live object graph:

- Runs scripts inside a real Ractor (via `load`, which is not
  proxied) and reports the actual exception.
- Requires a library, then sweeps every constant it introduced with
  `Ractor.shareable?`, and inspects every class and module for
  class-level ivars and class variables, with
  `const_source_location` attribution.
- Boots `config.ru` and serves one GET / entirely inside a Ractor,
  the per-worker model of Ractor web servers; then hammers it from
  4 Ractors x 25 requests to surface failures that only appear
  under concurrency.
- Boots Rails (`config/environment.rb`), eager-loads, and sweeps
  the application's namespaces.

## Agent skill

This repository ships a `ractor-readiness` skill that teaches
coding agents (Claude Code and friends) the full audition
workflow: audit, fix tiers, suite-parity verification, and
incremental adoption. It lives in
[skills/ractor-readiness/SKILL.md](https://github.com/yaroslav/audition/blob/main/skills/ractor-readiness/SKILL.md).

Install into Claude Code as a plugin:

```
/plugin marketplace add yaroslav/audition
/plugin install audition@audition
```

Or install the skill with the
[skills CLI](https://github.com/vercel-labs/skills):

```console
$ npx skills add yaroslav/audition
```

## Extending

Checks are written in a small declarative DSL and can be registered
from outside the gem:

```ruby
class NoSleep < Audition::Static::Checks::Base
  check_name "no-sleep"

  explain :sleepy,
          severity: :warning,
          message: "sleep inside potential Ractor code",
          why: "Blocking one Ractor blocks its whole OS thread.",
          fix: "Prefer Ractor::Port#receive with a timeout."

  on :call_node do |node|
    flag(node, :sleepy) if node.name == :sleep && !node.receiver
  end
end

Audition::Static::Checks.register(NoSleep)
```

`on` generates the Prism visitor and always continues traversal;
`explain` entries are a message catalog with `%{placeholders}`.

## Development

```console
bundle install
bundle exec rake spec       # RSpec suite
bundle exec rake standard   # standardrb lint
lefthook install            # pre-commit lint hook
bundle exec exe/audition --capabilities
```

Static scanning is Ractor-parallel on large targets (one worker
per core, minus one for the main Ractor); audition's own `lib/`
passes `audition lib` clean.

The design notes in `docs/design.md` include the empirically
verified Ruby 4.0 Ractor semantics table that the checks are
calibrated against.

## Assisted by

Claude Fable 5.

## License

MIT. See LICENSE.txt.

### Bundlebun

**bundlebun** bundles [Bun](https://bun.sh)—the all-in-one JavaScript runtime, package manager, and build tool—directly into your Ruby gem bundle. No Docker, no `curl | sh`, no Homebrew required.

* **Zero setup.** `bundle add bundlebun && rake bun:install` and you're done.
* **Team-friendly.** Everyone runs the same Bun version, pinned in `Gemfile.lock`.
* **Auto-updating.** New Bun release? bundlebun ships it automatically. Or freeze your preferred version.
* **Integrates automatically.** Works out of the box with vite-ruby, cssbundling-rails, jsbundling-rails, and ExecJS.

```sh
bundle add bundlebun
rake bun:install
bin/bun --version
```

---

## Rationale

**Modern frontend setup is needlessly complex** and may involve a lot of maintenance. Developers need _at the very least_ a JavaScript runtime (typically, Node.js), a package manager (could be npm, yarn, or pnpm), and a build tool (Vite, Webpack, esbuild, Parcel—dozens of them).

- One way forward is to dockerize development environments, creating unnecessary headaches for the development team—both frontend and backend engineers (especially if the team is not that large and the project is not that complex).
- Another approach is to declare front-ops bankruptcy and pursue the "no-build" route.

**What if we could simplify this?** **Bun** is a **JavaScript runtime**, optimized for speed and developer experience. Bun is _also_ a fast JavaScript **package manager**. Bun is _also_ a **build tool**. Bun is also distributed as a single executable file.

However, Bun still requires [some installation](https://bun.sh/docs/installation), and we need to make sure everyone on the team is using the same version.

So, how about we just pack it into a Ruby gem as a binary and allow developers to stay updated? Then, we'll be ready every time a new Bun version is out—or the user can freeze their desired version within their Ruby project. There are no setups, large READMEs with instructions, and no enforcing the Docker workflow.

**Enter bundlebun**. With a fast JavaScript runtime and a package manager included, you can even skip the build tool and use Bun itself.

## Install

bundlebun gem releases include a binary distribution of Bun for each supported Bun platform (macOS, Linux, Windows) and architecture. bundlebun is tested for Unix-like environments and Windows.

Add bundlebun to your `Gemfile`, placing it _after_ your existing frontend-related gems:

```ruby
# Frontend-related gems go here
# gem "vite_rails"

gem "bundlebun"
```

Then run:

```sh
bundle add bundlebun
rake bun:install
```

`rake bun:install` creates a `bin/bun` binstub and auto-detects which integrations to enable. It will also offer to migrate `package.json` scripts and `Procfile` entries to use `bin/bun`.

_If you're seeing a message like `Could not find gems matching 'bundlebun' valid for all resolution platforms (aarch64-linux, aarch64-linux-gnu <...> )`, [check this article](https://github.com/yaroslav/bundlebun/wiki/Could-not-find-gems-matching-'bundlebun'-valid-for-all-resolution-platforms)._

_Windows:_ `bin\bun.cmd` is created instead. If you joined a project with only the Unix binstub, run `rake bun:install` again.

## Usage

### Binstub

The easiest way to run Bun is via the `bin/bun` binstub:

```sh
bin/bun install
bin/bun add postcss
bin/bun run build
```

### PATH

bundlebun prepends the bundled Bun directory to your application's `PATH`. Tools that detect a `bun` executable (like vite-ruby) find it automatically—no extra configuration needed.

### Rake

Alternatively, you can use a Rake task. The syntax is far from perfect—that's a limitation of Rake—but it's an option if you cannot install the binstub. Note the quotes around the parameters:

```sh
rake bun[command]  # Run bundled Bun with parameters
```

```sh
> rake "bun[outdated]"
bun outdated v1.1.38 (bf2f153f)
...
```

### Ruby API

```ruby
Bundlebun.('install')               # exec: replaces the current Ruby process with Bun
Bundlebun.call(['add', 'postcss'])  # same thing, array form
```

**Note:** `Bundlebun.call` (and the `()` shortcut) replaces the current process—it never returns. Use `Bundlebun.system` to run Bun and continue executing Ruby:

```ruby
if Bundlebun.system('install')
  puts 'Dependencies installed!'
end

success = Bundlebun.system('test')
# => true if Bun exited successfully, false or nil otherwise
```

See the [API documentation](https://rubydoc.info/gems/bundlebun) for full details on `Bundlebun::Runner`.

### Instrumentation

When ActiveSupport is available, bundlebun emits events you can subscribe to:

```ruby
ActiveSupport::Notifications.subscribe('system.bundlebun') do |event|
  Rails.logger.info "Bun: #{event.payload[:command]} (#{event.duration.round(1)}ms)"
end
```

Events: `system.bundlebun` (for `Bundlebun.system`) and `exec.bundlebun` (for `Bundlebun.call`). Payload: `{ command: args }`.

## Integrations

bundlebun auto-detects and loads integrations when you run `rake bun:install`—as long as `gem "bundlebun"` is placed _after_ the relevant gems in your `Gemfile`.

### vite-ruby / vite-rails

[vite-ruby](https://github.com/ElMassimo/vite_ruby) and [vite-rails](https://vite-ruby.netlify.app/) are gems that make Ruby and Rails integration with [Vite](https://vite.dev/), a great JavaScript build tool and platform, seamless and easy.

The bundlebun integration is installed automatically with `rake bun:install`, or you can run it explicitly:

```sh
rake bun:install:vite
```

That will make sure you have a `bin/bun` binstub. Next, it installs a custom `bin/bun-vite` binstub to use in build scripts, and creates or updates `vite.json` to use that binstub for building. See the [Vite Ruby configuration manual](https://vite-ruby.netlify.app/config/index.html) for details on `vite.json`.

To enable the integration manually:

```ruby
Bundlebun::Integrations::ViteRuby.bun!
```

### cssbundling-rails / jsbundling-rails

[cssbundling-rails](https://github.com/rails/cssbundling-rails) and [jsbundling-rails](https://github.com/rails/jsbundling-rails) are Rails gems that support the traditional CSS and JS building pipeline for Ruby on Rails.

Be sure to check both gems for documentation on bootstrapping your frontend build pipeline instead of duplicating approaches here. cssbundling-rails, for instance, includes an excellent sample build configuration for Bun.

```sh
# Bootstrap cssbundling-rails
bundle add cssbundling-rails
bin/rails css:install:[tailwind|bootstrap|bulma|postcss|sass]

# Bootstrap jsbundling-rails
bundle add jsbundling-rails
bin/rails javascript:install:bun

# Ensure bundlebun integration
rake bun:install:bundling-rails
```

### ExecJS

[ExecJS](https://github.com/rails/execjs) runs JavaScript code straight from Ruby. It supports a number of runtimes it can launch—and get a result from. Bun runtime support already exists in ExecJS; bundlebun just ensures it uses the bundled version.

Works automatically when bundlebun loads after ExecJS in your `Gemfile`. To enable manually:

```ruby
Bundlebun::Integrations::ExecJS.bun!
```

## Versioning

bundlebun versions follow the `#{gem.version}.#{bun.version}` scheme. For example, `0.1.0.1.1.38` = gem version `0.1.0` + Bun `1.1.38`.

New gem versions are published automatically on each Bun release. Lock to a specific Bun version in your `Gemfile`, or leave it unspecified to always get the latest.

## Uninstall

```sh
bundle remove bundlebun
```

Then clean up:

- `bin/bun` (and `bin\bun.cmd` on Windows)
- `bin/bun-vite` if present
- `tasks/bundlebun.rake` if present
- Any `bin/bun` references in scripts and configs
- Any `Bundlebun` references in your code

## Acknowledgements

Bun binaries are sourced directly from [oven-sh/bun releases](https://github.com/oven-sh/bun/releases). [Bun](https://bun.sh) was created by Jarred Sumner ([@jarred-sumner](https://github.com/jarred-sumner)) and distributed under MIT.

Big thanks to Jason Meller ([@terracatta](https://github.com/terracatta)) for his work on Bun's Ruby on Rails ecosystem integration—jsbundling-rails, cssbundling-rails, turbo-rails, stimulus-rails, and ExecJS support—in [this Pull Request](https://github.com/rails/rails/pull/49241).

## Contributing

Make sure you have up-to-date Ruby. Run `bin/setup` to install gems, install [lefthook](https://github.com/evilmartians/lefthook), and run `rake bundlebun:download` to fetch a local Bun build for tests.

Run `rake rspec` to check tests. Open an issue or a PR.

## License

The gem is available as open source under the terms of the MIT License. See [LICENSE.txt](https://github.com/yaroslav/bundlebun/blob/main/LICENSE.txt).

### Callback Hell

> You live in it.

Callback Hell is a Ruby gem that analyzes your Ruby on Rails application models and provides actionable insights on callbacks and validations. Use it to identify models that might benefit from refactoring, spot callback pollution from gems and associations, and keep your models clean and maintainable.

## Why bother?

As Rails applications grow, callbacks can quickly spiral out of control. Callback Hell helps you:

- **Visualize callback complexity** across your entire application
- **Identify callback hotspots** that need refactoring attention
- **Track callback origins**: distinguish your code from Rails internals and gem callbacks
- **Spot inheritance issues** and understand callback propagation
- **Audit conditional callbacks** that might be hiding bugs

## Quick start

Add to your Rails application's `Gemfile`:

```ruby
gem "callback_hell", group: :development
```

And then:

```bash
bundle install
```

### Basic usage

Generate a complete analysis report (callbacks _and_ validations):

```bash
bin/rails ch
```

Sample output:

<img src="https://raw.githubusercontent.com/evilmartians/callback_hell/refs/heads/main/assets/report.png" width="758" height="767" alt="Callback Hell sample output">

Or run specific reports:

```bash
# Just callbacks
bin/rails ch:callbacks

# Just validations
bin/rails ch:validations
```

### Requirements

We support Ruby 3.0+ and Rails 7.0+.

## Usage

**Note:** you can use either `rake` or `bin/rails` as you wish.

### Command line options

All rake tasks support the following options:

#### Output formats

```bash
# Table format (default)
bin/rails ch:callbacks format=table

# Line format: detailed per-callback breakdown
bin/rails ch:callbacks format=line

# GitHub Actions format for CI/CD
bin/rails ch:callbacks format=github
```

#### Model filtering

```bash
# Analyze specific model by class name
bin/rails ch model=User

# Or by file path
bin/rails ch model=admin/user

# Works with namespaced models too
bin/rails ch model=Admin::User
```

#### Sorting

```bash
# Sort by callback count (default)
bin/rails ch sort=total

# Sort alphabetically
bin/rails ch sort=name
```

#### Analysis modes

```bash
# Default mode: your callbacks only
bin/rails ch mode=default

# Full mode: includes Rails internals and associations
bin/rails ch mode=full
```

#### Custom model paths

```bash
# Include models from engines or non-standard locations
bin/rails ch path=engines/admin/app/models,lib/models
```

### Understanding the output

#### Callback origins

- **Own**: Callbacks defined in your application code
- **Rails**: Built-in Rails framework callbacks
- **Gems**: Callbacks from external gems
- **Inherited**: Callbacks inherited from parent classes or modules

#### Special Categories

- **Conditional**: Callbacks with `:if` or `:unless` conditions
- **Associations**: Auto-generated by Rails associations (`has_many`, `belongs_to`, etc.)
- **Attributes**: Generated by Rails attribute features (encryption, normalization, etc.)

### Integration with CI/CD

You can try the GitHub Actions format to integrate with your CI pipeline:

```yaml
# .github/workflows/callback_analysis.yml
name: Callback Analysis
on: [push, pull_request]

jobs:
  analyze:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true
      - name: Analyze callbacks
        run: bin/rails ch format=github
```

## Credits and acknowledgements

Callback Hell is supposed to be a spiritual successor of the [arca](https://github.com/jonmagic/arca) gem by [Jonathan Hoyt](https://github.com/jonmagic).

The entire idea and a detailed specification for the gem were done by [Vladimir Dementyev](https://github.com/palkan) who initially offered it as a Ruby developer candidate take-home task for [Evil Martians](https://evilmartians.com/?utm_source=callback_hell).

Shout-out to [Adam Doppelt](https://github.com/gurgeous) for [table_tennis](https://github.com/gurgeous/table_tennis) that we use for formatting tables.

## Contributing

Bug reports and pull requests are welcome at [https://github.com/evilmartians/callback_hell](https://github.com/evilmartians/callback_hell).

## License

The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).

### Carbon Fiber

**Carbon Fiber is a Ruby Fiber Scheduler** with a native event loop (io_uring on Linux, kqueue on macOS). Install it, and your `Net::HTTP`, `TCPSocket`, `Mutex`, `Queue`, `Process.spawn` code becomes concurrent without any changes. Carbon Fiber also works as a plug-and-play backend for the **Async framework** ([async gem](https://github.com/socketry/async)).

In my benchmarks (see below), Carbon Fiber is the **fastest pure Ruby Fiber Scheduler** currently available.

Carbon Fiber is implemented using the **Zig** programming language and is powered by **[libxev](https://github.com/mitchellh/libxev)** by Mitchell Hashimoto, used in his **Ghostty** terminal emulator. It is one of the first Ruby native extensions written in Zig.

## Features

- **Very fast.** My benchmarks rank it as the fastest pure Ruby Fiber Scheduler overall. Uses **io_uring on Linux** and **kqueue on macOS**.
- **Plug and play with plain Ruby**, thanks to the Fiber Scheduler API—`Net::HTTP`, `TCPSocket`, `Process.spawn`, `Mutex`, `Queue`, DNS—all transparently concurrent under the scheduler; no gem-specific wrappers required.
- **Async (gem async) support** as a swappable backend. One call swaps the [Async](https://github.com/socketry/async) event loop for ours.
- **Pure-Ruby fallback**—when the native extension isn't available (Windows, for instance), Carbon Fiber uses a pure Ruby implementation behind the same API.

## Example

```ruby
require "carbon_fiber"
Fiber.set_scheduler(CarbonFiber::Scheduler.new)

urls.each do |url|
  Fiber.schedule { Net::HTTP.get(URI(url)) }  # plain Net::HTTP, runs concurrently
end
```

---

## Performance

AWS EC2 c7a.2xlarge, 8 dedicated vCPUs, Amazon Linux 2023, kernel 6.18.20, Ruby 4.0.2 + YJIT, io_uring. 5-run median.

Some benchmarks:

| Workload | Carbon Fiber | Async | Itsi | Carbon Fiber vs. Async |
|---|---|---|---|---|
| `http_server` | **49.380k req/s** | 30.823k req/s | 30.864k req/s | +60% |
| `http_client_api` | **19.500k req/s** | 16.721k req/s | timeout | +17% |
| `http_client_download` | **8.426k dl/s** | 7.062k dl/s | timeout | +19% |
| `tcp_echo` | **52.973k ops/s** | 32.330k ops/s | 32.046k ops/s | +64% |
| `cascading_timeout` | **4.668k ops/s** | 4.414k ops/s | error | +6% |
| `connection_pool` | **4.967k co/s** | 4.612k co/s | 4.954k co/s | +8% |

Wins on most workloads against Async, Itsi, fiber_scheduler, io-event, and libev. [See detailed benchmarks →](#benchmarks)

---

## Requirements

- Ruby 3.4 or 4.0.
- Linux (amd64 or arm64), macOS (Apple Silicon, x64 uses fallback*), Windows (fallback*).

Ships precompiled—no Zig compiler required.

_\* Fallback means pure Ruby code, worse performance._

## Installation

```
bundle add carbon_fiber
```

## Usage

### As a standalone fiber scheduler

Ruby's fiber scheduler protocol is built into the standard library. Install Carbon Fiber as the thread's scheduler and every blocking I/O call (`sleep`, `Net::HTTP.get`, `TCPSocket.new`, `Process::Status.wait`) yields automatically—no callbacks, no special wrappers.

See also: [Ruby Fiber Scheduler protocol](https://ruby-doc.org/core/Fiber/Scheduler.html).

#### Example: Parallel file downloads

Fetch a batch of files concurrently. Each fiber yields to the event loop while waiting for data; the scheduler runs all of them on a single thread.

```ruby
require "carbon_fiber"

require "net/http"
require "uri"

urls = %w[
  http://files.example.net/archive-1.tar.gz
  http://files.example.net/archive-2.tar.gz
  http://files.example.net/archive-3.tar.gz
]

scheduler = CarbonFiber::Scheduler.new
Fiber.set_scheduler(scheduler)

results = {}
urls.each do |url|
  Fiber.schedule do
    results[url] = Net::HTTP.get(URI(url))
  end
end

scheduler.run
Fiber.set_scheduler(nil)

puts "Downloaded #{results.size} files (#{results.values.sum(&:bytesize)} bytes total)"
```

The `Net::HTTP` code is unchanged from a sequential version. The scheduler intercepts every socket read and write, parks the fiber, and resumes it when the kernel signals readiness—all three downloads proceed in parallel.

#### Example: Parallel subprocess processing

Transcode a batch of video files using multiple `ffmpeg` processes, all monitored concurrently. `Process::Status.wait` yields to the scheduler while the child runs; no polling required.

```ruby
require "carbon_fiber"

jobs = {
  "output-1080p.mp4" => ["ffmpeg", "-i", "input.mov", "-vf", "scale=1920:1080", "output-1080p.mp4"],
  "output-720p.mp4"  => ["ffmpeg", "-i", "input.mov", "-vf", "scale=1280:720",  "output-720p.mp4"],
  "output-480p.mp4"  => ["ffmpeg", "-i", "input.mov", "-vf", "scale=854:480",   "output-480p.mp4"],
}

scheduler = CarbonFiber::Scheduler.new
Fiber.set_scheduler(scheduler)

statuses = {}
jobs.each do |label, cmd|
  Fiber.schedule do
    pid = Process.spawn(*cmd, out: File::NULL, err: File::NULL)
    statuses[label] = Process::Status.wait(pid)
  end
end

scheduler.run
Fiber.set_scheduler(nil)

statuses.each do |label, status|
  puts "#{label}: #{status.success? ? "ok" : "exit #{status.exitstatus}"}"
end
```

All three `ffmpeg` processes run in parallel. The scheduler uses a background thread per `process_wait` call and parks the fiber until the process exits.

---

### With the Async framework

Carbon Fiber implements the `IO::Event::Selector` interface, so it can replace Async's built-in event loop. Call `CarbonFiber::Async.default!` once at startup; every subsequent `Async { }` block uses our backend.

```ruby
require "async"
require "carbon_fiber/async"
CarbonFiber::Async.default!
```

Alternatively, set the environment variable: `IO_EVENT_SELECTOR=CarbonFiberSelector ruby app.rb`.

#### Example: Fan-out API calls

Call multiple upstream endpoints in parallel using `Async::Barrier`, then wait for all results.

```ruby
require "async"
require "async/barrier"
require "carbon_fiber/async"

require "net/http"
require "json"

CarbonFiber::Async.default!

ENDPOINTS = %w[/users /orders /products /inventory /settings]

Async do
  barrier = Async::Barrier.new
  results = {}

  ENDPOINTS.each do |path|
    barrier.async do
      response = Net::HTTP.get_response(URI("http://api.example.com#{path}"))
      results[path] = JSON.parse(response.body)
    end
  end

  barrier.wait

  puts "Loaded #{results.size} resources"
  results
end
```

#### Example: Rate-limited concurrent crawl

Fetch many pages concurrently while keeping no more than 10 connections open at once. `Async::Semaphore` limits in-flight requests; `Async::Barrier` tracks completion.

```ruby
require "async"
require "async/barrier"
require "async/semaphore"
require "carbon_fiber/async"

require "net/http"

CarbonFiber::Async.default!

pages = (1..200).map { |i| "http://data.example.com/records?page=#{i}" }

Async do
  barrier   = Async::Barrier.new
  semaphore = Async::Semaphore.new(10, parent: barrier)
  results   = []

  pages.each do |url|
    semaphore.async do
      results << Net::HTTP.get(URI(url))
    end
  end

  barrier.wait
  puts "Fetched #{results.size} pages"
end
```

---

## How It Works

The scheduler has two layers:

1. **Zig native core** (`ext/carbon_fiber_native/`) owns the event loop (libxev), ready queue, timer heap, and fiber chaining. Handles `io_wait`, `io_read`, `io_write`, `block`/`unblock`, timer-based sleep, and `timeout_after` directly in native code.

2. **Ruby shell** (`lib/carbon_fiber/scheduler.rb`) implements the Ruby Fiber Scheduler protocol, delegates to the native Selector, and falls back to a background thread for operations the native core doesn't cover (DNS, `process_wait`).

**Fiber chaining:** when a fiber parks, the native event loop calls `rb_fiber_transfer` directly to the next ready fiber, skipping a round-trip through the root fiber. This removes one context switch per scheduling decision on every `sleep`, `read`, and `write`.

If the native extension can't be loaded (on Windows, for example), a pure-Ruby fallback (`lib/carbon_fiber/native/fallback.rb`) provides the same Selector API using threads and condition variables.

---

## Benchmarks

AWS EC2 c7a.2xlarge, 8 dedicated vCPUs, Amazon Linux 2023, kernel 6.18.20, Ruby 4.0.2 + YJIT, io_uring. 5-run median.

### Ruby Fiber Schedulers (leading ones): Carbon Fiber vs. Async vs. Itsi

Measuring pure Ruby Fiber Scheduler performance (`Fiber.set_scheduler`).

| Workload | Unit | Carbon Fiber | Async | Itsi | Carbon Fiber vs. Async |
|---|---|---|---|---|---|
| `http_client_api` | req/s | **19,500** | 16,721 | timeout | +17% |
| `http_client_download` | dl/s | **8,426** | 7,062 | timeout | +19% |
| `http_server` | req/s | **49,380** | 30,823 | 30,864 | +60% |
| `tcp_echo` | ops/s | **52,973** | 32,330 | 32,046 | +64% |
| `connection_pool` | co/s | **4,967** | 4,612 | 4,954 | +8% |
| `fan_out_gather` | cyc/s | 2,022 | 1,923 | **2,094** | +5% |
| `db_query_mix` | qry/s | 1,660 | 1,623 | **1,662** | +2% |
| `cascading_timeout` | ops/s | **4,668** | 4,414 | error | +6% |

Enabling YJIT turned out to be very beneficial for Async as well—numbers here are with `--yjit` on both sides.

### Async framework: stock Async vs. Async + Carbon Fiber backend

Swapped the io-event selector for Carbon Fiber's native backend. Same Async code.

| Workload | Unit | Stock Async | Carbon Fiber | Delta |
|---|---|---|---|---|
| `http_client_api` | req/s | 17,119 | **17,323** | +1.2% |
| `http_client_download` | dl/s | 2,263 | **2,474** | +9.3% |
| `task_churn` | task/s | 119,477 | **120,985** | +1.3% |
| `condition_signal` | sig/s | 44,887 | **46,921** | +4.5% |
| `cascading_timeout` | ops/s | 4,414 | **4,472** | +1.3% |
| `tcp_throughput` | ops/s | 32,606 | **48,214** | +47.9% |

### Examples of how to run benchmarks

See [README](https://github.com/yaroslav/carbon_fiber/blob/main/benchmarks/README.md) in the `benchmarks/` directory.

```bash
# macOS smoke test
benchmarks/bench -t carbon,async -w http_client_api,tcp_echo

# Linux via Docker (io_uring)
benchmarks/core_docker -t carbon,async,itsi -r 5

# Async framework comparison
benchmarks/async_docker -r 5
```

Note that your results will differ depending on the operating system (io_uring or kqueue), system load and hardware, and there will be a lot of variance with certain workloads anyway. Try to benchmark on dedicated hardware, with setup (operating system, kernel) close to your production environment.

---

## Acknowledgements

Thanks to Mitchell Hashimoto for [libxev](https://github.com/mitchellh/libxev) (and Ghostty).

Thanks to furunkel for [zig.rb](https://github.com/furunkel/zig.rb).

Thanks to Samuel Williams for the [Async](https://github.com/socketry/async) framework and [ecosystem](https://github.com/socketry).

## License

MIT

### Inkmark

A very fast, feature-packed, AI-first Markdown gem for Ruby.

- **Very fast**. Up to 1.3× faster than redcarpet _(not CommonMark-conformant)_, about 3×–9× faster than other Ruby Markdown gems with native extensions. Built with Rust, based on [pulldown-cmark](https://github.com/pulldown-cmark/pulldown-cmark), uses SIMD.
- **No surprises**. CommonMark + GitHub Flavored Markdown conformance.
- **"Batteries included" approach**. Build lots of useful features, make them easy to use and as fast as possible.
- **Easy to use**. As simple as a one-method API. Pass options inline as a hash, set them one by one, or set default options for the entire application.
- **Feature-packed**. Server-side syntax highlighting with themes, frontmatter support, table of contents in Markdown and HTML, plain text export, extraction of headings/links/images, statistics (character and word count, likely document language, blocks count), lazy image loading attributes, emoji shortcodes, autolinks, heading IDs with Unicode-transliterated slugs, wikilinks, footnotes, tables, task lists, smart punctuation, hard wraps, "nofollow/noopener" on external links.
- **AI-first**. Two chunking primitives: heading-based with breadcrumbs and per-chunk character/word counts, and sliding-window with overlap for size-bounded chunks where headings are absent or uneven. Block-aware or word-aware truncation for context-window budgeting. Markdown-to-Markdown pipeline. Plain-text extraction for embedding models. Structured extraction of headings, images, links, code blocks—each carrying byte ranges back into the source.
- **Security conscious**. Raw HTML denied by default. Hostname and URL-scheme allowlists for both links and images. GFM tagfilter for dangerous tags. A Rust-backed gem.
- **Easy extension API**. Hook any element with a Ruby block—no subclassing, no intermediate AST, no HTML post-processing. Rewrite URLs, swap code blocks for your own renderer, drop subtrees, or just walk the document for analysis. Handlers fire inside the single-pass parser, so extension costs essentially nothing beyond the render itself—and far less than regexing over output HTML.

## Installation

    bundle add inkmark

Or in your `Gemfile`:

```ruby
gem "inkmark"
```

Ruby 3.3+ is supported.

The gem comes precompiled, a compiler toolchain is _not_ required for installation.

## Quick start

```ruby
require "inkmark"

# Class-method shortcut
Inkmark.to_html("**hello**")
# => "<p><strong>hello</strong></p>\n"

# Instance form
Inkmark.new("# Hello").to_html

# With options
Inkmark.to_html("hi <em>there</em>", options: { raw_html: true })

# Mutable options via accessor
g = Inkmark.new("# Table\n\n| a | b |\n|---|---|\n| 1 | 2 |")
g.options.tables = false
g.to_html  # tables render as paragraphs now
```

## Presets

Inkmark ships presets as opinionated shortcuts for common
rendering profiles. Pass one via `preset:` in the options hash; every
other option in the hash overrides the preset's values (deep-merging
for nested element-policy hashes). You can—and are recommended to!—override preset options as you see fit.

- **`:recommended`**: a curated profile for modern web content. On
  top of GFM, enables smart punctuation, auto heading IDs, lazy-loading
  images with an `http`/`https` scheme allowlist, autolinks,
  `rel="nofollow noopener"` on external links, a scheme allowlist for
  link destinations, emoji shortcodes, syntax highlighting, hard wraps,
  and frontmatter parsing.

  **This is a good starting point for most apps**. Still, you are expected to
  override individual options to match your specific needs (e.g. adding statistics and table of contents, tightening link/image allowlists to your own hostnames, turning off features you don't want).

- **`:trusted`**: `:recommended` plus raw HTML pass-through.
  **Dangerous.** Intended only for content you fully trust: internal,
  team-authored. With raw HTML on, Inkmark does no sanitization beyond
  the narrow GFM tagfilter (turn it off at your own risk); the caller is
  responsible for output safety. Do not apply this preset to anything a user can influence, directly or indirectly.

- **`:gfm`**: the bare default. CommonMark plus the core GFM extensions
  (tables, strikethrough, tasklists, footnotes, tagfilter). Strict,
  conservative, and matches the render profile of every other major
  GFM engine. Everything else is off.

- **`:commonmark`**: the minimum. Strict CommonMark. No GFM extensions, no
  typographics, nothing opinionated.

```ruby
# Recommended profile
Inkmark.to_html(md, options: { preset: :recommended })

# Recommended profile with stats and table of contents
Inkmark.to_html(md, options: { preset: :recommended, statistics: true, toc: true })

# Recommended profile, but disable smart punctuation
Inkmark.to_html(md, options: { preset: :recommended, smart_punctuation: false })

# Just GFM (the default)
Inkmark.to_html(md)
Inkmark.to_html(md, options: { preset: :gfm })     # equivalent

# Recommended profile with a tightened link-host allowlist
Inkmark.to_html(md, options: {
  preset: :recommended,
  links:  { allowed_hosts: ["*.example.com"] }
})

# Trusted content (raw HTML passes through—use with care)
Inkmark.to_html(internal_doc, options: { preset: :trusted })
```

## Options

GFM extensions are on by default; raw HTML rendering is off by default.
Pass a hash to `Inkmark.to_html` / `Inkmark.new`, or mutate an `Inkmark::Options`
instance via its accessors.

| Key | Default | Description |
|---|---|---|
| `gfm` | `true` | GFM conformance mode + tables, strikethrough, tasklists, and footnotes. |
| `gfm_tag_filter` | `true` | GFM "Disallowed Raw HTML" extension. When `gfm` and `raw_html` are both true, protects you from several predefined tags (`title`, `textarea`, `style`, `xmp`, `iframe`, `noembed`, `noframes`, `script`, `plaintext`). No effect when `raw_html: false`. |
| `tables` | `true` | GFM pipe tables with optional column alignment markers (`:---`, `:---:`, `---:`). |
| `strikethrough` | `true` | `~~text~~` renders as `<del>text</del>`. |
| `tasklists` | `true` | `- [ ]` and `- [x]` render as disabled checkboxes. |
| `footnotes` | `true` | `text[^1]` + `[^1]: body` renders as superscript links and footnote block. |
| `raw_html` | `false` | Pass raw HTML through unescaped. Off by default for untrusted-input safety. **When enabled, the caller is fully responsible for sanitizing the output—see the [Raw HTML](#raw-html) section.** |
| `smart_punctuation` | `false` | Convert `"..."` → `"..."`, `...` → `…`, `--` → `–`, `---` → `—`. |
| `headings` | `{ attributes: false, ids: false }` | Heading-related policy. `:attributes` enables `# Heading {#id .klass}` Markdown inline attribute syntax; `:ids` auto-generates `id="slug"` on every heading from its text, with automatic Unicode transliteration of non-English headings (duplicates get a counter suffix; user-supplied ids from `:attributes` win). Deep-merges over defaults—pass only the sub-keys you care about. |
| `images` | `{ lazy: false, allowed_hosts: nil, allowed_schemes: nil }` | Image-related policy. `:lazy` adds `loading="lazy" decoding="async"` to every `<img>`. `:allowed_hosts` is a glob allowlist for `<img src>` hostnames (see examples; non-matching images drop to alt text). `:allowed_schemes` is a URL-scheme allowlist—typical: `["http", "https"]` to block `data:` image URIs. Both allowlists default to `nil` (no filtering); `[]` deny-all-external. Deep-merges. |
| `links` | `{ autolink: false, nofollow: false, allowed_hosts: nil, allowed_schemes: nil }` | Link-related policy. `:autolink` auto-links bare URLs and emails with correct boundary detection. `:nofollow` adds `rel="nofollow noopener"` to external `<a>` tags. `:allowed_hosts` / `:allowed_schemes` are glob / scheme allowlists for `<a href>` (relative/anchor/mailto URLs are never filtered). Non-matching links unwrap to plain text. Deep-merges. |
| `emoji_shortcodes` | `false` | Replace gemoji-style `:shortcode:` sequences with their emoji character (`:rocket:` → 🚀). Unknown codes and codes inside code blocks are preserved. |
| `syntax_highlight` | `false` | Server-side syntax highlighting for fenced code blocks with a language tag. Uses the `syntect` Rust crate with CSS class output. Batteries included: pair with CSS from `Inkmark.highlight_css` for the theme stylesheet. |
| `hard_wrap` | `false` | Treat every single newline as a hard line break (`<br />`). By default a bare `\n` is a soft break rendered as a space. Enable for one-sentence-per-line content or when migrating from renderers that default to hard wraps. |
| `toc` | `false` | Collect a table of contents from headings. Accepts `true` / `false` for simple enable/disable, or a Hash like `toc: { depth: 3 }` to limit which heading levels appear in the rendered TOC (h1–h3 in that example; default is no limit). Enables `Inkmark#toc` which returns a `Inkmark::Toc` value object (`#to_markdown` / `#to_html` / `#to_s`). Implicitly enables `headings: { ids: true }`. Also populates a lightweight `Inkmark#statistics` with `heading_count`. Depth affects only the rendered TOC; `heading_count`, `extracts[:headings]`, and `chunks_by_heading` still see every heading. |
| `statistics` | `false` | Collect scalar document statistics during parsing: language detection, character/word counts, and `*_count` fields for headings, code blocks, images, links, and footnote definitions. See examples. For structured arrays of records, use `extract`. Implies `toc` and `headings: { ids: true }`. |
| `extract` | `nil` | Hash opting into structured extraction of specific element kinds. Keys: `:images`, `:links`, `:code_blocks`, `:headings`, `:footnote_definitions`—each `true`/`false`. When set, `Inkmark#extracts` returns a Hash keyed by the requested kinds, each with an Array of record Hashes including a `:byte_range`. `extract: { headings: true }` and `toc: true` trigger each other—one heading walk powers both surfaces. |
| `math` | `false` | Recognize `$inline and `$display$` math blocks. |
| `definition_list` | `false` | `term\n: definition` renders as `<dl>`. |
| `superscript` | `false` | `^text^` renders as `<sup>`. |
| `subscript` | `false` | `~text~` renders as `<sub>`. Conflicts with strikethrough—enable only one. |
| `wikilinks` | `false` | `[[Page]]` and `[[Page\|label]]` render as links. |
| `frontmatter` | `false` | Frontmatter (YAML metadata at the start of the document). Parsed and exposed via `Inkmark#frontmatter`; the block is stripped from rendered output. |

Options can be supplied in several ways:

```ruby
# As a hash at construction
Inkmark.to_html(md, options: { math: true, tables: false })

# Via mutable accessor
g = Inkmark.new(md)
g.options.math = true
g.options.tables = false
g.to_html

# Process-level defaults, to set in your application initializer
Inkmark.default_options.math = true
Inkmark.new(md).to_html  # picks up the default
```

Unknown option keys raise `ArgumentError` immediately, including via the
hash form—typos fail loudly:

```ruby
Inkmark.new("x", options: { taples: true })
# => ArgumentError: unknown Inkmark option: :taples
```

## Raw HTML

Raw HTML is suppressed by default. This is safe-by-default for rendering untrusted Markdown:

```ruby
Inkmark.to_html("<script>alert(1)</script>")
# => "<p>&lt;script&gt;alert(1)&lt;/script&gt;</p>\n"
```

Enable pass-through with `raw_html: true`; _only do this for trusted
input_:

```ruby
Inkmark.to_html("<em>keep me</em>", options: { raw_html: true })
# => "<p><em>keep me</em></p>\n"
```

> **Your responsibility.** With `raw_html: true` you are fully
> responsible for every `<tag>` that reaches the HTML output. Inkmark does not
> sanitize raw HTML beyond the narrow GFM tagfilter described below—it will
> happily emit `<img onerror="…">`, `<a href="https://github.com/yaroslav/inkmark/blob/main/javascript:…">`, `<style>`
> contents, and any other attack surface the source contains. Always pipe the
> output through a dedicated sanitizer (like [Loofah][] or
> [rails-html-sanitizer][]) before rendering untrusted content in a page.

[Loofah]: https://github.com/flavorjones/loofah
[rails-html-sanitizer]: https://github.com/rails/rails-html-sanitizer

Even with `raw_html: true`, the **GFM tagfilter** stays on by
default and escapes nine unsafe tag names—`title`, `textarea`, `style`, `xmp`,
`iframe`, `noembed`, `noframes`, `script`, `plaintext`. This is required for GFM conformance. Opt out with `gfm_tag_filter: false` (or `gfm: false`) if you need raw pass-through of those tags—trusted input only. The tagfilter is a narrow spec-compliance pass, **not** a sanitizer—the responsibility note above still applies in full.

```ruby
Inkmark.to_html("<script>alert(1)</script>", options: { raw_html: true })
# => "<p>&lt;script>alert(1)&lt;/script></p>\n"
```

## Host allowlists

Restrict which hostnames can appear in links and images by passing glob
patterns. Disallowed links have their `<a>` tags stripped (the link text
stays); disallowed images drop to their alt text (or disappear when alt
is empty). Relative URLs, anchors, `mailto:`, and other non-web schemes
pass through unchanged—only `http://` / `https://` URLs are matched.

```ruby
Inkmark.to_html(md, options: {
  links:  { allowed_hosts: ["example.com", "*.example.com"] },
  images: { allowed_hosts: ["{cdn,static,img}.example.com"] }
})
```

Patterns use glob syntax (same engine as `.gitignore`), **not regex**:

- `example.com`: exact host only
- `*.example.com`: any subdomain (matches `cdn.example.com`, `a.b.example.com`; does **not** match bare `example.com`)
- `{cdn,static}.example.com`: brace alternation for multiple explicit hosts
- `*.{example,trusted}.com`: combine wildcards and alternation

Hostnames are matched case-insensitively and ports are ignored. An empty
array `[]` blocks every external link or image while still allowing
relative URLs.

## URL scheme filtering

For rendering untrusted Markdown, opt in to scheme allowlists to block
`javascript:`, `data:`, and other dangerous URL schemes in links and
images:

```ruby
Inkmark.to_html(md, options: {
  links:  { allowed_schemes: ["http", "https", "mailto"] },
  images: { allowed_schemes: ["http", "https"] }
})
```

Disallowed links are unwrapped (text stays, `<a>` tags drop); disallowed
images drop to alt text. Relative paths, anchors, and protocol-relative
URLs pass through—no scheme to check.

```ruby
opts = { links: { allowed_schemes: ["http", "https"] } }

Inkmark.to_html("[click](javascript:alert(1))", options: opts)
# => "<p>click</p>\n"

Inkmark.to_html("![pic](data:image/svg+xml,<svg/onload=evil()>)",
               options: { images: { allowed_schemes: ["http", "https"] } })
# => "<p>pic</p>\n"   # dropped to alt text
```

**Scope:** scheme filtering applies to Markdown-emitted links and images
(`[text](https://github.com/yaroslav/inkmark/blob/main/url)` / `![alt](https://raw.githubusercontent.com/yaroslav/inkmark/main/url)`). Raw HTML `<a href>` / `<img src>` inside
`raw_html: true` content is *not* filtered—for that case use a
downstream HTML sanitizer like Loofah.

**Default:** filtering is off. Full CommonMark autolink conformance is
preserved (including uncommon schemes like `irc:` and `ftp:`). Add the
filter explicitly when rendering untrusted input.

## Statistics and extraction

Inkmark collects document metadata as a side effect of the single render pass.
Two independent options control what's exposed:

- **`statistics: true`** populates `Inkmark#statistics` with scalar counts and
  language detection—nothing you have to iterate.
- **`extract: { kind: true, ... }`** populates `Inkmark#extracts` with structured
  arrays of records. Opt into only the kinds you need; unasked-for arrays are
  never allocated.

```ruby
md = Inkmark.new(source, options: {
  statistics: true,
  extract: {
    images:               true,
    links:                true,
    code_blocks:          true,
    headings:             true,
    footnote_definitions: true
  }
})
md.to_html

md.statistics
# => {
#      heading_count:             2,
#      likely_language:           "eng",
#      language_confidence:       0.93,
#      character_count:           142,
#      word_count:                28,
#      code_block_count:          1,
#      image_count:               1,
#      link_count:                2,
#      footnote_definition_count: 1,
#    }

md.extracts[:code_blocks]
# => [{ lang: "ruby", source: "puts \"hello\"\n", byte_range: 78...101 }]

md.extracts[:headings]
# => [
#      { level: 1, text: "Hello World",  id: "hello-world",  byte_range: 0...14 },
#      { level: 2, text: "Code Example", id: "code-example", byte_range: 68...83 }
#    ]
```

### Extract record shapes

| Kind                      | Fields                                         |
|---------------------------|------------------------------------------------|
| `:images`                 | `src`, `alt`, `title`, `byte_range`            |
| `:links`                  | `href`, `text`, `title`, `byte_range`          |
| `:code_blocks`            | `lang`, `source`, `byte_range`                 |
| `:headings`               | `level`, `text`, `id`, `byte_range`            |
| `:footnote_definitions`   | `label`, `text`, `byte_range`                  |

`byte_range` is an exclusive `Range` (`start...end`) pointing into the original
source string—slice with `source.byteslice(r.begin, r.size)` to recover the
raw Markdown. `source` on `:code_blocks` is pulldown-cmark's pre-filter code
content, so enabling `syntax_highlight: true` does not mutate it.

### Mutual trigger: `toc` ↔ `extract[:headings]`

One heading walk powers both the TOC renderer and the heading extract, so the
two options trigger each other. Enabling either gives you access to both
`Inkmark#toc` (with `#to_markdown` / `#to_html`) and `Inkmark#extracts[:headings]`.

```ruby
Inkmark.new(source, options: { toc: true }).extracts[:headings]
# => [{ level: 1, text: "Hello World", id: "hello-world", byte_range: 0...14 }, ...]
```

## Chunks extraction (for RAG)

`Inkmark.chunks_by_heading` splits a document by heading into an ordered
Array of section Hashes. Each section's `:content` is **filter-applied
Markdown**—emoji expanded, URLs autolinked, allowlists applied—serialized
back through pulldown-cmark. Designed as the first stage of a
chunk → embed → retrieve pipeline.

```ruby
sections = Inkmark.chunks_by_heading(readme)
sections.each do |s|
  puts "#{'#' * s[:level]} #{s[:heading]} (#{s[:id]})"
  puts s[:content]
end
```

Each entry:

```ruby
{
  heading:    "From source",           # String, or nil for the preamble
  level:      3,                       # 1-6, or 0 for the preamble
  id:         "from-source",           # slug, or nil for the preamble
  breadcrumb: ["Docs", "Installation"], # ancestor heading texts, root to parent
  content:    "Run `bundle install`...\n"  # filter-applied Markdown
}
```

Sections are **hierarchical**: a `##` section's `:content` includes any
nested `###` subsections, which also appear as their own entries. Content
before the first heading (if any) becomes a preamble entry with
`heading: nil` and `level: 0`.

`:breadcrumb` carries the ancestor heading texts from root to immediate
parent. Root-level sections and the preamble have an empty array. Skipped
levels are omitted, so an `###` directly under an `#` has `breadcrumb:
["Top"]`, not `["Top", nil]`. RAG pipelines typically prepend the
breadcrumb to each chunk before embedding—it gives the vector model a
cheap signal about the chunk's place in the document.

Enable `statistics: true` to add `:character_count` and `:word_count` to
every section entry. Counts reflect the section's filter-applied text
content including any code-block bodies (code is content for embedding
purposes, not just prose). Numbers across sections won't sum to the
document total because sections overlap hierarchically—a parent section's
count includes its nested subsections.

```ruby
Inkmark.chunks_by_heading(doc, options: {statistics: true})
# => [
#   { heading: "Installation", level: 2, id: "installation",
#     breadcrumb: ["Intro"],
#     character_count: 180, word_count: 32,
#     content: "..." },
#   ...
# ]
```

```ruby
Inkmark.chunks_by_heading(readme).each do |s|
  next if s[:heading].nil?  # skip preamble
  context = (s[:breadcrumb] + [s[:heading]]).join(" > ")
  embed_and_store("#{context}\n\n#{s[:content]}", metadata: {id: s[:id]})
end
```

### Picking specific sections

`chunks_by_heading` always returns the full array. Use plain `Enumerable`
to slice it however you need:

```ruby
sections = Inkmark.chunks_by_heading(readme)

# Find one by heading text
sections.find { |s| s[:heading] == "Installation" }

# Filter by regexp
sections.select { |s| s[:heading]&.match?(/install|usage/i) }

# All top-level headings only
sections.select { |s| s[:level] == 1 }

# Skip the preamble
sections.reject { |s| s[:heading].nil? }
```

No filter kwarg on the method—`.select` / `.find` / `.reject` already
cover every filtering shape, and you can compose conditions freely
(heading AND level, or heading NOT in a blocklist, etc.). The preamble
is a regular entry with `heading: nil` and falls out of Regexp/String
filters naturally (`nil == "Foo"` is false; `nil&.match?(x)` is nil).

### RAG pipeline caveat: HTML-emitting filters

**Disable `syntax_highlight`, `images: { lazy: true }`, and `links: { nofollow: true }`
when chunking for RAG.** These filters embed raw `<pre>…`, `<img loading=…>`,
and `<a rel=…>` HTML into the serialized Markdown; the HTML noise hurts
embedding quality for downstream semantic search.

```ruby
sections = Inkmark.chunks_by_heading(doc, options: {
  emoji_shortcodes: true,    # keep—improves semantic signal
  links: {
    autolink:        true,                # keep—proper anchor Markdown
    allowed_schemes: %w[http https mailto], # keep—safe URLs
    nofollow:        false                 # off—would embed <a rel=...> HTML
  },
  images: { lazy: false },   # off—would embed <img loading=...> HTML
  syntax_highlight: false    # off—would embed <pre><span...> HTML
})
```

### Scope

`chunks_by_heading` is a **structural chunking primitive**, not a
complete RAG chunker. It splits a document along heading boundaries.
For documents without headings—or when you need a strict size
budget regardless of document structure—reach for
[`chunks_by_size`](#sliding-window-chunking) below.

Inkmark does not ship token-based budgeting (there is no embedded
tokenizer). Use `character_count` / `word_count` or your own tokenizer
to approximate. Prepending document titles or parent-heading
breadcrumbs to each chunk is a few lines of Ruby on top of the array
this method returns.

### Sliding-window chunking

`Inkmark.chunks_by_size` splits a document into fixed-size chunks with
optional overlap, walking the filter-applied Markdown sequentially. Use
this when headings are absent or uneven, or when you need a strict
size budget for embedding input.

```ruby
# Char-budgeted windows with overlap
Inkmark.chunks_by_size(doc, chars: 500, overlap: 50)

# Word budget, word-boundary cuts
Inkmark.chunks_by_size(doc, words: 120, overlap: 15, at: :word)

# Dual budget: cut at whichever is reached first
Inkmark.chunks_by_size(doc, chars: 1000, words: 200)
```

Each window:

```ruby
{
  index:   0,         # 0-based sequence position
  content: "..."      # filter-applied Markdown slice
  # character_count, word_count added when options: { statistics: true }
}
```

**Boundary modes.** `at: :block` (default) cuts only between top-level
Markdown blocks—output stays valid Markdown, and a single block that
exceeds the budget is emitted as its own window rather than silently
dropped. `at: :word` serializes the full filtered Markdown and cuts at
the last Unicode word boundary that fits—tighter fit but may split
open constructs.

**Overlap.** Measured in chars. Each new window begins with the
trailing `overlap` chars of the previous window, so adjacent chunks
share context—useful when an embedding model's attention benefits
from neighbor overlap. Must be less than `chars:` when both are set.

**Validation.** `chars` or `words` required (at least one). Both must
be positive. `overlap` defaults to 0, must be non-negative, and must be
less than `chars` when `chars` is set. Invalid combinations raise
`ArgumentError` at the Ruby boundary—silent clamping would mask
bugs like swapped args.

#### Heading vs size: which to use

`chunks_by_heading` for docs where headings encode meaningful
structure (articles, specs, READMEs). Each chunk carries heading,
level, id, and breadcrumb metadata—retrieval benefits from that
context.

`chunks_by_size` for unstructured or uneven-heading docs, or when a
hard size ceiling matters more than document structure. No structural
metadata; windows are just positioned slices.

You can compose them for a hybrid "heading-based, but size-capped"
pattern:

```ruby
Inkmark.chunks_by_heading(doc).flat_map do |c|
  if c[:content].size > 2000
    Inkmark.chunks_by_size(c[:content], chars: 500, overlap: 50)
  else
    [c]
  end
end
```

## Truncation

`Inkmark.truncate_markdown` caps a document at a character and/or word
budget, cutting at either a Markdown block boundary (valid structure) or
a Unicode word boundary (tighter fit, may split an open construct).
Designed for LLM context-window budgeting and RAG chunk normalization.

```ruby
# Block-boundary cut: last complete block that fits, output is valid Markdown
Inkmark.truncate_markdown(doc, chars: 4000, at: :block)

# Word-boundary cut: last word that fits, output may split open constructs
Inkmark.truncate_markdown(doc, chars: 4000, at: :word)

# Dual budget: cut at whichever limit is hit first
Inkmark.truncate_markdown(doc, chars: 4000, words: 500, at: :word)

# Suppress the marker
Inkmark.truncate_markdown(doc, chars: 4000, at: :block, marker: nil)

# Custom marker
Inkmark.truncate_markdown(doc, chars: 4000, at: :block, marker: "[…]")
```

Default marker is `"…"`. When appended, it counts toward the
budget—`chars: 4000` always yields output ≤ 4000 codepoints.

**Behavior:**

- **Source fits the budget**: returned unchanged (no marker).
- **First block alone exceeds the budget** (block mode): empty string.
  Honest to "no block fits"; fall through to word-mode truncation if you
  want a best-effort cut.
- **Marker too large for the budget**: raises `ArgumentError`.
- **Filter pipeline**: `emoji_shortcodes`, `links: { autolink: true }`, host/scheme
  allowlists etc. run before truncation, so the measured output matches
  what downstream tools consume.

### Per-section truncation

`chunks_by_heading` accepts a `truncate:` kwarg that applies the same
contract to every section's `:content` independently:

```ruby
Inkmark.chunks_by_heading(doc, truncate: {chars: 500, at: :block})
```

Each section's content is cut to the 500-char budget; metadata
(`:heading`, `:level`, `:id`, `:breadcrumb`) stays intact. When
`statistics: true` is also set, `:character_count` / `:word_count` are
recomputed against the truncated content.

```ruby
Inkmark.chunks_by_heading(doc,
  options: {statistics: true},
  truncate: {chars: 500, at: :block, marker: "…"}
)
# => each entry: { heading:, level:, id:, breadcrumb:,
#                  character_count:, word_count:, content: (≤ 500 chars) }
```

Because sections are hierarchical (a parent section's `:content`
includes nested subsections), applying the same budget to every entry
means each chunk stands alone as a self-contained, budget-capped unit.

## Plain-text extraction

`Inkmark#to_plain_text` strips all Markdown syntax and returns inline content as
plain text. Designed for embedding models, token counting, LLM input, and any
downstream consumer that treats Markdown formatting as noise.

```ruby
Inkmark.to_plain_text("**bold** and [a link](https://example.com)")
# => "bold and a link (https://example.com)\n"

g = Inkmark.new(source, options: { emoji_shortcodes: true, links: { autolink: true } })
g.to_plain_text
```

The same event-level filters (emoji replacement, autolink, host/scheme
allowlists, etc.) run before plain-text serialization, so preprocessing passes
apply consistently across `to_html`, `to_markdown`, and `to_plain_text`.

### Output grammar

| Element | Plain-text form |
|---|---|
| `**bold**`, `*italic*`, `~~strike~~` | inner text only |
| `` `code` `` | inner text (no backticks) |
| `[text](https://github.com/yaroslav/inkmark/blob/main/url)` | `text (url)` |
| `<https://x.com>` (autolink) | `https://x.com` (collapses when text == url) |
| `![alt](https://raw.githubusercontent.com/yaroslav/inkmark/main/src)` | `alt (src)` |
| `# Heading` | plain text with blank line above/below |
| `> quote` | every line prefixed with `> ` (email-style; nests) |
| `- item` / `1. item` | `- ` / `1. ` bullets; 2-space indent per nesting |
| `- [x] task` | `- task` (checkbox dropped) |
| tables | header row `\t`-joined, blank line, body rows `\t`-joined |
| ``` ```code``` ``` | raw content, blank line above/below |
| `---` | `---` surrounded by blank lines |
| `[^foo]` | `[foo]` |
| `[^foo]: body` | appended at document end as `[foo]: body` |
| soft break | space |
| hard break | `\n` |
| raw HTML | stripped by default; passes through when `raw_html: true` |

Blank lines inside a blockquote emit a bare `>` marker (matching email quoting
conventions; no trailing whitespace).

## Markdown-to-Markdown pipeline

`Inkmark#to_markdown` runs the same event-level filter pipeline as `to_html` and
serializes the result back to Markdown text. Use it as a preprocessing step in
pipelines that consume Markdown: LLM prompts, secondary renderers, content
storage, or any stage that needs clean Markdown rather than HTML.

```ruby
# Class-method shortcut
Inkmark.to_markdown("**bold** :rocket:", options: { emoji_shortcodes: true })
# => "**bold** 🚀"

# Instance form—the same options object drives both outputs
g = Inkmark.new(source, options: {
  emoji_shortcodes: true,
  links: { allowed_hosts: ["trusted.com", "*.trusted.com"] }
})
g.to_markdown   # filtered Markdown for pipeline
g.to_html       # rendered HTML for display
```

### Choosing filters for a Markdown pipeline

Inkmark's filters fall into two groups depending on what they emit:

**Markdown-native filters** transform the event stream without producing HTML.
Their output is standard Markdown and is safe to pass to any downstream
consumer:

| Filter | Effect in `to_markdown` |
|---|---|
| `emoji_shortcodes` | `:rocket:` → `🚀` in the output text |
| `links: { autolink: true }` | bare `https://x.com` → `[https://x.com](https://x.com)` |
| `links: { allowed_hosts:, allowed_schemes: }` | disallowed links unwrapped to plain text |
| `images: { allowed_hosts:, allowed_schemes: }` | disallowed images dropped to alt text |
| `smart_punctuation` | `"..."` → `"…"` etc. (text-only transformation) |

**HTML-emitting filters** synthesize raw `<...>` markup. When these are active
and you call `to_markdown`, that markup is embedded verbatim in the output. Raw
HTML blocks are valid CommonMark, but they may break or confuse downstream
consumers—especially LLMs and renderers that do not expect HTML inside
Markdown:

| Filter | What ends up in the Markdown |
|---|---|
| `syntax_highlight` | fenced code blocks become `<pre><code><span class=...>` HTML |
| `images: { lazy: true }` | images become `<img loading="lazy" decoding="async" ...>` HTML |
| `links: { nofollow: true }` | links become `<a rel="nofollow noopener" ...>` HTML |

**Recommendation:** disable HTML-emitting filters when calling `to_markdown`.
They are designed for final HTML output and produce hard-to-process markup in a
Markdown pipeline:

```ruby
Inkmark.to_markdown(source, options: {
  # Markdown-native—safe to enable
  emoji_shortcodes: true,
  links:  { allowed_schemes: %w[http https mailto], nofollow: false },
  images: { lazy: false },

  # HTML-emitting—turn off for clean Markdown output
  syntax_highlight: false,         # would embed <pre><span...> blocks
})
```

## Event handlers

Register handlers with `#on` to inspect or transform document elements as they
are parsed. Handlers fire **post-order**—children before parents—so when a
`:table` handler runs, its rows and cells are already available. Returns `self`
for chaining.

```ruby
md = Inkmark.new(source)

md.on(:heading) { |h| ... }
  .on(:image)   { |img| ... }
  .on(:link)    { |l| ... }
```

Two entry points trigger handlers:

- **`#walk`**—fires handlers without producing HTML. Use it for analysis:
  collecting specific elements, validating content, extracting structured data.
  For built-in heading/link/image/word-count collection, see `statistics: true`.
- **`#to_html`**—fires handlers then renders. Mutations made inside a handler
  change what ends up in the HTML.

### Collecting data with `#walk`

```ruby
# Check that every image has alt text
md = Inkmark.new(source)
missing_alt = []
md.on(:image) { |img| missing_alt << img.dest if img.text.empty? }
md.walk
raise "Images missing alt text: #{missing_alt.join(', ')}" if missing_alt.any?
```

```ruby
# Collect every fenced code block language used in the document
languages = Set.new
md.on(:code_block) { |c| languages << c.lang if c.lang && !c.lang.empty? }
md.walk
```

```ruby
# Validate that no link points to a deprecated domain
deprecated = /old-docs\.example\.com/
md.on(:link) { |l| warn "Deprecated link: #{l.dest}" if l.dest =~ deprecated }
md.walk
```

### Rewriting output with `#to_html`

#### Image CDN rewriting

Set `dest=` to redirect images to a CDN. The change is reflected in the
rendered `<img src>`:

```ruby
md = Inkmark.new(source)
md.on(:image) do |img|
  img.dest = "https://cdn.example.net/#{File.basename(img.dest)}"
end
html = md.to_html
```

#### Rewriting link destinations

```ruby
md.on(:link) do |l|
  if l.dest.start_with?("http")
    l.html = %(<a href="#{l.dest}" target="_blank" rel="noopener">#{l.text}</a>)
  end
end
```

#### Shifting heading levels

Bump every heading down one level so the document fits inside a layout that
reserves `<h1>` for the page title:

```ruby
md = Inkmark.new(source)
md.on(:heading) { |h| h.level = [h.level + 1, 6].min }
html = md.to_html
```

#### Custom code block rendering

Intercept fenced code blocks by language tag. Setting `html=` skips Inkmark's
default `<pre><code>` output—and the `syntax_highlight` filter, even if
enabled:

```ruby
md = Inkmark.new(source)
md.on(:code_block) do |c|
  case c.lang
  when "mermaid"
    c.html = %(<div class="mermaid">#{c.text}</div>\n)
  when "math"
    c.html = %(<div class="math">\\[#{c.text}\\]</div>\n)
  end
end
html = md.to_html
```

#### Custom directives in paragraphs

Match a special directive syntax and replace the paragraph with a component:

```ruby
# Markdown:
#   @available_since rails=7.1 ruby=3.2
#
md.on(:paragraph) do |p|
  next unless p.text =~ /\A@available_since\s+(.+)\z/
  attrs = $1.scan(/(\w+)=(\S+)/).map { |k, v| %( #{k}="#{v}") }.join
  p.html = %(<AvailableSince#{attrs} />\n)
end
```

#### Replacing with Markdown

Use `markdown=` when the replacement is itself Markdown rather than raw HTML.
The replacement is parsed with the same options as the main document—emoji
expansion, heading IDs, raw HTML suppression—and is subject to the same
post-render filters (`syntax_highlight`, allowlists, `images: { lazy: true }`, `links: { nofollow: true }`).
Handlers do **not** fire on elements within the replacement.
`html=` takes priority when both are set on the same event.

```ruby
md = Inkmark.new(source)
md.on(:paragraph) do |p|
  if p.text.start_with?("@note ")
    body = p.text.sub(/\A@note /, "")
    p.markdown = "> **Note:** #{body}"
  end
end
html = md.to_html
```

#### Suppressing elements

Call `delete` on any event to omit it from the output. Children are suppressed
along with their parent:

```ruby
md.on(:image)   { |img| img.delete }                                  # all images
md.on(:heading) { |h|   h.delete if h.text.start_with?("INTERNAL:") } # by content
```

#### Inline code annotation

`:code` fires for inline backtick spans. Use it to add links or decoration:

```ruby
md.on(:code) do |c|
  if c.text =~ /\A[A-Z][A-Za-z]+#\w+\z/    # e.g. String#split
    c.html = %(<a href="/api/#{c.text.tr('#', '/')}"><code>#{c.text}</code></a>)
  end
end
```

### Children and tree context

Container elements expose their child events (lazy, cached):

```ruby
md.on(:table) do |t|
  rows = t.children_of(:table_row)
  rows.each_with_index do |row, i|
    cells = row.children_of(:table_cell).map(&:text)
    puts "Row #{i}: #{cells.join(' | ')}"
  end
end
```

Use `parent_kind` and `ancestor_kinds` for context-sensitive decisions:

```ruby
# Skip decorative images that are already inside a link
md.on(:image) { |img| img.delete if img.ancestor_kinds.include?(:link) }

# Only process top-level paragraphs
md.on(:paragraph) do |p|
  next unless p.parent_kind.nil?
  # ... only top-level paragraphs reach here
end
```

`depth` gives the nesting level (0 = top-level block):

```ruby
md.on(:blockquote) { |b| puts "blockquote at depth #{b.depth}" }
md.on(:paragraph)  { |p| puts "paragraph at depth #{p.depth}" }
# A paragraph inside a blockquote has depth 1.
```

### Source byte ranges

`byte_range` is an exclusive Ruby Range (`start...end`) that lets you slice the
original source to recover the raw Markdown for any element:

```ruby
source = File.read("post.md")
md = Inkmark.new(source)
md.on(:heading) do |h|
  puts "#{h.byte_range}: #{source.byteslice(h.byte_range).inspect}"
end
md.walk
```

Populated for all container kinds and the leaf kinds `:code`, `:rule`,
`:inline_math`, `:display_math`. Returns `nil` for `:text`, `:soft_break`,
and `:hard_break`. Also `nil` for `:link` when `links: { autolink: true }` is enabled
(the autolink filter inserts new link events that would shift the offset queue).

### Event object reference

Every handler receives an `Inkmark::Event` with these fields and methods:

| Field / method | Type | Description |
|---|---|---|
| `kind` | `Symbol` | Element kind, e.g. `:heading`, `:image` |
| `text` | `String` | Plain text of all descendant text nodes |
| `depth` | `Integer` | Nesting depth; 0 = top-level block |
| `parent_kind` | `Symbol, nil` | Kind of the immediate parent, or `nil` at root |
| `ancestor_kinds` | `Array<Symbol>` | Ancestor kinds, nearest first |
| `byte_range` | `Range, nil` | Byte offsets in the original source string |
| `children` | `Array<Event>` | Direct child events (containers only) |
| `children_of(kind)` | `Array<Event>` | Children filtered by kind |
| `delete` |—| Suppress this element from output |
| `deleted?` | `Boolean` | True if `delete` was called |
| `html=` | `String, nil` | Replace output with a raw HTML string |
| `markdown=` | `String, nil` | Replace output by re-rendering a Markdown string |
| `dest=` | `String, nil` | Rewrite URL on `:link` / `:image` |
| `title=` | `String, nil` | Rewrite title attribute on `:link` / `:image` |
| `level=` | `Integer, nil` | Change heading level (1–6) on `:heading` |
| `id=` | `String, nil` | Change `id` attribute on `:heading` |

#### Per-kind field availability

**Container kinds**: handler fires after all children are processed:

| Kind | Readable | Mutable |
|---|---|---|
| `:heading` | `text`, `level`, `id` | `level=`, `id=`, `html=`, `markdown=` |
| `:paragraph` | `text` | `html=`, `markdown=` |
| `:blockquote` | `text` | `html=`, `markdown=` |
| `:list` |—| `html=`, `markdown=` |
| `:ordered_list` |—| `html=`, `markdown=` |
| `:list_item` | `text` | `html=`, `markdown=` |
| `:code_block` | `text` (alias `source`), `lang` | `html=`, `markdown=` |
| `:table` |—| `html=`, `markdown=` |
| `:table_head` |—| `html=`, `markdown=` |
| `:table_row` | `text` | `html=`, `markdown=` |
| `:table_cell` | `text` | `html=`, `markdown=` |
| `:emphasis` | `text` | `html=`, `markdown=` |
| `:strong` | `text` | `html=`, `markdown=` |
| `:strikethrough` | `text` | `html=`, `markdown=` |
| `:link` | `text`, `dest`, `title` | `dest=`, `title=`, `html=`, `markdown=` |
| `:image` | `text` (alt), `dest`, `title` | `dest=`, `title=`, `html=`, `markdown=` |
| `:footnote_definition` | `text` | `html=`, `markdown=` |

**Leaf kinds**: no children; handler fires on the event itself:

| Kind | Readable | Mutable |
|---|---|---|
| `:code` | `text` | `html=` |
| `:text` | `text` | `html=` |
| `:html` | `text` | `html=` |
| `:rule` |—| `html=` |
| `:soft_break` |—| `html=` |
| `:hard_break` |—| `html=` |
| `:footnote_reference` | `text` | `html=` |

All kinds expose `depth`, `parent_kind`, `ancestor_kinds`, `byte_range`,
`children`, `children_of`, `delete`, `deleted?`.

`:code_block` `text` and `source` are identical—`source` is an alias for
readability when treating the field as raw source code.

### Filter interaction

Enrichment filters run **before** handlers. Handlers always see:

- Emoji already resolved (`emoji_shortcodes: true`)—`h.text` contains `"🚀"`,
  not `":rocket:"`
- Bare URLs already autolinked (`links: { autolink: true }`)—they appear as `:link` events
- Heading `id` already set (`headings: { ids: true }`)—`h.id` is populated

Post-render filters (`syntax_highlight`, allowlists, `images: { lazy: true }`,
`links: { nofollow: true }`) run **after** handlers:

- `:code_block` events are still `:code_block`, not opaque HTML, even when
  `syntax_highlight: true`—setting `html=` on a code block overrides the
  highlighter
- Handler-set `dest=` values pass through host and scheme allowlists

## Benchmarks

Inkmark ships a benchmark harness comparing it against `kramdown`,
`commonmarker`, `redcarpet`, `markly`, and `rdiscount` on a sweep of real
Markdown inputs.

Measuring apples to apples: every adapter is tuned for **feature parity** with
Inkmark's defaults—CommonMark + core GFM (tables, strikethrough, tasklists,
footnotes, tagfilter), no typographics, no autolink, no syntax highlighting,
no heading-id slugging.

Run locally:

```bash
bundle config set with benchmark
bundle install
bundle exec rake benchmark
```

### Assets

| Asset | Size | What it exercises |
|---|---:|---|
| `commonmark-spec` | 201.3 KB | CommonMark spec—code-block-heavy, edge-case-heavy |
| `commonmarker-readme` | 17.0 KB | Real-world commonmarker README—options tables, fenced code |
| `redcarpet-readme` | 14.0 KB | Real-world redcarpet README—prose + code samples |
| `redcarpet-benchmark` | 8.0 KB | Classic redcarpet bench corpus—heavy emphasis / inline parsing |
| `large-4k` | 3.7 KB | dotenv README—mixed prose, code blocks, tables |
| `medium-1k` | 1.0 KB | Faraday README header—images, badges, inline links |
| `small-512b` | 0.5 KB | Short README section with headings and bullet lists |
| `tiny-256b` | 0.3 KB | 3-line CommonMark snippet—parser setup/overhead-bound |

See `benchmarks/NOTICE` for attribution on the vendored test inputs.

### Results

Numbers below are from AWS EC2 `c7a.large` (AMD EPYC), Ruby 4.0.2 with YJIT on.
Each engine uses its idiomatic "hot path"—Inkmark relies on its cached default
options, Redcarpet reuses one pre-built `Markdown` object. Iterations per
second, higher is better.

**`commonmark-spec` (201.3 KB)**
```
inkmark:        1,172 i/s
redcarpet:        908 i/s -  1.29x slower
markly:           453 i/s -  2.59x slower
commonmarker:     345 i/s -  3.40x slower
rdiscount:        212 i/s -  5.53x slower
kramdown:          26 i/s - 45.08x slower
```

**`commonmarker-readme` (16.9 KB)**
```
inkmark:       16,658 i/s
redcarpet:     12,988 i/s -   1.28x slower
commonmarker:   4,268 i/s -   3.90x slower
markly:         3,974 i/s -   4.19x slower
rdiscount:      2,676 i/s -   6.22x slower
kramdown:         113 i/s - 147.42x slower
```

**`redcarpet-readme` (14.0 KB)**
```
inkmark:       17,343 i/s
redcarpet:     13,587 i/s -  1.28x slower
markly:         5,455 i/s -  3.18x slower
commonmarker:   4,890 i/s -  3.55x slower
rdiscount:      3,336 i/s -  5.20x slower
kramdown:         208 i/s - 83.38x slower
```

**`redcarpet-benchmark` (8.0 KB)**
```
inkmark:       27,634 i/s
redcarpet:     23,777 i/s -  1.16x slower
markly:         9,346 i/s -  2.96x slower
commonmarker:   7,805 i/s -  3.54x slower
rdiscount:      6,201 i/s -  4.46x slower
kramdown:         367 i/s - 75.30x slower
```

**`large-4k` (3.7 KB)**
```
inkmark:       64,051 i/s
redcarpet:     58,420 i/s -   1.10x slower
markly:        22,500 i/s -   2.85x slower
commonmarker:  18,053 i/s -   3.55x slower
rdiscount:     13,839 i/s -   4.63x slower
kramdown:         624 i/s - 102.64x slower
```

**`medium-1k` (1.0 KB)**
```
redcarpet:    216,968 i/s
inkmark:      213,478 i/s -  1.02x slower
markly:        70,251 i/s -  3.09x slower
commonmarker:  46,357 i/s -  4.68x slower
rdiscount:     45,880 i/s -  4.73x slower
kramdown:       2,813 i/s - 77.13x slower
```

**`small-512b` (0.5 KB)**
```
inkmark:      388,266 i/s
redcarpet:    368,401 i/s -  1.05x slower
rdiscount:     74,032 i/s -  5.24x slower
markly:        61,175 i/s -  6.35x slower
commonmarker:  46,658 i/s -  8.32x slower
kramdown:       3,952 i/s - 98.25x slower
```

**`tiny-256b` (0.3 KB)**
```
redcarpet:    535,972 i/s
inkmark:      511,019 i/s -   1.05x slower
rdiscount:     99,001 i/s -   5.41x slower
markly:        96,159 i/s -   5.57x slower
commonmarker:  57,704 i/s -   9.29x slower
kramdown:       4,117 i/s - 130.18x slower
```

## Contributing

Bug reports and pull requests are welcome on GitHub at
https://github.com/yaroslav/inkmark.

## Acknowledgements

Inkmark is built with:

[pulldown-cmark](https://github.com/pulldown-cmark/pulldown-cmark) by Raph Levien, Marcus Klaas de Vries, Martín Pozo, Michael Howell, Roope Salmi and Martin Geisler;

[Magnus](https://github.com/matsadler/magnus) by Matthew Sadler;

[syntect](https://github.com/trishume/syntect) by Tristan Hume, Keith Hall, Google Inc and other contributors;

And other Rust crates—thanks to their authors.

Thanks to Julik Tarkhanov for short but useful brainstorming sessions.

## License

The gem is available as open source under the terms of the
[MIT License](https://github.com/yaroslav/inkmark/blob/main/LICENSE.txt). Third-party content (benchmark assets, CommonMark
spec) is attributed in `NOTICE` and `benchmarks/NOTICE`.

### Kino

**Kino** is a high-performance **Ractor** web server for Ruby 4.0+.

Ruby threads cannot run Ruby code in parallel, so production setups fork
a process per core and pay for each copy in memory. Kino runs your code
on every core in **one small process**. A **Rust** (tokio + hyper)
front-end owns the network, parallel **Ractors** run your Rack 3 app,
and a threaded fallback mode runs everything else, Rails included.

* **Fast.** On a real 8-core server, every Kino mode is **1.5-2×**
  ahead of a Puma fork cluster on I/O-light endpoints. Ractor mode also
  wins on pure CPU, **30%+**. [Benchmarks](#benchmarks) below.
* **A fraction of the memory.** About **~7×** on the simplistic bench 
  Ractor app, and about **4× less memory** than a Puma cluster serving Rails in fallback threaded mode.
* **Parallel without forking.** Ractor mode runs CPU work **more than
  5× faster** than Kino's own GVL-bound threaded mode, in the same
  small process.
* **Production plumbing included.** Graceful drain, crash supervision
  and respawn, bounded queues with 503 backpressure, request timeouts,
  hardened intake (slowloris and TLS-handshake deadlines, connection
  and body-size caps), an `on_error` hook for your error tracker,
  TLS (rustls), live stats, async access and app logging.
* **Tells you why.** `kino --check` lists exactly what blocks your app
  from ractor mode, finding by finding, so you do not have to decode
  `Ractor::IsolationError` yourself.
* **Puma-shaped.** The same `workers × threads` topology, a familiar
  config DSL, a `kino` CLI. If you can run Puma, you can run Kino.

**N.B.:** Ractors are officially **experimental** in Ruby 4.0, and so is this server. The threaded mode is solid. Still, Kino aims to be the best way to experiment with Ractors today—and the best Ractor server when they become stable.

---

## Why

The GVL allows only one Ruby thread to run at a time. To use all cores,
Ruby servers fork processes, and every fork costs a full copy of the
app. Ractors do not have this limit: each one has its own lock, so one
process can run Ruby in parallel. What was missing is a server that
dispatches requests to them. Ruby 4.0 reworked Ractors (`Ractor::Port`,
`shareable_proc`, less lock contention) and made this worth building.

Why a Ractor server has to be built this way, and which Rust parts make
Ractors fast here: [doc/why-kino.md](https://github.com/yaroslav/kino/blob/main/doc/why-kino.md). The full design
notes live in [doc/architecture.md](https://github.com/yaroslav/kino/blob/main/doc/architecture.md).

## Benchmarks

Measured on a real server: AWS **c7a.2xlarge** (8-core AMD EPYC 9R14,
16 GB, Amazon Linux 2023). This is a realistic app-server size.

**These tables run a tiny synthetic Rack app**—plaintext, a 10 KB body,
a CPU-bound `fib`, a 5 ms wait—deliberately small, to measure the server
rather than an app. It is Ractor-shareable, so Kino runs it in `:ractor`
mode (and `:threaded` for comparison). **A real Rails app is a different
story:** it is *not* Ractor-shareable, so it runs only in Kino's
`:threaded` fallback, with its own numbers—see [Rails](#rails) below.
Ruby 4.0.5 with YJIT, every server at its defaults: Puma forks 8 workers ×
3 threads, Kino stays in one process (8 workers; 1 thread each in ractor
modes, 3 in threaded). Numbers are req/s by wrk (8-second windows, 64
connections, same host). Methodology:
[doc/benchmarks.md](https://github.com/yaroslav/kino/blob/main/doc/benchmarks.md).

| endpoint    | Kino :ractor | + lanes | :ractor, `workers 32`² | Kino :threaded | Puma (cluster) |
|-------------|-------------:|--------:|-----------------------:|---------------:|---------------:|
| /plaintext  |      229,534 | **250,222** |         182,997 |        216,994 |        118,176 |
| /10k        |      178,083 | **189,862** |         151,034 |        160,400 |        106,768 |
| /cpu (fib)  |   **77,999**¹|  70,885 |          66,100 |         13,429 |         58,006 |
| /io (5 ms)  |        1,552 |   1,551 |       **5,888** |          4,709 |          4,693 |
| /io_native  |        1,570 |   1,571 |       **6,274** |          4,695 |          4,691 |

Memory tells two different stories depending on the app, both by **PSS**
(proportional set size; see note) after sustained load.

**The tiny benchmark app** (Ractor-shareable, so Kino runs it in `:ractor`
or `:threaded`). Kino is **~7× lighter in :ractor mode, ~10× in :threaded**
than the Puma cluster — the gap stays large because a trivial app is almost
all private per-worker heap, which copy-on-write can't share:

| tiny app, Kino  | Kino (one process) | Puma cluster (8 workers) | ratio |
|-----------------|-------------------:|-------------------------:|------:|
| :ractor (8×1)   |         **148 MB** |                 1,068 MB |  ~7×  |
| :threaded (8×3) |         **107 MB**³|                 1,068 MB | ~10×  |

**A real Rails app** (not Ractor-shareable—Kino's `:threaded` fallback
only, [below](#rails)). The gap is **~4×**, smaller because Rails' large
framework *is* shared copy-on-write across Puma's forks:

| Rails hello-world | Kino :threaded | Puma cluster (8 workers) | ratio |
|-------------------|---------------:|-------------------------:|------:|
| **PSS**           |      **92 MB** |               **389 MB** |  ~4×  |

"+ lanes" is the experimental per-worker-queue dispatcher (`lanes true`).
It posts the fastest plaintext/10k of any configuration here. Details:
[doc/benchmarks.md](https://github.com/yaroslav/kino/blob/main/doc/benchmarks.md#lane-dispatch-experimental-lanes-true).

¹ Stock settings, no tuning. Ractor mode beats the fork cluster on pure
CPU by +34% (+22% with lanes). Threaded mode shows the GVL ceiling that
every single-process Ruby server hits. The old CPU-tuning recipe is
retired: its `threads 1` half **is** the default now, and its
`tokio_threads 1` half costs −12% on real hardware; see
[doc/benchmarks.md](https://github.com/yaroslav/kino/blob/main/doc/benchmarks.md#cpu-bound-tuning).

² Wait-bound throughput is slots ÷ wait, and the default columns bring
8 single-thread workers against the cluster's 24 threads. Kino slots
are threads, not processes—when your app waits a lot, raise `workers`.
The `workers 32` column is that tuning: **+25% over the cluster on /io
(+34% via `Kino.sleep`)** while still ahead of it on pure CPU, all in
one small process. The cost is the CPU-light rows (32 ractors
oversubscribe 8 cores); pick the topology your app's wait profile
needs. See
[doc/benchmarks.md](https://github.com/yaroslav/kino/blob/main/doc/benchmarks.md#why-io-lags-in-ractor-mode-on-linux).

³ With `MALLOC_ARENA_MAX=2` (the standard Ruby deployment setting;
Heroku's default). Without it, 24 threads churning 10 KB responses
through one glibc heap balloon to ~670 MB—an arena-fragmentation
footgun, not a leak, and ractor mode sidesteps it. See
[doc/benchmarks.md](https://github.com/yaroslav/kino/blob/main/doc/benchmarks.md#memory-under-load-and-the-glibc-arena-footgun).

A common first idea is to keep your current server and wrap the app in
a ractor pool. We measured that too (same box; the analysis is in the
doc):

| endpoint   | Kino :ractor (8×3) | Puma + ractor wrapper | Falcon + ractor wrapper |
|------------|-------------------:|----------------------:|------------------------:|
| /plaintext |        **193,826** |                19,480 |                  99,776 |
| /cpu (fib) |         **68,061** |                17,755 |                  48,721 |
| /io (5 ms) |          **4,530** |                 1,454 |                   1,549 |

### Rails

Rails is not Ractor-shareable today, so Kino serves it in `:threaded`
fallback — one GVL-bound process. On the same box (`examples/rails-hello`,
edge Rails, production, 8×5):

| Rails hello-world            |  req/s | memory (PSS) |
|------------------------------|-------:|-------------:|
| Kino :threaded (one process) |  2,637 |    **92 MB** |
| Puma cluster (8 workers)     | 12,138 |       389 MB |

The honest trade-off: Puma's fork cluster uses all 8 cores, so it serves
~4.6× the throughput — at ~4× the memory. Ractor-mode Rails would close
the throughput gap at one-process memory cost; the upstream blockers are
tracked in [doc/rails-on-ractors.md](https://github.com/yaroslav/kino/blob/main/doc/rails-on-ractors.md).

In short: on the tiny synthetic app, ractor mode beats fork-level CPU parallelism (**5.8×** Kino's
own GVL-bound threaded mode, +34% over the cluster) in one process, at
about 1/7th of the cluster's memory by PSS (~4× on a real Rails app).
Every Kino mode is 1.5-2.1× ahead of the cluster on I/O-light endpoints. The macOS numbers
(secondary; everything there hits the loopback ceiling) and the
YJIT × Ractors gotcha are in [doc/benchmarks.md](https://github.com/yaroslav/kino/blob/main/doc/benchmarks.md).

Reproduce: `bench/run.sh [seconds] [concurrency]` for the main table,
`bench/studies.sh` for the follow-ups (CPU recipe, topology, scaling,
logging, memory).

## Install

You need Ruby >= 4.0. Add Kino to your application's bundle:

```sh
bundle add kino      # or: gem install kino (outside a bundle)
```

or put it in the `Gemfile` yourself:

```ruby
gem "kino", "~> 0.1"
```

Then generate a config and serve:

```sh
bundle exec kino --init    # writes kino.rb; every directive documented in place
bundle exec kino           # picks up config.ru + kino.rb, serves on :9292
```

(After a standalone `gem install`, the `kino` command works without
`bundle exec`.)

No Rust compiler needed: released versions ship precompiled native gems
for Linux (x86_64/aarch64, glibc and musl) and macOS (arm64). On other
platforms the gem compiles at install time; that needs a Rust toolchain,
plus clang/libclang on Linux.

## Usage

```ruby
require "kino"

# Ractor mode needs a Ractor-shareable app: capture nothing, freeze config.
app = Ractor.shareable_proc do |env|
  [200, { "content-type" => "text/plain" }, ["Hello from #{Ractor.current}"]]
end

Kino::Server.run(app, port: 9292)   # traps INT/TERM; Ctrl-C drains gracefully
```

Or embedded, with everything spelled out:

```ruby
server = Kino::Server.new(app,
  bind: "127.0.0.1",
  port: 9292,                 # 0 = ephemeral; read back via server.port
  workers: Etc.nprocessors,   # ractors (parallelism)
  threads: 1,                 # per worker; ractor default 1, threaded default 3
  mode: :auto,                # :auto | :ractor | :threaded
  queue_depth: 1024,          # bounded queue; overflow → 503
  queue_timeout: 5.0,         # seconds before 503 on a full queue
  request_timeout: nil,       # seconds before a slow response becomes a 504 (nil = off)
  max_connections: 8192,      # cap concurrent connections; default: most of ulimit -n
  max_body_size: 50 * 1024 * 1024,  # bytes before a 413; nil = let a proxy handle it
  on_error: ->(e, env) { ErrorTracker.capture(e) },  # after the client got its 500
  shutdown_timeout: 30,       # drain deadline
  tls: { cert: "cert.pem", key: "key.pem" },  # file paths or inline PEM
)
server.start
server.shutdown               # graceful: drain → deadline → abort stragglers
```

### Modes

- **`:ractor`**: `workers` Ractors × `threads` Threads each. The app must
  be `Ractor.shareable?` (frozen middleware, `shareable_proc` endpoints).
  Forcing `:ractor` with an unshareable app raises
  `Kino::UnshareableAppError`. A crashed ractor returns 500 to its
  in-flight requests right away, then respawns.
- **`:threaded`**: the same machinery on `workers × threads` plain
  Threads. Runs **any** Rack app, including Rails, today. Parallel for
  I/O, serialized by the GVL for CPU.
- **`:auto`** (default): `:ractor` when the app is shareable, otherwise
  a warning and `:threaded`. One caveat: a *class* used as a Rack app
  always counts as "shareable" (classes are), even if calling it touches
  unshareable state. Force `:threaded` for those.

## Config file and CLI

Settings can live in a Puma-style Ruby DSL file. Precedence: explicit
kwargs and CLI flags > config file > defaults.

```ruby
# kino.rb
port 9292
workers 8
threads 1
mode :ractor
```

```sh
kino --init                   # write a fully commented sample kino.rb
kino                          # config.ru + kino.rb, port 9292
kino --check                  # explain whether the app can run in :ractor mode
kino -C config/kino.rb -p 3000 -w 4 -m ractor my_app.ru
```

The generated sample documents every directive, including the Rails
settings and the performance notes.

## `kino --check`

When an app cannot run in `:ractor` mode, Kino can tell you why, instead
of leaving you with a bare `Ractor::IsolationError`. The check changes
nothing (it does not freeze your objects) and names each blocker:
captured variables with the place they were defined, instance variables
by path, and the class-level instance variable trap that catches
class-style apps:

```
$ kino --check
check: app is NOT Ractor-shareable
  - app (Proc at app.rb:12)—captures `cache` = {} (Hash) (unshareable)
  - app (HelloApp).@instance—class-level ivar holds #<HelloApp…>—classes
    pass Ractor.shareable?, but reading this from a worker ractor raises
    Ractor::IsolationError on the first request
  hints: freeze config at boot; build endpoints with Ractor.shareable_proc;
  keep per-worker resources in Ractor.store_if_absent; or run mode :threaded.
```

Exit status is 0/1, so it works in CI. The programmatic form is
`Kino::Check.report(app)`.

## Request timeouts

`request_timeout: seconds` (or `request_timeout 30` in `kino.rb`) limits
how long the app may take to produce a response. Past the deadline the
client gets an immediate **504** while the handler keeps running; its
late response is dropped without harm. Off by default. The handler is
deliberately *not* killed, because interrupting arbitrary Ruby mid-flight
is unsafe. A stuck handler still occupies its worker slot until it
returns, so set the deadline above your slowest legitimate endpoint and
watch `stats[:timeouts]`.

Timeouts guard your app; the network intake guards itself. New
connections past `max_connections` (default: most of `ulimit -n`) wait
in the kernel backlog; request bodies past `max_body_size` (default
50 MB, `nil` delegates to a fronting proxy) get a **413**; and fixed
deadlines drop slow-header clients (15 s), stalled TLS handshakes
(10 s), and uploads stalled mid-body (30 s). When a worker catches an
app or delivery error,
`on_error ->(error, env) { ErrorTracker.capture(error) }` is called
after the client got its 500—the only place a tracker sees errors
raised while the response was being written (in `:ractor` mode, build
the handler with `Ractor.shareable_proc`).

## Stats

`server.stats` returns a live snapshot: the configuration plus counters
from the native layer (one relaxed atomic per request, no measurable
cost):

```ruby
server.stats
# => {mode: :ractor, lanes: false, workers: 8, threads: 1, batch: 1,
#     respawns: 0, queued: 0, in_flight: 2, served: 1041, rejected: 0,
#     timeouts: 0}
# plus lane_depths: [...] when lane dispatch is on
```

From the outside, `kill -USR1 <pid>` prints the same snapshot as one line
(pair it with `pidfile` to find the pid):

```
Kino stats: mode=:ractor lanes=false workers=8 threads=1 batch=1 respawns=0 queued=0 in_flight=2 served=1041 rejected=0 timeouts=0
```

## Logging

With one log line per request, `Kino::Logger` sustained **2.4× the
throughput of a shared `::Logger`** (149k vs 63k req/s on the benchmark
box). There are two native pieces. Both write through a lock-free
channel to a Rust flusher thread, so request threads never take a log
mutex and never make a write syscall:

- **Access log** (`log_requests true`): one line per request to stdout,
  including the 503s that never reach your app. Recommended in
  development; cheap enough for production. On color terminals the
  lines are tinted by status class: 2xx green, 3xx yellow, 4xx maroon,
  5xx bright red:

  ```
  127.0.0.1 [Tue, 10 Jun 2026 13:39:56 GMT] "GET / HTTP/1.1" 200 0.1ms
  ```

- **`Kino::Logger`**: a `::Logger` over the same async sink, for your
  app's own logging (`Kino::Logger.new("log/production.log")`, or no
  argument for stdout). The raw IO-like device is `Kino::Logger::Device`,
  for integrations that want bytes without `::Logger` formatting. The
  device is frozen and Ractor-shareable, so one device serves every
  worker.

`Kino::Logger` in a **Rails** app: it is a real `::Logger` subclass, so
it fits anywhere Rails expects a logger:

```ruby
# config/environments/production.rb, simplest forms:
config.logger = Kino::Logger.new                          # stdout
config.logger = Kino::Logger.new("log/production.log")    # file
# both file and stdout:
config.logger = ActiveSupport::BroadcastLogger.new(
  Kino::Logger.new("log/production.log"), Kino::Logger.new
)
# tagged logging wraps it like any ::Logger:
config.logger = ActiveSupport::TaggedLogging.new(Kino::Logger.new)
```

From a plain **Rack** app, give middleware the logger, or hand
`Rack::CommonLogger` the raw device (it just calls `write`):

```ruby
# config.ru
use Rack::CommonLogger, Kino::Logger::Device.new   # access-style app log
run MyApp
```

(If you only want request lines, prefer Kino's own `log_requests true`.
It is free for your Ruby threads, and it also sees the 503s that never
reach Rack.)

Graceful shutdown drains both logs fully. A hard crash can lose the tail
of the buffer, and when you log faster than the disk can take (over 100k
lines/s), the sink drops lines instead of blocking request threads.
These trade-offs are measured in
[doc/benchmarks.md](https://github.com/yaroslav/kino/blob/main/doc/benchmarks.md#logging-costs).

## Timer waits

`Kino.sleep(seconds)` is a high-resolution sleep on the OS clock with
the GVL released. MRI's own `sleep` wakes up late inside non-main
ractors (details and numbers in [doc/benchmarks.md](https://github.com/yaroslav/kino/blob/main/doc/benchmarks.md)).
Use `Kino.sleep` for explicit timer waits in handlers. Ordinary blocking
I/O does not need it.

## Rack 3 compliance

The spec suite runs every test app under `Rack::Lint` over real sockets:
streaming request bodies (forward-only `rack.input`), enumerable and
callable (full-duplex stream) response bodies, lowercase and multi-value
headers, HEAD/204 semantics. Full hijack is left out on purpose; it is
optional in Rack 3.

## Rails

Rails (edge) runs on Kino today in `:threaded` mode; see
`examples/rails-hello`. Ractor-mode Rails is blocked upstream. The exact
blockers, the `Ruby::Box` findings, and what would unlock it are written
up in [doc/rails-on-ractors.md](https://github.com/yaroslav/kino/blob/main/doc/rails-on-ractors.md). The example
ships a probe script that re-tests against whatever Rails you bundle.

## Development

```sh
bin/setup
bundle exec rake                       # compile, Rust tests, specs, RBS, lint
RB_SYS_CARGO_PROFILE=dev bundle exec rake compile   # fast dev rebuilds
```

## Assisted by

Claude Code (Mythos, Opus).

## Contributing

Bug reports and pull requests are welcome on GitHub at
https://github.com/yaroslav/kino.

## License

The gem is available as open source under the terms of the
[MIT License](https://opensource.org/licenses/MIT).

### gem nosj

**gem nosj** is an **very fast JSON parser and generator for Ruby**, written in Rust on the first-party [nosj](https://github.com/yaroslav/nosj) crate and **SIMD-accelerated** on every platform (NEON on Apple Silicon, SSE2/AVX2 on x86-64).

> gem nosj is the powerful evil twin of the json gem.

- It is **faster** than gem json and every
third-party parser, including Oj, RapidJSON, FastJsonparser, Yajl. 1.0–1.8× faster than the bundled json gem, 1.3–11× faster than Oj, and up to 17×
faster than Yajl—[see Benchmarks](#benchmarks).
- It has **lazy documents**: `NOSJ.lazy` wraps a document and parses a value only when you touch it—repeated access costs nanoseconds, and everything you never read is never parsed.
- It has a **partial parsing mode**: JSON Pointer lookups that pull single values out of big documents in microseconds, skipping everything else.
- It has **file APIs**: parse, generate, dig, and lazy-wrap files directly—no throwaway file-sized Ruby String, and the partial modes memory-map the file so unread pages never even leave the disk.
- It does **byte-splicing edits**: `NOSJ.splice` changes a field inside a passing payload 10–51× faster than parse-mutate-generate, and RFC 6902 JSON Patch / RFC 7386 merge patch apply straight to the raw string.
- It is **great to debug with**: parse errors carry line, column, and a caret snippet pointing at the break, and `NOSJ.stats` X-rays a mystery blob (depth, value counts, key histogram) faster than parsing it.
- It accelerates a **Rails** application in both encoding and decoding.
- It comes **precompiled** (platform gems built with per-platform optimizations,
nothing to compile on install).
- Otherwise, same API and option names as gem json.

**And there's more**: validate documents without building a single Ruby object, minify and pretty-print without parsing, resolve whole batches of paths in one pass, stream NDJSON / JSON Lines in both directions, and accelerate an entire application with a one-line drop-in.

## Requirements

- **Ruby 3.3 or newer** (CRuby; tested on 3.3, 3.4, and 4.0).
- **Linux** (x86-64 and arm64, glibc and musl), **macOS** (Apple
  Silicon), or **Windows** (x64): these platforms install precompiled,
  per-platform-optimized gems with nothing to build.

## Getting started

```bash
bundle add nosj
```

```ruby
require "nosj"

NOSJ.parse('{"a":[1,true]}')       #=> {"a" => [1, true]}
NOSJ.generate({"a" => [1, true]})  #=> '{"a":[1,true]}'
```

That's it—if you know the `json` gem, you already know `nosj`.

### gem json compatibility

Want the speedup without touching your code? One line reroutes
`JSON.parse`, `JSON.generate`, `JSON.pretty_generate`, and `JSON.dump`
through nosj:

```ruby
require "nosj/json"
```

In a Bundler app (Rails included) that can live entirely in the
Gemfile you can do this:

```ruby
gem "nosj", require: "nosj/json"
```

A MultiJson adapter ships too: 

```ruby
require "nosj/multi_json"
```

And then `MultiJson.use NOSJ::MultiJsonAdapter`.

### Ruby on Rails

In a Rails app, use this for "Rails mode":

```ruby
gem "nosj", require: "nosj/rails"
```

That installs a nosj-backed ActiveSupport JSON encoder, so `obj.to_json`, `render json:`, and `ActiveSupport::JSON.encode` walk the object tree natively:
values that aren't JSON-native recurse through `as_json` exactly like
ActiveSupport's own encoder, non-finite floats encode as `null`, and
HTML-safety escaping (`escape_html_entities_in_json`) behaves
identically—verified differentially against ActiveSupport's encoder.
It loads the drop-in too, so `ActiveSupport::JSON.decode` and JSON
request-body parsing ride the fast path. 

Measured against ActiveSupport's own encoder: ×1.9 on small documents
up to ×5.2 on large trees and ×14 on HTML-heavy content—see
[Benchmarks → Rails mode](#rails-mode).

## What's in the box

### json API

The `json` gem API, on the `NOSJ` module:

```ruby
NOSJ.parse(src, symbolize_names: true)   # also: freeze, max_nesting,
                                         # allow_nan, allow_trailing_comma
NOSJ.generate(obj)                       # indent, space, object_nl, ...,
NOSJ.pretty_generate(obj)                # ascii_only, script_safe, strict
```

### Lazy documents

`NOSJ.lazy` wraps a document in a lazy view: read
a field and only that path is parsed—containers stay lazy, scalars
arrive as plain Ruby values, and repeated reads are cached:

```ruby
doc = NOSJ.lazy(json)
doc["users"][3]["name"]   # parses only this path
doc.dig("meta", "count")  # a whole path in one fused resolution
doc["users"].size         # counted without materializing anything
doc["users"][3].value     # materialize one subtree (parse options apply)
```

Pass a frozen string and creating the view is practically free—
nanoseconds, even on a megabyte document. Malformed content raises
when it is first read, not at wrap time.

### Partial parsing

Pull values out of a document without
materializing the rest—skipped content is stepped over at SIMD block
speed, so a lookup costs what it skips, not what the document weighs:

```ruby
NOSJ.dig(json, "users", 3, "name")        # Hash#dig-shaped
NOSJ.at_pointer(json, "/users/3/name")    # JSON Pointer

# Many lookups in one pass. A batch costs about as much as its
# single deepest member:
NOSJ.at_pointers(json, ["/users/3/name", "/meta/count"])
NOSJ.dig_many(json, [["users", 3, "name"], ["meta", "count"]])
```

Example: an early field resolves in ~0.35µs where `JSON.parse(json).dig(...)`
costs ~980µs on the same document—three orders of magnitude. A field
at the far end of a 570 KB document costs ~71µs, still 13× faster
than parse-then-dig. Misses return nil; matched subtrees materialize
with the same options as `parse` (`symbolize_names:`, `freeze:`).

### Files API

Every mode has a file-native form, so a document never
round-trips through a throwaway Ruby String:

```ruby
NOSJ.load_file("config.json")                 # 1.3× File.read + parse
NOSJ.write_file("out.json", obj)              # generate straight to disk
NOSJ.dig_file("huge.json", "users", 3, "name")   # never reads the rest
NOSJ.at_pointer_file("huge.json", "/meta/count")
doc = NOSJ.load_lazy_file("huge.json")        # lazy view over a memory map
```

The partial and lazy forms memory-map the file, so pages you never
read are never loaded from disk. Missing files raise the usual
`Errno` exceptions. Measured numbers live in
[Benchmarks → File APIs](#file-apis).

### Reformat without parsing

`NOSJ.minify` and `NOSJ.reformat` pipe the parser's events straight
into the emission kernels: **zero Ruby objects** are built for the
document (the specs assert it), SIMD in and SIMD out. Minifying the
631 KB twitter.json takes 409µs—3.4× faster than a
`generate(parse(x))` round-trip, 3.9× faster than gem json's cycle,
and only 1.4× the cost of `valid?`:

```ruby
NOSJ.minify(json)                        # compact
NOSJ.reformat(json, pretty: true)        # pretty_generate layout
NOSJ.reformat(json, ascii_only: true)    # escape-transcode, no parse
NOSJ.reformat_file("big.json")           # straight off a memory map
```

Output is exactly `generate(parse(json))`—canonical number spellings,
normalized escapes, same formatting options—except duplicate keys pass
through (a reformatter must not silently drop data) and lone-surrogate
strings re-escape instead of raising. Acceptance options apply too:
`minify(src, allow_trailing_comma: true)` normalizes the commas away.

### Byte-splicing edits and JSON Patch

Change a field in a passing payload without the parse → mutate →
generate cycle: `NOSJ.splice` skips to the target spans (one pass for
the whole batch) and rebuilds the string around them—every byte
outside the targets is copied untouched, so formatting, key order,
and number spellings survive exactly. On twitter.json this is
**10×–51× faster** than parse-mutate-generate (129µs for a late
field, 26µs for an early one, vs ~1.3ms):

```ruby
NOSJ.splice(json, "/config/timeout" => 30)
NOSJ.splice(json, "/a" => 1, "/b/c" => [true])   # batch, one pass
```

On top of the same machinery: **RFC 6902 JSON Patch** applied to the
raw string (`add`, `remove`, `replace`, `move`, `copy`, `test`—the
whole appendix-A suite is in the specs), and **RFC 7386 JSON Merge
Patch**:

```ruby
NOSJ.patch(json, [
  {"op" => "test", "path" => "/a", "value" => 1},
  {"op" => "replace", "path" => "/a", "value" => 2},
  {"op" => "add", "path" => "/list/-", "value" => "x"}
])
NOSJ.merge_patch(json, {"config" => {"legacy" => nil, "timeout" => 30}})
```

Missing splice targets raise `KeyError` (use `patch` `add` to
insert); failed patches raise `NOSJ::PatchError`; inserted values are
byte-identical to `NOSJ.generate`'s output.

### NDJSON / JSON Lines

Logs, data pipelines, LLM batch files—one JSON document per line.
`NOSJ.each_line` streams values out (1.6× faster than the
line-split-and-`JSON.parse` idiom on twitter statuses), and
`NOSJ.generate_lines` builds the whole stream in one buffer pass
(4.1× faster than map-generate-join):

```ruby
NOSJ.each_line(log) { |event| ingest(event) }   # or an Enumerator:
NOSJ.each_line(log).first(10)                   # walks only 10 lines

NOSJ.generate_lines(events)          #=> %({"a":1}\n{"b":2}\n...)
NOSJ.each_line_file("events.ndjson") { |e| }    # over a memory map
NOSJ.write_lines("out.ndjson", events)          # straight to disk
```

Blank lines are skipped, one value per line is enforced, and a
malformed line raises a [rich `ParserError`](#rich-parse-errors) whose
`#line` is the physical line in the stream. Parse options apply per
line—`NOSJ.each_line(log, freeze: true)` yields Ractor-shareable
values.

### Validation without parsing

`NOSJ.valid?` runs the full parser—tokenizers, string decode, 
number validation—into a null sink and allocates no Ruby objects
at all. It is 2-4× faster than `NOSJ.parse`:

```ruby
NOSJ.valid?('{"a":1}')                #=> true
NOSJ.valid?('{"a":}')                 #=> false
NOSJ.valid?(src, max_nesting: false)  # same options as parse
```

### Document statistics

`NOSJ.stats` answers "what is this 40 MB blob": one counting pass 
through the same null-sink machinery—no Ruby value is built for 
the document, and it costs _less_ than a parse (~1.3× faster on `twitter.json`):

```ruby
s = NOSJ.stats(blob)          # or NOSJ.stats_file("huge.json")
s[:byte_size]                 #=> 631514
s[:root]                      #=> :object
s[:max_depth]                 #=> 10
s[:values]                    #=> {total: 13914, objects: 1264, arrays: 1050,
                              #    strings: 4754, integers: 2108, ...}
s[:keys]                      #=> {total: 13345, unique: 94}
s[:key_histogram].first(3)    #=> [["id", 447], ["id_str", 447], ["urls", 364]]
s[:containers]                #=> {max_object_entries: 40, max_array_length: 100}
s[:strings]                   #=> {bytes: 200716, max_bytes: 463}
```

Nesting is unlimited by default (a deep blob is exactly what a
diagnostic should describe); malformed documents raise the usual rich
`ParserError`.

### Rich parse errors

A failed parse raises `NOSJ::ParserError`
carrying where the document broke—`#byte_offset`, `#line`, `#column`,
and a caret `#snippet`—computed only when a parse fails, so success
pays nothing. Positions stay absolute through partial parsing, lazy
documents, and the file APIs; unrescued errors print the snippet:

```ruby
NOSJ.parse(%({\n  "a": 1,\n  "b": }))
# NOSJ::ParserError: unexpected character at byte 19
#   e.line     #=> 3
#   e.column   #=> 8
#   e.snippet  #=>   "b": }
#                         ^
```

## Benchmarks

Every installed JSON gem, benchmark-ips: AWS EC2 c7a.2xlarge (AMD EPYC 9R14, Zen 4), Ruby 4.0.6 + YJIT, json 2.21.1, Oj 3.17.4, RapidJSON 0.4.0, FastJsonparser 0.6.0, Yajl 1.4.3, PGO build, 2026-07-16. `×N` = times slower than nosj. 

### Parse

| file | nosj (i/s) | json | Oj | FastJsonparser | RapidJSON | Yajl |
|---|---:|---:|---:|---:|---:|---:|
| activitypub | **12.4k** | ×1.18 | ×1.53 | ×1.89 | ×1.99 | ×4.57 |
| canada | **248** | ×1.10 | ×8.28 | ×1.45 | ×1.51 | ×4.96 |
| citm_catalog | **504** | ×1.03 | ×1.97 | ×2.07 | ×1.83 | ×4.93 |
| gsoc-2018 | **397** | ×1.30 | ×1.47 | ×1.81 | ×1.80 | ×4.78 |
| homebrew-formula | **15.2** | ×1.13 | ×1.54 | ×2.42 | ×2.02 | ×4.30 |
| homebrew-llvm | **49.3k** | ×1.60 | ×1.48 | ×1.50 | ×1.85 | ×5.11 |
| mesh | **1.1k** | ×1.30 | ×3.93 | ×1.82 | ×1.95 | ×6.96 |
| numbers | **5.9k** | ×1.19 | ×4.64 | ×1.43 | ×1.75 | ×7.04 |
| ohai | **14.4k** | ×1.47 | ×1.61 | ×2.11 | ×1.97 | ×4.71 |
| simple | **977k** | ×1.31 | ×1.77 | ×2.08 | ×1.63 | ×5.15 |
| small_mixed | **3.2M** | ×1.48 | ×2.85 | ×2.44 | ×1.82 | ×6.75 |
| tolstoy | **8.9k** | ×1.79 | ×1.96 | ×2.29 | ×2.10 | ×17.31 |
| twitter | **1.1k** | ×1.09 | ×1.83 | ×2.25 | ×2.61 | ×5.40 |

### Generate

| file | nosj (i/s) | json | Oj | RapidJSON | Yajl |
|---|---:|---:|---:|---:|---:|
| activitypub | **34.8k** | ×1.20 | ×1.78 | ×2.41 | ×5.84 |
| canada | **150** | ×0.98\* | ×10.98 | ×11.20 | ×10.87 |
| citm_catalog | **1.3k** | ×1.04 | ×1.44 | ×1.55 | ×2.85 |
| gsoc-2018 | **1.1k** | ×1.28 | ×2.64 | ×3.64 | ×11.00 |
| homebrew-formula | **20.9** | ×1.07 | ×1.25 | ×1.62 | ×3.03 |
| homebrew-llvm | **71.5k** | ×1.23 | ×2.22 | ×3.04 | ×5.80 |
| mesh | **613** | ×1.08 | ×9.24 | ×9.40 | ×9.26 |
| numbers | **2.1k** | ×1.03 | ×10.63 | ×11.00 | ×10.69 |
| ohai | **39.4k** | ×1.07 | ×1.29 | ×1.50 | ×3.37 |
| simple | **2.2M** | ×1.03 | ×1.58 | ×1.62 | ×4.36 |
| small_mixed | **5.6M** | ×1.09 | ×2.26 | ×1.83 | ×7.16 |
| tolstoy | **8.3k** | ×1.30 | ×4.15 | ×6.50 | ×14.54 |
| twitter | **2.9k** | ×1.09 | ×1.46 | ×2.01 | ×4.05 |

\* canada-generate is a statistical tie with the json gem (within
measurement error).

### File APIs

twitter.json (570 KB) from a warm page cache, medians of 7 alternating
rounds against the plain-Ruby composition on the same parser (Apple
Silicon dev box, Ruby 4.0.6 + YJIT, PGO build, 2026-07-16):

| operation | µs/op | vs the Ruby way |
|---|---:|---|
| `NOSJ.load_file` (parse the whole file) | 948 | ×1.33 vs `NOSJ.parse(File.read(path))` |
| `NOSJ.dig_file` (one deep field) | 246 | ×5.2 vs read + parse + dig |
| `NOSJ.load_lazy_file` + one field | 257 | ×5.0 vs read + parse + dig |

`NOSJ.write_file` measures at parity with
`File.write(NOSJ.generate(obj))` on this box—file-write timings swing
too much for an honest multiplier; what it saves is the intermediate
file-sized Ruby String.

### Rails mode

`ActiveSupport::JSON.encode` with the nosj encoder installed, against
stock ActiveSupport (Apple Silicon dev box, Ruby 4.0.6 + YJIT,
activesupport 8.1, medians of 5 interleaved per-process rounds,
outputs verified byte-identical first, 2026-07-17;
`rake bench:rails`):

| workload | nosj (i/s) | vs ActiveSupport |
|---|---:|---|
| twitter tree (570 KB) | 4.5k | ×5.2 |
| 100-record index (with timestamps) | 33.7k | ×1.7 |
| HTML-heavy user content | 177.7k | ×14.2 |
| Time/Date/BigDecimal hash | 1.1M | ×3.0 |
| small API hash | 4.2M | ×1.9 |
| small hash `to_json` | 4.1M | ×1.9 |

The HTML-safety escaping that dominates stock encodes of
user-generated content is fused into the SIMD string-emission kernels
here: escaped output costs the same single pass as unescaped.

Reproduce with `rake bench` (the parity-gated comparison, after a PGO retrain—the shipping configuration) or `rake bench:ips` (the multi-gem shoot-out).

## Switching from the json gem

You mostly don't have to do anything. Some differences:

- The legacy object-deserialization options (`create_additions`,
  `object_class`, `array_class`, `decimal_class`) raise ArgumentError;
  the `nosj/json` drop-in falls back to the original gem for them.
- Behaviors the `json` gem itself deprecates (JS comments, raw invalid
  UTF-8) follow the strict semantics instead.
- Unlike `Array#dig`, negative indices in `NOSJ.dig` return nil (JSON
  Pointer has no equivalent).
- Parse errors raise `NOSJ::ParserError` (`NOSJ::NestingError` past
  `max_nesting`, like the gem); messages use byte offsets rather than
  the gem's phrasing, and the exception carries `#line`, `#column`,
  and a caret `#snippet`.

Everything else—including the gem's exact float formatting, which is
not the shortest-round-trip form most libraries emit—matches
byte-for-byte and is verified continuously against the full corpus.

## How it works

Most fast parsers build their own tree first and convert it into Ruby
objects second, paying for every string and container twice. nosj is
built on the [nosj](https://github.com/yaroslav/nosj) Rust crate, an
event parser with no tree of its own:

- **No intermediate tree.** The crate parses with NEON/SSE2/AVX2 SIMD
  kernels and emits *events*; the extension builds interned hash keys,
  strings, and containers directly on Ruby's heap during the parse,
  with GC-safe value stacks and epoch-evicted key caches. Generation
  walks Ruby objects once, streaming through fused scan-and-store
  escape kernels.
- **Byte-exact floats.** Output reproduces the json gem's fpconv
  (Grisu2) float format digit for digit—round-tripping is verified,
  not assumed.
- **PGO everywhere.** Local builds, CI, and every precompiled platform
  gem train on the benchmark corpus before the shipping compile; the
  precompiled binaries use portable codegen with SIMD tiers detected at
  runtime.

## Development

```bash
bundle exec rake compile                # build the extension (applies a PGO profile if present)
bundle exec rake spec                   # the gem-parity suite
bundle exec rake bench                  # PGO retrain + the parity-gated sweep vs the json gem
bundle exec rake bench:fast             # the sweep without retraining
bundle exec rake "bench:ips[twitter]"   # multi-gem shoot-out (benchmark-ips); no args = full corpus
```

## Acknowledgements

Thanks to [Jean Boussier](https://github.com/byroot), [Florian Frank](https://github.com/flori), [Hiroshi Shibata](https://github.com/hsbt), [Nobuyoshi Nakada](https://github.com/nobu), [Étienne Barrié](https://github.com/etiennebarrie), and the other authors and maintainers of the [json gem](https://github.com/ruby/json)—for their work on the gem itself, for optimization ideas, and for some of the JSON documents in the benchmark corpus.

Thanks to [Mat Sadler](https://github.com/matsadler) for [magnus](https://github.com/matsadler/magnus).

Thanks to the authors of the projects credited in [NOTICE](https://github.com/yaroslav/nosj-ruby/blob/main/NOTICE) and the
crate's NOTICE: [fpconv](https://github.com/night-shift/fpconv) and
Florian Loitsch's Grisu2,
[fast_float](https://github.com/fastfloat/fast_float) and
[fast_double_parser](https://github.com/lemire/fast_double_parser)
(Daniel Lemire et al.),
[simdjson](https://github.com/simdjson/simdjson) (Daniel Lemire, Geoff
Langdale, and contributors), and the architectural lineage of
[yyjson](https://github.com/ibireme/yyjson),
[simd-json](https://github.com/simd-lite/simd-json), and
[sonic-rs](https://github.com/cloudwego/sonic-rs).

## Assisted by

Claude Fable 5, Claude Opus 4.8.

## License

MIT. The underlying Rust crate is `MIT AND BSL-1.0 AND Apache-2.0`; its
NOTICE file itemizes the derived components.

### nosj

**nosj is a very fast SIMD-accelerated JSON parser and writer** that never builds its own values. It hands you **events**—object begins, keys, strings,
numbers—and you build your structure directly: a runtime's heap
objects, an arena, an Arrow column. Strings arrive as borrowed slices
straight from the input buffer whenever no unescaping is needed.

2-19× faster than serde_json, ahead of sonic-rs and simd-json, up to 12 GB/s—[see Benchmarks](#benchmarks).

**Why:** if the values you need aren't Rust values, every DOM parser
makes you pay twice—once to build its tree, once to convert every node
into yours. That second pass re-touches every string, allocates
everything again, and routinely costs more than the fast parse saved.
nosj deletes the intermediate tree in both directions: the streaming
`Writer` takes events from your own traversal, so generation never
materializes a Rust-side copy either.

**And when you only need a few fields**, don't parse at all: the JSON
Pointer resolver takes a path like `/users/1/name` and walks the raw
bytes straight to it, skipping every sibling at 64-byte-block speed and
returning the value's raw text. A query costs what it skips, not what
the document weighs—a field deep in a 600 KB document resolves in
~100µs where parse-then-navigate costs milliseconds—and a whole set
of pointers resolves in one pass for about the price of the deepest
one.

**How it differs:** serde_json, simd-json, and sonic-rs produce *their*
representation (a `Value`, a tape, a lazy document). nosj produces
none—you get a push `Sink`, a pull `Reader`, a streaming `Writer`, and
raw-text JSON Pointer resolution. If you want a `Value` tree in Rust,
use serde_json; if you want your own values at SIMD speed, that is the
entire point of nosj.

> **Warning:**
> This crate was produced entirely by **Claude Fable 5**.
> It is extensively verified: differential fuzzing against serde_json,
> sanitizer-instrumented kernel tests, byte-exact cross-checks of every
> SIMD path against a scalar model, and continuous byte-for-byte
> output-parity gates in its host application.

## Show me

```rust
use nosj::{Buffers, Node, Reader, WriteOptions, Writer};

// Parse (pull): ask for events, build whatever you want.
let mut bufs = Buffers::new();
let mut p = Reader::new(r#"{"a": [1, true]}"#, &mut bufs);
assert!(matches!(p.next_node().unwrap(), Node::ObjectStart));
assert_eq!(p.object_first_key().unwrap(), Some("a"));

// Generate (write): emit events, the writer owns the syntax.
let mut out = Vec::new();
let mut w = Writer::new(&mut out, &WriteOptions::COMPACT);
w.begin_object();
w.key("a");
w.begin_array();
w.int(1);
w.boolean(true);
w.end_array();
w.end_object();
assert_eq!(out, br#"{"a":[1,true]}"#);
```

Grab one field out of a big document without parsing the rest:

```rust
let mut bufs = nosj::Buffers::new();
let doc = r#"{"users":[{"name":"ada"},{"name":"grace"}]}"#;
let raw = nosj::pointer(doc, "/users/1/name", &mut bufs).unwrap();
assert_eq!(raw, Some("\"grace\""));
```

## What's in the box

| Direction | Host-driven | Library-driven |
|-----------|-------------|----------------|
| Parse     | **pull**—`Reader` | **push**—`Sink` via `parse` / `parse_indexed` |
| Generate  | **write**—`Writer` |—|
| Partial   | **pointer**—`pointer` / `pointers` |—|

**The parser**: strict RFC 8259 (opt-in `NaN`/`Infinity` and trailing
commas on cold paths), one set of SIMD tokenizers, three ways to drive
it:

- `parse`—fused single-pass byte cursor (the yyjson architecture),
  pushing events into your `Sink`. The default and fastest way to build
  a complete structure; flat containers run in tight scalar loops with
  zero frame-stack traffic.
- `parse_indexed`—SIMD stage-1 pass indexes every token boundary, then
  one dispatch per token into the same `Sink`. The indexing phase is
  pure computation, exposed separately (`Buffers::preindex`) so hosts
  with a runtime lock (a GVL/GIL) can run it outside the lock.
- `Reader`—pull: ask for the next event, navigate objects key by key,
  `skip_value()` over subtrees you don't care about, grammar enforced
  as you walk.

**The writer** (`Writer`): streaming push writer—you emit events, it
owns separators, nesting, layout (compact through fully custom pretty),
and escaping, and it cannot fail (no `Result` on the hot path). Number
bytes are pinned (itoa integers, fpconv floats) with an opt-in
shortest-round-trip float mode.

**The partial parser** (`pointer`, `pointers`): resolves JSON Pointers
(`/users/1/name`, with the standard `~0`/`~1` escapes) against the raw
document and returns the matched value's raw text without parsing
anything else. Siblings are skipped at 64-byte block speed, so a query
costs what it skips, not what the document weighs; `pointers` resolves
a whole set of paths in one forward pass for about the price of its
deepest member. Misses are `Ok(None)` (`serde_json::Value::pointer`
semantics); skipped interiors are bracket-balance-checked, the resolved
target fully validated.

µs per query, AWS EC2 c7a.2xlarge (AMD EPYC 9R14, Zen 4;
`examples/pointer_bench`):

| shape | nosj pointer | sonic-rs get | serde parse+pointer |
|---|---:|---:|---:|
| twitter early (`/statuses/0/id`) | 0.1 | 0.2 | 1723.9 |
| twitter late (`/statuses/95/user/screen_name`) | **58.5** | 220.1 | 1715.7 |
| citm deep (`/performances/40/.../areaId`) | **45.9** | 128.4 | 3875.2 |

| nosj pointers (batch of 5) | nosj pointer ×5 (sequential) | sonic-rs get_many |
|---:|---:|---:|
| **66.3** | 217.3 | 238.4 |

## Benchmarks

`cargo run --release --features shortest-floats --example compare`:
alternating round-robin blocks over `benchmark/` (the classic simdjson
corpus plus real-world payloads), MB/s, higher is better, whole-binary
PGO with every contender trained equally. AWS EC2 c7a.2xlarge
(AMD EPYC 9R14, Zen 4; AVX2 paths active), rustc 1.97.0.

nosj appears more than once because it has no DOM: **events** drives
the fused cursor into a non-allocating sink (the design point);
**tree** builds a naive owned `Vec`/`String` tree so the comparison
with DOM libraries is apples-to-apples; **shortest floats** is the same
tree with the opt-in `FloatFormat::Shortest` instead of pinned fpconv
bytes.

### Parse (MB/s)

| file | nosj (events) | nosj (tree) | serde_json | sonic-rs | simd-json | simd-json (borrowed) | jiter | json-rust | json-steroids | asmjson |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| activitypub | **3080** | 672 | 512 | 2123 | 430 | 608 | 708 | 583 | 441 |—|
| canada | **1110** | 592 | 348 | 1014 | 348 | 350 | 346 | 566 | 364 | 959 |
| citm_catalog | **3317** | 822 | 567 | 2162 | 653 | 815 | 862 | 786 | 664 | 1974 |
| gsoc-2018 | **6666** | 1615 | 1212 | 5278 | 1183 | 2102 | 1315 | 875 | 879 |—|
| homebrew-formula | **3366** | 313 | 177 | 1654 | 166 | 289 | 422 | 382 | 246 |—|
| homebrew-llvm | 3475 | 1834 | 1350 | **4123** | 1543 | 2040 | 1427 | 1246 | 1121 | 957 |
| mesh | **1044** | 673 | 408 | 867 | 479 | 479 | 607 | 615 | 319 | 771 |
| numbers | 1294 | 978 | 595 | **1321** | 727 | 729 | 938 | 948 | 566 | 956 |
| ohai | **2229** | 433 | 408 | 1827 | 387 | 779 | 718 | 590 | 356 | 1356 |
| simple | **1246** | 377 | 367 | 726 | 253 | 357 | 508 | 498 | 414 | 670 |
| small_mixed | **455** | 153 | 145 | 198 | 105 | 111 | 171 | 183 | 153 | 213 |
| tolstoy | **12101** | 9584 | 4461 | 11202 | 3496 | 3799 | 368 | 895 | 399 | 485 |
| twitter | **3166** | 667 | 408 | 2452 | 530 | 955 | 811 | 687 | 484 |—|

### Generate (MB/s)

| file | nosj (tree) | nosj (shortest floats) | serde_json | sonic-rs | simd-json | json-rust | json-steroids |
|---|---:|---:|---:|---:|---:|---:|---:|
| activitypub | 2372 | 2353 | 1277 | **3317** | 2283 | 1350 | 2531 |
| canada | 302 | **805** | 766 | 776 | 612 | 1289\* | 613 |
| citm_catalog | 1077 | 1088 | 876 | **1138** | 1003 | 1068 | 1068 |
| gsoc-2018 | 4709 | 4745 | 1451 | **7307** | 4110 | 1528 | 4608 |
| homebrew-formula | **1288** | 1285 | 391 | 1082 | 592 | 661 | 780 |
| homebrew-llvm | 2870 | 2835 | 1265 | **2934** | 2466 | 1289 | 2595 |
| mesh | 381 | **681** | 615 | 617 | 440 | 871\* | 427 |
| numbers | 320 | **715** | 693 | 690 | 465 | 1230\* | 481 |
| ohai | 1042 | 1048 | 984 | **1574** | 1078 | 1154 | 1200 |
| simple | 668 | 671 | 667 | 658 | 756 | **888** | 722 |
| small_mixed | **345** | 341 | 339 | 319 | 330 | 334 | 264 |
| tolstoy | 9973 | 10085 | 1624 | **18947** | 7275 | 1631 | 907 |
| twitter | 1626 | 1633 | 1107 | **2374** | 1185 | 1284 | 1190 |

\* json-rust's float output is not byte-faithful. Blank cells: asmjson
(here running its native AVX-512 path) failed those files.

## How it is fast

**Parsing**

- Stage 1 in the simdjson lineage: shufti nibble-table classification,
  odd/even escape carries, PMULL/CLMUL prefix-XOR quote masks, unrolled
  index flatten. NEON on aarch64; SSE2 baseline and runtime-detected AVX2 on
  x86_64; a byte-exact scalar model cross-checked bit-for-bit in tests.
- Strings: zero-copy for the escape-free case; 32-byte SIMD scans with the
  control-character check folded in, so string bytes are read once.
- Numbers: fused SWAR validation + accumulation (eight digits per step),
  Eisel-Lemire float construction verified differentially against
  fast-float2 over hundreds of thousands of cases.

**Generation**

- Fused scan+store escaping: each 16-byte chunk is speculatively stored
  into reserved output capacity as it is scanned, so clean string data is
  touched once. SWAR tail and table-driven scalar path for short strings;
  overlapping-word copies instead of `memcpy` calls for tiny spans.
- Three escape modes: standard (RFC 8259 minimum), script-safe (`/`,
  U+2028/U+2029—safe for HTML `<script>` embedding), and ASCII-only
  (`\uXXXX` with surrogate pairs).
- Integers via `itoa`; floats in the **fpconv (Grisu2) format**—a
  line-faithful port of the widely deployed `fpconv.c`, byte-for-byte
  compatible with the C reference and pinned by differential tests.
  Deliberately not Rust's shortest-round-trip `Display`: reproducing the
  reference bytes exactly is the guarantee hosts need, with
  `FloatFormat::Shortest` as the opt-in alternative.

## Where it came from

nosj was born inside
[gem nosj](https://github.com/yaroslav/nosj-ruby), a Ruby JSON library
that set out to beat Ruby's bundled, heavily optimized C `json` gem.
Parsing speed was never the blocker—the post-parse conversion pass
was, so this crate became the parser that skips it. Exercised end to
end by the gem, it benchmarks faster than that C implementation across
the full classic suite in both directions, with byte-for-byte output
parity verified continuously.

How everything works—architecture, pipeline internals, the memory
safety model, and why the unusual decisions are deliberate—is written
up top to bottom in [doc/DESIGN.md](https://github.com/yaroslav/nosj/blob/main/doc/DESIGN.md).

## UTF-8 contract

`&str` entry points are safe. `*_utf8_unchecked` entry points skip
whole-input validation for hosts whose runtime already tracks string
validity; string content is still validated structurally during the parse.
One deliberate leniency, matching widely deployed parsers: a lone *low*
surrogate escape decodes to raw WTF-8 bytes, delivered via
`Sink::str_bytes` (lossy-converted unless overridden); a lone *high*
surrogate is an error.

## Testing and fuzzing

Unit tests cross-check every SIMD kernel against its scalar reference and
pin the two push drivers to identical event streams. Three cargo-fuzz
targets extend that continuously: driver-vs-cursor trace equality,
writer→parser round-trips across all layout configurations, and a
value-for-value differential against serde_json (with its
`float_roundtrip` feature, so both sides are correctly rounded). CI runs
each target briefly on every push.

## Installation

```sh
cargo add nosj
```

## Acknowledgements

Thanks to [Jean Boussier](https://github.com/byroot),
[Florian Frank](https://github.com/flori),
[Hiroshi Shibata](https://github.com/hsbt),
[Nobuyoshi Nakada](https://github.com/nobu),
[Étienne Barrié](https://github.com/etiennebarrie), and the other
authors and maintainers of the [json gem](https://github.com/ruby/json)—for
their work on the gem itself, for optimization ideas, and for some of
the JSON documents in the benchmark corpus.

Thanks to the authors of the projects credited in [NOTICE](https://github.com/yaroslav/nosj/blob/main/NOTICE):
[fpconv](https://github.com/night-shift/fpconv) and Florian
Loitsch's Grisu2,
[fast_float](https://github.com/fastfloat/fast_float) and
[fast_double_parser](https://github.com/lemire/fast_double_parser)
(Daniel Lemire et al.),
[simdjson](https://github.com/simdjson/simdjson) (Daniel Lemire, Geoff
Langdale, and contributors), and the architectural lineage of
[yyjson](https://github.com/ibireme/yyjson),
[simd-json](https://github.com/simd-lite/simd-json), and
[sonic-rs](https://github.com/cloudwego/sonic-rs).

## Assisted by

Claude Fable 5.

## License

MIT, with two derived components under their upstream licenses (SPDX:
`MIT AND BSL-1.0 AND Apache-2.0`):

- `src/grisu2.rs` is a line-faithful port of
  [fpconv](https://github.com/night-shift/fpconv)—Boost Software
  License 1.0 (`LICENSE-BSL-1.0`).
- The correctly-rounded float parser (`src/float.rs`,
  `src/el_table.rs`) is ported from
  [fast_float](https://github.com/fastfloat/fast_float) /
  fast_double_parser, and `src/stage1.rs` uses structural-indexing
  techniques from [simdjson](https://github.com/simdjson/simdjson)—
  Apache License 2.0 (`LICENSE-APACHE-2.0`).

Every derivative, plus architectural lineage acknowledgments, is
itemized in the `NOTICE` file.

### Posts

# Posts

- [Inkmark: a very fast, feature-packed, AI-first Markdown gem for Ruby](/posts/inkmark-fast-ai-first-markdown.md) — Markdown is the lingua franca of the AI internet. Inkmark is a very fast Rust-backed Ruby Markdown gem, has features galore, and comes with first-class support for LLM/RAG pipelines.
- [Bundlebun bundles Bun, and a tour of asset pipelines and JavaScript runtimes Rails has been through](/posts/bundlebun-bundles-bun.md) — Bundlebun is a Bun bundler for Rails. The why, the how, and a tour of every asset pipeline and JavaScript runtime Rails has been through to get here.

### Bundlebun bundles Bun, and a tour of asset pipelines and JavaScript runtimes Rails has been through

This post introduces [Bundlebun][bundlebun], a gem that packs Bun—all-in-one JavaScript runtime, package manager, and build tool—directly into your `Gemfile`. No Docker, no `curl | sh`, no Homebrew required, and everyone gets a JavaScript runtime after `bundle install` with the exact version pinned.

Check the [repository and README][bundlebun], or run `bundle add bundlebun` followed by `rake bun:install` to try it.

But first, we go down memory lane to remember how asset pipelines worked with every Rails version and Rails' relationship with JavaScript runtimes, and the #nobuild movement introduced in the latest versions of Rails.

[bundlebun]: https://github.com/yaroslav/bundlebun

## How it started

It all started with a Node version mismatch.

There was an internal Ruby on Rails application I joined, initially implemented by [an engineer wildly overqualified for the job][palkan]. The app had a `Dockerfile`; in fact, there's a sprawling, once-a-year-updated [write-up on how to dockerize Rails applications][ruby-on-whales]. Problem is, not many people love spinning up Docker containers for development, especially on tiny Rails applications with barely any frontend—and this one had a simple frontend.

Long story short, one of us had an LTS version of Node.js, another had the latest stable one. The app only worked with a specific version of Node.js, assets weren't building properly, and the bootstrap process broke completely: the app didn't start. It took us some time to figure out why and fix it.

This got me thinking: how is it possible we're two decades into Ruby on Rails, and overqualified engineers still stumble into the problem of agreeing on a specific version of a JavaScript runtime and packaging assets for an application with a _simple_ frontend? Surely there's a better way—one developers would willingly reach for.

[palkan]: https://x.com/palkan_tula
[ruby-on-whales]: https://evilmartians.com/chronicles/ruby-on-whales-docker-for-ruby-rails-development

## Ways to solve the issue

Let's write the problem down:

- Make sure every developer of the app has a version of JavaScript runtime and the necessary tooling;
- Make sure that everyone is on exactly the same version of said runtime and tooling.

First of all, **Docker** definitely solves both issues, but I don't know that many developers who _love_ to spin a Docker container for every local Rails app, no matter how small. For larger apps, sure, but we're focusing on small and mid-sized apps with a simpler frontend for now.

**`curl | sh`** scripts are both optional and don't guarantee a specific version is installed.

**Linux package managers** like apt, dnf, or pacman come with the OS, but you can't assume which one—and the version they ship often lags upstream.

**Homebrew** is optional, tracks upstream closely, but isn't really built for project-scoped version pinning.

**Mise**, **asdf**, and other version managers solve the version pinning problem for sure. Still optional by design, though: we cannot force developers to install them other than forcing them to run the bootstrap script.

Now, in case you are reading this from the future where **Nix** and [Flakes][nix-flakes] are the winning way to run development environments, I salute you. Sadly, we're not there yet. Still the same "optional tool" issue.

**Devcontainers** are a massive step in the right direction, but shine only when run from an IDE with Devcontainers support (VS Code, IntelliJ IDEA, Zed). Others are left with the [CLI][devcontainers-cli], which is a bit lacking.

[nix-flakes]: https://nixos.wiki/wiki/Flakes
[devcontainers-cli]: https://github.com/devcontainers/cli

Well, there should be a simpler way to ship a JavaScript runtime and builder with Ruby on Rails, and the community has tried for the best part of twenty years. Why aren't we there yet?

## Rails history with asset pipelines and JavaScript runtimes

In **Rails 2**, there was no asset pipeline; JavaScript and CSS lived in `public/javascripts` and `public/stylesheets` with supporting helpers. 

Sprockets is where the JavaScript runtime requirement was born, in **Rails 3.1**. The asset pipeline had `app/assets/javascripts`, manifest files, and CoffeeScript and Sass as the default authoring formats. CoffeeScript needed to be compiled to JavaScript, Sass to CSS, and that compilation happened at request time in development and at deploy time in production. To do that, Sprockets needed a JavaScript engine, ExecJS: a small gem that abstracts over a currently available JavaScript runtime, picking V8 via therubyracer or mini_racer, Node.js, or JavaScriptCore. This is where the JavaScript runtime requirement starts.

**Rails 4** introduced Turbolinks, which made full-page swaps feel like an SPA without the full-blown JavaScript framework, and continued to use Sprockets.

**Rails 5** added ActionCable for WebSockets, but we were still using Sprockets. While we're on this topic, the JavaScript world grew dramatically; everyone used npm, Webpack, and Babel.

In **Rails 5.1**, that was addressed: Webpacker was introduced, a gem wrapper for Webpack, with Yarn for package management. It was opt-in and you could use both it and Sprockets.

In **Rails 6.0**, Webpacker became the default for JavaScript: new apps got `app/javascript`, a `package.json`, a `node_modules` directory, and the requirement that the deploy machine has Node. At this point, we had a **single app with two pipelines** (Sprockets was still the default for CSS). Webpack itself moved fast and broke configs; the Webpacker maintainers tried to keep up but the dependency tree of a fresh Rails 6 app was sometimes massive even for an empty app.

**Rails 7.0** deprecated Webpacker and replaced it with _three_ new gems:
importmap-rails (ESM without a bundler), jsbundling-rails (a wrapper that allowed you to choose esbuild, rollup, or Webpack), and cssbundling-rails (same, but for Tailwind, PostCSS, dart-sass, or Bootstrap). At this point, the [Hotwire stack][hotwire] became the default for Rails frontend. Propshaft was introduced as a slimmer Sprockets alternative.

[hotwire]: https://hotwired.dev/

**Rails 8.0** committed to the **no-build approach** (branded as #nobuild) fully: Propshaft became the default asset pipeline, the standalone Tailwind binary handled CSS without Node.js, and the new-app generator pointed at importmap-rails by default. The pitch: a new Rails 8 app needs Ruby and nothing else—no JavaScript runtime for deployment.

Is this the end of the story and do Hotwire and #nobuild end the cycle of reinventing frontend within Rails? Stimulus, one of the key components of Hotwire, had its latest point release in... August 2023, almost three years ago. Version 3.2 was released in November 2022, and version 3.0 saw the light in September 2021. Are we mid-cycle with something new coming in a year or two to repeat the cursed cycle? Who knows.

But, more importantly, what's wrong with #nobuild?

## \#nobuild

I wish things always worked as advertised—the return to simple, no-build, no-Node.js frontend applications would be romantic. It's the closest thing to a circa-2007 `application.js` we've had. 

For some apps, no-build works. For most apps that touch any meaningful JavaScript ecosystem, the limits start piling up.

### HTTP/2 does not save you

The strongest argument for no-build is HTTP/2 **multiplexing**. Webpack was invented, among other things, to amortize the connection cost of fetching dozens of small files, and HTTP/2 made that cost roughly free, so why bundle them anymore?

In practice, the issue is the shape of an ES module graph rather than the cost per request. The browser parses your `app.js`, finds `import { foo } from "./foo.js"`, and can't request `foo.js` until it has finished parsing `app.js`. When `foo.js` imports more modules, those can't start until `foo.js` parses too. HTTP/2 makes the requests at any one level cheap, but it doesn't change the fact that the levels run in series. A typical React-shaped app—framework, state library, UI library, utilities, components—has a dependency several levels deep. On a 50 ms RTT connection that's a few hundred milliseconds of pure waterfall before the first frame paints. `<link rel="modulepreload">` helps, but only if you hand-list every module the entry point will eventually reach. A bundle replaces all of that with one round trip and a single parse pass. It's tolerable for prototypes and internal tooling, but not great otherwise.

**Compression** doesn't help unbundled code either. Brotli on a 200 KB bundle reaches ratios in the 4–5× range because the algorithm has the whole graph as its corpus and finds repeated patterns across files. Split that same code into a hundred separate ESM files, each compressed independently, and you lose the cross-file dictionary while still paying per-response framing overhead. Wire bytes for an unbundled app routinely run 30%–50% above the bundled equivalent on identical source.

There's also **tree-shaking**, which doesn't happen in importmap-land. If a library exports ten functions and you import one, the browser fetches and parses the other nine. Lodash, date-fns, RxJS, lucide-react, anything with "icons" in its name—a lot of the modern frontend ecosystem ships generously, on the assumption that the build pipeline will pick what's used.

The "importmaps win on repeat visits" claim depends on releases rarely changing the cached parts, which is true for leaf modules and not very true for entry points and shared utilities. A hash change near the top of the graph means the browser revalidates downstream files even when their contents are unchanged. A bundle ships one new hash, busts one cache entry, and is done.

Overall, #nobuild ends up slower on first visit, slower on most repeat visits, heavier on the wire, and adds parser pauses that bundlers eliminated a decade ago.

### And then there's the ecosystem

Performance is half of it. The other half is that the JavaScript ecosystem most teams actually want to use simply doesn't run in importmap-land.

TypeScript is the most common collision. A lot of modern JavaScript libraries, from utility kits to entire frameworks, ship `.ts` source as the canonical form and let consumers transpile. Importmaps don't load `.ts`; you either avoid TypeScript libraries (a shrinking option) or bring back a compile step, which means you have a bundler. JSX, Vue templates, and Svelte components have the same problem. Every popular component framework expects a transform pass before its source becomes browser-loadable JavaScript. Pre-compiling to `.js` and committing the output to your static directory technically works, but you've now reinvented bundling by hand.

The npm package ecosystem assumes a bundler in subtler ways too. A surprising fraction of libraries still ship CommonJS, sometimes alongside ESM, sometimes only CommonJS. Bundlers paper over this with interop shims; importmaps can't, and the workarounds put your dependency graph at the mercy of third-party CDN-built ESM mirrors.

Tailwind is its own category. The standalone CLI handles vanilla utilities fine, and the moment you reach for a plugin that doesn't ship as a binary—DaisyUI, custom variants, component sets—you're back to Node. Tailwind 4 packs more into the binary, but plugin authors can only follow as far as the embedded toolchain reaches.

And then the libraries themselves: `shadcn/ui`, Three.js loaders, Zod schemas, the whole `@tanstack/*` family. Each one expects a build pipeline. The gap between "works in importmap" and "works in any modern Node tutorial" widens with every release. The cost isn't only the learning curve—it's that the set of packages your team can adopt without an architectural debate keeps shrinking.

The thing that doesn't get mentioned often is that #nobuild doesn't fully eliminate building. It moves the work. Importmap pin lists, modulepreload hints, CDN provider choices, version freezes spread across HTML files, the script that regenerates the importmap manifest when you upgrade a dependency. It's all still configuration.

#nobuild is genuinely great if the app you are building has a simple frontend and you don't need many frontend libraries. The moment this changes, you're back to picking—or reinventing—a bundler.

So, let's pick a runtime and a bundler.

## Why Bun

[Bun][bun] is a JavaScript runtime—much like Node.js or Deno—created by Jarred Sumner in 2021. Lately, Bun has been everywhere in the news because [Anthropic (of Claude fame) acquired it in 2025](https://bun.com/blog/bun-joins-anthropic). In fact, Claude Code, potentially your favorite tool at the moment, [works on Bun](https://www.anthropic.com/news/anthropic-acquires-bun-as-claude-code-reaches-usd1b-milestone).

[bun]: https://bun.sh

Bun is a joy to work with because it prioritizes developer experience, and, specifically, the fun of development. To quote their 1.0 release post,

> Bun's goal is simple: eliminate slowness and complexity without throwing away everything that's great about JavaScript. Your favorite libraries and frameworks should still work, and you shouldn't need to unlearn the conventions you're familiar with.

Bun bills itself as the fastest JavaScript runtime available. But Bun is not only a runtime—it is also a super-fast package manager that can replace npm, yarn, or pnpm. It is _also_ a bundler, albeit at the moment not as featureful as, say, Vite, but more than enough for simpler frontend applications. So, the entirety of Node.js + npm + esbuild + ts-node is replaced by a single product.

What also helps is that Jarred used to do Rails. More than that:

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">A lot of Bun is inspired by Ruby on Rails</p>&mdash; Jarred Sumner (@jarredsumner) <a href="https://twitter.com/jarredsumner/status/1833102987817836617?ref_src=twsrc%5Etfw">September 9, 2024</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

What's also important for us: Bun is distributed as a single binary. So, a single executable file that is a JavaScript runtime, a package manager, and a builder—bingo.

Still, to get all that beauty, you need to install it on every developer machine, and make sure all developers have the same version—as Bun can have releases as frequent as every couple of weeks. If the app was tested with a specific version of Bun, we need to make sure that every developer has it installed, and that it is possible to use it for deployment as well. We need to pin a version and make sure installation works.

We've already gone through some options to install and maintain version pinning, and most of them are not a perfect match. So what do we do?

Pack the executable into a gem.

## Meet Bundlebun

[Bundlebun][bundlebun] is a Ruby gem that contains the Bun binary—the official artifact published by the Bun team, for your CPU architecture and operating system, packaged inside the gem's release file. You add it the way you add any gem:

```
bundle add bundlebun
```

And then you run

```
rake bun:install
```

The first command pulls the gem with the right Bun for your machine. The second drops a `bin/bun` binstub into your repo, and installs integrations—we'll get back to that later. From then on, anyone who clones the project has Bun the moment they run `bundle install`. They don't install Node. They don't install Bun separately.

Now, about version pinning. A gem version `0.1.0.1.1.38` parses as gem version `0.1.0` shipping Bun `1.1.38`—the first three numbers are the gem's, the last three are Bun's, glued on automatically every time Bun publishes a release. Bun moves and the gem moves with it; there's no maintainer in the middle deciding when to catch up. Your `Gemfile.lock` records exactly which Bun version your team is on, the same way it records Rails. Want the latest? Leave the gem unconstrained. Want to pin? The `~>` operator you already know keeps you on specific Bun versions. Or, freeze the version completely. The mechanism is the one Ruby developers have used since 2010—it now covers JavaScript too.

### What does this buy you in practice

A new developer joins the project. They clone the repo, run `bundle install`, and have Bun on their laptop. Five minutes from clone to running app, with no detour through a "set up your environment" document. The same `Gemfile.lock` that pins Rails pins their Bun.

CI loses a step. No `setup-node@v4`, no separate cache key for Node modules, no version drift between local `bun --version` and what GitHub Actions thinks Bun is. The `bundle install` step that was always there now produces a working Bun, so the step that runs `bin/bun build` just works. A workflow that used to spend twenty seconds setting up Node spends two extracting the gem cache.

Production deploys lose a `RUN apt-get install -y nodejs` line from the `Dockerfile`, or whichever PaaS-buildpack equivalent you'd been using. The image gets smaller. There's one fewer "where did this binary come from" question for your security team to ask.

The compound effect can be real for any team that's spent meaningful time on environment-setup issues—which is probably most teams.

### What it does to existing Rails-Node bridges

The bigger payoff is that the gems already gluing Rails to Node-shaped tools just use Bun now. You don't switch frontend stacks—you swap the runtime underneath them.

The integrations are already installed once you run `rake bun:install`. You can also run them selectively.

Running `bun:install` offers to automatically update your `package.json` and `Procfile` (if you have one) so they invoke the new, bundled Bun in place of whatever was previously wired up. Decline the prompt to leave them as-is.

If your project uses [vite-ruby](https://vite-ruby.netlify.app/), the install step alters your `vite.json` to point at the bundled binstub, and the next `bin/vite` runs Vite on Bun. Vite is still Vite—same plugin ecosystem, same config, same hot reload. The package install step that used to take twenty seconds now takes two, and builds shave a few seconds off too.

If you use `cssbundling-rails` or `jsbundling-rails`—the Hotwire-era gems for delegating frontend tooling to Node—Bundlebun replaces that delegation. During the installation process, the underlying toolchain is swapped to `bin/bun`. The `bin/dev` Rails teams have been running for years keeps working; the asset pipeline just stops requiring a separate Node install.

ExecJS lets Ruby execute JavaScript on the server—the engine behind libraries like `react-rails`, the legacy template-rendering choices a lot of teams have forgotten they have, and a long tail of "we only need to run JavaScript at boot to compile a thing once" use cases. If you load Bundlebun after ExecJS in your `Gemfile`, ExecJS will detect the bundled Bun as a runtime, and your server-side JavaScript executes under bundled Bun.

### Using it

`bin/bun` is the day-to-day surface, and it behaves exactly like the Bun CLI:

```
bin/bun install
bin/bun add postcss
bin/bun run build
```

For one-off commands inside Rails's task surface, there's a Rake wrapper (just notice the Rake syntax):

```
rake "bun[outdated]"
```

For Ruby code that needs to call Bun from inside the application—a release task, a custom build step, a cron job—there's a small Ruby API:

```ruby
Bundlebun.('install')      # exec form, replaces the Ruby process, 
                           # for use in binstubs, shell files, tasks

Bundlebun.system('test')   # spawns Bun, returns control to Ruby
```

ActiveSupport instrumentation publishes `system.bundlebun` events, so the same dashboard that watches your N+1 queries can watch your Bun commands:

```ruby
ActiveSupport::Notifications.subscribe('system.bundlebun') do |event|
  Rails.logger.info "Bun: #{event.payload[:command]} (#{event.duration}ms)"
end
```

However, most of the time, you would only use `bin/bun` and automatic integrations.

### Behind the scenes

The user-facing part of the gem is rather trivial: executing Bun one way or another, plus hacks for integrating with existing libraries. What is more interesting are some gotchas I've discovered along the way. All the interesting stuff happens in build and testing.

#### Automatic releases

If there are no code changes in the gem itself, the release process is _fully automatic_. When a new Bun release is out, there is no gem author pushing a button behind the scenes.

Instead, there is a task that is run every day by a cron job in GitHub Actions.

The task, using the GitHub API, checks if there is a new Bun release out, by comparing it with the last group of digits from our own latest release. Meaning, if the latest Bun release is `v1.3.14` and our own gem version is `v0.5.0.1.3.13`, there is a mismatch, and a new gem version should be released.

For every supported platform, a Bun executable is downloaded and packed into a platform- and architecture-specific gem. Next, again, for every platform, smoke tests are run by asking Bundlebun, essentially, what is 2 + 2 (`bun -e 'console.log(2+2)'`). If the script is satisfied with the result, a gem release is pushed. 

The workflow and packing tasks can save you some time if you are thinking about doing a similar binary-in-gem project.

#### Integration-integration tests

An important part of Bundlebun is its plug-and-play integration into an existing Ruby on Rails application, hijacking the calls to a JavaScript runtime in existing libraries.

We _have_ to test the behavior between various Ruby and Rails versions. Unit testing the hijack module would mostly be useless code duplication; we must be sure that overloading works in practice and the Bun executable is run with specific parameters—or that the result of some asset pipeline looks exactly as expected.

That's why instead of going the traditional integration testing route with something like the `combustion` gem, in some of the tests I had to automatically generate an actual Rails application, add `bundlebun` to `Gemfile`, install gems, and run the install Rake task. Next, it is possible to compare the result of, say, going through an asset pipeline and building CSS with the expected one. Real Rails application, real assertions, no surprises when a new vite-ruby or Rails version comes out and something breaks.

There were several gotchas with how Bundler works in this regard (`bundle` runs our testing suite, which generates a directory where another `bundle` should run, quite messy), so in case you are writing a Rails plugin that requires this level of testing, again, re-using that code could possibly save some time.

#### Native Windows support

Lots of Ruby gems ignore the existence of Windows by not running a Windows machine on CI at all. A typical answer to that is to use WSL or Docker. In my opinion, not good enough—if it is *possible* to do a gem build for Windows, we should also test its correctness on the native platform and not rely on WSL. We need to take inspiration from the Rust community, where lots of top-tier developers use Windows natively and the OS is not considered a "second-class citizen".

Now, Bun supports Windows natively, so it was a separate task for me to ensure the gem works natively on Windows. That means: `bun.cmd` binstubs instead of just `bun`, `PATH` gotchas and other issues.

## Time to try it

Finally, time to try Bundlebun! In an existing Rails app:

```
bundle add bundlebun
rake bun:install
```

Don't forget to read the `README` at GitHub: [yaroslav/bundlebun][bundlebun].

And don't hesitate to [ping me on X][yaroslav-x] with feedback and questions.

[yaroslav-x]: https://x.com/yaroslav

### Inkmark: a very fast, feature-packed, AI-first Markdown gem for Ruby

Meet [Inkmark][inkmark], a very fast, featureful, AI-first Markdown Ruby gem. It's a Rust-backed gem—built on [pulldown-cmark][pulldown-cmark]—ready to replace your existing Markdown gem, providing lots of features, as well as first-class primitives for AI: chunking documents for retrieval, truncating to context windows, mapping output back to source bytes for citation.

It is currently the fastest CommonMark- and GFM-conformant Markdown engine for Ruby across the board. If you're already running Redcarpet, kramdown, or Commonmarker, you can swap it in: `bundle add inkmark`—check the [README][inkmark] first.

But before we get to the gem, a step back: the reason I built this is that Markdown means something different now than it did when most of the existing Markdown libraries were designed.

[inkmark]: https://github.com/yaroslav/inkmark
[pulldown-cmark]: https://github.com/pulldown-cmark/pulldown-cmark

## Markdown is the lingua franca of the AI internet

Since John Gruber and Aaron Swartz's inception of the Markdown project in 2004, it has become incredibly popular: READMEs, posts, comment boxes, and developer-built static sites—beating all existing simple text-to-HTML formats and even becoming more popular than authoring content in WYSIWYG HTML boxes.

Then, large language models showed up and quietly made Markdown the default wire format of the AI internet:

- **LLM prompts** are Markdown: System prompts, user messages, assistant responses.
- **LLM outputs** are Markdown by default. Headings, lists, code fences. ChatGPT, Claude, Gemini—their answers render as Markdown.
- **RAG corpora** are stored as Markdown. Everything gets normalized to Markdown before chunking and embedding. It's the universal intermediate format because it preserves enough structure (headings, code, lists) without dragging in HTML's complexity.
- **Agent meta files** (`CLAUDE.md`, `AGENTS.md`, and others) are Markdown.

Markdown won the AI ecosystem the same way JSON (the "previous lingua franca", in a way) won APIs: good enough to be useful, simple and open enough for everyone to tolerate it.

### "User interviews"

When I did "user interviews" of a sort before designing the library—asking my Ruby developer friends and past co-workers about their Markdown parsing experience and requirements—it became quite obvious that they belong in two large groups.

The first group did not use Markdown with LLMs other than for rendering simple streaming "answers" from AI. There were no specific requirements other than to maybe provide support for extending Markdown syntax (I think we have more than enough Markdown syntax spin-offs that require feature flags in parsers; instead, Inkmark has the extension API). The second group, though... The second group was full of developers who worked with RAG pipelines. They _required_ a good API. They _needed_ chunking, and I've even seen several DIY chunking snippets based on other gem APIs. It is clear that the "market" for AI-first features, so to speak, is there.

### New priorities for Markdown libraries

That changes what a Markdown library has to be good at:

First of all, **pure speed for bulk processing.** In the past, a Markdown library used to render one document per page request—or render a page or a comment once and store the resulting HTML in the database. Now it renders ten thousand documents at index time, and again every time the corpus refreshes. Previously, one could argue that speed of the Markdown processor does not matter much because the processing only happens once, on creation or update. Now, a Markdown processor _has_ to be fast to deal with everything that is happening.

Second, it would be nice—and lately, almost a must-have—to see **LLM/RAG-friendly primitives**: chunking that respects document structure, truncation that doesn't destroy code blocks or even words. We need plain-text export for embedding models (and for search indices, too). None of this is the kind of thing an old-school Markdown-to-HTML renderer was designed for.

### Ruby AI ecosystem

Existing Ruby Markdown gems were built for rendering to HTML and supporting a lot of syntax gotchas. Some also have a nice API to them, allowing traversal and changing the document. They are not optimized for the LLM reality, and there is still space to make a Markdown processor even faster.

That's the gap Inkmark fills.

It slots into the rest of Ruby's AI stack—[RubyLLM][rubyllm] for multi-provider LLM calls, [langchainrb][langchainrb] for LangChain primitives, [neighbor][neighbor] for pgvector and sqlite-vec on ActiveRecord, [informers][informers] for local transformer inference—so you can build a real RAG pipeline without leaving Ruby.

Inkmark is faster than competing Ruby Markdown gems, with a trivial enough API so it is easy to replace them in minutes. With Inkmark, it is possible to avoid writing complex transformation classes in Ruby and slicing documents as HTML. In any case, the event API should be enough to traverse and even change Markdown on the fly.

[rubyllm]: https://github.com/crmne/ruby_llm
[langchainrb]: https://github.com/patterns-ai-core/langchainrb
[neighbor]: https://github.com/ankane/neighbor
[informers]: https://github.com/ankane/informers

## Inkmark is very fast

No matter how many features we need, and if we even have a RAG pipeline, 9 out of 10 users would still use a Markdown gem like this:

```ruby
html = MarkdownEngine.parse(source)
```

So if we're not fast enough, or, ideally, the fastest, what's the point?

Inkmark is built on [pulldown-cmark], a CommonMark and GFM parser written in Rust that uses SIMD. The result is the fastest CommonMark-conformant Markdown engine available for Ruby today.

Let's see the numbers against other Ruby Markdown gems from the [benchmark suite][inkmark-bench]. I've included the benchmarks the gems themselves use, plus some Markdown files of a specific size (about 4 KB, about 1 KB, and so on). With Inkmark as the reference, how much faster or slower are the others?

| Benchmark | Redcarpet | Markly | Commonmarker | rdiscount | kramdown |
|---|---:|---:|---:|---:|---:|
| CommonMark spec (201 KB) | 1.29× slower | 2.59× slower | 3.40× slower | 5.53× slower | 45× slower |
| Redcarpet README (14 KB) | 1.28× slower | 3.18× slower | 3.55× slower | 5.20× slower | 83× slower |
| Redcarpet's own benchmark (8 KB) | 1.16× slower | 2.96× slower | 3.54× slower | 4.46× slower | 75× slower |
| dotenv README (4 KB) | 1.10× slower | 2.85× slower | 3.55× slower | 4.63× slower | 103× slower |
| Faraday README (1 KB) | 1.02× faster (ballpark) | 3.09× slower | 4.68× slower | 4.73× slower | 77× slower |
| README section (0.5 KB) | 1.05× slower (ballpark) | 6.35× slower | 8.32× slower | 5.24× slower | 98× slower |

Some notes. First, Redcarpet wins by a hair on inputs under a kilobyte—parser setup overhead dominates at that size; it varies by the run, and I would say we're in the same ballpark. Above ~1 KB Inkmark pulls ahead, and the gap widens with document size. What is also important is that Inkmark beats Redcarpet at _its own benchmark_ taken from their repository.

Second, Redcarpet itself is _not_ CommonMark conformant; it's the speed leader of the previous generation, but you pay in subtle compatibility differences. Inkmark gives you top-tier speed _and_ CommonMark + GFM conformance.

Third, the gap to kramdown and Commonmarker can be massive on real inputs. Commonmarker is a modern, nicely built Ruby gem, also powered by Rust. It is based on the Comrak Rust crate, which, in a way, sacrifices performance to be able to generate an AST from a Markdown file; the library Inkmark uses is specifically built for speed and does not parse the document into an AST.

If you're running Redcarpet because it's the fast option, it should be safe to switch now. Equal-or-better throughput with stricter spec compliance and many more features. For other gems, unless you use their specific features, like AST parsing with Commonmarker, you will be getting better speeds (3×–9×), more features, and comfort working with AI pipelines.

[inkmark-bench]: https://github.com/yaroslav/inkmark#benchmarks

## Inkmark has features galore

Let's explore the feature set, from the basics to advanced magick.

### The basics

What most people would use, and, possibly, not even touch other features. A one-method, one-line Markdown parser/renderer:

```ruby
Inkmark.to_html("**hello**")

# or
Inkmark.new("# Hello").to_html

# or: mutable options accessor for tweaks at runtime
md = Inkmark.new(src)
md.options.tables = false
md.to_html
```

You can also set process-level defaults in something like a Rails initializer:

```ruby
Inkmark.default_options.preset = :recommended
Inkmark.default_options.statistics = true
```

### Presets

Now, unlike other libraries, to avoid juggling a dozen different flags and options (although you can still fine-tune everything), Inkmark ships several _presets_:

```ruby
# Modern-web profile: smart punctuation, autolinks, lazy images, syntax
# highlighting, nofollow on external links, scheme allowlist on 
# link destinations, emoji shortcodes, heading IDs, hard wraps, 
# frontmatter
Inkmark.to_html(md, options: { preset: :recommended })

# `:recommended` plus raw HTML pass-through. 
# Use ONLY for content you fully trust.
Inkmark.to_html(md, options: { preset: :trusted })

# See other presets in README
```

And then override—tighten link allowlists to your own hostnames, turn off features you don't use, dial in what you want.

```ruby
Inkmark.to_html(md, options: {
  preset: :recommended,
  links: { allowed_hosts: ["*.example.com"] }
})
```

### Features at a glance

The single-method API hides a long list of built-in features—server-side syntax highlighting with CSS class output, GFM tables and tasklists and strikethrough and footnotes, smart punctuation, emoji shortcodes, autolinks, lazy-loading images, heading IDs with Unicode-transliterated slugs, frontmatter parsing, definition lists, math blocks, wikilinks, hard wraps, superscript and subscript, table of contents, statistics. Each has a single [option flag in the README's grid][readme-options]. Most flags do what their name says.

[readme-options]: https://github.com/yaroslav/inkmark#options

### Safety and policy

Rendering Markdown from untrusted input is a sanitization problem dressed up as a formatting problem. Inkmark is safe-by-default; the policy controls let you tighten further.

#### Raw HTML

Raw HTML pass-through is *off by default*. Anything that looks like HTML in the source gets escaped:

```ruby
Inkmark.to_html("<script>alert(1)</script>")
# => "<p>&lt;script&gt;alert(1)&lt;/script&gt;</p>\n"
```

You can enable pass-through only for trusted content, like the one that was authored by you or your team.

```ruby
Inkmark.to_html("hi <em>there</em>", options: { raw_html: true })
# => "<p>hi <em>there</em></p>\n"
```

When `raw_html: true` is on alongside GFM, the **tagfilter** still strips a small set of dangerous tags, the GFM spec's own minimal hardening. You can disable the tagfilter, but at that point, sanitization is up to you.

#### Allowlists for hosts and schemes

You can lock link destinations and image sources to a list of hostnames using glob syntax. This can be useful if, for instance, you only want to allow images from your CDN hosts—something users uploaded using your own image processing tools.

```ruby
Inkmark.to_html(md, options: {
  links:  { allowed_hosts: ["example.com", "*.example.com", "github.com"] },
  images: { allowed_hosts: ["cdn.example.com", "imgur.com"] }
})
```

Default is no filtering. Pass `[]` for deny-all-external, useful when rendering content that should never link out.

You also can—and probably should—drop unsafe URL schemes: `javascript:`, `data:`, anything you don't expect:

```ruby
Inkmark.to_html(md, options: {
  links:  { allowed_schemes: %w[http https mailto] },
  # block `data:` URIs in img src
  images: { allowed_schemes: %w[http https] }  
})
```

### Frontmatter

Inkmark supports parsing frontmatter: it parses a leading YAML block and exposes it via `Inkmark#frontmatter`. Useful when Markdown files are the source of truth for structured metadata—blog posts, docs sites, or content collections.

```ruby
md = Inkmark.new(post, options: { frontmatter: true })
md.to_html       # => "<h1>Body</h1>\n"
md.frontmatter   # => { "title" => "Hello", "tags" => ["ruby", "markdown"] }
```

### Statistics and extraction

Say you need document statistics: word and character count, or how many blocks of a specific type there are. Inkmark collects document metadata. Two independent options control what's exposed: `statistics: true` counts elements and even does language detection. `extract: { kind: true, ... }` populates structured arrays of records:

#### Statistics and structured extraction

```ruby
md = Inkmark.new(source, options: { statistics: true })
md.to_html

md.statistics
# => {
#      heading_count:   2,
#      character_count: 142,
#      word_count:      28,
#      code_block_count: 1,
#      image_count:     1,
#      link_count:      2,
#      footnote_definition_count: 1,
#      likely_language:    "eng",
#      language_confidence: 0.93
#    }
```

`extract` opts into structured arrays of records; you ask for the kinds you need. You can go over every heading to make a custom table of contents (more on that later), collect all links, collect all images to make a gallery—all without writing a line of code to parse the original Markdown text.

```ruby
md = Inkmark.new(source, options: {
  extract: {
    headings:             true,
    links:                true,
    images:               true,
    code_blocks:          true,
    footnote_definitions: true
  }
})
md.to_html

md.extracts[:headings]
# => [
#      { level: 1, text: "Hello World",  id: "hello-world",  byte_range: 0...14 },
#      { level: 2, text: "Code Example", id: "code-example", byte_range: 68...83 }
#    ]

md.extracts[:code_blocks]
# => [{ lang: "ruby", source: "puts \"hello\"\n", byte_range: 78...101 }]
```

Every record carries a `byte_range` pointing into the original source string, if needed; `source.byteslice(r.begin, r.size)` recovers the raw Markdown. Useful for RAG-readiness: you can map the block back to exact source bytes—for grounding, for highlighting, for verification—without substring searches that fail on paraphrase or near-duplicate sections.

#### Table of contents

Set `toc: true` to collect a table of contents alongside the render. The resulting `toc` object exposes `#to_markdown`, `#to_html`, and `#to_s` for usage in whatever format you need.

```ruby
md = Inkmark.new(source, options: { toc: true })
md.to_html
md.toc.to_html         # rendered <nav><ol>...</ol></nav>
md.toc.to_markdown     # ordered list with anchor links
```

`toc: { depth: 3 }` limits which heading levels appear (h1–h3 in that example). Enabling `toc` implicitly enables `headings: { ids: true }` so anchors work, and triggers heading extraction, so `md.extracts[:headings]` is populated as well!

### Quality-of-life features

A few of the remaining flags hide features you'd otherwise reach for a second gem to get—or hand-roll over the post-rendered HTML.

For **typography**, `smart_punctuation: true` rewrites ASCII punctuation during the parse: `--` → en dash, `---` → em dash, `...` → ellipsis, straight `"..."` → matched curly quotes. One flag, no second-pass typography pipeline, all done in native code.

`images: { lazy: true }` adds **lazy-loading images support**: `loading="lazy" decoding="async"` on every `<img>` Inkmark emits. Browsers defer off-screen images natively—no JS, no IntersectionObserver scaffold. For long-form Markdown this is free page-load latency reduction.

**Autolinking** is also available: `links: { autolink: true }` promotes bare URLs and email addresses to anchors with correct boundary detection—it won't swallow the trailing period of a sentence. `links: { nofollow: true }` adds **`rel="nofollow noopener"`** to anchors pointing off-domain, the standard hygiene step for user-generated content.

`headings: { ids: true }` auto-generates **heading anchors**: `id` slugs from heading text, transliterating non-ASCII characters so non-English headings survive the trip: `## Привет, мир` produces a slug like `privet-mir` rather than blank or mangled output. Duplicates within a document get a counter suffix. Pair with `headings: { attributes: true }` to opt into explicit `# Heading {#custom-id .klass}` syntax when you want to pin a slug yourself.

`emoji_shortcodes: true` expands **gemoji-style `:shortcode:`** to emoji characters inline. Codes inside code blocks stay verbatim; unknown codes pass through untouched.

**Syntax highlighting** with `syntax_highlight: true` runs fenced code blocks through [syntect](https://github.com/trishume/syntect)—TextMate-grammar accuracy across dozens of languages, CSS-class output (not inline styles, so you own the theme). No client-side JavaScript, no flash of unstyled code, no Pygments shell-out.

## Inkmark is AI-first

With all that in mind, statistics and extraction are only half of Inkmark's AI/RAG story; the other half is the chunking, truncation, and serialization primitives. None of these exist as first-class methods in any other Markdown gem.

### Heading-based chunking

When working with RAG, we need to split a document into chunks before embedding. _What's a chunk, though?_ Either a structural unit (heading section) or a hard size budget (sliding window). And Inkmark gives you both.

`Inkmark.chunks_by_heading` walks the document and emits one entry per heading section, with the heading hierarchy as breadcrumbs:

```ruby
sections = Inkmark.chunks_by_heading(readme)

sections.first
# => {
#      heading:    "Installation",
#      level:      2,
#      id:         "installation",
#      breadcrumb: ["Inkmark", "Getting started"],
#      content:    "Run `bundle install`...\n"
#    }
```

The breadcrumb is the chunk's structural address: ancestor headings, root to immediate parent. When you embed this chunk for retrieval, prepend the breadcrumb to the content—`"Inkmark > Getting started > Installation\n\n..."`—and the embedding model gets the document's structural context for free. Without breadcrumbs, you're either embedding flat text (lossy: every "Installation" section in your corpus collides) or hand-rolling a heading walker on top of an AST.

Note that when enabling `statistics: true`, every section _also_ carries `:character_count` and `:word_count`—useful for size budgeting at retrieval time without re-counting.

The full RAG pattern is short:

```ruby
Inkmark.chunks_by_heading(readme).each do |s|
  next if s[:heading].nil?  # skip the preamble
  context = (s[:breadcrumb] + [s[:heading]]).join(" > ")
  embed_and_store("#{context}\n\n#{s[:content]}", metadata: { id: s[:id] })
end
```

`chunks_by_heading` always returns the full array, including a preamble entry (`heading: nil`, `level: 0`) for content before the first heading.

```ruby
sections = Inkmark.chunks_by_heading(readme)
sections.find    { |s| s[:heading] == "Installation" }
sections.select  { |s| s[:heading]&.match?(/install|usage/i) }
sections.reject  { |s| s[:heading].nil? }   # skip preamble
```

### Sliding-window chunking

For documents without clean heading structure, like transcripts or OCR output, heading-based chunking has nothing to grip on. `Inkmark.chunks_by_size` is the answer: fixed-size windows with overlap, walking the filter-applied Markdown sequentially.

```ruby
# Char budget with overlap
Inkmark.chunks_by_size(doc, chars: 500, overlap: 50)

# Word budget, cuts at word boundaries
Inkmark.chunks_by_size(doc, words: 120, overlap: 15, at: :word)

# Dual budget: cut at whichever is reached first
Inkmark.chunks_by_size(doc, chars: 1000, words: 200)
```

Each window comes back as a hash with an `:index` and `:content`, plus `:character_count` / `:word_count` when `statistics: true` is set.

Two boundary modes. `at: :block` (the default) cuts only between top-level Markdown blocks—your output stays valid Markdown, and a single block that exceeds the budget gets emitted as its own oversized window rather than silently sliced. `at: :word` serializes the full filtered Markdown and cuts at the last Unicode word boundary that fits—tighter packing, but it can split open constructs.

You can also compose the two for a "heading-based, but size-capped" hybrid: split by heading, fall through to sliding-window for any oversized section.

```ruby
Inkmark.chunks_by_heading(doc).flat_map do |c|
  if c[:content].size > 2000
    Inkmark.chunks_by_size(c[:content], chars: 500, overlap: 50)
  else
    [c]
  end
end
```

Inkmark doesn't ship a tokenizer; budgets are in characters or Unicode words. However, you can already get the likely document language, character and word count from statistics to do a ballpark estimation.

### Block- and word-aware truncation

There is also truncation: taking one document and capping it at a budget without breaking it. Useful when you're stuffing a doc into an LLM context window, when a chunk turns out larger than expected, when you're previewing.

The hand-rolled truncation would likely break code blocks, and sentences mid-word, mid-table. `Inkmark.truncate_markdown` does it correctly:

```ruby
# Cuts at the last complete block that fits—output stays valid Markdown
Inkmark.truncate_markdown(doc, chars: 4000, at: :block)

# Word-boundary cut—tighter, but may split open constructs
Inkmark.truncate_markdown(doc, chars: 4000, at: :word)

# Dual budget, custom marker
Inkmark.truncate_markdown(doc, chars: 4000, words: 500, at: :word, marker: "[…]")
```

The marker (default `"…"`) counts toward the budget, so `chars: 4000` always returns ≤ 4000 codepoints. Pass `marker: nil` to suppress the marker entirely.

`chunks_by_heading` accepts a `truncate:` keyword that applies the same contract per section—useful when you want every section to stand alone as a self-contained, budget-capped unit:

```ruby
Inkmark.chunks_by_heading(doc, truncate: { chars: 500, at: :block })
# => each entry's :content is ≤ 500 chars; metadata stays intact;
#    statistics counts are recomputed against the truncated content
```

### Plain-text extraction

Strip all Markdown syntax and return inline content as plain text. Designed for embedding models, token counting, LLM prompts, anything downstream that treats Markdown formatting as noise:

```ruby
Inkmark.to_plain_text("**bold** and [a link](https://example.com)")
# => "bold and a link (https://example.com)\n"
```

The same filter pipeline runs before serialization, so `to_plain_text`, `to_markdown`, and `to_html` see the same emoji expansion, autolink rewrites, and allowlist policy applied consistently:

```ruby
md = Inkmark.new(source, options: { emoji_shortcodes: true, links: { autolink: true } })
md.to_plain_text   # plain text with :rocket: → 🚀, bare URLs as text
md.to_html         # HTML with the same transforms applied
```

The output grammar is documented in the README and predictable: bold/italic/strike unwrap to inner text, links serialize as `text (url)` (collapsed when text equals url), images as `alt (src)`, code fences as their raw bodies, blockquotes as email-style `>` prefixes, lists as `- ` / `1. ` bullets with two-space-per-level indent.

### Markdown-to-Markdown pipeline

`#to_markdown` runs the same filter pipeline as `#to_html` and serializes the result back to Markdown. Use it as a preprocessing step in pipelines that consume Markdown rather than HTML—LLM prompts, secondary renderers, content storage, anything that wants clean, normalized Markdown:

```ruby
# Class-method form for one-shots
Inkmark.to_markdown("**bold** :rocket:", options: { emoji_shortcodes: true })
# => "**bold** 🚀"

# Instance form: same options object drives both outputs
md = Inkmark.new(source, options: {
  emoji_shortcodes: true,
  links: { allowed_hosts: ["trusted.com", "*.trusted.com"] }
})
md.to_markdown   # filtered Markdown for the next stage
md.to_html       # rendered HTML for display
```

## Inkmark has an extension API

The RAG primitives above—and most of Inkmark's own filter pipeline—are built on the same underlying extension API your own code uses. The model is event handlers. You register a block for any element kind, and Inkmark fires it during the parse, post-order—children before parents.

### Rewriting output

Set `dest=`, `level=`, `id=`, `html=`, or `markdown=` on the event to rewrite what ends up in the output. Setting `html=` skips the default rendering for that element, including any post-render filters; setting `markdown=` re-renders a Markdown string with the same options as the main document.

Rewrite images to be served from a CDN, rearrange headings, do custom markup for code blocks and filter markup, easily:

```ruby
md = Inkmark.new(source)

# Image CDN rewriting
md.on(:image) do |img| 
  img.dest = "https://cdn.example.com/#{File.basename(img.dest)}"
end

# Heading shifts (for fitting docs into a layout that owns <h1>)
md.on(:heading) { |h| h.level = [h.level + 1, 6].min }

# Custom code block rendering by language tag
md.on(:code_block) do |c|
  case c.lang
  when "mermaid" then c.html = %(<div class="mermaid">#{c.text}</div>\n)
  when "math"    then c.html = %(<div class="math">\\[#{c.text}\\]</div>\n)
  end
end

# Suppress elements entirely
#   all images
md.on(:image)   { |img| img.delete }
#   by content
md.on(:heading) { |h|   h.delete if h.text.start_with?("INTERNAL:") }   

# Replace a paragraph with a re-parsed Markdown snippet
md.on(:paragraph) do |p|
  next unless p.text.start_with?("@note ")
  p.markdown = "> **Note:** #{p.text.sub(/\A@note /, "")}"
end

html = md.to_html
```

Six things happening in one parse, no AST walking, no regex.

### Walking for analysis

`#walk` fires the handlers without rendering. Use it for pure analysis: validation, collection, instrumentation.

```ruby
# Verify every image has alt text
md = Inkmark.new(source)
missing_alt = []
md.on(:image) { |img| missing_alt << img.dest if img.text.empty? }
md.walk
raise "Missing alt: #{missing_alt.join(', ')}" if missing_alt.any?

# Collect every fenced code block language used in the document
languages = Set.new
Inkmark.new(source).on(:code_block) { |c| languages << c.lang if c.lang }.walk
```

Note: for built-in heading/link/image/word-count collection, `statistics: true` will likely solve most of your collection tasks; the handler API is for the cases where you need something custom.

### Tree context

Container elements expose their child events lazily:

```ruby
md.on(:table) do |t|
  rows = t.children_of(:table_row)
  rows.each_with_index do |row, i|
    cells = row.children_of(:table_cell).map(&:text)
    puts "Row #{i}: #{cells.join(' | ')}"
  end
end
```

Use `parent_kind`, `ancestor_kinds`, and `depth` for context-sensitive decisions:

```ruby
# Skip decorative images that are already inside a link
md.on(:image) { |img| img.delete if img.ancestor_kinds.include?(:link) }

# Process only top-level paragraphs
md.on(:paragraph) do |p|
  next unless p.parent_kind.nil?
  # ... only top-level paragraphs reach here
end
```

`depth` is 0 at the top level and increments once per nesting level—a paragraph inside a blockquote has `depth: 1`.

### Source byte ranges

Every event carries a `byte_range` pointing into the original source, the same one extraction records use. Useful when handlers need to correlate elements back to source positions—for syntax-highlighting injection, source maps, diffing:

```ruby
source = File.read("post.md")
md = Inkmark.new(source)
md.on(:heading) do |h|
  raw = source.byteslice(h.byte_range)
  puts "#{h.byte_range}: #{raw.inspect}"
end
md.walk
```

## Time to try it

```
bundle add inkmark
```

Or in your `Gemfile`:

```ruby
gem "inkmark"
```

No Rust toolchain is needed for installation. Inkmark supports Ruby 3.3+.

Definitely read the [README at GitHub][inkmark]—it has the full options grid, the full event-handler surface, and the benchmark methodology in case you want to reproduce the numbers above.

Don't hesitate to [ping me on X][yaroslav-x] with feedback.

[yaroslav-x]: https://x.com/yaroslav