@amigo-labs/stemmer

Porter/Snowball stemmer powered by rust-stemmers — batch-only API. Single-word stemming is intentionally not exposed.

18 languages (Arabic, Danish, Dutch, English, Finnish, French, German, Greek, Hungarian, Italian, Norwegian, Portuguese, Romanian, Russian, Spanish, Swedish, Tamil, Turkish).

Install

pnpm add @amigo-labs/stemmer

Usage

import { Stemmer, stemOnce } from '@amigo-labs/stemmer'

const s = new Stemmer('english')

// Stem a list of words in a single FFI crossing.
s.stemMany(['running', 'cats', 'jumping'])
// ['run', 'cat', 'jump'] (Porter output)

// Tokenize and stem in one call — the realistic RAG / search hot-path.
s.tokenizeAndStem('The quick brown fox jumps over the lazy dog', {
  lowercase: true,
  minTokenLength: 2,
  stopwordsEn: true,
})
// ['quick', 'brown', 'fox', 'jump', 'lazi', 'dog']

// Buffer in, Buffer out — zero-copy for pipelines that stay in Rust.
s.tokenizeAndStemToBuffer('running quickly jumping')
// <Buffer 72 75 6e ...> → "run\nquickli\njump"

Install for the browser

The same import works in Angular, React, Vite, esbuild, and webpack ≥ 5 — the bundler picks the WASM build via the browser conditional export:

import { Stemmer } from '@amigo-labs/stemmer'

rust-stemmers is small (~80 KB gzipped), well under the 500 KB budget.

Why no stem(word)?

Single-word stemming in JavaScript is ~30 ns of work. The NAPI FFI floor alone is ~109 ns — you’d pay 3× the algorithm’s cost just to cross the boundary. Calling stemMany(['word']) loops with no FFI crossing, which is the correct shape. See docs/post-mortems/levenshtein.md for the measured precedent.

stemOnce(lang, word) exists for ad-hoc usage (tests, REPL) and is explicitly documented as a slow path.

Migration from natural

See MIGRATION.md. The stemmer subset of natural is nearly drop-in, but single-word PorterStemmer.stem(word) is not supported by design.

License

MIT

Perf-Review: @amigo-labs/stemmer

Status: 🟢 Green (measured) · Reviewed: 2026-07-02 · Version: 0.1.1

Verdict

7.03–7.85× across all four scenarios vs. natural (bench 2026-05-28). The batch-first API shape does exactly what the candidate review demanded: one FFI crossing per corpus chunk instead of one per token, so the Snowball compute in rust-stemmers dominates and the multiplier stays flat from 1k-token batches up to 100 KB tokenize-and-stem runs. Exceeds every candidate gate (≥3× batch-1000, ≥5× batch-10k, ≥3× tokenizeAndStem-10KB).

Evidence

Measured speedup (docs/benchmarks/stemmer.json, 2026-05-28, commit 45d29b4)

Scenario	@amigo-labs/stemmer	natural	Speedup
stemMany × 1 000	1 832 Hz	260.7 Hz	7.03×
stemMany × 10 000	190.2 Hz	26.67 Hz	7.13×
tokenizeAndStem 10 KB	1 313 Hz	172.2 Hz	7.62×
tokenizeAndStem 100 KB	133.1 Hz	16.95 Hz	7.85×

• docs/packages.json speedup: "7–7.9× faster".
• Install size: 26 KB vs natural’s 65 MB node_modules (natural bundles the whole NLP toolkit; we ship only stemming).

What shipped vs. the candidate prediction

• Batch-only Stemmer class: stemMany, tokenizeAndStem, tokenizeAndStemToBuffer; stemOnce exists as the documented slow path.
• 18 languages (candidate said 17 — rust-stemmers added Tamil in the interim).
• English-only stopword list in v0.1.

Divergences

rust-stemmers implements a newer Snowball revision than natural’s hand-ported stemmers — a small set of words stems differently (documented with examples). Not a natural drop-in; it replaces only the stemming corner of it. See crates/stemmer/__conformance__/divergences.md.

Pre-port assessment: natural.md

References

• Crate: crates/stemmer
• Bench shard: docs/benchmarks/stemmer.json
• docs/packages.json speedup: "7–7.9× faster"