/Docs · Louvain Engine

At scale

Load speed, search latency, disk and memory at 10,000 and 100,000 passages, measured

In short: on a six-core desktop CPU, 10,000 passages load in 51 minutes with the default embedder and search in under 250 ms at the 95th percentile. 100,000 passages, loaded without the embedder, take 32 minutes and 1.6 GB of disk, and a keyword search takes about 0.6 s. The embedder sets the load speed. Each row below was measured; no row is estimated from another.

The machine#

An Intel Core i5-8500B at 3.0 GHz, 6 cores, 24 GB of memory, in a Linux virtual machine with Node 22 and Docker 29. The stack is the one ./engine up starts, in its default configuration: the CPU embedder, the keyword index and Postgres, with no reranker and no answerer. Nothing in the run is paid for and nothing leaves the machine.

The corpus and the queries#

The corpus is made up from a fixed seed (packages/engine/scripts/scale-bench/corpus.ts): one document per passage, each with a title, a key and a paragraph drawn from a seeded word list. 50 picks at random choose the targets (a document picked twice counts once, so 10,000 has 49), and each target gets three queries:

  • keywords: two of its rarer words;
  • key: its key;
  • question: "What do the title say?".

The first 10 queries run once as an untimed warm-up, then every query runs once, timed, one at a time. Found counts the queries whose target is among the page's documents. A keywords query often misses because other documents share the same two words, so its found count describes the corpus, not a defect.

Loading#

PassagesVectorsLoad timePassages a second
10,000embedded by the default CPU embedder51 min3.3
100,000keyword-only: a stand-in vector, no embedder call32 min51.6

The load goes through the records source and the engine's one ingest path, in batches. At 10,000 the embedder is the ceiling: loading runs at its pace on this CPU, and it embeds about 7 passages a second at best on a faster one. Everything else in the load costs about 0.02 s a passage, which is what the keyword-only row measures. To load faster, serve an embedder on a GPU at LOUVAIN_EMBEDDINGS_URL (Configuration). 1,000,000 passages were not run: this machine has neither the time nor the disk for them.

Searching#

Times are in milliseconds.

PassagesModeQueryFoundp50p95
10,000hybridkeywords20 of 49173245
10,000hybridkey49 of 49108135
10,000hybridquestion49 of 49185242
10,000keywordkeywords25 of 4984141
10,000keywordkey49 of 497887
10,000keywordquestion49 of 4993159
100,000keywordkeywords15 of 505681,098
100,000keywordkey50 of 50544581
100,000keywordquestion50 of 50596682

A keyword search at 100,000 passages takes about seven times as long as one at 10,000. This run does not say which stage the extra time goes to. The 100,000 row has no hybrid search because its vectors are stand-ins.

Disk and memory#

PassagesPostgresKeyword indexPeak memory while loadingPeak memory while searching
10,000150 MB16 MBengine 167, Postgres 181, embedder 1,023, keyword index 82engine 191, Postgres 255, embedder 1,024, keyword index 76
100,0001,419 MB144 MBengine 223, Postgres 427, embedder 1,032, keyword index 165engine 148, Postgres 233, embedder 1,021, keyword index 211

Disk is the Postgres database's total relation size plus the workspace's keyword index directory. Memory is in MiB, the highest of samples taken every two seconds: the engine's resident set, and each service's figure from docker stats. The embedder holds its model in memory whether it is called or not.

Running it#

From an install's folder:

Code
./engine up
node --import tsx packages/engine/scripts/scale-bench.ts --scales 10000,100000

Each scale loads into a database of its own and prints one RESULT line, which holds every figure on this page. --embed-up-to (10,000 by default) sets the largest scale loaded with the real embedder, --targets the number of target documents, and --seed the corpus.

We use Google Analytics, and Google’s and LinkedIn’s advertising tags, to see how this site is used and which ads bring people here. They set cookies only if you accept. Cookie Notice.