Performance and accessibility checks

The 0.8 qualification (issue #68) asks for measured performance and keyboard-accessible workflows. Its budgets were frozen by the maintainer from the measurements below (see Budgets). This page describes the tools that produce those measurements, and records the first.

The corpus

tests/data/corpus/ holds a versioned, public-safe set of drawings as .vsk documents. manifest.json records each case’s SHA-256 and what it holds, so a measurement names exactly what it measured:

Case	Kind	Atoms	Species	Reactions
small-ethanol	small	3	1	0
small-stereochemistry	small	13	5	0
small-mechanism	small	12	3	1
small-three-step	small	40	6	3
large-peptide	large	201	1	0
large-scheme-30	large	1,020	31	30
large-chain-2000	large	2,000	1	0
crowded-numbered-60	crowded	534	60	0
crowded-grid-150	crowded	2,685	150	0

The small cases are the manual’s example documents; scripts/make_corpus.py generates the others deterministically and checks the files (--check). Changing a case means a new corpus version, and measurements of different versions are not compared. The test suite checks the manifest and reopens every case through .vsk and QVF with the same species and reactions, so work on speed cannot quietly change graph or stereo fidelity.

The harness

python scripts/benchmark.py --output perf.json
python scripts/benchmark.py --case large-scheme-30 --repeat 10

For each case it times loading, deriving species and reactions, cleanup, Tidy scheme (when there are reactions), SVG, PNG and PDF rendering and QVF export: once cold and --repeat times warm (5 by default), keeping every observation, with the peak memory Python allocated in a separate run (tracemalloc, which leaves out what RDKit allocates in C++). An operation that does not apply or whose optional part is not installed is recorded as skipped, a deliberate refusal as refused, and anything else as a failure with a nonzero exit. The record names the operating system, architecture, processor model, logical CPUs, memory, Python, RDKit, vibe-sketcher and corpus versions, never the machine’s name.

First measurement (2026-10-22)

Apple M4 Pro, 12 logical CPUs, 24 GiB, macOS (Darwin 27.0), Python 3.14.8, RDKit 2026.09.1, vibe-sketcher 0.3.0 (main), corpus version 1; warm medians of three runs, in seconds:

Case	Load	Derive	Cleanup	Tidy	SVG	PNG	PDF	QVF
small-ethanol	0.001	0.001	0.001	–	0.000	0.004	0.000	0.006
small-stereochemistry	0.001	0.004	0.010	–	0.001	0.033	0.004	0.067
small-mechanism	0.000	0.002	0.003	0.029	0.002	0.010	0.002	0.014
small-three-step	0.001	0.010	0.033	0.046	0.002	0.072	0.026	0.033
large-peptide	0.002	0.023	0.216	–	0.019	0.148	0.037	0.048
large-scheme-30	0.007	0.107	0.463	0.877	0.070	1.459	0.160	0.383
large-chain-2000	0.012	0.228	20.928	–	0.153	refused	refused	0.933
crowded-numbered-60	0.005	0.143	0.394	–	0.036	0.405	0.108	0.321
crowded-grid-150	0.021	0.221	1.204	–	0.138	1.062	0.217	1.142

What it showed:

• Everything but cleanup of the 2,000-atom chain stayed near or below 1.5 s. That cleanup took about 21 s and peaked at about 217 MB of Python memory.

• The chain’s PNG and PDF are refused: the figure would be wider than the 14,400 pt publication limit. That refusal is by design; SVG and QVF work.

The maintainer judged that cleanup over budget. Profiling showed that most of it went to three whole-molecule layouts tried one after another, whose cost grows faster than the molecule. Per candidate, in seconds (rdkit / mimic / coordgen), they took 0.10 / 0.08 / 0.13 at 500 atoms, 0.72 / 0.47 / 0.60 at 1,000 and 5.2 / 1.8 / 2.1 at 2,000. The overlap count built an n × n matrix, about 130 MB at 2,000 atoms. Cleanup now:

• lays out a molecule of more than 500 atoms with the distance-matrix “mimic” candidate alone, which is the fastest at that size and follows the drawing most closely (on the chain all three tie, and mimic was the one chosen);

• leaves a molecule of more than 3,000 atoms as drawn, saying why (mimic alone takes 3.5 s at 3,000 atoms and 10 s at 5,000);

• counts overlaps row by row, in memory linear in the atoms.

After the cleanup limits (2026-10-22)

The same machine and corpus, warm medians of three runs, in seconds:

Case	Load	Derive	Cleanup	Tidy	SVG	PNG	PDF	QVF
small-ethanol	0.000	0.001	0.001	–	0.000	0.001	0.000	0.004
small-stereochemistry	0.001	0.003	0.003	–	0.001	0.010	0.002	0.017
small-mechanism	0.001	0.002	0.003	0.010	0.001	0.005	0.001	0.011
small-three-step	0.001	0.004	0.010	0.026	0.002	0.029	0.008	0.021
large-peptide	0.002	0.005	0.110	–	0.009	0.080	0.015	0.028
large-scheme-30	0.007	0.058	0.221	0.388	0.028	0.708	0.082	0.164
large-chain-2000	0.010	0.113	2.535	–	0.080	refused	refused	0.400
crowded-numbered-60	0.004	0.047	0.130	–	0.018	0.189	0.043	0.152
crowded-grid-150	0.017	0.113	0.592	–	0.067	0.499	0.113	0.442

The chain’s cleanup now takes 2.5 s instead of about 21 s, and its peak Python memory is 64 MB instead of 217 MB. It still cleans, with the same result, and the smaller cases are unchanged. The 50-atom cleanup target of #18 is met by a wide margin.

Budgets

The maintainer froze these budgets on 2026-10-23 from the record above, with about twice its numbers as headroom. They apply to the warm median on the reference machine of that record, per operation, for corpus version 1:

Scope	Budget
Small cases	0.1 s
Large and crowded cases	1.5 s
Cleanup of one molecule over 1,000 atoms	5 s
Peak Python memory (tracemalloc)	128 MB

The browser budgets were frozen on 2026-10-30 from the first browser measurement, again at about twice its numbers. They apply to the warm median in Chromium on the reference machine, for corpus version 1:

Browser	Small cases	Large and crowded cases
Open to drawn and listed	1.5 s	7.5 s
Edit until listed	1.6 s	3.2 s
JS heap after the cold open	450 MB	450 MB

Cold runs are recorded but not budgeted. A release candidate re-measures with scripts/benchmark.py --browser, and an exceeded budget blocks the release. A deliberate refusal, such as the chain’s PNG and PDF or cleanup above 3,000 atoms, is an outcome of its own, not a pass.

scripts/benchmark.py --check compares a saved record with these budgets:

python scripts/benchmark.py --browser --output perf.json
python scripts/benchmark.py --check perf.json

It prints within budget and exits 0, or names each failure and exits 1. A failure is an operation over its time or memory budget, and also a record that cannot show the budgets are met: a case not measured, an operation that failed or was skipped for want of an optional part, no warm runs (--repeat 0), a corpus file or version that differs from this checkout’s, or no browser measurement. The cleanup budget of 5 s applies to a case whose largest molecule has more than 1,000 atoms, so the 2,685-atom grid of small molecules keeps the 1.5 s budget. Loading is measured once, cold, and is not budgeted. Refusals are listed as notes, and so is a record from a processor other than the reference one, which the check treats as a guide only. The 2026-10-24 measurement of corpus version 1 on the reference machine is within budget; its slowest operation is the chain’s cleanup at 2.6 s.

In the browser

The budgets above measure vibe-sketcher’s Python side. What a user waits for in the shell also includes the canvas engine drawing in the browser, so --browser measures each case in Chromium too (maintainer decision, 2026-10-30):

python scripts/benchmark.py --browser --output perf.json

It serves the built shell (web/dist) from a local server and, for each case, once cold and --repeat times warm, with a fresh copy of the case each time:

• open: from loading the shell on the document until Save is enabled, the canvas has drawn at least one path per bond and the panel lists every species;

• edit: from placing a cubane template on the canvas until the server has derived the change and the panel lists as many species as the server’s answer. It includes the shell’s fixed 0.5 s pause before it sends a drawing change. The shell ignores the canvas engine’s echo for 0.8 s after it shows a document; if a placement falls into that window, nothing is sent, so the harness places another cubane and times that one, recording the attempts.

The JS heap Chromium reports after the cold open is recorded too, with the browser, Playwright and canvas-engine versions and the window size, in a browser section of the same record. --check checks it against the browser budgets, frozen from the record below, and fails a record without one, measured in another engine, or with a case’s open or edit failed, over budget or measured without warm runs. An edit that needed more than one placement is noted.

First browser measurement (2026-10-30)

The reference machine (Apple M4 Pro, 12 logical CPUs, 24 GiB, macOS), corpus version 1, vibe-sketcher 0.6.3, Chromium 156.0.8078.4 through Playwright 1.64.0, Ketcher 3.18.0, a 1440 × 1000 window; warm medians of three runs, in seconds, and the JS heap after the cold open:

Case	Atoms	Open	Edit	JS heap (MB)
small-ethanol	3	0.728	0.749	72
small-mechanism	12	0.740	0.766	72
small-stereochemistry	13	0.739	0.782	72
small-three-step	40	0.756	0.798	92
large-peptide	201	0.859	0.793	92
large-scheme-30	1,020	1.739	1.578	188
large-chain-2000	2,000	2.125	0.788	199
crowded-numbered-60	534	1.241	1.097	139
crowded-grid-150	2,685	3.624	0.844	225

Every run was within a few hundredths of a second of its median, and no placement needed a second attempt. About 0.7 s of opening any case is the shell and the canvas engine starting; the rest grows with the drawing, the 150-molecule grid taking longest. An edit costs about 0.75 s on a small drawing, two thirds of it the fixed pause, and up to 1.6 s on the 30-step scheme, whose panel lists the most.

Keyboard and screen readers

The browser shell’s tests check, in a real browser:

• the WCAG 2.2 A and AA rules axe-core can test, on the shell, its panels and Ketcher’s palettes in several states, including a selected atom tool (tests/test_browser_accessibility.py). The drawing surface itself is left out;

• menus, the tool bar and the panel by keyboard alone, with focus kept where the user was when the panel updates;

• that every control the Tab key reaches shows a 2 px focus ring;

• that information is announced politely and fades, while an error interrupts screen readers (role="alert") and stays until the next message, a click or Escape, so nobody has to read it in time or needs a pointer to clear it;

• that each tool in the compact palette whose face is a letter includes that letter in its name, for example Carbon (C), so speech input can say what it shows;

• a whole session by keyboard alone: import a structure, generate its 3D structure, submit a calculation and cancel it, save with Ctrl+S, export through the Export menu and open a file, each control reached with Tab from where the last one left the focus;

• that Tab and Shift+Tab both leave the canvas, in either palette.

Canvas limitations

The drawing canvas is Ketcher’s (EPAM). Where it falls short, the shell offers another way:

Limitation	What to use instead
Atoms and bonds are drawn with a pointer; there is no way to place them by keyboard.	Import ▾ → SMILES…, Name… or File…, all by keyboard; or the SDK, CLI, REST API and MCP tools.
A screen reader cannot read the drawing itself.	The Properties panel lists each species with its formula, compound number, multiplicity and name, and each reaction with its sides; Skip to properties and inputs jumps there. vibe-sketcher info and the REST API report the same.
Ketcher’s palettes cycle Tab among themselves, and its clipboard area takes the focus back on Shift+Tab, which would trap a keyboard user.	The shell moves the focus itself: Tab and Shift+Tab pass through the canvas’s controls and out of it.
Ketcher’s palettes add about 47 Tab stops.	The compact palette (?palette=compact) shows the shell’s own tool bar instead: one Tab stop, arrow keys inside it.
Ketcher’s drawing surface is outside the axe-core audit, which cannot judge it.	The shell, the panels, the menus and Ketcher’s palettes are audited. The adapter names two controls Ketcher leaves unnamed (the copy button’s triangle and its clipboard area). It also darkens five palette labels, O, S, P, F and Cl, to at least 4.6:1.
Ketcher’s copy button has a 9 px triangle for more formats, below WCAG 2.2’s 24 px target size.	Export ▾ → Copy as MOL, Copy as CDXML, Copy as SVG or Copy as PNG, or Export ▾ → KET for Ketcher’s own format.
The drawing colours atom labels by element. Red O, gold S, orange P, and green F and Cl are below 4.5:1 on white.	Every journal export prints the labels in black. The Properties panel gives each species’ formula and SMILES as text.

A manual review with a screen reader is still to come, as #68 asks.