Under both: Cloud. Sandboxes, services and simulated computers.
An illustration · drag to orbit
Two customers.One loop between them.
Each side supplies what the other lacks. Labs lack real, under-specified engineering work and real experts. Factories lack models that have been measured on that kind of work. A person approves every change.
1 / 3
Real work becomes tasks and graders.
Real work, opted in and rights-cleared, becomes tasks, graders and data.
2 / 3
Tasks train and evaluate models.
Those tasks, graders and data train and evaluate models.
3 / 3
Better models go back to work.
Better models go back to work and produce more of it. A person approves every change.
Every number comes from a computation,never from a language model.
The same engines serve as tool, grader and data generator.
Physics
Structural and thermal analysis on CAD files, validated against closed-form answers
Search over documents and manuals with sources and unit conversion
For engineers
Experiments
Design of experiments and campaigns that propose the next runs
For r&d engineers
Monitor
Control charts, anomaly detection, root cause, remaining life
For quality, maintenance
Fluids
Learned model of surface aerodynamics with uncertainty
For engineers, labs
Engineer
From a requirement to a verified design package
For design engineers
OHRQ Bench: nine tracks.Two at release 0.1.
Nine tracks from CAD and G-code to plant operations. Every answer is graded by a deterministic grader or a solver, never by a model. Two tracks are at release 0.1, with floors and ceilings measured.
Track
What it tests
Release
Engineering
Design and manufacturing work with one checkable answer: CAD programs, G-code, PLC programs, wiring, tolerances, schedules, inspection plans
Release 0.1
Computer Use
Office and engineering-adjacent work on simulated computers; runs replay to the same state
Release 0.1
CAD Desktop
The same part-modelling tasks, solved by driving real CAD software from screenshots
Factory Assistant
An assistant inside a running plant system, graded on outcomes
Physics
Predict stress, temperature and surface pressure on shapes the model has not seen
Process
Mass and energy balances, heat integration, pump and pipe sizing, equipment cost
Operations
Scheduling and inventory decisions in a simulated plant
Design
From a requirement to a part a solver and the manufacturing rules both accept
Real Work
Tasks from real engineering work at companies that opted in
Seven synthetic data products, each with a card written from measured numbers and a free sample pack.
Checked rows. Every row is re-checked by its source’s grader or solver in a fresh process before release.
No overlap with test sets. Anything matching a hidden benchmark task or a held-out engine task is dropped, so sold data cannot contaminate an evaluation.
Traceable deliveries. Deliveries use signed, expiring links, and each order can be traced if a copy leaks.
The data catalogue
eng-sft
Engineering tasks with checked answers, for supervised training
eng-pref
Preference pairs over engineering answers
eng-traj
Step-by-step trajectories on engineering tasks
cu-transcripts
Computer-use episodes on simulated computers
proc-qa
Process-engineering questions with computed answers
Frontier agents already pass 32 of 35 synthetic engineering tasks in pilot runs, which is why synthetic-only benchmarks saturate and why OHRQ builds on real work. Every number here comes from OHRQ’s own recorded runs.
Experiments
36 of 42
Process-optimisation tasks passed. Simple methods pass 4 to 12.
Monitor
76 of 102
Diagnosis and quality tasks passed: 76 of the 83 it attempts.
Machining
61 of 63
Machining tasks passed, on a seed it had not run before.
Knowledge
91% of 120 queries
Queries where it finds the known item first. Plain substring search finds 86%.