How Accurate Is It?

Every figure below is measured against published reference data, on cases the model never saw during fitting. Where we have not measured something, we say so.

How These Numbers Are Produced

A simulation that is tuned on a dataset and then scored on that same dataset is testing its own memory, not its ability to predict. Every percentage on this page is measured on held-out data — propellers, motors or battery packs excluded from the fit. Several are leave-one-out. Where a figure only holds inside a certain range, the software withdraws it outside that range rather than quoting a number that does not apply.

Propulsion

Benchmarked against the UIUC propeller database, manufacturer bench reports and published datasheets.

Measured

P1 — Thrust Mapping

Coefficients fitted to 210 static wind-tunnel tests from the UIUC propeller database, scored on propellers held out of the fit.

  • Thrust within ~15%
  • ~6% when the product line is named
  • Two 10×4.7s from different makers differ by ~22% — a printed size does not identify a blade
Measured

P2 — Hover Power

Every figure is leave-one-motor-out, so it is a prediction rather than a memory of the data.

  • Within 7–8% for rotors carrying 5–40 N each
  • Cross-maker check: within 4.4%
  • Hobbywing X8, 30-inch propeller: within 6.3%
Measured

P3 — Forward Flight

Scored on propellers the fit never saw. Outside 9–19 inches the page says so and withdraws these figures.

  • Thrust within 14%
  • Efficiency within 0.07
  • Roughly halves the error of the model it replaced
No figure — deliberately

P4 — ESC Thermal

Nobody publishes a logged temperature trace from a drone ESC in flight, so there is no error percentage here and this page does not invent one.

  • Current is validated — same torque balance as P1
  • Temperature is anchored, not fitted: returns the manufacturer's own limit at the continuous rating
  • Rise time is an order-of-magnitude figure, and is labelled as one

Avionics & Control

Control loops have no wind tunnel. The reference is the closed-form solution — which is exact, and a stronger check than a measurement, not a weaker one.

Measured

A1 — PID Attitude

Integrated with RK4 under a zero-order hold on torque, which is what a real controller applies between updates.

  • Matches the analytic solution to about 1×10-12 radians
Measured

A2 — Altitude Hold & EKF

  • Filter gains match Kalata's published closed form to about one part in a million
  • Sensor noise modelled on published Bosch datasheets
Measured

A3 — Yaw / Magnetometer

  • Earth's field matches NOAA's own printing precision across all 100 of their published reference values
  • Gyro-bias behaviour verified against the filter's closed form
Measured

A4 — Waypoint Guidance

  • Closed-loop damping matches the damping parameter to better than 1%
  • Measured by logarithmic decrement, not curve fitting
Measured

A5 — Failsafe

  • Every rate is read out of ArduPilot's source, with a test pinning each to the file it came from
  • Priority logic checked exhaustively across all 24 fault combinations

AI / ML

Scored against real measured datasets, not against data shaped like the model.

Measured

AI1 — Anomaly Detection

F1 scores over 400 randomised flights. No method wins everywhere, and the page says so.

  • Altitude — Z-score 0.88 best
  • Battery — IQR 1.00 best
  • Vibration — IQR 0.93 best
Measured

AI2 — Battery Health

The most useful number on this page is the gap between the two rows below.

  • Synthetic packs: 3.9 cycles
  • 14 real NASA packs: 77 cycles
  • Error is biased −69 cycles — it retires a pack early, the safe direction to be wrong in
Measured

AI3 — Landing Pad Vision

  • Under 2 cm ground position error at 4 m with a 200 px marker
  • ~1.5 cm in motion with noise, whenever the marker is found
Measured

AI4 — Swarm Simulation

  • Exact for the model it states — a study of a control policy, not a prediction of a real aircraft
  • Policies checked against published literature behaviour

What We Do Not Claim Yet

13 of the 25 experiments currently publish a measured accuracy figure. These 12 do not.

Validation in progress

These experiments are built, tested and in daily classroom use. What they do not yet have is a published error figure against external reference data.

Performance — PF1 to PF5
Structures — S1 to S5
Propulsion — P5 (battery pack under load)
AI/ML — AI5 (tracking & prediction)

Why publish this at all?

Because a vendor who claims everything is validated is telling you nothing. A list of what is measured is only meaningful next to a list of what is not.

Every experiment states its own accuracy on screen, in the software, next to the result — including the ones that say “not measured”. Students see the same caveats you do, which is the point: knowing what a model does not cover is part of the engineering.

This page is updated as validation work lands.

Reference Sources

Every figure above traces to published data. We link to the sources; we do not redistribute them.

Propellers & Motors

Avionics & Sensors

AI / ML Datasets

Aerodynamics & Aircraft

Individual experiments cite further sources in the software itself, on the page where the figure is used. Datasets remain the property of their publishers and are linked, never rehosted.

Check It Yourself

We will walk your department through any figure on this page and show where it comes from.

Request a Demo