Checking the model against reality
Your code agreeing with MuJoCo to the sixteenth decimal place proves that two programs computed the same thing. It says nothing about whether that thing is your pendulum. This module separates the two questions and answers the second one on a bench, with an honest error bar.
8 formulas · 10/10 checks pass · 22 sources · fact-checked and reviewed
Key ideas
Verification and validation are different questions
Verification asks whether the code solves the model correctly. Validation asks whether the model represents the real world well enough for its intended use. As Oberkampf and Trucano put it, "Verification does not address whether the conceptual model has any relationship to the real world."[1]
A perfect match between two programs checks the arithmetic, not the inputs
Hand-computed energy matching MuJoCo's to 8×10⁻¹⁷ J (F6.10) shows that both used the same , and and the same formula. A wrong mass, a forgotten screw or a mismeasured length goes into both and cancels out of the comparison.[1]
Measure what the equations use
The swing depends on two numbers, and , and both can be measured directly: from a static balance on a kitchen scale (F9.01), then from the period (F9.02). Neither needs the shapes or masses of the parts, only one length, the lever arm, which turns out to dominate the error budget. Any mass the parts list missed ends up in the measured and .[2][3]
The error budget comes before the measurement
Small independent relative errors add in quadrature (F9.03), and a squared quantity counts double: , so 0.1 % on the period is 0.2 % on . Work the budget out first. It tells you which instrument to improve, and what size of disagreement the test can detect at all.[4][5]
9.81 is nobody's gravity in particular
Standard gravity, 9.80665 m/s², is a defined constant (F9.08), and MuJoCo's default is 9.81. Normal gravity at sea level runs from 9.7803 m/s² at the equator to 9.8322 m/s² at the poles, and equals 9.81 only near 49.2° latitude (F9.04). The measured is a scale reading times , so the wrong biases by the same fraction.[7][8]
Agreement has a definition
The bench and the model agree when their difference is within about twice their combined standard uncertainty (F9.07). A larger gap means something is unaccounted for, in the model or in the bench's own budget (or, about one time in twenty, chance). A smaller one means the test cannot tell. "Close enough", without a number, is neither.[4]
Change one thing at a time
Measure the pendulum on a free bearing first, then on the motor with its windings open, then with the driver connected. Each step adds a short, known list of effects (the rotor's inertia, the motor's bearings and cogging, then electrical braking), so each difference has few possible causes.[9][10]
Formulas
The same entries as the formula sheet. Open an entry's details for its symbols and the checks behind it.
Gravity stiffness from a static balance
Pendulum held horizontal, pivot free to turn (off the motor, so cogging cannot hold part of the load). Pivot friction can carry up to its static torque either way: average a reading taken after lowering onto the scale with one taken after pressing down and releasing.
Symbols
Symbol Meaning Unit force on a scale under the far end N reading of a scale calibrated with a known mass on the bench kg pivot to the scale's contact point m Inertia the dynamics actually see
Includes every rotating part (rotor, hub, screws). Correct the measured period for amplitude first (F6.09).
Symbols and checks
- ✓ I from the same swing, k / (wd^2 + sigma^2)expected 0.0058208 · got 0.00582245 · relative error 2.8×10⁻⁴5 deg amplitude lengthens the period by ~0.05 %, so ~0.1 % on I
Error budget for the measured inertia
From I = m_read g L_s T²/(4π²): each input enters once, the period squared. Independent, small relative errors (first-order propagation, added in quadrature). Use the worst-case sum if errors may be correlated.
Symbols and checks
- ✓ quadrature budget vs Monte Carlo spread of Iexpected 0.0032209 · got 0.00322052 · relative error 1.2×10⁻⁴400000 random draws of scale, length, period and g
Local gravity (normal gravity on the ellipsoid, plus height)
Normal gravity is a smooth model of the whole Earth. Real gravity at a site departs from it by local anomalies, quoted in milligals (1 mGal = 10⁻⁵ m/s², about 10⁻⁶ of g). Where the anomaly is around 100 mGal (10⁻⁴ of g), it is several times smaller than the error of assuming 9.81 m/s² at most mid-latitudes, though near 49° latitude 9.81 happens to be right.
Symbols and checks
Symbol Meaning Unit geodetic latitude rad height above the ellipsoid m equatorial normal gravity 9.7803253359 m/s² normal gravity constant 0.00193185265241 dimensionless first eccentricity squared 0.00669437999013 dimensionless - ✓ g_e derived vs 9.7803253359expected 9.78033 · got 9.78033 · relative error 4.2×10⁻¹³
- ✓ k_s derived vs 0.00193185265241expected 0.00193185 · got 0.00193185 · relative error 1.5×10⁻¹²
- ✓ e^2 derived vs 0.00669437999013expected 0.00669438 · got 0.00669438 · relative error 1.7×10⁻¹²
- ✓ Somigliana vs 1980 series, 40 N 100 mexpected 9.80139 · got 9.80139 · relative error 2.1×10⁻⁷
- ✓ normal gravity at 45 deg, sea level, vs standard gravity 9.80665expected 9.80665 · got 9.8062 · relative error 4.6×10⁻⁵standard gravity was based on sea level at 45 deg
How sharply a zero crossing and a peak can be timed
Small swing, θ = A cos ωt, and δθ ≪ A. The angle moves fastest at a zero crossing and not at all at a peak, so time the period between zero crossings. Δt grows as the swing decays (Δt_zero ∝ 1/A).
Symbols and checks
Symbol Meaning Unit time to cross one count at a zero crossing s time spent in the top count at a peak s angle resolution (one encoder count) rad swing amplitude rad - ✓ time spent in the top count at a peak (s)expected 0.0487567 · got 0.04885 · relative error 0.001912-bit encoder, 5 deg swing, MuJoCo RK4
- ✓ time to cross one count at a zero crossing (s)expected 0.00228547 · got 0.00228 · relative error 0.0024
Period uncertainty from timing many cycles
Independent crossing errors and a good clock. With damping the last crossing is the less certain (F9.05), so for a period taken from the first and last crossings, timing more cycles stops helping: with decay rate σ the error is smallest near N = x/(σT), where x solves x = 1 + e^(−2x) (x ≈ 1.11). A least-squares fit to every crossing has no such optimum.
Symbols and checks
Symbol Meaning Unit uncertainty of the first crossing time s uncertainty of the last crossing time s whole cycles between them dimensionless - ✓ best number of cycles: x = 1 + exp(-2x) vs brute forceexpected 15.8008 · got 15.801 · relative error 1.5×10⁻⁵with the M8 viscous example, b = 1e-3
When the bench and the model agree
Independent uncertainties, roughly normal. With coverage factor 2 this is the E_n ≤ 1 criterion of ISO 13528, written with standard uncertainties. A larger gap means something is unaccounted for, in the model or in the bench's budget, or, about one time in twenty, chance. A smaller gap means the test cannot tell.
Symbols
Symbol Meaning Unit standard uncertainty of each value units of x Standard gravity is a defined constant, not your gravity
Adopted in 1901, based on sea level at 45° latitude. MuJoCo's default 9.81 is close to it; neither is the gravity on your bench (F9.04).
Symbols and checks
- ✓ normal gravity at 45 deg, sea level, vs standard gravity 9.80665expected 9.80665 · got 9.8062 · relative error 4.6×10⁻⁵standard gravity was based on sea level at 45 deg
Worked example: closing the gap between code and bench
The measurement that answers the question the energy match could not. Predictions come from the model. The instruments are illustrative, not a recommendation: a kitchen scale reading to 0.1 g, a ruler read to 0.5 mm, a 12-bit encoder (4096 counts per turn), and a 5° swing timed over 16 cycles, on a bench at 40° N and 100 m above the ellipsoid. The hinge has module 08's viscous damping, N·m·s/rad, so the swing decays while it is timed, and dry friction small enough to ignore. (With module 08's dry-friction example as well, a 5° swing would stop after 30 half-swings, before the timing below was done, and friction would shift each crossing.) Every number is computed by tools/robotics/verify.py.
Static balance, for k
Hold the pendulum horizontal on a free bearing, off the motor so that cogging cannot carry any of the load, with the scale under the bob's centre, 168 mm from the pivot (F9.01). A scale calibrated on the bench with a known mass reads 208.9 g whatever the local gravity, because it was calibrated in that same gravity; multiply by local to get the force. Pivot friction can carry part of the moment either way: a friction torque the size of module 08's dry-friction example, 5×10⁻⁴ N·m, would be 0.15 % of , more than the scale, period and gravity terms below put together (0.12 %). Average a reading taken after lowering the pendulum onto the scale with one taken after pressing it down and letting it come back; friction acts in opposite directions in the two, so the mean cancels most of it and half the difference estimates it.[2][9][11]
Time the swing
Release from 5° and time 16 full cycles between same-direction zero crossings, about 13.1 s. Each crossing time is uncertain by the time the angle takes to cross one count: 2.3 ms at the start, but 7.0 ms at the end, because by then the swing has decayed to 1.6°. The period is good to 0.057 % (F9.06). Why 16 cycles? Taking the period from the first and last crossings, about 16 is the best number with this damping: time more and the blurred last crossing costs more than the extra cycles gain. A least-squares fit to every crossing keeps improving with more cycles, at the price of more code. Correct for the amplitude during the timed cycles with F6.09, and for damping with (F9.02).
Inertia from k and the period
If the hardware matches the model, this returns exactly the model's 0.0058208 kg·m²: the bench's lower gravity lowers and lengthens by amounts that cancel. Mass the model missed changes both and ; a part centred on the axis, such as a motor's rotor, changes only .
The error budget
Each instrument's resolution is used as its uncertainty: generous for a reading that is only rounded (the GUM would use the resolution divided by ), realistic for a ruler read by eye. Pivot friction is assumed averaged out (step 02). The ruler dominates. Measuring the lever arm with calipers to 0.1 mm brings the total down to 0.14 %, and then the period is the largest term. A brute-force Monte Carlo over the same inputs agrees with the quadrature total (the check under F9.03).[4][5][15]
Relative error on I, term by term (F9.03) Term Instrument Error on I Scale 0.1 g on 208.9 g 0.048 % Lever arm 0.5 mm on 168 mm 0.30 % Period (counts twice) 16 decaying cycles, 12-bit encoder 0.11 % Gravity a 100 mGal anomaly, assumed 0.01 % Total, in quadrature 0.32 % Decide
With about 0.32 % standard uncertainty on the bench value, the test can resolve a disagreement of about 0.64 % or more (F9.07), and the model's own uncertainty, from its weighed masses and measured lengths, widens that. A larger gap is a finding: check the bench budget first (pivot friction, amplitude, gravity), then weigh the parts again, look for what turns with the pendulum, and put a rotor's inertia into MuJoCo's
armature.[4][17][18]
The energy match verified the code. The scale, the ruler and the encoder can validate the model, to within the 0.64 % this bench can resolve (more once the model's own uncertainty is added), and only for the job of predicting this pendulum's swing. Until the measurement is made, that is a plan, not a result.
Best practice
Write the prediction, with its uncertainty, before you measure.
A number written down first cannot drift towards the measurement afterwards. It also forces the error budget early, while changing instruments is still cheap.
Measure k statically, off the motor.
Cogging is a detent torque that acts with no current. On the motor, it can carry part of the static load and corrupt the reading.[9]
Calibrate the scale where you use it.
A scale senses force and reports mass using the gravity it was calibrated in. Calibrate with a known mass on your bench and the reading no longer depends on ; multiply by your local to get the force.[11]
Use local gravity, in the model and in the arithmetic.
MuJoCo's 9.81 is 0.088 % high at this bench. Set
gravityin the model'soptionto your local value, so that simulated and measured periods are comparable.[8][18]Time between zero crossings, over many cycles, but not too many.
Crossings are 21.3 times sharper than peaks here (F9.05). Using only the first and last crossings, the period error falls with the number of cycles until the decaying swing blurs the last one; with this damping, about 16 cycles is best (F9.06). A least-squares fit to every crossing has no such limit.[6]
Spend effort on the largest term in the budget.
Here the ruler contributes 0.30 %, and the other three together 0.12 %. A better scale would change nothing; calipers would, and after them, better timing.[4]
Average out pivot friction in the static balance.
Take one reading after lowering the pendulum onto the scale and one after pressing it down and letting it come back. Friction acts in opposite directions in the two, so their mean cancels it, and half their difference tells you how much there was. This assumes the bearing really slips both ways and its friction is the same in both directions; if the half-difference is not small, carry part of it in the budget.[2]
Keep the raw data, with units and the code version.
Log time, angle and instrument settings as CSV with units in the header, plus the commit of the code that produced it. A result you cannot regenerate is an anecdote.
Questions
- My code matches MuJoCo to 10⁻¹⁶. Is my model right?
- It is consistent. Both programs were fed the same masses and lengths, so the match cannot see a mistake in them. Rightness needs a measurement of the real thing.[1]
- Why not just weigh and measure the parts?
- Do that too; the model needs it. But the swing depends only on and , and measuring those directly catches everything a parts list misses: glue, screws, cable, a rotor.
- What if I cannot take the pendulum off the motor?
- Take the static reading at several rotor positions and compare them. If they disagree, cogging is in the reading, and none of them is .[9]
- Why does my bench period differ from MuJoCo's by a few hundredths of a percent?
- Check the amplitude first: a 5° release alone adds 0.048 %, so compare runs at the same amplitude. After that, probably gravity: at this bench, local makes the real period 0.044 % longer than MuJoCo's at 9.81. Set the model's gravity to your local value before comparing.[8][18]
- How much does the amplitude matter?
- At 5° the period is 0.048 % long, which is 0.095 % on : comparable to the period term in the budget. Correct it with F6.09, or release from a smaller angle if the encoder can still time the crossings well.[19]
- How do I get the rotor's inertia?
- Time the swing twice with the same pendulum: once on a free bearing, once on the motor shaft with the windings open. The gravity stiffness is the same, so the difference in is the rotor plus any coupling, provided cogging is small: on the motor it adds a position-dependent stiffness of its own, which this calculation would fold into . Check the static readings at several rotor positions first. The difference goes in the joint's
armature(direct drive, so no gear ratio to square).[18] - Is normal gravity my gravity?
- Not exactly. Normal gravity is a smooth model of the whole Earth, from its shape and spin (F9.04). Real gravity departs from it by local anomalies, quoted in milligals (10⁻⁵ m/s², about 10⁻⁶ of ). The budget assumes an anomaly of 100 mGal, which is an assumption rather than a measurement; at this bench it is 8.8 times smaller than the error of assuming 9.81.[8][15]
Watch
- listed
The Character of Physical Law: Seeking New Laws (opens in a new tab)[20]
Richard P. Feynman · Messenger Lectures, Cornell University, 1964 (BBC film); streamed by Caltech
How a guessed law is tested: compute what it predicts, then compare with experiment. The spirit of this whole module, from 1964.
- listed
Lecture 20: Linear System Modeling a Single Degree of Freedom Oscillator (opens in a new tab)[21]
J. Kim Vandiver · MIT OpenCourseWare, 2.003SC Engineering Dynamics, Fall 2011
Damped free response, including how to determine the damping ratio by experiment.
- listed
Modern Robotics, Chapter 8.9: Actuation, Gearing, and Friction (opens in a new tab)[22]
Kevin M. Lynch · Northwestern University, Modern Robotics video series (YouTube)
Apparent inertia through gearing, which is what the
armaturemeasurement captures.
Read
- read
Verification and validation in computational fluid dynamics (opens in a new tab)[1]
William L. Oberkampf, Timothy G. Trucano · Progress in Aerospace Sciences 38(3), 209–272 (2002). Sandia report SAND2002-0529, doi:10.2172/793406
The definitions, and a long argument for why validation needs experiments designed for it. Sections 1 and 2 are enough.
- listed
JCGM 100:2008, Evaluation of measurement data — Guide to the expression of uncertainty in measurement (GUM) (opens in a new tab)[4]
Joint Committee for Guides in Metrology / BIPM. doi:10.59161/JCGM100-2008E
The GUM, the international guide to stating measurement uncertainty. Free from BIPM.
- listed
An Introduction to Error Analysis, 2nd edition (opens in a new tab)[5]
John R. Taylor · University Science Books, 1997. ISBN 978-0-935702-42-2
A gentle introduction to propagating errors through a calculation.
- read
World Geodetic System (1984) (opens in a new tab)[8]
AHRS (Attitude and Heading Reference Systems) documentation, version 0.4.0
Normal gravity, with the constants used in F9.04.
Sources
Numbered in order of first use on this page. read: checked against the claim. listed: exists and covers the topic, wording not checked. derived: worked out here and checked numerically.
- [1]read
Verification and validation in computational fluid dynamics (opens in a new tab)
William L. Oberkampf, Timothy G. Trucano · Progress in Aerospace Sciences 38(3), 209–272 (2002). Sandia report SAND2002-0529, doi:10.2172/793406
§2.1 (AIAA definitions of verification and validation); §1.1 and §2.3 (verification as a mathematics issue, after Roache)
Verification: the implementation accurately represents the conceptual model and its solution. Validation: the degree to which the model represents the real world for its intended use. 'Verification does not address whether the conceptual model has any relationship to the real world.'
- [2]read
University Physics Volume 1, §12.1 Conditions for Static Equilibrium (opens in a new tab)
William Moebs, Samuel J. Ling, Jeff Sanny · OpenStax, 2016. CC BY 4.0
Eq. 12.5 (net torque zero); weight acts at the centre of mass in a uniform field
The static moment balance used to measure m·g·d.
- [3]listed
Engineering Vibration, 4th edition (opens in a new tab)
Daniel J. Inman · Pearson, 2014. ISBN 978-0-273-78521-7 (international edition)
Standard reference for viscous damping, logarithmic decrement and Coulomb damping. Listed for further reading; every formula that cites it also cites a source checked on this page.
- [4]listed
Joint Committee for Guides in Metrology / BIPM. doi:10.59161/JCGM100-2008E
Combining independent uncertainties in quadrature (law of propagation of uncertainty); expanded uncertainty with a coverage factor of 2 for a level of confidence of about 95 %.
- [5]listed
An Introduction to Error Analysis, 2nd edition (opens in a new tab)
John R. Taylor · University Science Books, 1997. ISBN 978-0-935702-42-2
Propagation of small relative errors for products and powers (the factor 2 on the period).
- [6]derived
Time spent within one angle step of a zero crossing and of a peak
F9.05, checked against a MuJoCo RK4 swing.
- [7]read
Standard gravity (opens in a new tab)
Wikipedia
Standard gravity is the defined constant 9.80665 m/s², set by the 3rd CGPM in 1901 and originally based on free fall at sea level at 45° latitude; actual gravity varies with place.
- [8]read
World Geodetic System (1984) (opens in a new tab)
AHRS (Attitude and Heading Reference Systems) documentation, version 0.4.0
Somigliana closed form for normal gravity; height correction. Constants re-derived here from the four WGS84 defining parameters.
- [9]read
Cogging torque (opens in a new tab)
Wikipedia
Cogging is the interaction between the rotor magnets and the stator slots; it is a detent (no-current) torque, especially prominent at low speed.
- [10]read
University Physics Volume 2, §13.6 Electric Generators and Back Emf (opens in a new tab)
Samuel J. Ling, Jeff Sanny, William Moebs · OpenStax, 2016. CC BY 4.0
Back emf and Fig. 13.32
A turning motor generates a back emf that is zero at rest and rises in proportion to the motor's angular velocity.
- [11]read
The Difference Between Force and Mass (opens in a new tab)
Morehouse Instrument Company, 7 January 2025
The force a 1 kg mass exerts varies by up to 0.539 % worldwide with local gravity; an instrument calibrated with mass weights and no gravity correction is calibrated for mass, not force.
- [12]read
University Physics Volume 1, §15.5 Damped Oscillations (opens in a new tab)
William Moebs, Samuel J. Ling, Jeff Sanny · OpenStax, 2016. CC BY 4.0
Eq. 15.24 (underdamped solution, exponential envelope), Eq. 15.26 (damped frequency); under-, critically and over-damped conditions
Exponential envelope e^(−bt/2m), damped frequency, damping regimes. (Translational form; the rotational form swaps m→I, k→mgd.)
- [13]read
International Gravity Formula (opens in a new tab)
AHRS documentation
1980 series coefficients; cites Moritz (1980) and Hinze et al. (2013).
- [14]listed
National Imagery and Mapping Agency, 1997
The WGS84 normal gravity model (cited by MathWorks' gravitywgs84; the URL is that citing page).
- [15]read
Gravity anomaly (opens in a new tab)
Wikipedia
Normal gravity accounts for the whole Earth, corrected for its idealised shape and rotation; anomalies are measured in milligals; the free-air correction is 0.3086 mGal per metre of height.
- [16]read
Free-air correction (opens in a new tab)
SLB Energy Glossary
The free-air correction is 0.3086 mGal/m (0.09406 mGal/ft) of height.
- [17]read
ISO 13528 En Score (Dataplot Reference Manual) (opens in a new tab)
NIST Information Technology Laboratory
Cites ISO 13528:2005, pp. 27–28
The En score divides a difference by the root-sum-square of the two expanded uncertainties; with coverage factor 2, a critical value of 1 matches a z-score of 2, a warning signal.
- [18]read
MuJoCo XML Reference (3.17.0) (opens in a new tab)
Google DeepMind
option-timestep (default 0.002 s; RK4 costs about four times per step); option-gravity (default 0 0 −9.81); option-integrator; option-flag-energy; option-flag-eulerdamp (implicit joint damping in Euler, on by default); body-joint-armature (reflected inertia, gear ratio squared; positive armature improves stability); actuator-general-armature (rotor inertia, scaled by gear squared); body-joint-damping (Euler treats joint damping implicitly); body-joint-frictionloss; compiler-inertiafromgeom; body-geom-density (default 1000 kg/m³, used only when mass is not given)
MuJoCo defaults, energy definitions, armature, damping and friction-loss behaviour. Read from the 3.17.0 source of the docs (doc/XMLreference.rst).
- [19]read
The AGM Simple Pendulum (opens in a new tab)
Mark B. Villarino · arXiv:1202.2782v2 [math.CA], 2014
exact period T = 4√(l/g)·K(k), k = sin(α/2); series 1 + α²/16 + 11α⁴/3072 + 173α⁶/737280 + …
Large-amplitude period, exact and as a series. Applies to the compound pendulum with l replaced by the equivalent length ℓ = I/(md).
- [20]listed
The Character of Physical Law: Seeking New Laws (opens in a new tab)
Richard P. Feynman · Messenger Lectures, Cornell University, 1964 (BBC film); streamed by Caltech
The series' closing lecture in the book's order. The Caltech page lists the lectures in its own order.
The lecture on how a guessed law is tested by computing its consequences and comparing them with experiment.
- [21]listed
Lecture 20: Linear System Modeling a Single Degree of Freedom Oscillator (opens in a new tab)
J. Kim Vandiver · MIT OpenCourseWare, 2.003SC Engineering Dynamics, Fall 2011
Lecture video on damped free response and on determining the damping ratio by experiment (topics confirmed from the lecture page's description).
- [22]listed
Modern Robotics, Chapter 8.9: Actuation, Gearing, and Friction (opens in a new tab)
Kevin M. Lynch · Northwestern University, Modern Robotics video series (YouTube)
The book’s own video for §8.9: motors, gearing, apparent (reflected) inertia and friction models. Linked from the book site’s chapter 8 video list.