Science in practice · Validation & tuning

How we earn the numbers.

A measurement you can't trust is worse than none at all. So before any figure reaches you, it has to survive a deliberate process — one that keeps building, tuning and proving strictly apart, sizes its datasets with real statistics, and only signs off a number once it agrees with an independent reference.

In development & validation

This is the method, not a finished report. The datasets it describes are still being built, and no accuracy claim has been published — that's the point. Nothing ships until the figures clear the bars set out below. The working notes live on GitHub ↗

The core framework

Three disciplines, never conflated

Most measurement mistakes come from blurring three different questions. We keep them strictly apart, because answering one with another's data is how you fool yourself.

  • 1

    Verification — did we build the maths right?

    Bit-for-bit golden tests, determinism checks and closed-form synthetic cases prove the code is logically correct before it ever touches a real swing.

  • 2

    Tuning — which settings work best?

    Parameters like filter gains and detection thresholds are optimised only on a dedicated tuning slice of the data, kept walled off from everything else.

  • 3

    Validation — did we build the right measurement?

    The cardinal sin is reporting accuracy on the data you tuned on. We measure agreement only against independent reference data the tuning never saw.

VERIFICATION Maths & code, golden tests TUNING Parameter optimisation VALIDATION — held-out data only Agreement with an independent reference

Illustrative — tune and verify freely; validate only on untouched data.

Measurement validity

A ladder of validity

A number on its own means nothing. We build trust by climbing the classic measurement-science ladder, one rung at a time.

  • 1

    Face & content

    The metrics sit inside published anatomical ranges (roughly ≤54° flexion, ≤60° extension) and cover every degree of freedom the wrist actually has, per the recognised biomechanics standard.

  • 2

    Construct & concurrent

    Deliberately scripted faults must separate cleanly in the data, and the readings must agree with an independent device measuring the same swing at the same time.

  • 3

    External & predictive

    The measurements have to hold up across different bodies and swings — and, ultimately, help predict what the ball actually does.

EXTERNAL & PREDICTIVE CONCURRENT CONSTRUCT FACE & CONTENT

Illustrative — each rung has to hold before the next is claimed.

Statistical power

How many swings is enough?

Dataset sizes aren't guessed — they're set by the statistic being computed, so each claim comes with a known margin rather than a hopeful one.

  • 1

    Pass-rates & fault recall

    For a yes/no rate, binomial power sets the size: n = z²·p(1−p) / E². A ±5% margin needs about 138 swings; to claim a fault is caught ≥95% of the time takes roughly 60 clean trials of that fault.

  • 2

    Agreement limits

    The precision of a Bland–Altman agreement band tightens with sample size; around 50 paired swings pins its limits down tightly enough to trust.

  • 3

    Repeatability floor

    Pinning down a metric's typical error needs roughly 25–30 repeated swings before the error estimate itself is stable.

n = 15 — wide margin n = 30 — repeatability floor n = 50 — tight agreement band

Illustrative — the margin narrows as the sample grows.

Concurrent agreement

Held up against an independent reference

To prove concurrent validity we capture the same swing with an established, independent wrist-measurement device and compare them axis by axis — using statistics that don't flatter.

  • 1

    Agreement, not just correlation

    We report the mean bias and the 95% limits of agreement, plotting the difference against the average to expose any drift that grows with the size of the angle.

  • 2

    A coefficient that punishes offsets

    A plain correlation can look superb even with a constant offset. An absolute-agreement statistic instead penalises any systematic shift, so a steady few-degree bias can't hide.

  • 3

    Thresholds declared in advance

    The pass/fail bars are set before we look: flexion and extension target RMSE ≤ 4–5° with bias inside ±2°; the noisier radial/ulnar hinge targets RMSE ≤ 8°.

Difference vs average +1.96 SD mean bias −1.96 SD Bias small, scatter inside the limits = agreement

Illustrative — a Bland–Altman view of two methods on one swing.

Calibration

Lining the sensors up with the bones

Sensors strap to soft tissue, not bone. A few degrees of mounting error quietly leaks one axis into another — so we measure that error and correct it out.

  • !

    Raw sensor → true joint angle

    Each reading is transformed from the sensor's own frame into the limb's anatomical frame. Get the mount 5° out and flexion bleeds straight into the radial/ulnar hinge reading.

  • A one-second calibration pose

    A brief, still address pose lets the system measure the mounting offset against gravity and freeze the correction before the swing begins.

  • Measured, not inferred

    We read forearm roll directly from a forearm-plus-hand sensor pair rather than inferring it from a single sensor — so some disagreements with other devices are real anatomy, not error.

X (bone) Y (bone) X (sensor) Y (sensor) Calibration maps the sensor's axes onto the bone's

Illustrative — the mounting offset between sensor and bone.

Signal integrity

When the swing overwhelms the sensors

Orientation filters assume the accelerometer only ever feels gravity. A golf swing breaks that assumption violently — so the maths has to adapt rather than believe it.

  • 1

    Trust gravity less under load

    As swing forces grow, the accelerometer stops being a reliable "down". The filter detects this and leans on the gyroscope until things settle.

  • 2

    Blank through impact

    At the strike the sensor slams into its ±16g limit and clips. Corrections are frozen across a short window around impact so that shock never poisons the orientation.

  • 3

    Re-anchor the yaw

    Gravity can pin down tilt but not rotation in the horizontal plane, which slowly drifts. That yaw is re-aligned during the still calibration pose.

trust swing dynamics → Address — full trust Transition — down-weighted Impact — frozen

Illustrative — how far the accelerometer is trusted through a swing.

Honest statistics

Why more swings isn't more data

Fifty swings from one golfer aren't fifty independent data points — they're deeply correlated. Treating them as independent would badly overstate how much we actually know.

  • !

    Repeats inflate confidence

    Because one golfer's swings resemble each other, 50 of them can carry the statistical weight of only about two independent swings. Ignore that and the error bars look far tighter than they are.

  • !

    Within-subject ≠ population

    Early datasets prove the kit is consistent for one golfer. That's necessary, but it can't speak for everyone — a different claim needs different data.

  • A multi-golfer floor

    Only by measuring 8–12 different golfers do we capture real differences in anatomy and sensor placement — the bar before any generalisable accuracy claim.

Golfer A Golfer B Effective sample ≈ 2, not 50

Illustrative — tight clusters per golfer, few truly independent points.

Tuning governance

Tuning without fooling ourselves

With dozens of tunable parameters and limited data, overfitting is the constant danger. The process is built specifically to stop it.

  • 1

    A three-way split

    Parameters are searched on the tuning set, chosen on a separate validation set, and judged exactly once on a held-out set that stays locked until the code is frozen.

  • 2

    The right optimiser for the surface

    Independent knobs, tangled interactions and continuous gain tables each call for a different search method — matched to the maths rather than forced into one.

  • 3

    No regressions, full stop

    At freeze the gate is hard: a new default must lift the average without breaking a single swing that already worked.

Held-out test set locked until the code is frozen

Illustrative — the final test set stays sealed until freeze.

Science-led, kept open

Trust is earned, not claimed.

Every method here is drawn from the published literature, kept open to inspect, and held against real swings before it ships. PinPoint Studio is a prototype under validation, with a first release targeted for late 2026 — once, and only once, the figures hold up.