Does it agree with a medical device?
The camera's heart rate was compared with a fingertip pulse oximeter on 15 recordings, and only 6 of them count: the released model was trained on the first 72 per cent of the subject list, so on the other 9 it would be measuring memory rather than skill.
6 unseen people, ten-second windows. The oximeter's recorded pulse, put through the same processing as the camera signal.
The number the device itself displays, which our code cannot influence — the stricter of the two. Across all 15 it is 6.08, inflated by two recordings where the device's own readout is wrong by over 20 BPM.
Whether the two rise and fall together across subjects. Blind to a constant offset, which the plot below is not.
73 % land within 3 BPM. One reading per recording, all subjects.
Agreement plot
Bland-Altman: the standard way clinicians check whether a new measurement agrees with an established one. All 15 recordings, 9 of which the model has seen.
Each dot is one person. The solid line is the average gap between camera and reference — -0.59 BPM, so the camera does not systematically read high or low. Nineteen readings in twenty should fall between the dashed lines at -7.9 and 6.8 BPM, and that width, not the average, is what a monitoring claim would have to live with.
Camera against reference
One reading per recording
Dots on the dashed line are exact; above it the camera read fast, below it slow. The spread along the line is the range of resting heart rates in the group, not error.
Who it fails on
Worst first, whole recording
10 of 15 sit at zero, drawn as a hairline. That is a caveat, not applause: the frequency analysis resolves about 0.88 BPM, so zero means the same bin. The worst unseen person is subject47 at 8.8 BPM — and 0.46 once the same recording is scored in ten-second windows.
Can it tell when it is struggling?
Signal quality against error, one dot per person
Signal quality is how far the pulse stands above the noise, and it can be measured live with no reference to compare against. If a low score reliably meant a large error, the model could refuse to answer — a monitoring product has to be able to say “I do not know” rather than guess. It cannot yet: everything above 10 dB is exact, but below that an exact reading and an 8 BPM miss sit side by side.
Every recording
All 15, unseen subjects first. Nothing averaged away.
| Person | Split | Reference | Camera | Off by | Signal | Windowed |
|---|---|---|---|---|---|---|
| subject5 | held out | 101.1 | 101.1 | 0.00 | 16.5 | 0.26 |
| subject44 | held out | 87.9 | 87.9 | 0.00 | -1.4 | 0.62 |
| subject46 | held out | 91.4 | 91.4 | 0.00 | 1.6 | 0.35 |
| subject47 | held out | 105.5 | 114.3 | 8.79 | 0.3 | 0.46 |
| subject48 | held out | 91.4 | 91.4 | 0.00 | 10.6 | 0.49 |
| subject49 | held out | 86.1 | 86.1 | 0.00 | 14.6 | 0.16 |
| subject14 | trained on | 70.3 | 70.3 | 0.00 | 1.5 | 2.16 |
| subject15 | trained on | 119.5 | 112.5 | 7.03 | -2.9 | 0.14 |
| subject18 | trained on | 128.3 | 124.8 | 3.52 | 3.8 | 0.23 |
| subject24 | trained on | 94.9 | 87.0 | 7.91 | -3.2 | 0.36 |
| subject25 | trained on | 113.4 | 113.4 | 0.00 | 12.6 | 0.12 |
| subject27 | trained on | 111.6 | 111.6 | 0.00 | 4.0 | 1.48 |
| subject30 | trained on | 102.8 | 102.8 | 0.00 | 5.9 | 0.32 |
| subject32 | trained on | 116.9 | 116.9 | 0.00 | 0.4 | 0.51 |
| subject35 | trained on | 116.0 | 116.9 | 0.88 | -1.6 | 0.28 |
Reference, camera and off-by are in BPM; signal is in dB. The last column re-scores the same recording in ten-second windows — heart rate moves within a minute, so one number per video averages away the thing being measured.
How the video was processed
Every step between the raw frames and the numbers above
- face detection
- Haar cascade (HC), first frame only
- large box coef
- 1.5
- resize
- 128 × 128
- interpolation
- INTER_AREA
- pixel range
- 0-255 (no /255 normalisation, matching the toolbox)
- clip length
- 128
- data type
- Raw
- method
- repo calculate_metric_per_video
- steps
- detrend (lambda=100), Butterworth bandpass 0.75-2.5 Hz (45-150 BPM), FFT peak
- diff flag
- 0
One line there decides whether any of this works: pixel values stay in the 0–255 range. The reference implementation never divides by 255, so normalising raises no error and silently returns a plausible-looking waveform that is noise.