Model comparison

Is the deep model worth it?

Same 15 videos, same cropped face, same scoring — the only difference is the algorithm: PHASE-Net, the deep model; the reference POS implementation, the standard classical baseline; and a simplified POS written from scratch here.

PHASE-Net
2.96BPM off
0.39against the reference waveform

Average gap from the oximeter — the fingertip clip's own reading — on the 6 people held out, meaning never used in training.

POS reference
2.87BPM off
1.01against the reference waveform

Fixed maths, no training. Level with the deep model on the same people.

POS written here
3.74BPM off
2.01against the reference waveform

Same projection maths as above, one detail missing. About 30 per cent worse.

Worst person, all 15
2.2vs22.4BPM

Largest error each made on any single person. This, not the average, is the real difference.

Head to head

Average error in beats per minute on the 6 held-out people. Lower is better.

10 s windows · 1 s step

Left is the score papers publish — each method against the pulse trace the finger sensor recorded — and the deep model wins by 2.6×. Right is the same recordings against the heart rate the device reports itself, untouched by our code, and the three tie. PHASE-Net was trained to reproduce that pulse trace, so the left group marks it on its own homework.

Worst case, and why it is the headline

Largest error any single person produced, across all 15.

An average hides the failures that matter: a monitor that is usually right and occasionally 22 BPM wrong is not usable. Caveat — nine of these fifteen sit inside PHASE-Net's training data; on the held-out six alone it is 0.62 against 3.30 BPM, smaller, same direction.

Where the averages come from

Error against the reference pulse trace, one group per person, held-out people first.

faded = trained on

Two people set the whole scale: on subjects 25 and 27 both classical methods lose the pulse entirely. On everyone else all three sit under 4 BPM.

Every person, every method

Median heart rate and both errors, in beats per minute. Bars share one scale, so any two can be compared by eye.

SubjectSplitDevice HRPHASE-Net HRvs refvs devicePOS reference HRvs refvs devicePOS written here HRvs refvs device
subject 44held out8280.1
0.62
5.41
78.2
3.30
4.99
78.3
8.37
10.08
subject 46held out9794.3
0.35
1.66
93.8
0.72
1.29
94.7
1.45
1.34
subject 47held out112111.2
0.46
4.82
109.4
0.72
4.97
110.4
1.10
5.16
subject 48held out8891.5
0.49
3.69
91.1
0.40
3.61
91.1
0.39
3.60
subject 49held out8787.1
0.16
1.27
87.2
0.63
1.61
86.9
0.45
1.43
subject 5held out9999.9
0.26
0.91
99.6
0.29
0.77
99.7
0.27
0.81
subject 14trained on7775.3
2.16
8.64
74.8
2.99
8.54
74.8
3.04
9.50
subject 15trained on116117.3
0.14
1.10
113.8
1.72
2.35
113.8
2.92
3.55
subject 18trained on122123.9
0.23
1.87
123.9
0.43
1.86
124.1
0.43
1.85
subject 24trained on97110.4
0.36
7.31
109.5
3.27
7.50
104.4
8.50
9.10
subject 25trained on92113.8
0.12
23.93
114.0
1.55
22.85
113.3
10.83
20.13
subject 27trained on89111.6
1.48
19.28
92.1
22.41
20.72
78.4
27.23
16.34
subject 30trained on97100.1
0.32
5.58
100.1
0.54
5.66
100.2
2.84
4.39
subject 32trained on107110.4
0.51
3.46
110.3
3.54
5.97
110.5
7.09
9.36
subject 35trained on106109.3
0.28
2.33
109.8
0.61
2.51
109.8
0.42
2.38

On subjects 25 and 27 all three methods differ from the device by about 20 BPM in the same direction, and two independent readings of the device's own waveform side with the methods. A broken reference, not three simultaneous failures — and both are trained-on, so the held-out figures are untouched.

Why the usual score flatters the deep model

PHASE-Net was trained to reproduce the reference pulse trace, so scoring it against that same trace asks how well it hit its own training target.

POS was never trained on anything — it is fixed maths from the optics of light through skin. Hence a 2.6× gap on the left chart and none against the device.

Our POS is 30 per cent worse, and we know why

Both implementations compute the identical projection. The reference applies it inside a 1.6 second sliding window; ours fits the whole minute at once.

Lighting and skin appearance drift over a minute and one global fit cannot track that. The baseline is not weak because the maths is simple.

10_compare_models.py, cached signals for all 15 subjects·6 held out·generated Sat, 29 Aug 2026 20:23:15 GMT