Arbab Latif

Contactless Vitals: Pulse Extraction with rPPG

A Python package that extracts a pulse waveform from ordinary face video using remote photoplethysmography (rPPG). It tracks the face, measures the colour of three skin regions, and separates the pulse from motion and lighting changes. It stops at the waveform and does not estimate heart rate yet.

View the code on GitHubFull results

correlation with a contact sensor on one real recording
0.79
dB SNR for POS under in-band flicker, where GREEN gives -15.9
18.5
automated tests
139
Waveforms under in-band brightness flicker: GREEN follows the flicker while CHROM and POS follow the true pulse
With brightness flicker inside the pulse band, GREEN follows the flicker while CHROM and POS follow the pulse.

Problem

Each heartbeat changes skin colour by well under 1% of its brightness. A camera can see this, which makes it possible to monitor a pulse without touching anyone. The difficulty is that head movement, flickering lights and shine change the picture far more, and some of them repeat at heart-rate speeds, so a frequency filter alone cannot remove them.

Approach

  • MediaPipe FaceLandmarker follows 478 landmarks through the video.
  • Landmarks define the forehead and both cheeks, and a skin mask uses each region's own median colour, so no skin tone is assumed.
  • Each region becomes one mean RGB value per frame, resampled to even time steps. Long gaps stay empty, so nothing is invented.
  • GREEN, CHROM and POS each turn the RGB trace into a pulse signal, followed by detrending and a zero-phase 0.7 to 4 Hz band-pass.

Results

  • Distortion inside the pulse band is the real test. GREEN falls by about 42 dB, while CHROM and POS stay near their clean scores.
  • With no pulse in the video, every method stays below 0 dB, so there is no false pulse.
  • Quality falls on darker skin in the simulation. This is a real weakness and is reported as one.
  • One real UBFC-rPPG recording exposed timestamp problems that synthetic video could not.

Limitations

  • One real person so far, which is a first check and not a measure of accuracy.
  • Synthetic videos prove the code is correct, not that it works well on people.
  • Processing is offline. Low light, heavy compression and large head turns are untested.
  • This is a research project, not a medical device, and must not be used for diagnosis or monitoring.

Built with

  • Python
  • MediaPipe
  • NumPy
  • SciPy
  • Matplotlib
  • scikit-image
  • pytest

Related projects