Validation and Benchmarking

Colab Notebook

In this tutorial we will learn how to validate and benchmark the fiducial point extraction of photoplethysmogram (PPG) signal.

Our objectives are to:

  • Compare the fiducial points annotations of two manual annotations

  • Validate the fiducial points detection of pyPPG

  • Benchmark the fiducial points detection of pyPPG, PulseAnalyse, PPGFeat

  • Make Bland-Altman analysis for the annotations and detection results

  • Saving all results

Download and extract the manual annotations of fiducial points by accessing the provided link: PPG-BP manual annotations.

Additionally, acquire and extract the MATLAB codes essential for evaluating PPGFeat and PulseAnalyse by using the following link: MATLAB codes.

Ensure that you place the extracted contents in the designated folder associated with this script.

Import Python packages:

  • Install the pyPPG toolbox for PPG analysis

pip install pyPPG==1.0.67

  • Import required components from pyPPG

import pyPPG.validation.pw_anal as PW

  • Import other packages

import pandas as pd
from datetime import datetime
import os

Initialisation:

  • Initialise the following input parameters to the validation and benchmarking

# Initialise the pulse wave package
pwex = PW.PulseWaveAnal()

# Initialise the correction
correction = pd.DataFrame()
corr_on = ['on', 'v', 'w', 'f']
corr_off = ['dn', 'dp']
correction.loc[0, corr_on] = True
correction.loc[0, corr_off] = False

# Save time for the results of the output folders
date = datetime.now()
dname = str(date.year) + '_' + str(date.month) + '_' + str(date.day) + '_' + str(date.hour) + '_' + str(date.minute)

Validation of the fiducial points:

  • Evaluate the manual annotations and pyPPG detection of the fiducial points

# Run PPG-BP Evaluation for the manual annotation and pyPPG
pwex.eval_PPG_BP(plts=True, correction=correction, dname=dname, prnt=False)

Extract other fiducial points:

  • Extract the detected the fiducial points of PPGFeat and PulseAnalyse

# Command to run MATLAB script for PPGFeat
current_directory = os.getcwd()
script_folder = current_directory+os.sep+'PPGFeat'
scipt='get_PPGFeat_fps'
pwex.run_matlab_script(script_folder,scipt,dname,'')

# Command to run MATLAB script for PulseAnalyse
script_folder = current_directory+os.sep+'PulseAnalyse'
scipt='get_PA_fps'
pwex.run_matlab_script(script_folder,scipt,dname,'')

Benchmarking:

  • Compare the results of PPGFeat and PulseAnalyse with pyPPG

# Run Benchmarking
pwex.benchmark_PPG_BP(detector='PPGFeat', dname=dname, plt=True, prnt=False)
pwex.benchmark_PPG_BP(detector='PulseAnal', dname=dname, plt=True, prnt=False)

# Run Bland-Altman analysis
script_folder = current_directory+os.sep+'BlandAltman'
scipt='BlandAltman_anal'
pwex.run_matlab_script(script_folder, scipt, dname, 'MG_PC')
pwex.run_matlab_script(script_folder, scipt, dname, 'pyPPG')
pwex.run_matlab_script(script_folder, scipt, dname, 'PPGFeat')
pwex.run_matlab_script(script_folder, scipt, dname, 'PulseAnal')

The resulting figures and outcomes are stored within the results folder, which is automatically generated within the project directory.