First set of facial skin images and decision of methods for preprocessing
Gdańsk University of Technology
- Publisher
- Zenodo
- Language
- en
Abstract
Image Preprocessing Algorithms The preprocessing pipeline (described in Deliverable D2.1) combines several steps designed to ensure the spatial and temporal consistency of facial sequences before their analysis through the STREAM-Net model. First, the RGB images are resized to a fixed resolution of 144 × 144 px, and face cropping (ROI extraction) is applied to remove background regions and focus the analysis on skin areas relevant to blood flow estimation. Then, a temporal normalization between consecutive frames is performed, computed as the difference divided by the sum of consecutive frames, in order to highlight dynamic color variations associated with blood pulsation and reduce the influence of illumination or skin tone. This processing generates two complementary inputs —a spatial and a temporal one— that feed the dual-branch architecture of STREAM-Net. Additionally, a Bayesian uncertainty estimation mechanism using Monte Carlo Dropout is integrated to assess the model’s robustness to stochastic variations and quantify prediction confidence. Finally, to further improve image quality and facilitate more accurate physiological signal extraction, super-resolution and deblurring techniques based on deep learning (e.g., EDSR, LapSRN, SwinIR, and FaceSR) are applied, enhancing both fine-detail recovery and structural fidelity according to PSNR and SSIM metrics. PURE Dataset The PURE (Pulse Rate Detection) dataset provides facial video sequences used for the training and validation of remote pulse estimation algorithms such as those implemented in STREAM-Net. It includes recordings of 10 volunteers (8 male, 2 female) performing six controlled motion conditions —steady, talking, slow translation, fast translation, small rotation, and medium rotation— introducing different levels of motion and illumination variability. Each sequence lasts one minute and was recorded using an eco274CVGE camera (SVS-Vistek GmbH) at 30 fps and 640 × 480 px resolution, while reference pulse signals were simultaneously acquired using a Pulox CMS50E oximeter at 60 Hz. The participants were seated approximately 1.1 m from the camera under natural daylight illumination, with slight variations caused by cloud movement. These recordings capture subtle color changes on the skin surface as well as realistic head movements, providing an ideal dataset for evaluating remote photoplethysmography (rPPG) pipelines and assessing the robustness of preprocessing algorithms against motion artifacts, illumination fluctuations, and facial expressiveness.
Full text
http://www.tu-ilmenau.de Ansprechpartner Univ.-Prof. Dr.-Ing. Horst-Michael Groß Head of department Telefon +49 3677 692858 E-Mail senden Pulse Rate Detection Dataset - PURE Description This data set consists of 10 persons performing different, controlled head motions in front of a camera. During these sentences the image sequences of the head as well as reference pulse measurements were recorded. The 10 persons (8 male, 2 female) that were recorded in 6 different setups resulting in a total number of 60 sequences of 1 minute each. Recording Setup The videos were captured with a eco274CVGE camera by SVS-Vistek GmbH at a frame rate of 30 Hz with a cropped resolution of 640x480 pixels and a 4.8mm lens. Reference data have been captured in parallel using a finger clip pulse oximeter (pulox CMS50E) that delivers pulse rate wave and SpO2 readings with a sampling rate of 60 Hz. The test subjects were placed in front of the camera with an average distance of 1.1 meters. Lighting condition was daylight trough a large window frontal to the face with clouds changing llumination conditions slightly over time. The six different setups were as follows: Steady The subject was sitting still and looks directly into the camera avoiding head motion. Talking Simulated video sequence, where the subjects were asked to talk while avoiding additional head motion. This setup equals a video conference situation in a real robot application. Slow Translation These sequences comprise head movements parallel to the camera plane. Therefore, the images recorded by the camera were displayed on screen and shown to the subjects. A Pulse Rate Detection Dataset - PURE https://www.tu-ilmenau.de/neurob/data-sets-code/pulse/ 1 z!2 18/02/2020, 17:06
moving rectangle of the size of the face was added to the image, and the subjects were asked to keep their face inside. The rectangle was moving horizontally at a controlled speed and with a predefined pattern, thus the sequences of all individuals are repeatable. The average speed was 7% of the face height per second, where the average face height was 100 pixels. Fast Translation This dataset has the same setup as slow translation, except twice the speed of the moving target. Small Rotation This setup comprises different targets that were placed at 35 cm around the camera. The subjects were told to look at these targets in a predefined sequence. They were asked to move not only there eyes but orient their head. See Fig. \ref{fig:setup} for an impression of the setup. The one minute sequence of the targets is shown in the little clock in the figure. Random times ensure that the motion artifacts are not periodically. Depending on the distance between the camera and the subject, that roughly varies between 1 m and 1.3 m, the head rotation angles are round about 20°. Medium Rotation These sequences had the same setup as for small rotation, but with targets placed 70 cm around the camera resulting in average head angle of 35°. The pulse rate of the test persons varies slightly between and during sequences that do have a length of 1 Minute each. Minimum pulse rate measured using the oximeter is at 42 BPM and the maximum rate was 148 BPM. Although, we have had very high pulse rates from one subject, all recording were taken during rest. Citations If you consider using the data sets on this page, please reference the following: Stricker, R., Müller, S., Gross, H.-M. Non-contact Video-based Pulse Rate Measurement on a Mobile Service Robot in: Proc. 23st IEEE Int. Symposium on Robot and Human Interactive Communication (Ro-Man 2014), Edinburgh, Scotland, UK, pp. 1056 - 1062, IEEE 2014 Download Please contact Ronny[email protected] for requests. © 2004-2020 TU Ilmenau Pulse Rate Detection Dataset - PURE https://www.tu-ilmenau.de/neurob/data-sets-code/pulse/ 2 z!2 18/02/2020, 17:06
