Virtual reality and machine learning in the automatic photoparoxysmal response detection

S.I.: COMPUTATIONAL-BASED BIOMARKERS FOR MENTAL AND EMOTIONAL HEALTH(CBMEH2021) Virtual reality and machine learning in the automatic photoparoxysmal response detection Fernando Moncada 1 •Sofı ´a Martı ´n 2 •Vı ´ctor M. Gonza ´lez 1 •Vı ´ctor M. A ´lvarez 2 •Beatriz Garcı ´a-Lo ´pez 3 • Ana Isabel Go ´mez-Mene ´ndez 3 •Jose ´R. Villar 2 Received: 9 November 2021 / Accepted: 4 January 2022 The Author(s) 2022 Abstract Photosensitivity, in relation to epilepsy, is a genetically determined condition in which patients have epileptic seizures of different severity provoked by visual stimuli. It can be diagnosed by detecting epileptiform discharges in their electroencephalogram (EEG), known as photoparoxysmal responses (PPR). The most accepted PPR detection method—a manual method—considered as the standard one, consists in submitting the subject to intermittent photic stimulation (IPS), i.e. a flashing light stimulation at increasing and decreasing flickering frequencies in a hospital room under controlled ambient conditions, while at the same time recording her/his brain response by means of EEG signals. This research focuses on introducing virtual reality (VR) in this context, adding, to the conventional infrastructure a more flexible one that can be programmed and that will allow developing a much wider and richer set of experiments in order to detect neurological illnesses, and to study subjects’ behaviours automatically. The loop includes the subject, the VR device, the EEG infrastructure and a computer to analyse and monitor the EEG signal and, in some cases, provide feedback to the VR. As will be shown, AI modelling will be needed in the automatic detection of PPR, but it would also be used in extending the functionality of this system with more advanced features. This system is currently in study with subjects at Burgos University Hospital, Spain. Keywords Electroencefalogram Virtual reality Photoparoxysmal response Machine learning &Vı ´ctor M. Gonza ´lez [email protected] Fernando Moncada [email protected] Sofı ´a Martı ´n [email protected] Vı ´ctor M. A ´lvarez [email protected] Beatriz Garcı ´a-Lo ´pez [email protected] Ana Isabel Go ´mez-Mene ´ndez [email protected] Jose ´R. Villar [email protected] 1 Electrical Engineering Department, University of Oviedo, Asturias, Spain 2 Computer Science Department, University of Oviedo, Asturias, Spain 3 Neurophysiology Department, Burgos University Hospital, Burgos, Spain 123 Neural Computing and Applications https://doi.org/10.1007/s00521-022-06940-z(0123456789().,-volV)(0123456789().,-volV) 1 Introduction Virtual reality (VR) is increasingly becoming part of our daily lives and its applicability is widening [29]: from video games and entertaining, to education, medical treatment and rehabilitation, military applications, architectural and urban design, digital marketing and activism, engineering and robotics, fine arts, heritage and archaeology, occupational safety, social sciences, psychology and many more. For this reason, it is not only important to understand how this technology can affect our brains, but also its potential to extend the current scope of neurological diseases detection methods. Using VR in new specific triggering scenarios will allow controlling and extending the available functionalities of the visual stimulation systems used for the detection of such pathologies [4]. This reflection is not new. Back in the 1990s, research performed to detect the risk of television and video games exposure resulted in a set of recommendations for TV manufacturers and video games developers. Simultaneously, the studies performed were the basis to many new uses of these techniques in education [9] or in rehabilitation [8], to name a few. Photosensitivity is an abnormal visual sensitivity of the brain resulting in a photoparoxysmal response (PPR), i.e. a brain epileptic discharge consisting of a cortical spike or a spike-and-wave provoked by a flash or a visual stimuli [26]. There are four different types of PPR resulting from different brain responses to intermittent light stimulation [31]. Even though PPR can be found in non-epileptic electroencephalogram (EEG) recordings, it is strongly associated with epilepsy [16]. The relevance of the PPR relies on its association with specific epileptic syndromes [34] and monitoring of treatment in a clinical context [20]. The photosensitivity range is related to the likelihood of occurrence of reflex seizures in daily life; thus, it is of crucial interest to early detect PPR. The most commonly used procedure to detect PPR, known as intermittent photic stimulation (IPS), is described in [23]. It proposes to submit the subject to a series of light flashes while simultaneously monitoring the brain activity using EEG signals, according to the European consensus group methodology for visual stimulation defined in 2012 [27]. The light flashes frequency is first increased from a minimum to a maximum or until a PPR is observed (whatever happen first). If no PPR is observed, the process finishes. Should a PPR occur, the procedure is repeated in an inverse manner, i.e. starting with a top flickering frequency that is gradually decreased until a minimum is reached or a PPR happen. The aim is to detect the minimum and the maximum frequencies at which the subject shows PPR, if any, with the minimal exposure. The detection of PPR is usually performed by physicians, i.e. clinical neurophysiologists and nurses, who manually review the EEG signals’ variability in search of PPR [2,14,22], taking into account each subject’s clinical context such as age, seizure and family history. To our best knowledge, no automated method for the detection of PPR has been developed so far. This research is mainly focused on the design of a new and safe procedure for the automatic detection of PPR within EEG signals using digital biomarkers implemented using VR and AI techniques. In this sense, [25] designed a PPR detection method by analysing the potential and oscillation of the response provoked by a flashing stimulation, but following a different stimulation pattern from the standard one. There are other recent studies that analyse the photosensitivity and epilepsy based on other generalized discharges or seizures than PPRs: in [18], a detection method based on the band amplitude fluctuation computed from a high-frequency and a low-frequency components of the EEG windows in each EEG channel is proposed; [30] applied the extreme gradient boost technique for the classification of seizures in two different ways (applying a standard partitioning of the data and applying a leave-one-out cross-validation scheme), while a channel-independent long short-term memory network is used in [5]; the information extracted from EEG and electrocardiogram (ECG) signals is used in [35]ina multi-modal neural network which analyse the data in three different ways (only EEG data with a convolutional LSTM network; only ECG data with a residual convolutional network; and a fused network which combines the outputs of the individual networks to perform the final classification); in [6], K-nearest neighbours and artificial neural networks are used for the detection of ictal discharges and inter-ictal states; [32] proposed an EEG single-channel analysis applying three types of visibility graphs (basic, horizontal and difference) to represent different EEG patterns. Other studies make use of additional and different biometric measures for the same purpose, such us electrocardiograms (ECG) [10,11,28], electromyograms (EMG) [3,36] or magnetoencefalograms (MEG) [24]. This study proposes an alternative to the conventional IPS procedure for PPR detection using VR and machine learning (ML). This research is focused on the most frequent PPR type; thus, the PPR detection still needs more research work as it is not completely solved. However, introducing VR would eventually allow to study and to develop new and safer procedures for PPR detection. Since the VR infrastructure is much more flexible than the standard one, it can be easily configured to carry out new assays and stimulation paradigms, allowing for an advanced IPS/Visual stimulation system. Our proposal includes a VR device with a wireless connection to a Neural Computing and Applications 123 computer that has access to the data gathered by the EEG sensors. The subject must wear at the same time both the EEG cap and a head mounted display (HDM). Furthermore, a plausible solution for the automatic PPR detection is proposed using some features extracted from the EEG signals in an average montage, and classic ML techniques. An average montage is used so that the electronegative PPR discharge will express with an upward deflection of the EEG signal in the affected channel. The main contributions of this research are: – To introduce VR in a close loop with AI and ML models, so medical procedures could be revisited and enhanced. This contribution can lead in the near future to more advanced diagnose tests and procedures. – To provide the neurophysiology department at Burgos University Hospital with a novel instrument to analyse the impact of VR in relation to photosensitivity by integrating this solution in its daily work. – To develop ML models to detect anomalies in the EEG recordings when the patient is flashed using either VRML IPS or conventional IPS. The structure of this study is as follows. The next section focuses on the description of the proposal, detailing the different elements in the loop. Section 3gives details of the experimentation set-up that has been carried out at the proposal, while Sect. 4includes all the obtained results and the discussion on them. The final section draws the conclusion of this research. 2 A prototype for VR-ML IPS procedure The proposed solution complements the conventional setup by means of introducing a VR device that the subject must wear along with an EEG cap (see Fig. 1). The signals from the EEG sensors are analysed using well-known ML techniques. The intelligent module will eventually control the VR contents to gain increased capabilities and to perform more complex assays. This section gives details on each of the main modules: the VR part (next subsection) and the ML module (Sect. 2.2). 2.1 VR design for IPS and PPR detection Flashing lights are one of the main triggers of photosensitive responses. VR-Photosense [15] is a software designed to detect photic-driving and PPR while using VR and wearing a head-mounted display (HMD). VR-Photosense offers a VR scenario with IPS in order to measure brain responses to flashing lights at various frequencies and using different sequences. The main goal of this software is to simulate conventional IPS tests in a virtual reality environment. Conventional IPS places the light stimulator at a very short distance from the patient’s eyes, creating high exposure to flickering lights which are perceived with intensity even when the patient’s eyes are closed. In order to emulate the exposure and sensory effect caused by the conventional IPS light stimulator, a virtual reality scene has been designed as a 3D enclosing spherical dome environment with the patient’s vision placed at its centre (see Fig. 4). This design suppresses patient’s peripheral vision and increases the focus on the visual stimuli even with the eyes closed. Fig. 1 To the left, the conventional set for IPS procedure. To the right, the new VR-ML set for IPS procedure, including automatic EEG analysis and PPR detection Neural Computing and Applications 123 The VR-Photosense’s set-up is fairly simple. After downloading and starting the app, the cardboard viewer, into which a smartphone is inserted, is secured to the subject’s head using the adaptable straps, leaving the upper part of the head clear. Then, the EEG cap is easily set up on the subject’s head covering all necessary points of contact as shown in Fig. 2. The software system is divided into two parts: the light stimulation and the monitoring software. The VR-Photosense’s default configuration stimulates with white light combined with dark black background, emulating the conventional IPS. Introducing an innovation to conventional IPS, VR-Photosene allows the colour of the flickering light and background to be changed from the default configuration, so we designed two coloured settings in addition to the white one: i) one with bright red flashes and deep blue background; ii) and another with deep blue flashes with dark black background. These tones may influence the brain discharges, and combined may be more or less provocative when compared to the default one, allowing to study brain behaviour reacting to the stimulation with both of these scenarios. VR-Photosense has also been designed to resemble the conventional stimulation set-up and to facilitate the work of physicians, both clinical neurophysiologists and nurses, while conducting photosensitivity tests at the hospital. This system includes a monitoring feature that allows to observe in real time what is happening on the VR stimulation via a web server. This is thanks to the use of Websockets, a communications protocol that offers full-duplex communication channels over a single TCP connection making it faster and with low latency to update system. This monitoring feature along with EEG recordings translates into a full coverage IPS scenario that closely resembles the conventional set-up, with the difference of replacing the traditional stimulation device with a low cost VR headset and the VR-Photosense software. The EEG setup to carry out VR-Photosense testing at the hospital consists of using Natus Brain monitoring and Neuroworks software for EEG recording. Furthermore, the different hardware and software elements can be easily mixed and replaced as they are completely independent from each other. For example, the VR-Photosense can be configured to work with higher end HDMs such as Oculus; or EEG recording can be performed using a different device such us OpenBCI 3D printed device and open source software, which were used at the university EEG laboratory for preliminary experiments. All in all, VR-Photosense offers an innovative, low-cost and cross-platform IPS scenario in virtual reality, with more upcoming features to be included aimed to achieve a more detailed diagnosis. This light stimulation software has been implemented in Unity 3D using the programming language C#. Unity is a video game development engine that allows designing 3D scenes by means of a visual editor and the programming of gameplay events via scripting. These scripts are associated with the game objects included in the scene so that they behave in the desired way. Within the scene, the assets included are structured in software components as shown in Fig. 3. – Controls: auto-generated script and input system. The new InputSystem 1.0.2 of Unity has been used, which allows to set up the desired inputs through an interface. Different inputs can be entered from different devices for the same action. This makes configuration and connectivity with different types of input hardware, such as keyboards and game controllers, seamless. Fig. 2 VR-Photosense set-up with a cardboard viewer and an OpenBCI EEG headset Fig. 3 VR-Photosense software package diagram Neural Computing and Applications 123 – GoogleVR: an SDK for Android that allows the creation of virtual reality applications to be used with Google Cardboard HMD. – Plugins: set of Android plugins needed to export applications for devices using this operating system. – Resources: all the scripts developed for lighting and connection with the monitoring side. This package also includes shaders and materials used in the scene. – Scenes: the designed virtual reality scene (Fig. 4). – XR: default Unity configuration for extended (virtual and augmented) reality apps. The developed scene has four objects or GameObjects: the camera, two point lights and a sphere (see Fig. 4). The purpose of the sphere is to provide a black background, placing the camera inside it and inverting the normals, thus avoiding transparency. This inversion process is done by adding a custom shader to a new material assigning it to the sphere. As for the spotlights, one of them is the main white light and the other provides a blue background for the custom colour functionality. All developed scripts use the MonoBehaviour built-in class, since Unity uses a standard Mono run-time implementation. These classes are considered blueprints and each time they are associated with a GameObject, a new instance of the object defined by that blueprint is created. In them, two methods are predefined: Start and Update. In this case, only Start is used, which will be called by Unity when loading the scene. It is also important to note the use of corrutines. Corrutines are functions that allow pausing and resuming the execution in the frame in which it was paused. In this case they are used to implement the flickering effect in the lights and the stimulation sequences. As for the input system, two options were considered: remote control and keyboard. Considering the number of configurable parameters and the controlled actions to be performed by the person in charge of the stimulation, the use of only a VR remote control was considered insufficient. Bearing in mind the end user and the technology they use on a daily basis, the use of a Bluetooth keyboard was chosen. As a result, more configurations are available and the commands are set in the way these experts consider more comfortable to perform the stimulation in the most efficient possible manner. However, in order to enable the use of the stimulation with the Oculus Quest 2 glasses, this configuration has also been adapted for the two remote controls associated with this HMD. Finally, the list of commands available is shown in Table 1. 2.1.1 Monitoring In addition to simulating the IPS environment, it is deemed necessary to know what is happening inside the scene during the tests. To address this problem, a monitoring component—a web page (https://vrphotosense.herokuapp. com)—was added to the system. This solution allows the physicists to see in real time what is happening in the VR stimulation, e.g. the evolution of the sequences and the commands entered by the user. Another possibility was to implement a sound system to communicate the situation through audio, but it was considered less efficient as it was volatile and did not have a visual record of the simulation. To implement the monitoring system, a WebSokets component was used, which allows real-time and fast communication between the smartphone application and the web page avoiding the use of a database. The desired functionality is as shown in Fig. 5. Client 1, the smartphone application, sends information about the stimulation to the server, and the server passes it to client 2, the web page, which displays it on the screen along with a time stamp. WebSockets is a protocol that provides bidirectional, full duplex communication over a single TCP socket. The environment in which the VR-Photosense application is used is a fairly sensitive one for medical testing, thus imposing an as fast as possible exchange of information exigence. WebSockets is probably the most popular protocol for this type of use cases where data needs to be sent in real time. As we have seen before, Unity is based on the Mono platform as scripting engine, which means that it works in .NET (C#). This framework provides a default support for this protocol that is also supported by Mono, Fig. 4 Scene diagram implemented in Unity 3D Neural Computing and Applications 123 through the System.Net.WebSockets namespace. The operation in this case would be as shown in Fig. 6. The server has been developed using Node. It is a simple server that receives information from the client and returns it. In this case, nothing is done in Unity with the returned Table 1 List of available commands in VR-photosense application Available commands Action Command (keyboard) Command (Quest 2) Start Enter Right trigger Pause Space bar Right grip Reset Backspace Secondary touched (R) Increment Hz Numpad ?Primary touched (R) White light B Start (L) Blue light A Left grip Red-blue combination R Left trigger Upward pattern Up arrow Primary touched (L) Downward pattern Down arrow Secondary touched (L) Exit Escape Start (R) Fig. 5 Theoretical functionality of the monitoring system Fig. 6 Sequence diagram of the system using WebSockets Neural Computing and Applications 123 information; it is the monitoring client that makes use of it. This client is an HTML page that connects to the server, opens a connection, gets the information coming from the server that is part of a message and displays it on the screen. This HTML client has been implemented in the simplest possible way since it is an add-on to the main work and its appearance in terms of design is not relevant for the EEG laboratory staff. 2.2 ML-based PPR detection PPR can be found in epileptic syndromes that present seizures with or without visual stimulus trigger, and in some subjects both types of seizures are observed. Waltz classification [31] is used to define the expression of the PPR from an electroencephalographic point of view, introducing up to four different types of PPR: •Type-1: spikes within the occipital rhythm. •Type-2: parieto-occipital spikes with a biphasic slow wave. •Type-3: parieto-occipital spikes with a biphasic slow wave and spread to the frontal region. •Type-4: generalized spikes and waves or polyspikes and waves. All of them are depicted in Fig. 7. Type-4 PPR is the most frequently found in epileptic syndromes where photosensitivity constitutes a clinical concern, as it seems to have a strong association—higher than 90%—with epileptic seizures; the detection of this type of PPR represents the challenge focused on this research. Besides, in clinical practice, it is frequent that expression of PPR is variable, and in many times we obtain PPR that not necessarily fit within only one category of those initially defined by Waltz. Even more, the morphological characteristics of a given subject’s PPR may vary because of clinical variables as doses of anti-epileptic treatment, sleep quality, etc. To identify Type-4 PPR we propose the use of sliding windows—one second length and a tenth of a second shift—followed by a pre-processing stages that subtracts the window average and performs a feature extraction. Well-known ML techniques consider these features to propose a final label to the window. These ML techniques are applied in three parts that will be described in this section. First, the set of transformations that will be applied and their rationale are explained in Sect. 2.2.1. Then, the design and deployment of the ML models are detailed in Sect. 2.2.2 and, finally, the training of the ML part is described in Sect. 2.2.3. Fig. 7 The four types of PPR: atype-1: spikes within the occipital rhythm; btype-2: parieto-occipital spikes with biphasic slow wave; ctype-3: parieto-occipital spikes with biphasic slow wave and spread to the frontal region; dtype-4: generalized spikes and waves Neural Computing and Applications 123 2.2.1 Feature extraction The selection of mathematical transformations for signals representation is, per se, a problem that needs to be carefully addressed. The main point is to select features that, in conjunction, include, as a whole, the information that the experts use to make a decision. Therefore, all the possible windows must be analysed and significant differences must be stated between the anomalies and the normal signal state. Nowadays, the problem of feature transformation is dealt with deep learning (DL) and, more specifically, with auto-encoders; however, due to the lack of data to train the networks, we left this issue for future research. For this study, we focused on channels Fz and O2 because these are the channels were PPRs more frequently appear regardless their type. We pay attention to their normal and abnormal behaviour for the considered PPR type. Some examples of the windows that might be faced are depicted in Fig. 8. From the analysis of the signals, the following features set has been selected to represent each EEG data window, where wis the width of the window, t refers the time stamp for which the feature is computed, c is the channel—either Fz or O2—and dc tis the EEG cchannel’s signal with an average montage—the PPR expresses with an upward deflection—at time stamp t: •Cumulative First derivative, also known as the intensity of the signal, computed as CFDc t¼1 wPw1 i¼0jdc tþiþ1dc tþij=D. This feature has been chosen because PPR present high rate of change in the value of the channel. Drepresents the interval between consecutive samples, which is kept constant. •Cumulative Second derivative computed as CSDc t¼1 wPw1 i¼0jdc tþiþ22dc tþiþ1þdc tþij=D2.This feature has been selected because there is also a high rate of change in the first derivative, but not so high as for artefacts. •Number of relevant peaks using the S1 measurement proposed in [19] and computed as follows. Equation 1 defines the calculation of S1, where kis the predefined number of samples and pis the current sample timestamp for which we are determining whether it is a peak or not. The S1transformation represents a scaling of the TS, which makes the peak detection easier using a predefined threshold a. S1ðpÞ¼1 2max p1 i¼pkðdc pdc iÞþmax pþk i¼pþ1ðdc pdc iÞ  ð1Þ So, for each point for which S1ðpÞcan be computed within the EEG sliding window we compute S1ðpÞ;a peak occurs in time pif the value Spis higher than aand is the highest in its 2kneighbourhood. In the original report, all the parameters (k,a) where carefully determined for each problem in order to optimize the peak detection. In this research, kis set to 10 and ais set to three times the standard deviation of the EEG channel values when no activity is shown (upper right corner in Fig. 8). •Sum of the absolute values, to measure the area under the curve of the EEG signal. Fig. 8 Three EEG fragments from different conditions: the left-most and the centre recordings are considered normal conditions, while the recording at the right shows a PPR. The recordings include, from top to bottom, signals from the F3-AVG, Fz-AVG, F4-AVG, O1-AVG and O2-AVG channels, respectively, where AVG stands for the average of all recorded electrodes Neural Computing and Applications 123 •Maximum differences, also known in some fields as amount of movement [1], which measures the differences between the highest and the smallest values— expected to be a high value. It is calculated as MDc t¼jmaxi2½t;tþwðdc iÞmini2½t;tþwðdc iÞj. •Average Energy as proposed in [33], as the sum of the squared discrete FFT components magnitudes of the signal. All the features are standardized; given a data set, the average and the standard deviation are computed and used to transform the values into a normal distribution with mean 0.0 and standard deviation 1.0. 2.2.2 Designing and deploying the ML part The goal in this stage is to obtain models able to label the pre-processed EEG signal windows as normal or as PPR. Due to the data imbalance, the most interesting approach is to use unsupervised learning, so anomalies can be detected. However, this might generate too many false positive, so it could be interesting to also develop a complementary supervised learning solution. Therefore, for this research we proposed to use, first, unsupervised learning to obtain a model that signals those anomalous windows and then to classify the anomalous windows as PPR or normal using a supervised approach. The unsupervised learning is specific for a given subject, while the supervised learning is a generalised model. For this stage, we will develop models following the workflow proposed in Fig. 9. Data from the EEG sensors is windowed as explained before. For each window, the average is calculated and subtracted; the transformations from the previous subsection are calculated afterwards. A one-class classifier, learned for the current subject data, labels the window as normal or not. In case a window is labelled as an anomaly, then the two-class classifier, learned from other subjects, labels the window as PPR or not. For the one-class classifier, an unsupervised one-class k-nearest neighbours model (1C-KNN) [12,17] is tested: this model has been selected for its fast training and evaluation times while still performing sufficiently accurate. For the two-class classifier we propose K-nearest neighbour (2C-KNN) due to the small number of instances in the available data set. Both classifiers are from the scikit-learn library for Phyton [21]. 2.2.3 Training the ML part Training the models has two main stages as can be seen in Fig. 10: (i) training the one-class model and (ii) training the two-class model. At this moment we have two collections of data: (a) a collection of windows from the current subject (CPData), all labelled as normal, and (b) a collection of windows from the historical records (HRData), each window with its corresponding normal or PPR label. CPData is used in the one-class training, while HRData is used in the two-class training. The first part of the training is the 2C-KNN learning using the HRData; in case of highly imbalance of the data set, SMOTE will be used. Different values of the parameter K are tested for both classifiers to find the best performing model. When analysing the data recorded for the current subject, an incremental training is proposed. The idea is repeating the one-class classifier training until a real PPR be detected at a certain flashing frequency. That is, in case the frequency to be tested is increased for example from 4 to 6 Hz, if no PPR is detected in this new stimulation frequency, then the 1C-KNN is trained including the windows gathered from the first frequency range (1–6 Hz), and then, the next flashing frequency is evaluated (8 Hz) and the process is repeated again until a PPR is detected. The stimulation frequency at which the first PPR is detected is the cut frequency (fc). The process is illustrated in Fig. 11. This process has been described for the following flashing frequency increase sequence—standard Fig. 9 The workflow of the designed approach. The data gathered from the current subject are pre-processed. When a data window comes from frequencies smaller or equal to fc (the cut frequency, which is the stimulation frequency value at which the first PPR appears), the window is preserved for the training of the one-class models; otherwise, the window is labelled as normal or as anomaly. In this latter case, the two-class classifier labels the window as PPR or not. However, when no window is labelled as including a PPR for the current frequency, the windows for this frequency are also considered and the one class model is re-trained Neural Computing and Applications 123 amount of available data allows us to do so, such as autoencoders plus dense layers or long short-term memory networks. All of these improvements represents future research work. Acknowledgements This research has been funded by the Spanish Ministry of Science and Innovation under project MINECOTIN2017-84804-R, PID2020-112726RB-I00 and the State Research Agency (AEI, Spain) under grant agreement No RED2018-102312-T (IA-Biomed). Additionally, by the Council of Gijo ´n through the University Institute of Industrial Technology of Asturias grant SV-21GIJON-1-19. Funding Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. Conflict of interest The authors declare that they have no conflict of interest. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons. org/licenses/by/4.0/. References 1. A ´lvarez A, Trivin ˜o G, Cordo ´n O (2011) Body posture recognition by means of a genetic fuzzy finite state machine. In: IEEE 5th international workshop on genetic and evolutionary fuzzy systems (GEFs), pp 60–65 2. Beniczky S, Aurlien H, Franceschetti S, da Silva AM, Bisulli F, Bentes C, Canafoglia L, Ferri L, Kry ´sl D, Peralta AR, Ra ´cz A, Cross JH, Arzimanoglou A (2020) Interrater agreement of classification of photoparoxysmal electroencephalographic response. Epilepsia. https://doi.org/10.1111/epi.16655 3. Beniczky S, Conradsen I, Henning O, Fabricius M, Wolf P (2018) Automated real-time detection of tonic-clonic seizures using a wearable EMG device. Neurology. https://doi.org/10.1212/WNL. 0000000000004893 4. Brooks AL, Brahman S, Kapralos B, Nakajima A, Tyerman J, Jain LC (2021) Recent advances in technologies for inclusive well-being. Virtual patients, gamification and simulation. Intelligent systems reference library series, 196, 1 edn. Springer 5. Chakrabarti S, Swetapadma A, Pattnaik PK (2021) A channel independent generalized seizure detection method for pediatric epileptic seizures. Comput Methods Programs Biomed. https:// doi.org/10.1016/j.cmpb.2021.106335 6. Choubey H, Pandey A (2021) A combination of statistical parameters for the detection of epilepsy and eeg classification using ann and knn classifier. Signal Image Video Process. https:// doi.org/10.1007/s11760-020-01767-4 7. Cobb S (1947) Photic driving as a cause of clinical seizures in epileptic patients. Arch Neurol Psych 58(1):70–71 8. Hall C, Vivanti A, Abbey K (2019) Impact of television on nutritional intake in communal dining room settings among those with acquired brain injury a pilot study. Nutrition and dietetics. J Diet Australia 77(4):444–448. https://doi.org/10.1111/17470080.12526 9. Hoffman BL, Hoffman R, Wessel CB, Shensa A, Woods MS, Primack BA (2018) Use of fictional medical television in health sciences education: a systematic review. Adv Health Sci Educ 23(1):201–216. https://doi.org/10.1007/s10459-017-9754-5 10. Jahanbekam A, Baumann J, Nass RD, Bauckhage C, Hill H, Elger CE, Surges R (2021) Performance of ecg-based seizure detection algorithms strongly depends on training and test conditions. Epilepsia Open. https://doi.org/10.1002/epi4.12520 11. Jeppesen J, Fuglsang-Frederiksen A, Johansen P, Christensen J, Wu ¨stenhagen S, Tankisi H, Qerama E, Hess A, Beniczky S (2019) Seizure detection based on heart rate variability using a wearable electrocardiography device. Epilepsia 60:2105–2113. https://doi.org/10.1111/epi.16343 12. Khan SS, Ahmad A (2018) Relationship between variants of oneclass nearest neighbours and creating their accurate ensembles. IEEE Trans Knowl Data Eng 30:1796–1809 13. Kiloh LG (2013) and McComas. Clinical electroencephalography. Butterworth-Heinemann, A.J., Osselton, J.W 14. Larsen PM, Wu ¨stenhagen S, Terney D, Gardella E, Alving J, Aurlien H, Beniczky S (2021) Photoparoxysmal response and its characteristics in a large eeg database using the score system. Clin Neurophysiol. https://doi.org/10.1016/j.clinph.2020.10.029 15. Martı ´nS,A ´lvarez V, Garcı ´a-Lo ´pez B, Gonza ´lez VM, Vilar JR (2021) Vr-photosense: a virtual reality photic stimulation interface for the study of photosensitivity. In: Proceedings of 16th international conference on soft computing models in industrial and environmental applications (SOCO 2021) soft computing models in industrial and environmental applications, pp 178–186. Springer 16. Michelucci R, Pasini E, Riguzzi P, Andermann E, Ka ¨lvia ¨inen R, Genton P (2016) Myoclonus and seizures in progressive myoclonus epilepsies: pharmacology and therapeutic trials. Epilep Disorder. https://doi.org/10.1684/epd.2016.0861 17. Munroe D, Madden MG (2005) Multi-class and single-class classification approaches to vehicle model recognition from images. In: Proceedings of AICS-05: Irish conference on artificial intelligence and cognitive science, pp 1–10 18. Omidvarnia A, Warren AE, Dalic LJ, Pedersen M, Jackson G (2021) Automatic detection of generalized paroxysmal fast activity in interictal eeg using time-frequency analysis. Comput Biol Med. https://doi.org/10.1016/j.compbiomed.2021.104287 19. Palshikar GK (2009) Simple algorithms for peak detection in time-series. Technical report, Tata Research Development and Design Centre 20. Panayiotopoulos CP (2010). A clinical guide to epileptic syndromes and their treatment. https://doi.org/10.1007/978-1-84628644-5 21. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830 22. Rathore C, Prakash S, Makwana P (2020) Prevalence of photoparoxysmal response in patients with epilepsy: effect of the underlying syndrome and treatment status. Seizure. https://doi. org/10.1016/j.seizure.2020.09.006 23. Rubboli G, Parra J, Seri S, Takahashi T, Thomas P (2004) Eeg diagnostic procedures and special investigations in the assessment of photosensitivity. Epilepsia 45(5):35–39. https://doi.org/ 10.1111/j.0013-9580.2004.451002.x Neural Computing and Applications 123 24. Soriano MC, Niso G, Clements J, Ortı ´n S, Carrasco S, Gudı ´nM, Mirasso CR, Pereda E (2017) Automated detection of epileptic biomarkers in resting-state interictal meg data. Front Neuroinform. https://doi.org/10.3389/fninf.2017.00043 25. Strigaro G, Gori B, Varrasi C, Fleetwood T, Cantello G, Cantello R (2021) Flash-evoked high-frequency eeg oscillations in photosensitive epilepsies. Epilep Res. https://doi.org/10.1016/j.eplep syres.2021.106597 26. Trenite DKN (2019) Photosensitivity and epilepsy. In: Clinical electroencephalography, pp 487–495. Springer 27. Trenite ´DKN, Rubboli G, Hirsch E, Martins Da Silva A, Seri S, Wilkins A, Parra J, Covanis A, Elia M, Capovilla G, Stephani U, Harding G (2012) Methodology of photic stimulation revisited: updated European algorithm for visual stimulation in the eeg laboratory. Epilepsia. https://doi.org/10.1111/j.1528-1167.2011. 03319.x 28. Ufongene C, Atrache RE, Loddenkemper T, Meisel C (2020) Electrocardiographic changes associated with epilepsy beyond heart rate and their utilization in future seizure detection and forecasting methods. Clin Neurophysiol. https://doi.org/10.1016/ j.clinph.2020.01.007 29. Vailshery LS (2021) Ar/vr headset shipments worldwide 2020–2025. https://www.statista.com/statistics/653390/world wide-virtual-and-augmented-reality-headset-shipments/ 30. Vanabelle P, Handschutter PD, Tahry RE, Benjelloun M, Boukhebouze M (2020) Epileptic seizure detection using eeg signals and extreme gradient boosting. J Biomed Res. https://doi. org/10.7555/JBR.33.20190016 31. Waltz S, Christen HJ, Doose H (1992) The different patterns of the photoparoxysmal response—a genetic study. Electroencephalogr Clin Neurophysiol 83(2):138–145. https://doi.org/10. 1016/0013-4694(92)90027-F.https://www.sciencedirect.com/sci ence/article/pii/001346949290027F 32. Wang L, Long X, Arends JB, Aarts RM (2017) Eeg analysis of seizure patterns using visibility graphs for detection of generalized seizures. J Neurosci Method. https://doi.org/10.1016/j.jneu meth.2017.07.013 33. Wang S, Yang J, Chen N, Chen X, Zhang Q (2005) Human activity recognition with user-free accelerometers in the sensor networks. In: Proceedings of international conference on neural networks and brain ICNN&B’05, pp 1212–1217 34. Wolf P, Goosses R (1986) Relation of photosensitivity to epileptic syndromes. J Neurol Neurosurg Psych. https://doi.org/ 10.1136/jnnp.49.12.1386 35. Yang Y, Truong N, Maher C, Kavehei O, Truong ND, Eshraghian JK, Nikpour A (2021) A multimodal ai system for out-of-distribution generalization of seizure detection. bioXRiv. https://doi. org/10.1101/2021.07.02.450974 36. Zibrandtsen IC, Kidmose P, Kjaer TW (2018) Detection of generalized tonic-clonic seizures from ear-eeg based on emg analysis. Seizure. https://doi.org/10.1016/j.seizure.2018.05.001 Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Neural Computing and Applications 123