Generally, the clinical analysis is performed by specialised healthcare professionals, using a video-EEG system – namely synchronised video and electroencephalograms, at epilepsy monitoring units (EMUs), among other diagnostic techniques. "During clinical diagnosis, the clinicians utilise these videos to visually recognise movements of interests defined by movement features, which indicate the origin of the seizures", explained Tamás Karácsony, a researcher at the Institute for Systems and Computer Engineering, Technology and Science (INESC TEC) and Faculty of Engineering of the University of Porto (FEUP), Portugal, and at the Robotics Institute of Carnegie Mellon University, Pittsburgh, USA through the CMU Portugal Affiliated PhD program.
However, the semiology assessment is limited by a high inter-rater variability among said professionals, and despite being promising, the automatic and semi-automatic approaches using computer vision still depend on considerable “human in the loop” effort. “A patient is usually monitored for several days, which must be fully reviewed afterwards for the seizures. This requires a lot of time and effort from the clinical staff”, added the researcher.
In this context, a team of researchers from INESC TEC, in Portugal, and from the Neurology Department of Ludwig Maximilian University of Munich, tested a solution based on deep learning to classify epileptic events, using an infrared radar and 3D videos. The study presents an innovative approach, and is the first to explore the classification, in near real-time, from two-second video samples.
The solution presented is "inspired by the way experts analyse the semiology of seizures, taking into account not only the presence of specific movements of interest in different parts of the patients' bodies, but also their dynamics and their biomechanical aspects, such as speed or acceleration patterns, or range of motion", said Tamás Karácsony.
The researchers used the largest database of 3D videos of epileptic seizures synchronized with electroencephalograms and extracted data relating to 115 episodes, having developed a semi-specialised and automatic pre-processing algorithm to remove unnecessary elements. In practical terms, two image cropping methods – depth and Mask R-CNN – are combined, leading to a clean image, consequently improving the extraction of relevant data from the available videos, minimising unrelated variations, and improving the seizure classification process.
"Our solution utilises an action recognition approach, namely an I3D (Inflated 3D Network) based feature extraction architecture, which is a widely adopted human action recognition 3D Convolutional neural network (3D CNN), combined with a long short-term memory (LSTM) based classifier, to further exploit temporal features. Here the I3D uses 3D convolution to extract spatiotemporal information directly from videos, namely the movement features of the seizures, which are then classified by the LSTM into the three classes. Additionally, we utilized an intelligent 3D cropping of the scene to remove unrelated information, such as clinicians moving around the patients. By removing them, our method significantly improved classification performance", concluded the researcher.
The present research also proved the possibility of a solution to support online monitoring – using Artificial Intelligence based on the recognition of actions to differentiate epileptic seizures. In addition, the proposed solution can be used in addition to other 3D video datasets for seizure analysis and monitoring.
According to João Paulo Cunha, co-author of the study, a pioneer in the area of movement quantification in epilepsy that has published the first studies in this area more than 20 years ago, professor at the Faculty of Engineering of the University of Porto (FEUP) and scientific director of the CMU-Portugal program, "this work shows the feasibility of our novel action-recognition approach to distinguish three classes of epileptic events - two classes of epilepsy and a third class of non-epileptic events - with only two seconds of samples". He added that "the solution we presented can be applied to other sets of 3D videos, in the analysis of motor-related issues, e.g., associated with essential tremors or Parkinson's disease".
In this sense, by translating this knowledge into better diagnosis and treatment, this approach serves two purposes: "the knowledge gain can be used in the diagnostic step of EEG-Video-Monitoring for epilepsy and its differential diagnoses. It will increase diagnostic accuracy, make the stay more efficient and gain insight on the relationship of the underlying epilepsy and its core symptom of seizures. In further development, it may be used for home monitoring for seizures in therapy refractory epilepsy", said Prof. Rémi, head of the Epilepsy Monitoring Unit at the University of Munich, and co-author of the paper.
The near real-time detection also allows to issue a warning directed to the teams of healthcare professionals or caregivers, allowing a quick intervention, to reduce the consequences of the seizure. “The solution also allows a faster response and may reduce the associated risks and sudden death caused by epilepsy in the near future", said Tamás Karácsony.
MEDICA-tradefair.com; Source: INESC Brussels HUB
