Related publications

Published papers related to the MAMEM Project.
1.

Vangelis P. Oikonomou, Georgios Liaros, Kostantinos Georgiadis, Elisavet Chatzilari, Katerina Adam, Spiros Nikolopoulos and Ioannis Kompatsiaris, Comparative evaluation of state-of-the-art algorithms for SSVEP-based BCIs, Technical Report - eprint arXiv:1602.00904, February 2016

Abstract Brain-computer interfaces (BCIs) have been gaining momentum in making human-computer interaction more natural, especially for people with neuro-muscular disabilities. Among the existing solutions the systems relying on electroencephalograms (EEG) occupy the most prominent place due to their non-invasiveness. However, the process of translating EEG signals into computer commands is far from trivial, since it requires the optimization of many different parameters that need to be tuned jointly. In this report, we focus on the category of EEG-based BCIs that rely on Steady-State-Visual-Evoked Potentials (SSVEPs) and perform a comparative evaluation of the most promising algorithms existing in the literature. More specifically, we define a set of algorithms for each of the various different parameters composing a BCI system (i.e. filtering, artifact removal, feature extraction, feature selection and classification) and study each parameter independently by keeping all other parameters fixed. The results obtained from this evaluation process are provided together with a dataset consisting of the 256-channel, EEG signals of 11 subjects, as well as a processing toolbox for reproducing the results and supporting further experimentation. In this way, we manage to make available for the community a state-of-the-art baseline for SSVEP-based BCIs that can be used as a basis for introducing novel methods and approaches.
2.

Zoe Katsarou, MD, Meir Plotnik, PhD, Gabi Zeilig, MD, Amihai Gottlieb, MSc, Rachel Kizony, PhD and Sevasti Bonstantjopoulou, Computer uses and difficulties in Parkinson’s disease, The MDS 20th International Congress of Parkinson's Disease and Movement Disorders, Berlin-Germany, June 2016

Abstract Thirty five PD pts with a long experience in computer operation were included in the study. Their mean age was 59.5 (SD 8.27) years. Most of them were in Hoehn and Yahr stage II. PD pts uses, habits, and difficulties with the computer were explored by means of a structured interview which provided information in the form of yes/no answers to questions relevant to a wide range of usual computer uses and applications as well as difficutlties in performing various tasks relevant to computer operation. Two quantitative scales one referring to the contribution of the computer in social life, every day activities, emotional well-being (total score: 9=not important/45-very important) and the other exploring the disease impact on various aspects of computer operation (total score: 11=no effect/55 maximum effect) were also employed.
3.

S. Bostantjopoulou, M. Plotnick, G. Zeilig, A. Gottlieb, R. Kizony, S. Chlomissiou, A. Nichogiannopoulou, Z. Katsarou, Computer use aspects in patients with motor disabilities, 2nd Congress of the European Academy of Neurology (EAN'2016), Copenhagen, Denmark, May 28-31, 2016

Abstract Three groups of neurological patients were studied:a)25 patients with Parkinson's disease (PD),b)23 patients with spinal cord injury (SCI) and c)19 with neuromuscular disorders ((NMD).All patients were assessed by means of two scales, one referring to the contribution of the computer in social life,everyday activities,emotional well-being (CCLS) [total score:9=not important/45=very important] and the other exploring the disease impact on various aspects of computer operation (DICOS) [total score:11=no effect/55=maximum effect]. Reliability of both scales was excellent (Cronbach's alpha was 0.87 for CCLS and 0.93 for DICOS).Between groups comparisons showed that NMD patients regarded conputer use as most important and SCI patients had the major difficulty.Mean total scores (SD) were as follows for a)CCLS:PD patients=23.28(7.22);SCI patients=20.78(9.72);NMD patients=32.84(5.12) [p=0.000] and b)DICOS:PD patients=25.9(9.9);SCI patients=31.22(15.0);NMD patients=20.53(5.15)[p=0.017]. Our preliminary results show that patients with motor disabilities regard computer use as an important aspect of their life and their disability has a significant effect in their ability to operate it satisfactorily.This information is important for the development of innovating technology helping patients to overcome their specific disabilities.
4.

Zeilig Gabi, Gottlieb Amihai, Kizony Rachel, Katsarou Zoe, Bostantzopoulou Sevasti, Nichgiannopoulou Ariana, Chlomissiou Sissy and Plotnik Meir, MAMEM – A novel computer brain interface platform for enhancing social interaction of people with disabilities – Clinical requirements resulting from focus groups and literature survey, 20th European Congress of Physical and Rehabilitation Medicine, Lisbon Portugal, April 2016

Abstract Health professionals with experience in the field of Parkinson Disease, neuromuscular conditions and tetraplegia following spinal cord injury, from three medical centers, from two countries, participated. We performed a literature survey, focusing on the characteristics of the study population, their computer and internet use habits, existing solutions, and specific challenges related to EEGs and EMs - based –computerassistive devices. We conducted three focus groups, with six health professionals per group. We also performed a qualitative analysis of the focus groups transcripts. The clinical requirements that resulted at the end of this phase have been then summarized, prioritized and coded with numbers from 1 (minimal) to 7 (maximal importance) by the health professionals from each site
5.

Sofia Fountoukidou, Jaap Ham, Peter Ruijten, and Uwe Matzat, Using personalized persuasive strategies to increase acceptance and use of HCI technology, Adjunct Proceedings of the 11th International Conference on Persuasive Technology, Salzburg-Austria, April 2016

Abstract It has been recognized that the adoption of a certain technology is not only dependent on the technological excellence. End-users can be reluctant to use it even though they are aware of its benefits. Personalized persuasive technology could be a key to the successful adoption of the MAMEM technology. Previous research has identified various persuasive strategies that can effective for behavior and/or attitude change. However, it is still unclear which persuasive strategies are most effective for which type of person. Thus, one of the objectives of MAMEM is to adapt persuasive technology interventions to the target group characteristics, in order to increase their effectiveness. For its realization, user profiles and sets of personas for the three target groups are created. Next, the Intervention Mapping framework is used for developing and implementing effective persuasive interventions tailored both to the user audience characteristics and to the specific target behaviors. All in all, MAMEM will execute extensive research, in order to contribute to the current state of the art of designing persuasive technologies taking into consideration both the user characteristics and the behavior in request
6.

R. Menges, C. Kumar, K. Sengupta, S. Staab, eyeGUI: A Novel Framework for Eye-Controlled User Interfaces, 9th Nordic Conference on Human-Computer Interaction, NordiCHI 2016

Abstract The user interfaces and input events are typically composed of mouse and keyboard interactions in generic applications. Eye-controlled applications need to revise these interactions to eye gestures, and hence design and optimization of interface elements becomes a significant feature. In this work, we propose a novel eyeGUI framework, to support the development of such interactive eye-controlled applications with many vital aspects, like rendering, layout, dynamic modification of content, support of graphics and animation
7.

C. Kumar, R. Menges and S. Staab, Eye-Controlled Interfaces for Multimedia Interaction, in IEEE MultiMedia, vol. 23, no. 4, pp. 6-13, Oct.-Dec. 2016. doi: 10.1109/MMUL.2016.52

Abstract In the digitized world, interacting with multimedia information occupies a large portion of everyday activities; it’s now an essential part of how we gather knowledge and communicate with others. It involves several operations, including selecting, navigating through, and modifying multimedia, such as text, images, animations, and videos. These operations are usually performed by devices such as a mouse or keyboard, but people with motor disabilities often can’t use such devices. This limits their ability to interact with multimedia content and thus excludes them from the digital information spaces that help us stay connected with families, friends, and colleagues. In this paper, we primarily focus on the gaze-based control paradigm that we’ve developed as part of our work at the Institute for Web Science and Technologies (WeST) within the scope of MAMEM project. We outline the particular challenges of eye-controlled interaction with multimedia information, including initial project results. The objective is to investigate how eye-based interaction techniques can be made precise and fast enough to not only allow disabled people to interact with multimedia information but also make usage sufficiently simple and enticing such that healthy users might also want to include eye-based interaction.
8.

Korok Sengupta, Raphael Menges, Chandan Kumar and Steffen Staab, GazeTheKey: Interactive Keys to Integrate Word Predictions for Gaze-based Text Entry, Demo paper at the 22nd annual meeting of the intelligent user interfaces community, ACM IUI 2017, March 13 - 16, 2017 Limassol, Cyprus

Abstract In the conventional keyboard interfaces for eye typing, the functionalities of the virtual keys are static, i.e., user’s gaze at a particular key simply translates the associated letter as user’s input. In this work we argue the keys to be more dynamic and embed intelligent predictions to support gazebased text entry. In this regard, we demonstrate a novel "GazeTheKey" interface where a key not only signifies the input character, but also predict the relevant words that could be selected by user’s gaze utilizing a two-step dwell time.
9.

Zoe Katsarou, Gabi Zeilig, Meir Plotnik, Amihai Gotlieb, Rachael Kizony and Sevasti Bonstantjopoulou, Parkinson’s disease impact on computer use. A patients’ and caregivers perspective, The American Academy of Neurology 69th Annual Meeting, Neurology April 18, 2017 vol. 88 no. 16 Supplement P6.009, Boston, MA

Abstract The mean total score of PD paents on the CCSL scale was 22.7±6.9 Single items that scored high were relevant to interpersonal interacon, educaon, work and employment. The DICOS scale yielded a mean total score of 24.7± 10.0.Single items that had a significant impact on the whole score were speed of computer operaon and accuracy of performance. Caregivers’ mean scores on the CCSL and DICOS scales were similar to those of the paents (p=0.324).
10.

Anastasios Maronidis, Vangelis Oikonomou, Spiros Nikolopoulos and Ioannis (Yannis) Kompatsiaris, Steady State Visual Evoked Potential Detection Using Subclass Marginal Fisher Analysis, Proceedings of the 8th International IEEE EMBS Conference on Neural Engineering, May 25-28, 2017, Shanghai China (accepted for publication)

Abstract Recently, SSVEP detection from EEG signals has attracted the interest of the research community, leading to a number of well-tailored methods. Among these methods, Canonical Correlation Analysis (CCA) along with several variants have gained the leadership. Despite their effectiveness, due to their strong dependence on the correct calculation of correlations, these methods may prove to be inadequate in front of potential deficiency in the number of channels used, the number of available trials or the duration of the acquired signals. In this paper, we propose the use of Subclass Marginal Fisher Analysis (SFMA) in order to overcome such problems. SMFA has the power to effectively learn discriminative features of poor signals, and this advantage is expected to offer the appropriate robustness needed in order to handle such deficiencies. In this context, we pinpoint the qualitative advantages of SMFA, and through a series of experiments we prove its superiority over the state-of-the-art in detecting SSVEPs from EEG signals acquired with limited resources.
11.

Vangelis P. Oikonomou, Anastasios Maronidis, Georgios Liaros, Spiros Nikolopoulos and Ioannis Kompatsiaris, Sparse Bayesian Learning for Subject Independent Classification with Application to SSVEP-BCI, Proceedings of the 8th International IEEE EMBS Conference on Neural Engineering, May 25-28, 2017, Shanghai China (accepted for publication)

Abstract Sparse Bayesian Learning (SBL) is a widely used framework which helps us to deal with two basic problems of machine learning, to avoid overfitting of the model and to incorporate prior knowledge into it. In this work, multiple linear regression models under the SBL framework are used for the problem of multiclass classification when multiple subjects are available. As a case study, we apply our method to the detection of Steady State Visual Evoked Potentials (SSVEP), a problem that arises frequently into the Brain Computer Interface (BCI) paradigm. The multiclass classification problem is decomposed into multiple regression problems. By solving these regression problems, a discriminant vector is learned for further processing. In addition the adoption of the kernel trick and the special treatment of produced similarity matrix provide us with the ability to use a Leave-One-Subject-Out training procedure resulting in a classification system suitable for subject independent classification. Extensive comparisons are carried out between the proposed algorithm, the SVM classifier and the CCA based methodology. The experimental results demonstrate that the proposed algorithm outperforms the competing approaches, in terms of classification accuracy and Information Transfer Rate (ITR), when the number of utilized EEG channels is small.
12.

Raphael Menges, Chandan Kumar, Daniel Mueller, Korok Sengupta, GazeTheWeb: A Gaze-Controlled Web Browser, (TPG challenge winner) Proceedings of the 14th Web for All Conference. ACM, 2017

Abstract Web is essential for most people, and its accessibility should not be limited to conventional input sources like mouse and keyboard. In recent years, eye tracking systems have greatly improved, beginning to play an important role as input medium. In this work, we present GazeTheWeb, aWeb browser accessible solely by eye gaze input. It effectively supports all browsing operations like search, navigation and bookmarks. GazeTheWeb is based on a Chromium powered framework, comprising Web extraction to classify interactive elements, and application of gaze interaction paradigms to represent these elements.
13.

Chandan Kumar, Raphael Menges, Daniel Mueller, Steffen Staab, Chromium based Framework to Include Gaze Interaction in Web Browser, (honourable mention) Proceedings of the 26th International Conference Companion on World Wide Web (pp. 219-224). International World Wide Web Conferences Steering Committee, 2017

Abstract EnablingWeb interaction by non-conventional input sources like eyes has great potential to enhance Web accessibility. In this paper, we present a Chromium based inclusive framework to adapt eye gaze events in Web interfaces. The framework provides more utility and control to develop a fully featured interactive browser, compared to the related approaches of gaze-based mouse and keyboard emulation or browser extensions. We demonstrate the framework through a sophisticated gaze driven Web browser, which e effectively supports all browsing operations like search, navigation, bookmarks, and tab management.
14.

Fotis Kalaganis, Elisavet Chatzilari, Kostas Georgiadis, Spiros Nikolopoulos, Nikos Laskaris and Yiannis Kompatsiaris, An Error Aware SSVEP-based BCI, 30th IEEE International Symposium on Computer-based Medical Systems, Special Track on Multimodal Interfaces for Natural Human Computer Interaction: Theory and Applications, IEEE CBMS 2017, June 22-24, 2017 Thessaloniki - Greece (to appear)

Abstract ErrPs have been used lately in order to improve several existing BCI applications. In our study we investigate the contribution of ErrPs in a SSVEP based BCI. An extensive study is presented in order to discover the limitations of the proposed scheme. Using Common Spatial Patterns and Random Forests we manage to show encouraging results regarding the incorporation of ErrPs in a SSVEP system. Finally, we provide a novel methodology (ICRT) that can measure the gain of a BCI system by incorporating ErrPs in terms of time efficiency.
15.

Vangelis Oikonomou, Kostas Georgiadis, Georgios Liaros, Spiros Nikolopoulos and Yiannis Kompatsiaris, A comparison study on EEG signal processing techniques using motor imagery EEG data, 30th IEEE International Symposium on Computer-based Medical Systems, Special Track on Multimodal Interfaces for Natural Human Computer Interaction: Theory and Applications, IEEE CBMS 2017, June 22-24, 2017 Thessaloniki - Greece (to appear)

Abstract Brain-computer interfaces (BCIs) have been gaining momentum in making human-computer interaction more natural, especially for people with neuro-muscular disabilities. Among the existing solutions the systems relying on electroencephalograms (EEG) occupy the most prominent place due to their non-invasiveness. In this work we provide a review of various existing techniques for the identification of motor imagery (MI) tasks. More specifically we perform a comparison between CSP related features and features based on Power Spectral Density (PSD) techniques. Furthermore, for the identification of MI tasks two well-known classifiers are used, the Linear Discriminant Analysis (LDA) and the Support Vector Machine (SVM). Our results confirms that PSD features demonstrate the most consistent robustness and effectiveness in extracting patterns for accurately discriminating between left and right MI tasks.
16.

Korok Sengupta, Jun Sun, Raphael Menges, Chandan Kumar and Steffen Staab, Analyzing the Impact of Cognitive Load in Evaluating Gaze-based Typing, 30th IEEE International Symposium on Computer-based Medical Systems, Special Track on Multimodal Interfaces for Natural Human Computer Interaction: Theory and Applications, IEEE CBMS 2017, June 22-24, 2017 Thessaloniki - Greece (to appear)

Abstract Gaze-based virtual keyboards allow people with motor disability a method for text entry by eye movements. The effectiveness and usability of these keyboards have traditionally been evaluated with conventional text entry performance measures such as words per minute, keystroke saving, error rate, accuracy, etc. However, in comparison to the conventional text entry approaches, gaze-based typing involves natural eye movements that are highly correlated with human brain cognition. Employing eye gaze as an input could lead to excessive mental demand, and in this work we argue the need to include cognitive load as an eye typing evaluation measure. We evaluate three variations of gaze-based virtual keyboards, which have variable design in terms of word suggestion positioning. The conventional text entry metrics indicate no significant difference in the performance of the different keyboard designs. However, STFT (Short-time Fourier Transformation) based analysis of EEG signals indicate variances in the mental workload of participants while interacting with these designs. Moreover, the EEG analysis provides us insights into the user’s cognition variation in different typing phases and intervals, which should be considered to improve eye typing usability.
17.

Chandan Kumar, Raphael Menges and Steffen Staab, Assessing the Usability of a Gaze-Adapted Interface with Conventional Eye-based Emulation, 30th IEEE International Symposium on Computer-based Medical Systems, Special Track on Multimodal Interfaces for Natural Human Computer Interaction: Theory and Applications, IEEE CBMS 2017, June 22-24, 2017 Thessaloniki - Greece (to appear)

Abstract In recent years, eye tracking systems have greatly improved, beginning to play a promising role as an input medium. Eye trackers can be used for application control either by simply emulating the mouse device in the traditional graphical user interface, or by customized interfaces for eye gaze events. In this work we evaluate these two approaches to assess their impact in usability. We present a gaze-adapted Twitter application interface with direct interaction of eye gaze input, and compare it to the Twitter in a conventional browser interface with gaze-based mouse and keyboard emulation. We conducted an experimental study, which indicates a significantly better subjective user experience for the gazeadapted approach. Based on the results, we argue the need of user interfaces interacting directly to eye input to provide an improved user experience, more specifically in the field of accessibility.
18.

Vangelis Oikonomou, George Liaros, Spiros Nikolopoulos and Ioannis Kompatsiaris, Sparse Bayesian Learning for Multiclass Classification with application to SSVEP- BCI, 7th Graz Brain-Computer Interface Conference, September 18th – 22nd, 2017, Graz, Austria

Abstract Sparse Bayesian Learning (SBL) is a basic tool of machine learning. In this work, multiple linear regression models under the SBL framework (namely MultiLRM), are used for the problem of multiclass classification. As a case study we apply our method to the detection of Steady State Visual Evoked Potentials (SSVEP), a problem we encounter into the Brain Computer Interface (BCI) concept. The multiclass classification problem is decomposed into multiple regression problems. By solving these regression problems, a discriminant feature vector is learned for further processing. Furthermore by adopting the kernel trick the model is able to reduce its computational cost. To obtain the regression coefficients of each linear model, the Variational Bayesian framework is adopted. Extensive comparisons are carried out between the MultiLRM algorithm and several other competing methods. The experimental results demonstrate that the MultiLRM algorithm achieves better performance than the competing algorithms for SSVEP classification, especially when the number of EEG channels is small.
19.

Sofia Fountoukidou, Jaap Ham, Cees Midden, and Uwe Matzat, Using Tailoring to Increase the Effectiveness of a Persuasive Game-Based Training for Novel Technologies, Proceedings of the Personalization in Persuasive Technology Workshop, Persuasive Technology 2017, Amsterdam, The Netherlands, April 2017

Abstract A vast majority of people with motor disabilities cannot be part of the today’s digital society, due to the difficulties they face in using conventional interfaces (i.e., mouse and keyboard) for computer operation. The MAMEM project aims at facilitating the social inclusion of these people by developing a technology that allows computer operation, solely by using the eyes and mind. However, training is one of the key factors affecting the users’ technology acceptance. Game-based computer training including persuasive strategies could be an effective way to influence user beliefs and behaviours regarding a novel system. Tailoring these strategies to an individual level is a promising way to increase the effectiveness of a persuasive game. In the current paper, we briefly discuss the theoretical development of a persuasive game-based training for the MAMEM technology, as well as how we used tailored communication strategies to further enhance user technology acceptance. The development of such a tailored persuasive game will be essential for increasing acceptance and usage of assistive technology but also for the scientific insights in personalization of persuasion.