Towards Noninvasive Hybrid Brain–Computer Interfaces: Framework, Practice, Clinical Application, and Beyond

Muller-Putz, Gernot,Leeb, Robert,Tangermann, Michael,Hohne, Johannes,Kubler, Andrea,Cincotti, Febo,Mattia, Donatella,Rupp, Rudiger,Muller, Klaus-Robert,Del R Millan, Jose IEEE 2015 Proceedings of the Institute of Electrical and Ele Vol.103 No.6

<P>In their early days, brain-computer interfaces (BCIs) were only considered as control channel for end users with severe motor impairments such as people in the locked-in state. But, thanks to the multidisciplinary progress achieved over the last decade, the range of BCI applications has been substantially enlarged. Indeed, today BCI technology cannot only translate brain signals directly into control signals, but also can combine such kind of artificial output with a natural muscle-based output. Thus, the integration of multiple biological signals for real-time interaction holds the promise to enhance a much larger population than originally thought end users with preserved residual functions who could benefit from new generations of assistive technologies. A BCI system that combines a BCI with other physiological or technical signals is known as hybrid BCI (hBCI). In this work, we review the work of a large scale integrated project funded by the European commission which was dedicated to develop practical hybrid BCIs and introduce them in various fields of applications. This article presents an hBCI framework, which was used in studies with nonimpaired as well as end users with motor impairments.</P>

Editorial IEEE Brain Initiative Special issue on BMI/BCI Systems

Muller, Klaus-Robert,Carmena, Jose M. IEEE 2017 IEEE transactions on neural systems and rehabilita Vol.25 No.10

<P>The objective of this special issue is to address and disseminate state-of-the-art research and development in BMI/BCI Systems. It contains a selection of some of the latest technical and paradigmatic developments for invasive BMI and noninvasive BCI systems, both, as reviews and dedicated technical papers. The manuscripts were chosen after a careful peer review process for which we are indebted to the voluntary expert reviewers.</P>

Correction to “Concurrent Adaptation of Human and Machine Improves Simultaneous and Proportional Myoelectric Control” [Jul 15 618-627]

Hahne, Janne M.,Dahne, Sven,Hwang, Han-Jeong,Muller, Klaus-Robert,Parra, Lucas C. IEEE 2015 IEEE transactions on neural systems and rehabilita Vol.23 No.6

<P>In the above-named work [ibid., vol. 23, no. 4, pp. 618–627, Jul. 2015], the affiliation for Klaus-Robert Mueller should have appeared as follows: K-R. Mueller is with the Machine Learning Laboratory, Berlin Institute of Technology, D-10587 Berlin, Germany, and also with the Bernstein Center for Computational Neuroscience (BCCN), D-10587 Berlin, Germany, and also with the Department of Brain and Cognitive Engineering, Korea University, Anam-dong, Seongbuk-gu, Seoul 136-713, Korea; (e-mail: klaus-robert.mueller@tu-berlin.de).</P>

Motion-Based Rapid Serial Visual Presentation for Gaze-Independent Brain-Computer Interfaces

IEEE 2018 IEEE transactions on neural systems and rehabilita Vol.26 No.2

<P>Most event-related potential (ERP)-based brain–computer interface (BCI) spellers primarily use matrix layouts and generally require moderate eye movement for successful operation. The fundamental objective of this paper is to enhance the perceptibility of target characters by introducing motion stimuli to classical rapid serial visual presentation (RSVP) spellers that do not require any eye movement, thereby applying them to paralyzed patients with oculomotor dysfunctions. To test the feasibility of the proposed motion-based RSVP paradigm, we implemented three RSVP spellers: 1) fixed-direction motion (FM-RSVP); 2) random-direction motion (RM-RSVP); and 3) (the conventional) non-motion stimulation (NM-RSVP), and evaluated the effect of the three different stimulation methods on spelling performance. The two motion-based stimulation methods, FM- and RM-RSVP, showed shorter P300 latency and higher P300 amplitudes (<I>i.e.</I>, 360.4–379.6 ms; 5.5867– <TEX>$5.7662~\mu {V}$</TEX>) than the NM-RSVP (<I>i.e.</I>, 480.4 ms; <TEX>$4.7426~\mu {V}$</TEX>). This led to higher and more stable performances for FM- and RM-RSVP spellers than NM-RSVP speller (<I>i.e.</I>, 79.06±6.45% for NM-RSVP, 90.60±2.98% for RM-RSVP, and 92.74±2.55% for FM-RSVP). In particular, the proposed motion-based RSVP paradigm was significantly beneficial for about half of the subjects who might not accurately perceive rapidly presented static stimuli. These results indicate that the use of proposed motion-based RSVP paradigm is more beneficial for target recognition when developing BCI applications for severely paralyzed patients with complex ocular dysfunctions.</P>

Wasserstein Stationary Subspace Analysis

IEEE 2018 IEEE journal of selected topics in signal processi Vol.12 No.6

<P>Learning under nonstationarity can be achieved by decomposing the data into a subspace that is stationary and a nonstationary one [stationary subspace analysis (SSA)]. While SSA has been used in various applications, its robustness and computational efficiency have limits due to the difficulty in optimizing the Kullback-Leibler divergence based objective. In this paper, we contribute by extending SSA twofold: we propose SSA with 1) higher numerical efficiency by defining analytical SSA variants and 2) higher robustness by utilizing the Wasserstein-2 distance (Wasserstein SSA). We show the usefulness of our novel algorithms for toy data demonstrating their mathematical properties and for real-world data 1) allowing better segmentation of time series and 2) brain–computer interfacing, where the Wasserstein-based measure of nonstationarity is used for spatial filter regularization and gives rise to higher decoding performance.</P>

German Participation of International Exhibitions & Exhibition Concept of German Pavilion in Yeosu-Expo 2012

Dietmar Schmitz,Robert Muller 국제지역연구학회 2012 국제지역학논총 Vol.5 No.1

Low-temperature formation of source–drain contacts in self-aligned amorphous oxide thin-film transistors

Manoj Nag,Robert Muller,Soeren Steudel,Steve Smout,Ajay Bhoolokam,Kris Myny,Sarah Schols,Jan Genoe,Brian Cobb,Abhishek Kumar,Gerwin Gelinck,Yusuke Fukui,Guido Groeseneken,Paul Heremans 한국정보디스플레이학회 2015 Journal of information display Vol.16 No.2

We demonstrated self-aligned amorphous-Indium-Gallium-Zinc-Oxide (a-IGZO) thin-film transistors (TFTs) where the source–drain (S/D) regions were made conductive via chemical reduction of the a-IGZO via metallic calcium (Ca). Due to the higher chemical reactivity of Ca, the process can be operated at lower temperatures. The Ca process has the additional benefit of the reaction byproduct calcium oxide being removable through a water rinse step, thus simplifying the device integration. The Ca-reduced a-IGZO showed a sheet resistance (RSHEET) value of 0.7 k/sq., with molybdenum as the S/D metal. The corresponding a-IGZO TFTs exhibited good electrical properties, such as a field-effect mobility (μFE) of 12.0 cm2/(V s), a subthreshold slope (SS−1) of 0.4 V/decade, and an on/off current ratio (ION/OFF) above 108.

Why Does a Hilbertian Metric Work Efficiently in Online Learning With Kernels?

Yukawa, Masahiro,Muller, Klaus-Robert IEEE Signal Processing Society 2016 IEEE signal processing letters Vol.23 No.10

<P>The autocorrelation matrix of the kernelized input vector is well approximated by the squared Gram matrix (scaled down by the dictionary size). This holds true under the condition that the input covariance matrix in the feature space is approximated by its sample estimate based on the dictionary elements, leading to a couple of fundamental insights into online learning with kernels. First, the eigenvalue spread of the autocorrelation matrix relevant to the hyperplane projection along affine subspace algorithm is approximately a square root of that for the kernel normalized least mean square algorithm. This clarifies the mechanism behind fast convergence due to the use of a Hilbertian metric. Second, for efficient function estimation, the dictionary needs to be constructed in general by taking into account the distribution of the input vector, so as to satisfy the condition. The theoretical results are justified by computer experiments.</P>

Accurate Maximum-Margin Training for Parsing With Context-Free Grammars

Bauer, Alexander,Braun, Mikio,Muller, Klaus-Robert IEEE 2017 IEEE transactions on neural networks and learning Vol.28 No.1

<P>The task of natural language parsing can naturally be embedded in the maximum-margin framework for structured output prediction using an appropriate joint feature map and a suitable structured loss function. While there are efficient learning algorithms based on the cutting-plane method for optimizing the resulting quadratic objective with potentially exponential number of linear constraints, their efficiency crucially depends on the inference algorithms used to infer the most violated constraint in a current iteration. In this paper, we derive an extension of the well-known Cocke-Kasami-Younger (CKY) algorithm used for parsing with probabilistic context-free grammars for the case of loss-augmented inference enabling an effective training in the cutting-plane approach. The resulting algorithm is guaranteed to find an optimal solution in polynomial time exceeding the running time of the CKY algorithm by a term, which only depends on the number of possible loss values. In order to demonstrate the feasibility of the presented algorithm, we perform a set of experiments for parsing English sentences.</P>

M3BA: A Mobile, Modular, Multimodal Biosignal Acquisition Architecture for Miniaturized EEG-NIRS-Based Hybrid BCI and Monitoring

von Luhmann, Alexander,Muller, Klaus-Robert IEEE 2017 IEEE Transactions on Biomedical Engineering Vol.64 No.6

<P>Objective: For the further development of the fields of telemedicine, neurotechnology, and brain-computer interfaces, advances in hybrid multimodal signal acquisition and processing technology are invaluable. Currently, there are no commonly available hybrid devices combining bioelectrical and biooptical neurophysiological measurements [here electroencephalography (EEG) and functional near-infrared spectroscopy (NIRS)]. Our objective was to design such an instrument in a miniaturized, customizable, and wireless form. Methods: We present here the design and evaluation of a mobile, modular, multimodal biosignal acquisition architecture (M3BA) based on a high-performance analog front-end optimized for biopotential acquisition, a microcontroller, and our openNIRS technology. Results: The designed M3BA modules are very small configurable high-precision and low-noise modules (EEG input referred noise @ 500 SPS 1.39 μV<SUB>pp</SUB>, NIRS noise equivalent power NEP<SUB>750 nm</SUB> = 5.92 pW<SUB>pp</SUB>, and NEP<SUB>850 nm</SUB> = 4.77 pW<SUB>pp</SUB>) with full input linearity, Bluetooth, 3-D accelerometer, and low power consumption. They support flexible user-specified biopotential reference setups and wireless body area/sensor network scenarios. Conclusion: Performance characterization and in-vivo experiments confirmed functionality and quality of the designed architecture. Significance: Telemedicine and assistive neurotechnology scenarios will increasingly include wearable multimodal sensors in the future. The M3BA architecture can significantly facilitate future designs for research in these and other fields that rely on customized mobile hybrid biosignal modal biosignal acquisition architecture (M3BA), multimodal, near-infrared spectroscopy (NIRS), wireless body area network (WBAN), wireless body sensor network (WBSN).</P>