Neural Engineering

Bin He Editor
Neural
Engineering
Third Edition
Neural Engineering
Bin He
Editor
Neural Engineering
Third Edition
Editor
Bin He
Department of Biomedical Engineering
Carnegie Mellon University
Pittsburgh, PA, USA
ISBN 978-3-030-43394-9 ISBN 978-3-030-43395-6 (eBook)

https://doi.org/10.1007/978-3-030-43395-6
© Springer Nature Switzerland AG 2013, 2020

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole
or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way,
and transmission or information storage and retrieval, electronic adaptation, computer software,
or by similar or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are
exempt from the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in
this book are believed to be true and accurate at the date of publication. Neither the publisher
nor the authors or the editors give a warranty, express or implied, with respect to the material
contained herein or for any errors or omissions that may have been made. The publisher remains
neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
There has been tremendous progress in the engineering of tools designed to

interact with the nervous systems—from signal detection and processing to
restoration and enhancement of function. Neural engineering (or its equiva-
lent, neuroengineering) is a rapidly expanding interdisciplinary field bridging
neuroscience and engineering. It spans cellular, tissue, and systems level
research, and has become a core discipline within biomedical engineering and
beyond. This book is aimed at serving as a textbook for undergraduate and
graduate level courses in neural engineering within a biomedical engineering
or bioengineering curriculum. It is our intent to provide a comprehensive
review of the principles, concepts, theories, methods, and state-of-the-art
research in selected areas of neural engineering. It is also suitable as an
introduction to engineers or neuroscientists who are interested in entering the
field of neural engineering or acquiring knowledge about the current state of
the art in this rapidly developing discipline.
Chapter 1 provides a general introduction to human neuroanatomy and
neurophysiology. The chapter was written mainly for readers with back-
grounds in engineering and physical sciences. While readers who are familiar
with neuroanatomy and neurophysiology may skip this chapter, it will be
useful for those with little to no previous exposure or as a review. The chapter
includes over 60 original figures drawn for educational purposes.
Biopotential measurement, which yields information about the brain and
the peripheral nervous system, remains an important area of research in
neural engineering for the recording of both neuronal and neural circuit level
behavior. Chapter 2 covers the basic principles and techniques for electrodes
that can be used for neural recording and stimulation, as well as the bioelectric
circuits that can be used for various neural measurements. Various types
of electrodes required for measurement, and the most appropriate circuit
architectures needed to amplify and process these signals, are discussed.
An important aspect of neural engineering is to properly analyze and inter-
pret neural signals—a step that plays a vital role for sensing and controlling
neural prostheses and other neural interfacing devices, as well as understand-
ing the mechanisms of neural systems. Chapter 3 provides a comprehensive
review of electroencephalography (EEG) signal processing with a focus on
time and frequency domain analyses. After a general overview of EEG, time,
frequency, and wavelet signal processing techniques are reviewed in detail.
v
vi Preface
Brain-computer interfaces (BCIs) have emerged as a novel technology

that bridges the brain with external devices. BCIs have been developed to
decode human intention, leading to direct brain control of a computer or
device, bypassing the neuromuscular pathway. Bidirectional brain-computer
interfaces not only allow device control but also open the door for modulating
the central nervous system through neural interfacing. Chapter 4 provides
an introduction and comprehensive review of the concepts, principles, and
methods of BCI technology. Using various recorded brain signals that reflect
the “intention” of the brain, BCI systems have shown the capability to
control external devices, such as computers and robots. This chapter reviews
the concepts, principles, and various building blocks of BCIs, from signal
acquisition, signal processing, feature extraction, and feature translation, to
device control, and various applications. Examples of noninvasive BCIs are
discussed to provide an in-depth understanding of the noninvasive BCI tech-
nology. Chapter 5 reviews the intracortical BCI, or brain-machine-interface
(BMI), and discusses the four basic components of an intracortical BMI: intra-
cortical neural recording, a decoding algorithm, an output device, and sensory
feedback. The unique features of intracortical signals and the algorithms
for processing and decoding them are discussed in detail. Brain-controlled
functional electrical stimulation that can directly activate a patient’s own
paralyzed muscles, reanimating their arm, is also reviewed.
Neuromodulation, a principle discipline within neuroengineering research,
has rapidly become an important option in treating a variety of neurolog-
ical and mental disorders. It also represents an encoding technology for
bidirectional neural interfaces. The following three chapters cover three
widely applied neuromodulation techniques in clinical studies: deep brain
stimulation (DBS), transcranial magnetic stimulation (TMS), and transcranial
electrical stimulation (tES). Chapter 6 provides an overview of the principles,
electrodes and instrumentation used, mechanisms, and applications of DBS.
DBS is a neurosurgical technique that consists of the continuous delivery of
electrical pulses through chronically implanted deep electrodes connected
to a neurostimulator, programmable in amplitude, pulse width, frequency,
and stimulation channels. This chapter discusses engineering approaches that
have the potential to improve clinical outcomes of DBS, focusing on the
development of novel temporal patterns, innovative electrode designs, com-
putational models to guide stimulation, closed-loop DBS, emerging clinical
indications, and future noninvasive strategies. Chapter 7 provides an in-depth
coverage of TMS, a noninvasive neuromodulation technique that is based
on electromagnetic induction principles. This chapter explains the principles
of TMS devices including the electrical circuit topologies and efficiency of
the pulse generator, as well as the design of the stimulation coil; reviews
the underlying physics and its modeling, including the magnetic field of the
coil and the impact of the subject’s anatomy on the induced electric field;
and describes the biophysics of neuronal activation due to TMS. The chapter
concludes with an overview of stimulation paradigms encompassing single-
pulse, paired-pulse, and repetitive TMS, along with their applications in basic
research and the clinic. Chapter 8 covers tES neuromodulation, where electric
current is applied to electrodes on the head to modulate brain function. Various
Preface vii
tES devices are applied to conditions spanning neurological and psychiatric

disorders, neurorehabilitation after injury, and altering cognition in healthy
adults. All tES devices share certain common features including a waveform
generator, disposable electrodes or electrolyte, and headgear or an adhesive
to position the electrodes. Computational current flow models which support
device design and programming by informing dose selection for a given
outcome are also addressed.
Chapter 9 discusses optogenetic neural stimulation, which has been widely
used in animal models for neuroscientific research. In optogenetics, the
activity of specific cell types, such as neurons or astrocytes, is modulated
by exposing target cells to pulses of appropriate wavelengths. Prior to op-
togenetic stimulation experiments, specific genetic constructs are delivered
to target cells in order to express light activated ion-channels or ion-pumps
and produce light sensitivity. Unique features of optogenetic stimulation,
including specific cell-type targeting or bidirectional control of cellular activ-
ity, have allowed researchers to use this method in studying brain networks,
finding projections, or dissociating circuitries of neurological and psychiatric
disorders.
Chapter 10 deals with peripheral neural interfacing. This chapter examines
the possibility of detecting peripheral nerve signals and using these voluntary
signals to restore function in patients suffering from stroke, amputation,
or paralysis. The ability to obtain signals from peripheral nerves would
have significant benefits such as detection of motor intent in patients with
amputation. Similarly, decoding signals from the autonomic nervous system
would allow continuous monitoring of organ function. In this chapter, various
types of neural interfaces such as cuff electrodes, intra- and extrafascicular
electrodes, as well as regeneration electrodes, are reviewed, with a focus on
the flat interface nerve electrode.
Neuroimaging has played an important role in the understanding of neural
functions and in the aiding of clinical diagnoses and treatments. Recent
developments in functional neuroimaging have led to important tools for
the better understanding of, as well as aiding in the restoration of, neural
functions. Chapters 11, 12, 13, 14, and 15 cover five important approaches in
neuroimaging. Chapter 11 discusses the principles, methods, and applications
of functional MRI, with an emphasis on its biophysical and physiological
basis, experimental design and analysis methods, and applications to human
and animal studies. The use of functional MRI in combination with carefully
designed task paradigms has enabled scientists to map perceptual, cognitive,
and behavioral functions onto brain regions and networks. Spontaneous
activity observed with functional MRI in task-free resting states has also
been used to reveal intrinsic functional networks that collectively depict
the brain’s functional architecture or connectome. Chapter 12 reviews an
emerging neuroimaging technique, photoacoustic tomography (PAT), which
is playing an increasingly important role in brain studies. PAT’s unique
scalability provides an opportunity to examine the brain at multiple spatial
scales using the same contrast mechanism, bridging microscopic insights with
macroscopic observations of the brain. This chapter reviews the principles of
PAT, presents the major implementations, and summarizes the representative
viii Preface
neuroscientific applications. Existing challenges are also discussed in trans-

lating PAT to human brain imaging, elucidating its potential promise. Chap-
ter 13 reviews the basic principles and applications of electrophysiological
mapping and source imaging. Applying electromagnetic theory and signal
processing techniques, electrophysiological neuroimaging offers high tem-
poral resolution and good spatial resolution in mapping brain dynamics from
EEG or magnetoencephalogram (MEG). Knowledge of the spatiotemporal
dynamics of source distributions associated with neural activity aids in the
understanding of the mechanisms of neural systems, and provides a noninva-
sive probe of the complex central nervous system. Multimodal neuroimaging,
which integrates functional MRI and EEG/MEG, is also discussed. Chapter 14
covers functional and causal connectivity analysis and imaging, with the
goal of not only discovering where brain activity occurs but also how neural
information processing is performed. The concepts of functional and causal
connectivity are introduced, and mathematic models behind the causality
analysis are presented. Causal connectivity approaches using various signals
are discussed, with focus on translations to human subjects using electrophys-
iological recordings such as EEG, MEG, or electrocorticography. Chapter 15
covers deep learning models and their applications to brain image analysis.
Deep learning has received increasing attention in brain image analysis. In
this chapter, the preprocessing steps for brain images and the fundamental
concepts of deep neural networks are first introduced. After that, four typical
types of deep neural networks used for brain image analysis are discussed,
including (i) convolutional neural networks (CNNs) and their variants, (ii)
recurrent neural networks (RNNs) and their variants, (iii) autoencoders, and
(iv) generative adversarial networks (GANs) and their variants, as well as
their applications in brain image classification, segmentation, registration, and
image synthesis/augmentation.
Computational models of neural systems provide a quantitative perspec-
tive in neurophysiology and neural engineering by coupling experimental
observations to mathematical formulations. Chapters 16, 17, 18, and 19
deal with neural modeling, which is an important tool for understanding
neural mechanisms. Chapter 16 provides an introduction to neuronal mod-
eling, laying the foundation for several basic models and surveying key
topics. These include the properties of electrically excitable membranes, the
Hodgkin–Huxley model, and how such a model can be extended to describe
a variety of excitable membrane behaviors, including axonal propagation,
dendritic processing, and synaptic communication. Chapter 16 also covers
mathematical models that replicate basic neural behaviors through more
abstract mechanisms and explores efforts to extend single-neuron models
to the network level. Chapter 17 overviews the linear systems theory as a
useful tool for capturing biophysically relevant parameters of neural activity
and connectivity, and for analytical and numerical study. This chapter begins
with a brief overview of state-space representations and linearization of neural
models for nonlinear dynamical systems. After deriving core concepts in the
theory of linear systems such as the impulse and controlled responses to
external stimuli, and achieving desired state transitions, controllability, and
minimum energy control, recent advances are discussed in the application of
Preface ix
linear systems theory to structural and functional brain data across multiple
spatial and temporal scales, along with methodological considerations and
limitations. Chapter 18 focuses on modeling and analysis of neuronal pop-
ulations. This body of work has opened up avenues of inquiry that range
from primarily theoretical (How do neurons represent information?) to highly
practical (How can we design a robust BCI?). This chapter reviews the history
of analytic approaches and neuroscience research aimed at deciphering the
population code, from early work with single neurons and pairs to more recent
approaches leveraging the newest technology to measure tens to hundreds
of neurons simultaneously. Chapter 19 focuses on the clinical applications
of modeling and machine intelligence to forecast seizures in epilepsy pa-
tients. This chapter discusses computational modeling and machine learning
algorithms, in the context of seizure prediction and detection, as well as in
other applications, such as antiepileptic drug efficacy. Also discussed are
common methods of feature extraction––particularly focusing on wavelet
phase coherence, and cross-frequency coupling.
The retina represents an important component of the peripheral nervous
system. Chapters 20 and 21 discuss retinal prostheses and bioengineering.
As a successful neural interface, retinal prosthesis can provide a sense of
sight to people with severe visual impairment due to retinal photoreceptor
degeneration. Chapter 20 discusses the concepts and applications of retinal
prosthesis in patients, and clinical research studies that have shown that while
retinal implants can provide people with improved navigational skills, they
cannot restore normal reading abilities. Improvements in visual acuity may be
possible through denser electrode arrays or image processing strategies that
yield more focused, natural responses from the retina. Chapter 21 provides
a comprehensive review of the neural structure and function of the retina,
including its associated vasculature, major retinal diseases, and the modeling
and engineering approaches to understanding retinal physiology and patho-
physiology. The mathematical modeling of neural responses in the retinal
microenvironment as well as the restoration of retinal function is reviewed.
Because of its simpler structure and lack of significant neurofeedback, the
retina has long served as a model for understanding complex parts of the
nervous system and as a mainstay for the translation of neuroscience discov-
eries to clinical applications, embodying one of the unique features of neural
engineering research.
Tissue engineering is the use of engineering methods to replace, replicate,
or improve biological tissues. Neural tissue engineering involves the inte-
grated use of biomaterials, cellular engineering, and drug delivery technolo-
gies with the purpose of protecting, repairing, or regenerating cells and tissues
of the nervous system. Through the introduction of biochemical, topographic,
immunomodulatory, and other types of cues, tissues can be therapeutically
controlled to direct growth and tissue function toward overcoming biological
constraints on tissue repair and regeneration. These strategies can be applied
when injury or disease occurs in the brain, spinal cord, or peripheral nerves, or
to improve chronic functionality of implantable neural interfaces. Chapter 22
presents an overview of neural tissue engineering using examples of thera-
peutic systems including nerve conduits, implantable hydrogels, the delivery
x Preface
of neurotrophic factors and stem cells, genetic approaches to tissue engineer-

ing, immunomodulation, and electrical stimulation.
Through this collection of carefully selected chapters written by world
renowned experts in neural engineering, we wish to provide a general picture
of the field and to outline the fundamental underpinnings that will make
it a core discipline in biomedical engineering, while conveying many of
its exciting aspects. Neural engineering not only represents an interface
between neuroscience and engineering, but, more importantly, has led to great
advancements in basic and clinical neuroscience, many of which would not
have been possible without the integration with engineering principles.
This book is a collective effort by researchers and educators who specialize
in the field of neural engineering. I am very grateful to them for taking the time
out of their busy schedules and for their patience during the entire process.
It should be noted that the field of neural engineering is developing rapidly
and that there are many worthwhile topics that could not be included in this
book, as the book aims to serve as textbook for a semester-long or year-long
neural engineering course. Nevertheless, our intention is to provide a general
overview that covers the basics and important areas of neural engineering
research. A unique feature of this edition is to provide a rich set of homework
problems that can be used for classroom teaching. Instructors may contact the
Publishers for access to the solutions to the homework problems.
I am indebted to Merry Stuber and Murugesan Tamilselvan of Springer
for their support and great effort during this project. I would also like to ac-
knowledge the National Institutes of Health (R01EB021027, RF1MH114233,
R01NS096761, R01AT009263) and Carnegie Mellon University (Trustee
Professorship in Biomedical Engineering) for partial financial support.
Pittsburgh, PA, USA Bin He

Contents
1 Introduction to Neurophysiology . . . . . . . . . . . . . . . . . . . . . . . . . 1
Paul A. Iaizzo
2 Biopotential Measurements and Electrodes . . . . . . . . . . . . . . . . 65
Abraham Akinin, Akshay Paul, Jun Wang, Alessio Buccino,
and Gert Cauwenberghs
3 EEG Signal Processing: Theory and Applications . . . . . . . . . . 97
David L. Sherman and Nitish V. Thakor
4 Brain–Computer Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Bin He, Han Yuan, Jianjun Meng, and Shangkai Gao
5 Intracortical Brain–Machine Interfaces . . . . . . . . . . . . . . . . . . . 185
Emily R. Oby, Jay A. Hennig, Aaron P. Batista, Byron M. Yu,
and Steven M. Chase
6 Deep Brain Stimulation: Emerging Technologies
and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
Aysegul Gunduz
7 Transcranial Magnetic Stimulation: Principles
and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
Lari M. Koponen and Angel V. Peterchev
8 Transcranial Electrical Stimulation . . . . . . . . . . . . . . . . . . . . . . . 271
Dennis Q. Truong, Niranjan Khadka, and Marom Bikson
9 Optogenetics: Novel Brain Interface Technology That
Originates in Bioprospecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
Ramin Pashaie
10 Selective Chronic Recording in the Peripheral Nervous
System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Dominique M. Durand and Thomas Eggers
11 Functional Magnetic Resonance Imaging . . . . . . . . . . . . . . . . . . 331
Zhongming Liu and Jiayue Cao
12 Photoacoustic Tomography of Neural Systems . . . . . . . . . . . . . 349
Lei Li, Junjie Yao, and Lihong V. Wang
xi
xii Contents
13 Electrophysiological Mapping and Source Imaging . . . . . . . . 379

Bin He, Lei Ding, and Abbas Sohrabpour
14 Exploring Functional and Causal Connectivity
in the Brain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
Mingzhou Ding and Bin He
15 Deep Learning Models with Applications to Brain Image
Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433
Dinggang Shen, Luping Zhou, and Mingxia Liu
16 Neural Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
Michael N. Economo, Jad Noueihed, Joan J. Martinez,
and John A. White
17 Linear Dynamics and Control of Brain Networks . . . . . . . . . . 497
Jason Z. Kim and Danielle S. Bassett
18 Deciphering the Neuronal Population Code . . . . . . . . . . . . . . . 519
Sanjeev B. Khanna and Matthew A. Smith
19 Machine Intelligence-Based Epileptic Seizure Forecasting . . 535
Vasily Grigorovsky, Uilki Tufa, Daniel Jacobs,
and Berj L. Bardakjian
20 Retinal Prosthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567
James Weiland and Mark S. Humayun
21 Retinal Bioengineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581
Robert A. Linsenmeier and John B. Troy
22 Neural Tissue Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639
Johnathan G. Lyon, Lohitash Karumbaiah,
and Ravi V. Bellamkonda
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669
Contributors
Abraham Akinin Department of Bioengineering, UC San Diego, La Jolla,

CA, USA
Nanovision Biosciences, La Jolla, CA, USA
Berj L. Bardakjian Institute of Biomaterials and Biomedical Engineering,
University of Toronto, Toronto, ON, Canada
Edward S. Rogers Sr. Department of Electrical & Computer Engineering,
Danielle S. Bassett Departments of Bioengineering, Electrical and Systems
Engineering, Physics and Astronomy, Neurology, and Psychiatry, University
of Pennsylvania, Philadelphia, PA, USA
Santa Fe Institute, Santa Fe, NM, USA
Aaron P. Batista Department of Bioengineering, University of Pittsburgh,
Pittsburgh, PA, USA
Center for the Neural Basis of Cognition, University of Pittsburgh and
Carnegie Mellon University, Pittsburgh, PA, USA
University of Pittsburgh Brain Institute and Systems Neuroscience Center,
Pittsburgh, PA, USA
Ravi V. Bellamkonda Department of Biomedical Engineering, Pratt School
of Engineering, Duke University, Durham, NC, USA
Marom Bikson Department of Biomedical Engineering, The City College
of New York of CUNY, New York, NY, USA
Alessio Buccino Department of Bioengineering, UC San Diego, La Jolla,
CA, USA
Centre for Integrative Neuroplasticity, University of Oslo, Oslo, Norway
Jiayue Cao Department of Biomedical Engineering, University of Michi-
gan, Ann Arbor, MI, USA
Gert Cauwenberghs Department of Bioengineering, UC San Diego, La
Jolla, CA, USA
Steven M. Chase Center for the Neural Basis of Cognition, University of
Pittsburgh and Carnegie Mellon University, Pittsburgh, PA, USA
Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA, USA
Department of Biomedical Engineering, Carnegie Mellon University,
Pittsburgh, PA, USA
xiii
xiv Contributors
Lei Ding Stephenson School of Biomedical Engineering, University of Ok-

lahoma, Norman, OK, USA
Mingzhou Ding The J. Crayton Pruitt Family Department of Biomedical
Engineering, University of Florida, Gainesville, FL, USA
Dominique M. Durand Neural Engineering Center, Department of
Biomedical Engineering, Case Western Reserve University, Cleveland, OH,
USA
Michael N. Economo Department of Biomedical Engineering, Boston Uni-
versity, Boston, MA, USA
Thomas Eggers Neural Engineering Center, Department of Biomedical En-
gineering, Case Western Reserve University, Cleveland, OH, USA
Shangkai Gao Department of Biomedical Engineering, Tsinghua Univer-
sity, Beijing, China
Vasily Grigorovsky Institute of Biomaterials and Biomedical Engineering,
Aysegul Gunduz J. Crayton Pruitt Family Department of Biomedical Engi-
neering, University of Florida, Gainesville, FL, USA
Bin He Department of Biomedical Engineering, Carnegie Mellon Univer-
sity, Pittsburgh, PA, USA
Jay A. Hennig Center for the Neural Basis of Cognition, University of
Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA,
USA
Mark S. Humayun Ophthalmology, Biomedical Engineering, and Integra-
tive Anatomical Sciences, Ginsburg Institute for Biomedical Therapeutics,
USC Roski Eye Institute, University of Southern California, Los Angeles,
CA, USA
Paul A. Iaizzo Visible Heart® Laboratories, Department of Surgery and the
Institute for Engineering in Medicine, University of Minnesota, Minneapolis,
MN, USA
Daniel Jacobs Institute of Biomaterials and Biomedical Engineering, Uni-
versity of Toronto, Toronto, ON, Canada
Lohitash Karumbaiah Regenerative Bioscience Center, University of
Georgia, Athens, GA, USA
Niranjan Khadka Department of Biomedical Engineering, The City Col-
lege of New York of CUNY, New York, NY, USA
Sanjeev B. Khanna Department of Bioengineering, University of
Pittsburgh, Pittsburgh, PA, USA
Jason Z. Kim Department of Bioengineering, University of Pennsylvania,
Philadelphia, PA, USA
Contributors xv
Lari M. Koponen Department of Psychiatry & Behavioral Sciences, Duke

University, Durham, NC, USA
Lei Li Caltech Optical Imaging Laboratory, Department of Electrical Engi-
neering and Andrew and Peggy Cherng Department of Medical Engineering,
California Institute of Technology, Pasadena, CA, USA
Robert A. Linsenmeier Department of Biomedical Engineering, North-
western University, Evanston, IL, USA
Department of Neurobiology, Northwestern University, Evanston, IL, USA
Department of Ophthalmology, Northwestern University, Chicago, IL, USA
Mingxia Liu Department of Radiology and BRIC, University of North
Carolina at Chapel Hill, Chapel Hill, NC, USA
Zhongming Liu Department of Biomedical Engineering, University of
Michigan, Ann Arbor, MI, USA
Department of Electrical Engineering and Computer Science, University of
Johnathan G. Lyon Department of Biomedical Engineering, Pratt School
of Engineering, Duke University, Durham, NC, USA
Joan J. Martinez Technology Ventures, Columbia University, New York
City, NY, USA
Jianjun Meng School of Mechanical Engineering, Shanghai Jiao Tong Uni-
versity, Shanghai, China
Jad Noueihed Department of Biomedical Engineering, Boston University,
Boston, MA, USA
Emily R. Oby Center for the Neural Basis of Cognition, University of
Department of Neurobiology, University of Pittsburgh, Pittsburgh, PA, USA
University of Pittsburgh Brain Institute and Systems Neuroscience Center,
Pittsburgh, PA, USA
Ramin Pashaie Electrical Engineering and Computer Science Department,
University of Wisconsin-Milwaukee, Milwaukee, WI, USA
Akshay Paul Department of Bioengineering, UC San Diego, La Jolla, CA,
USA
Angel V. Peterchev Department of Psychiatry & Behavioral Sciences, Duke
Department of Biomedical Engineering, Duke University, Durham, NC, USA
Department of Electrical & Computer Engineering, Duke University,
Durham, NC, USA
Department of Neurosurgery, Duke University, Durham, NC, USA
Dinggang Shen Department of Radiology and BRIC, University of North
David L. Sherman Johns Hopkins University, Baltimore, MD, USA
xvi Contributors
Matthew A. Smith Department of Biomedical Engineering and Neuro-

science Institute; Carnegie Mellon University, Pittsburgh, PA, USA
Abbas Sohrabpour Department of Biomedical Engineering, Carnegie Mel-
lon University, Pittsburgh, PA, USA
Nitish V. Thakor Johns Hopkins University, Baltimore, MD, USA
John B. Troy Department of Biomedical Engineering, Northwestern Uni-
versity, Evanston, IL, USA
Dennis Q. Truong Department of Biomedical Engineering, The City Col-
lege of New York of CUNY, New York, NY, USA
Uilki Tufa Institute of Biomaterials and Biomedical Engineering, University
of Toronto, Toronto, ON, Canada
Jun Wang Department of Bioengineering, UC San Diego, La Jolla, CA,
USA
Lihong V. Wang Caltech Optical Imaging Laboratory, Department of Elec-
trical Engineering and Andrew and Peggy Cherng Department of Medical
Engineering, California Institute of Technology, Pasadena, CA, USA
James Weiland Biomedical Engineering and Ophthalmology, University of
John A. White Department of Biomedical Engineering, Boston University,
Boston, MA, USA
Junjie Yao Photoacoustic Imaging Laboratory, Department of Biomedical
Engineering, Duke University, Durham, NC, USA
Byron M. Yu Center for the Neural Basis of Cognition, University of
Department of Biomedical Engineering, Carnegie Mellon University,
Pittsburgh, PA, USA
Department of Electrical and Computer Engineering, Carnegie Mellon Uni-
versity, Pittsburgh, PA, USA
Han Yuan Stephenson School of Biomedical Engineering, University of
Oklahoma, Norman, OK, USA
Luping Zhou School of Electrical and Information Engineering, The Uni-
versity of Sydney, Sydney, NSW, Australia
Introduction to Neurophysiology
1
Paul A. Iaizzo
Abstract Hypothalamus · Homeostasis ·

Electroencephalograms
Neurophysiology is a critical and exciting
topic to study and understand in great detail
for those working in any field associated
with neuroengineering—basic or applied 1.1 Overview of Neurons,
research, device design and development, Synapses, Neuronal Circuits,
and/or neurology or neurosurgical clinical and Central Nervous System
subspecialties. The purpose of this chapter Anatomy
is to provide a general introduction to the
field of neurophysiology, that is, a high-level Cells within the central nervous system (CNS)
overview of the anatomy and workings of are like most other cells in the human body and
the human central nervous system (CNS). contain various components/organelles, includ-
One can explore other sources to find more in- ing surface membranes (which contain ion chan-
depth discussions related to many of the topics nels and biochemical receptors), nuclei (contain-
introduced in this chapter as well as learn the ing chromosomes and DNA), mitochondria, ribo-
specifics of state-of-the-art neuroengineering somes, endoplasmic reticulum, Golgi complexes,
concepts related to each topic. lysosomes, etc. The cell populations defined as
nerve cells (neurons) are considered as the func-
Keywords tional units within the human nervous system; see
The brain · Spinal cord · Neurons · Action Fig. 1.1. These cells also typically have dendrites,
potentials · Neural circuits · Reflexes · axons, and axon terminals. Neurons under rest-
Sensory systems · Motor control · Posture · ing conditions have an electrical potential across
Equilibrium · Autonomic nervous system · their plasma membranes, with the inside of these
cells being negatively charged with respect to
the outside (extracellular spaces). This is defined
as the resting membrane potential, which ranges
Electronic Supplementary Material: The online version between −40 and −90 mV in healthy neurons;
of this chapter (https://doi.org/10.1007/978-3-030-43395- by convention, the extracellular fluid is assigned a
6_1) contains supplementary material, which is available voltage of zero. In general, the resting membrane
to authorized users.
potential can be considered to hold steady, unless
P. A. Iaizzo () altered by changes in local electrical currents.
Visible Heart® Laboratories, Department of Surgery and These potentials exist due to an excess of negative
the Institute for Engineering in Medicine, University of
Minnesota, Minneapolis, MN, USA
ions inside the cells and an excess of positive
e-mail: [email protected] ones on the outside. One can consider that it is
© Springer Nature Switzerland AG 2020 1

B. He (ed.), Neural Engineering, https://doi.org/10.1007/978-3-030-43395-6_1
2 P. A. Iaizzo
If the concentration gradient for any ion is

known, then the relative equilibrium potential
across the plasma membrane for that ion can
be calculated by means of the Nernst equation,
that is, one can estimate the electrical potential
necessary to balance a given ionic concentration
gradient across a membrane (the net flux for this
ion is zero). The Nernst equation is
Eion = 61/Z · log (Cout /Cin )
where Eion is the equilibrium potential for a given

ion (mV); Cin is the intracellular concentration
of the ion; Cout is the extracellular concentra-
tion of the ion; Z is the valence of the ion; and
61 is a constant value that takes into account
the universal gas constant, temperature (37 ◦ C),
and Faraday’s electrical constant. If each one of
these three main ions become totally permeable
across a given membrane, then ENa = +60 mV,
EK = −90 mV, and ECl = −80 mV. Note that
nerve cells have negative resting membrane po-
tentials, suggesting that it is primarily determined
by either the chloride or potassium ion distribu-
Fig. 1.1 Although nerve cells throughout the central ner- tions. Yet, by measurements of ion movements,
vous system take hundreds of unique forms and shapes, it has been shown that chloride ions are typi-
most of the cells have common cellular components. cally passively distributed across a given neu-
Shown here are the major structural features of an ide-
ron’s surface membrane, and thus, chloride cur-
alized neuron: dendrites (receiving synapses from other
cells), the cell body, the axon hillock, myelination, an rents have negligible roles under resting condi-
axon, and the axon terminals (forming synapses onto other tions. This leaves potassium as the dominant ion
cells) species in determining the overall resting mem-
brane potentials in most nerve cells. It should
be noted that neurons typically contain a vari-
the distribution of three major mobile ions across
ety of ion selective channels within their surface
a neuron’s plasma membrane that sets up the
membranes, with differing neuron types having
possibility for a change in potential: (1) Na+ with
unique compositions. The term gating is used to
145 mmol/L extracellular and 15 mmol/L intra-
refer to the triggered openings of such channels.
cellular concentrations; (2) Cl− with 100 mmol/L
More specifically, voltage-gated ion channels re-
extracellular and 7 mmol/L intracellular concen-
spond to changes in local membrane potentials
trations; and (3) K+ with 5 mmol/L extracellular
of a given cell, and ligand-gated ion channels
and 150 mmol/L intracellular concentrations. The
are those that respond to specific biochemical
excess of charged ions collects near the plasma
factors (receptor activated by agonists). Note that
membrane, and their movement during excitation
spontaneously active ion channels will elicit ran-
of the cell underlies the development of an action
dom frequencies of opening and closing, whereas
potential, which then propagates from the point
leak channels seem to be more continuously open
of excitation along the surface membranes (e.g.,
(though only allowing typically low ion flows).
down a neuron’s axon). See Fig. 1.2 for defini-
In addition to classifications based on control
tions of excitation states.
mechanisms, channels are also classified by their
1 Introduction to Neurophysiology 3
Fig. 1.2 Shown here is a

general action potential
waveform. Depolarizing,
repolarizing,
hyperpolarizing, and
overshoot changes in
membrane potential are
shown in relation to the
resting membrane potential
(horizontal red line)
ion selectivities (e.g., Na+ , K+ , Ca2+ , or cation populations beyond neurons that make up the
nonspecific) and/or the directions in which such CNS that are known to be vital for its proper
ions pass through them (e.g., inward or outward). function.
Action potentials are elicited in nerve cells due Such cells are grouped into a population
primarily to transient changes in the cellular per- known as glia cells (or neuroglia); as such they
meabilities of both Na+ and K+ ions. An initial play critical roles to maintain cerebral tissue
local electrical depolarization (i.e., the surface homeostasis, form myelin, and provide both
membrane reaches a threshold voltage of +10 support and protection for the brain’s neurons.
to +30 mV above the given resting potential) The main purpose of a myelin layer, or a myelin
then causes the transient openings of voltage- sheath around a nerve cell’s axon, is to increase
dependent Na channels. This brief (1–2 ms) in- the speed at which generated electrical impulses
crease in sodium permeability (conductance) fur- propagate (e.g., generator or action potentials).
ther depolarizes the cell and drives the mem- Myelin is essential for the proper functioning of
brane potential toward the sodium equilibrium the nervous system. The major subpopulations of
potential; shortly (within approximately a mil- glia are (1) astrocytes (or astroglia), which are
lisecond), these channels are actively inactivated. star-shaped cells that have been shown to provide
This depolarization, in turn, activates voltage- physical and nutritional support for neurons,
gated K channels, which allows for efflux from clean up brain “debris,” assist in the transport of
the cell and thus drives the membrane potential nutrients, regulate the contents of the extracellular
back toward the potassium equilibrium potential space, and/or form a structural scaffolding to
(more negative); see Fig. 1.2. This excitation can help hold neurons in place; (2) microglia which,
also be considered as typically self-propagating like astrocytes, are important in removing waste
(excite adjacent cell membrane areas, e.g., action from cellular debris; and (3) oligodendroglia cells
potential propagation down the nerve axon); see which provide the insulation (myelin) for neurons
Fig. 1.3. that lie within the CNS (i.e., enhance action
Importantly, neurons form connections potential propagation rates). Note that Schwann
between themselves (e.g., via synapses, chemical, cells provide insulation (myelin) to neurons
or electrical), and this is the primary mechanism that lie within the peripheral nervous system. It
for information transfer within the CNS (Figs. should also be emphasized that cell populations
1.4 and 1.5). Nevertheless, there are other cell within the brain are dynamic structures which
4 P. A. Iaizzo
Fig. 1.3 Shown here is the schematic representation of voltage-gated Na channels in the dark blue (far right)
the one-way propagation of an action potential down a portion of the axon, i.e., the action potentials propagated
nerve cell’s axon. Local currents generated within the cell rapidly down the axon. As in the initial segments of the
body subsequently resulted in an action potential being axon membranes, the Na current becomes near zero, and
generated in the far left region of the axon (known as the initiated voltage-gated K current will allow for repo-
the axon hillock). This excitation then propagated to the larization back to the original resting membrane potential
middle region of the axon which, in turn, activated the (e.g., −70 mV)
continually turn over their structural components as important CNS processes such as synaptic
and/or alter their shapes to create new neural plasticity and synaptogenesis. Furthermore, it is
connections; hence, the CNS elicits a high degree believed that glial cells play essential roles in
of “plasticity.” In the CNS, the nerve cell body the regulation of repair of neurons and neural
and dendrites receive most of their inputs from pathways after injury (i.e., physical injury or an
other neurons. It is the branching of the dendrites ischemic event). More recently, it has been shown
with greatly increases a given cell’s surface area: that astrocytes can communicate with neurons
some neurons can elicit as many as 400,000 and even modify the signals they send or receive.
dendrites. In addition, dendritic extensions called Therefore, glial cells can affect the processing of
dendritic spines further enhance these surface information as well as the signaling that occurs
areas and are highly dynamic processes. The at a given synapse (i.e., the sites of connections
presence of ribosomes and protein-synthesis between neurons).
machinery in these spines allows them to remodel In humans, approximately one-half of an in-
their shapes in responses to variations within dividual’s gene pool contributes to building the
synaptic activities, which in turn is considered to brain and its various cell populations. It is esti-
play key roles in complex CNS processes such as mated that a healthy adult contains 1010 neurons
memory and learning. (10,000,000,000 neurons). Furthermore, the mul-
In the average human brain, it is estimated that titude of synapses between these neurons utilize
there is roughly one glia for every neuron, with a on the order of 100 different neurotransmitters,
ratio of about two neurons for every three glia in second messengers, and growth factors. Addi-
the cerebral cortex or gray matter [2]. Therefore, tionally, one can identify some 300 different re-
the brain’s populations of glial cells should be ceptor molecules in the brain; to further compli-
regarded more as partners to neurons, to optimize cate matters, these receptors can be configured as
overall brain function. Glial cells surround the receptor complexes (i.e., dimers, trimers, or more
somas, axons, and dendrites of neurons and pro- complex heteromers). Thus, one can only imagine
vide them with both physical and metabolic sup- the daunting task it would be to model such a
port. Additionally, glia are also considered crucial system from a neuroengineering standpoint.
for normal nervous system development, as well
Fig. 1.5 A schematic representation of the general

anatomy of a chemical synapse. Shown is a single presy-
naptic terminal adjacent to the postsynaptic membrane of
the post-somatic neuron. The region between the two cells,
the synaptic cleft, will have typical distances of 200 to
300 angstroms. The presynaptic terminal possesses vesi-
Fig. 1.4 Shown here are three different nerve cells that cles containing neurotransmitter molecules which, when
network together to spread information via their relative released into the synaptic cleft, will bind to the receptor
synaptic connections. For example, the arrows indicate protein (ligand channels) of the postsynaptic membrane.
the potential propagation of excitatory information via the Typically, when activated, this allows for the gating of ions
spread of action potential from one cell to the next by through these channels and thus focal voltage changes.
inducing voltage changes within the cell bodies of the This signaling can either be excitatory (i.e., an excitatory
adjacent cell postsynaptic potential, EPSP) or inhibitory (inhibitory
postsynaptic potential, IPSP)
In general, neurons can be divided into three

main functional classes: afferents, efferents, and tenuate overly intense activity, emphasize con-
interneurons. Afferent neurons convey informa- trasts, maintain rhythms (e.g., involuntary control
tion (sensory) from the tissues and organs of the of respiration), and/or keep a group of neurons
body into the CNS (i.e., the spinal cord and/or functional in their optimal working range (e.g.,
brain). In contrast, efferent neurons convey infor- by feedback adjustments of their gains). These
mation away from the CNS to the effector cells circuits may be contained within a given region
(they induce an action or response), that is, within of the brain or extend throughout the brain; al-
muscles, glands, or other types of nerve cells. ternately, the circuits may project to various re-
Interneurons connect the various nerve cells to- gions of the systemic nervous system. Hence,
gether within the CNS into networks. It is roughly such transported information can be considered to
estimated that for each afferent neuron bringing (1) “diverge” to multiple brain regions so to have
information into the CNS, there are ten associated a global impact or (2) “converge” on a single cell
efferents and 200,000 interneurons [15]. or group of similar cells (e.g., nuclei or ganglion)
Simplistically, one can envision the human to activate or inhibit a given neural function.
brain to be composed of millions of neural cir- The concepts of “divergence” and “conver-
cuits which serve to amplify weak signals, at- gence,” which are interposed via neural networks
6 P. A. Iaizzo
Fig. 1.6 A schematic representation of the concept of is the concept of convergence of excitatory and inhibitory
divergence of neural networks, that is, divergence of infor- signals from higher motor centers onto a final common
mation from afferents to spinal neurons (left). Also shown pathway onto a motor neuron (right)
throughout the CNS, can also be considered to 1.1.1 Temporal and Spatial
occur at more global levels relative to overall Facilitation
CNS function. In other words, information from
multiple brain regions (given sites or from groups Neural discharge patterns and network structures
of neurons or nuclei) may have simultaneous/in- within the CNS can have unique design properties
stantaneous function impacts, as divergent or con- that serve to achieve various signaling functions.
vergent responses, on given neural activities asso- Two examples of such known properties are de-
ciated with specific brain processes. For example, scribed here—temporal and spatial facilitation
our ability to wake from sleep results from the (Fig. 1.7):
large divergence of neural excitation arising from
the reticular activating system located within our • Temporal facilitation: Repetitive stimulation
brain stems. of an axon may subsequently result in the elic-
Divergence of neural information can occur itation of an action potential. Excitatory post-
via axon collaterals, which serve to make such synaptic potentials (EPSPs) triggered in rapid
information accessible simultaneously to various succession are additive, with the accumula-
parts of the CNS (Fig. 1.6, left panel). For exam- tive effect eventually becoming suprathresh-
ple, the same sensory information can be utilized old; this is possible because the durations of
for reflex responses as well as for mediating a the EPSPs are longer than the axonal refractory
sensory experience. In addition, a single motor periods; that is, summation of EPSPs resulting
neuron innervates numerous muscle fibers; they in an action potential is not affected by the
function as a contractile unit. axonal refractory periods.
Regarding convergence, thousands of axon • Spatial facilitation: The activation of a
collaterals can converge onto the cell body of a single axon produces subthreshold EPSPs,
single neuron. It then depends on the sum and but several axons innervating the same
directions depolarizing or hyperpolarizing of the neuron triggered simultaneously give rise
synaptic processes acting at each moment in to a suprathreshold potential and thus the
time, whether or not that particular neuron will elicitation of an action potential.
elicit an action potential and send information
(i.e., its signal) onward (Fig. 1.6, right panel). Neurons in the CNS receive inputs from
Convergence allows a neuron to process or dozens to thousands of axons, for example, an
integrate incoming excitatory and inhibitory average of 6000 collateral branches terminate
signals occurring at its membrane within a short (form synapses) on a single motor neuron. These
period of time (msec). inputs can include both excitatory and inhibitory
Fig. 1.7 Two modes of synaptic facilitation. Temporal required. Spatial facilitation occurs when multiple excita-
facilitation occurs when multiple stimuli are delivered to tory inputs (different nerves) are delivered at relatively the
a nerve in rapid enough succession to reach threshold to same time point
initiate an action potential; in this case, three stimuli were
information (inhibitory postsynaptic potentials nium, and the spinal cord is protected by a verte-
or IPSPs); hence, it is the net amount of overall bral column. Relative to other animals, the human
positive excitation (depolarization) at a given brain has a highly developed frontal cortex (Fig.
point in time that will ultimately determine if an 1.9), which is associated with executive functions
action potential will be triggered (suprathreshold such as self-control, planning, reasoning, and ab-
response). In other words, if an equal number of stract thought. The portion of the brain devoted
IPSPs and EPSPs occur on a neuron at a given to vision is also greatly enlarged in humans, as
time, then the net response will not elicit an action compared to other animals. Viewed outwardly or
potential. Thus, when there is a higher proportion via imaging methods such as MRI, the human
of EPSPs that affect the given cell membrane at cerebral cortex is nearly symmetrical, composed
a given time such that the cell reaches threshold, of left and right hemispheres. Each hemisphere
this will in turn causes the elicitation of an action is then conventionally divided into four “lobes”
potential. including the frontal, parietal, temporal, and oc-
In humans, the brain is considered as the con- cipital lobes (Figs. 1.8 and 1.9). These lobes are
trol center of the nervous system. Yet, when one named after the bones of the skull that overlie
speaks of the CNS, we typically include the spinal them, with one exception—the border between
cord and brain together (Figs. 1.8, 1.9, and 1.10). the frontal and parietal lobes is shifted backward
The brain is protected by a well-enclosed cra- to the central sulcus (a deep fold that marks the
8 P. A. Iaizzo
Fig. 1.8 The relative location of a normal brain within the skull; major regional brain areas are also indicated
border between the primary motor cortex and the ties to a specific brain region and/or to monitor
somatosensory cortex); see also Fig. 1.9. abnormal functions within a given region.
From a functional standpoint, a given brain
region may contribute to specific nervous system
activity, for example, the motor cortical areas on 1.1.2 Special Neural Circuits
the right side of the brain (anterior to the central
sulcus) control motor functions on the left side As noted above, neural circuitries by their inher-
of the body and vice versa. In a second example, ent organization can serve to amplify or attenu-
the brain areas primarily responsible for three- ate incoming signals to a given brain area or a
dimensional spatial resolution (i.e., knowledge of given neuron itself. Below are several simplified
your body relative to the surrounding environ- examples that illustrate various defined neural
ment) in most individuals are attributed to the circuits:
right temporal cortex. Much of what we know
about both the anatomy and functioning of the • Antagonist inhibition: In this example, the Ia
human nervous system has come from the study afferents of a muscle spindle (length sensor)
of human motor disorders and experimental stud- in a given muscle in which the spindle lies
ies of animals in which specific lesions of focal in parallel with the muscle fibers (extrafusal)
injuries were placed in a given neuronal tract, make excitatory synapses with the motor neu-
brain nuclei, and/or brain center. More recently, rons innervating that muscle (agonistic excita-
advances in functional MRI have been used to tion), and by way of interneurons, it also has
associate an individual’s functional neural abili- inhibitory synapses with motor neurons inner-
Fig. 1.9 Shown here is the gross anatomy of the central the middle top; sagittal on the right). The various brain
nervous system (brain and spinal cord) in several differ- regions and spinal cord regions are indicated: C cervical,
ent anatomical planes (horizontal on the left; coronal in T thoracic, L lumbar, and S sacral)
vating the antagonistic muscles. This process • Synaptic potentiation: Repeated use of a
is also called reciprocal inhibition. The antag- synapse can cause considerable enlargement
onistic muscles in this case received forward of the synaptic potentials; such changes
inhibition (Fig. 1.11, left panel). can be associated with biochemical changes
• Feedback inhibition: This occurs when the in- within that given cell (e.g., phosphorylation of
hibitory interneurons act on the cells by which various proteins).
they themselves were activated (e.g., Renshaw • Synaptic depression: The situation in which
inhibition, Fig. 1.11, center panel). the postsynaptic potentials during or follow-
• Lateral inhibition: A form of feedback inhibi- ing a tetanic stimulation become smaller than
tion in which the inhibitory interneurons are those initiated by a single stimulus.
connected in such a way that they act not
only on the excited cell itself but also on the
neighboring cells with the same function (Fig. 1.1.3 Reflexes
1.11, right panel).
• Positive feedback: In such circuits, interneu- Circuits that include both sensory pathways (af-
rons send excitatory signals back to the cells ferent) and an effector response (efferents) are
which they received the signal from and per- typically considered as reflexes. In other words,
haps also neighboring cells with similar func- reflexes may also denote a complete neuronal
tions. circuit extending from the peripheral receptor
10 P. A. Iaizzo
Fig. 1.10 Shown here is a depiction of how afferent infor- the lateral part of the brainstem (into the pontine areas);
mation typically enters various levels of the sensory axis of their cell bodies are commonly located in the trigeminal
the nervous system. For example, primary sensory neurons ganglion. Such sensory information will ascend to various
in three different regions of the body (foot, hand, and brain regions (divergence) and will also terminate in the
face) carry sensory information (e.g., pain or temperature, sensory cortical regions on the opposite side of the brain
noted by yellow stimulus triggers) into the dorsal parts of (i.e., these pathways are considered to be crossed within
the spinal cord (foot and hand). The cell bodies for these the CNS). On the left are three different transverse sections
neurons are primarily located in the dorsal root ganglia. through the corresponding neuroaxis; on the right is a
Note that the afferents carrying sensory information from coronal section view of the spinal cord and brain
the facial regions project through the trigeminal nerves to
through the CNS and back to the peripheral ef- an EPSP which, in turn, causes action potentials
fects (Fig. 1.12). In general, it can be considered (no interneurons are involved). Yet, there are also
that all receptors participate in reflexes of some synapses on each muscle fiber involved, known
kind. as the neuromuscular junctions (Fig. 1.13).
The most basic reflex found in humans within In the case of reflex facilitation, augmentations
the CNS is the monosynaptic reflex. This is also of the stretch reflex within the leg can occur by
noted as the stretch reflex which is elicited by increasing the neural gain in that given circuit,
a skeletal muscle stretch; in this case, there is such as by increasing one’s voluntary activities
one synapse in the CNS, that is, between the within one’s arms (i.e., the Jendrassik maneu-
afferent and efferent neurons. More specifically, ver). Hence, via excitatory synapses coming from
the Ia afferents from the muscle spindle send neurons associated with the arms, the neurons to
collaterals directly onto alpha motor neurons as be activated in these reflexes to become closer
Stimulus Stimulus Stimulus

Interneurons
Antagonist
Agonist
Muscle
Muscle
Fig. 1.11 A schematic representation of several com- antagonistic inhibition (left panel), Renshaw inhibition
mon inhibitory circuits within the central nervous system. (negative feedback; center panel), and lateral inhibition
Three inhibitory interneuron circuits are shown: those of (right panel)
Fig. 1.12 Shown here is the basic neural circuitry of a lower part of the figure is the simplest one in the CNS,
reflex arc (top). The receptor brings information into the the monosynaptic reflex. The Ia afferents innervating the
central nervous system (CNS) via an afferent pathway muscle spindle fibers (intrafusal) synapse within the CNS
and then out via the efferent pathway via neurons asso- directly on the alpha motoneurons, which, via their motor
ciated with the effector response. The arc shown on the axons, project and diverge to innervate the skeletal muscle
fibers (extrafusal)
12 P. A. Iaizzo
Fig. 1.13 Shown here is

the pathway of a
monosynaptic stretch
reflex. A tap of a reflex
hammer on the patella
tendon causes stretch of the
quadriceps muscle as well
as the afferent stretch
receptors within the muscle
spindles; this then elicits an
effector response. The
reaction to the length
changes is a contraction
within the muscle that was
itself stretched. The
afferent to efferent pathway
underlying this reflex is
diagrammed—from
spindle to the alpha motor
neuron to the
neuromuscular junctions
on the muscle fibers
eliciting contractions
to their threshold potentials for extended periods • Cough reflex: Stimulation of receptors in mu-
and fewer additional excitatory inputs (e.g., from cosa of the trachea and bronchi elicits not only
afferent activations) are needed to elicit reflex coughing but also conscious sensations. The
responses. feeling of a slight tickling or scratching will
It should be noted that most reflexes within the usually cause coughing, but this response does
human CNS are composed of numerous neural not necessarily occur immediately.
networks with many interneurons (polysynaptic • Clonus or tremor: In part, these involuntary
pathways) that have both convergent inputs and motor responses are due to manifestations of
divergent projections. In other words, except for simple stretch reflexes. Physiological tremors
the monosynaptic stretch reflex, all reflex arcs and/or physiological clonus can be elicited in
in humans contain several interneurons in series normal humans, yet they can become con-
(receptor ➔ interneuron ➔ interneuron ➔ effec- tinuous or more pronounced in pathological
tor); thus, these are called polysynaptic reflexes. conditions (e.g., due to a spinal cord injury).
Examples include the following: • Flexor reflex: In this reflex reaction, there are
contractions of the flexor muscles and relax-
• Suckling reflex: The receptors in this highly ations or inhibition of the extensor muscles.
complicated polysynaptic reflex include Thus, this reflex has the action to pull a body
touch-sensitive structures in the skin of the part away from an acute painful stimulus.
lips (mechanoreceptors). The effectors include • Crossed extensor reflex: About 0.2–0.5 sec-
the muscles of the lips, cheeks, tongue, throat, onds after stimulation of a given flexor reflex,
thoracic cage, and diaphragm. The effector extension occurs in the opposite limb. The
responses within this polysynaptic reflex primary purpose of this reflex is to push the
involve ingestion movements that are also entire body away and to prepare to support the
coordinated with respiration (i.e., to prevent body as another part is flexing. For example,
aspiration). you step on a piece of glass with your right foot
and immediately and involuntarily pull your mentary position of one’s body and the progress
lower leg away from the glass (flexor reflex), of the desired movement. On the other hand,
while this induced an extension of your left leg, certain kinds of sensory information can only be
so to prevent you from falling over (stabilizing acquired with appropriate motor acts, for exam-
your upright position). ple, move the head to view a given object or move
• Scratch reflex: This is an example of the com- your hand to touch something.
bination of several varied reflex components,
which have different functions, for example,
position sensing (location of irritation on the 1.2.1 Properties of a Particular
body) and a subsequent involuntary but appro- Stimulus
priately located scratching response. It should
be noted that typically you can voluntarily Relative features of the sensory systems within
suppress these reflex responses. the human body can be described by terms such as
modality, quality and quantity, and/or perception.
In general, a modality can be defined as a
1.1.4 Reflex Time group of similar sensory impressions mediated by
a particular sensory organ or groups of similar
The concept of measuring reflex times depends neurons. Modalities (or impressions), which arise
on numerous factors. Briefly, it can be described from our external environment, include the clas-
as the time between the onset of a stimulus and sic five—sight, hearing, touch, taste, and smell.
the action of the effect, which in turn is chiefly Additionally, we can detect the modalities of cold,
determined by the overall conduction times which warmth, vibration, and pain. Modalities are also
involve (1) transformation of the sensory stim- mediated by grouped sensory neurons, which re-
ulus; (2) transmission of information across a flect internal states within our bodies and include
synapse; (3) transmission of information between those that we are consciously aware of (equilib-
(in series) interneurons, if present; (4) transmis- rium, limb positions, loads, tensions, etc.) and
sion of information from the effector pathway to also those which we typically do not have skilled
the effector organ; and (5) the axon lengths and awareness of (unconscious internal modalities)
conduction velocities of the involved neurons. such as osmotic pressure of the blood, blood CO2
tension, lung capacity, and the relative stretch of
one’s stomach. Yet, with training and biofeed-
1.2 Sensory Systems back, one can learn to perceive these modalities
and perhaps even consciously control or respond
We experience our environment and the events to them.
that take place within our bodies not directly, not A sensory quality is defined as a distinctive
in their entirety, but by way of specialized sense property of a given modality. For instance, quali-
organs (e.g., eyes, ears, nose, etc.). In general, ties of vision might include the colors red, green,
each receptor organ is constructed so that it re- and blue or given lightness or hue of a color.
sponds to a particular range of environmental in- Pitches or tones are specific qualities related to
fluences, which then transmit the corresponding our hearing, and we are familiar with the qualities
information to higher integrating centers within of sweet, sour, salty, savory, and bitter (taste).
the brain (CNS). Each quality is related to “specific sensory stim-
The human sensory (afferent) systems and the uli”; it is a reaction to different types of stimuli
motor systems (effectors) are intimately inter- by the detecting cells within the same sense or-
meshed. For movements to be carried out in func- gan (i.e., “receptors”). In other words, specific
tionally appropriate ways, all the structures in- populations of sensory cells within an organ are
volved in their production require and receive adapted to respond more strongly and/or more
information from the periphery regarding the mo- specifically to a given type of sensory stimulus.
14 P. A. Iaizzo
Stimulation of these populations of specialized • Free nerve endings (touch, pressure, pain, and
receptors causes the generation of one or more temperature)
action potentials, which then propagate along the • Merkel’s disks (touch, pressure)
various afferent nerve axons to the appropriate • Hair follicle receptor (touch, pressure)
CNS sensory centers. It should be noted that the • Pacinian corpuscles in skin and deep body
properties of these generated action potentials are tissue (pressure, stretch)
the same for all types of qualities. Therefore, the • Meissner’s corpuscles (touch, discriminative)
information they contain is determined entirely • Ruffini endings (touch, pressure, internal reg-
by the receptor type from which the relevant nerve ulation, warmth)
arises and the relative areas of the brain which • Golgi tendon organ (tension, force)
become activated. • Muscle spindles (length)
The quantity of a given modality can be de- • Krause’s corpuscles (hot and cold)
fined as the relative intensity of a specific kind • Olfactory neurons (smell)
of sensory impression (i.e., with a given modal- • Auditory neurons (sound)
ity and quality). As the receptor potential in- • Hair cells in the semicircular canals (accelera-
creases, typically so does the frequency of actions and/or gravitational changes)
tion potential discharges. A threshold stimulus is • Rod and cone cells in the eyes (light and vari-
commonly defined as the smallest stimulus that ous colors)
produces a detectable response (i.e., elicitation of
at least one action potential). Yet, an important
concept related to a sensory quantity is the be-
havior known as accommodation, which means 1.2.2 Functional Organization
that in some cases not all receptors generate a of a Receptor
proportional frequencies of action potentials (or
static responses) for a given stimulus intensity. In general, although receptor populations have
For example, if a receptor exhibits accommo- different functions and unique overall structures,
dation, the relative frequency of discharge will most receptors have similar underlying primary
decrease over time. There are two basic types structures. Each receptor requires a stimulus for
of accommodation—slowly adapting (tonic) and activation (a physical stimulus), which is then
rapidly adapting (phasic) responses. processed by the afferent cells that are commonly
Finally, it is important to define an overall sen- composed of the following: (1) a filter that can
sory perception. First, any combination of several modify the physical stimulus and thus detect a
sensory impressions is considered as a sensation. given quality; (2) a transducer that processes the
A perception then interprets these sensations with stimulus into an electrical response, via ion chan-
reference to an individual’s learned experience. nel activities; (3) an encoder that processes the
For example, this author prefers a warm, chewy transduced signal into an ultimate effect, for ex-
chocolate-chip cookie with the chocolate slightly ample, a hyperpolarization or depolarization; and
melted. The human body contains hundreds of (4) the subsequent generation of action potentials
different types of sensory receptors, and there that typically occurs at the first node of Ranvier of
can be multiple types of these specific receptors a given primary sensory cell. Note that secondary
within a given tissue or organ. Although some sensory cells do not elicit action potentials them-
populations of these receptors serve the same selves but, via their induced changes in mem-
modality, they can have differing qualities. Listed brane potential, cause the generation of action
below are various types of receptors found within potentials of their innervating cells, for example,
the human body and some of their associated via synaptic activation (see additional examples
modalities: below).
1.2.3 The Relative Distributions The relative densities of a receptor popula-

of Receptors Within tions can vary within the human body. Note that
the Human Body some receptors are located only in specific sites
within the human body, whereas other receptors
One can define the regions within given tissues or (e.g., touch and pain receptors) can be found in
structures that are innervated by various sensory nearly all tissue. Other receptor populations can
receptor as its receptive field. More often than not, be found in specific tissue types, but in turn they
innervation areas of adjacent receptors overlap; in can have highly varied densities, thus making
other words, it is typical that there are regions of one region more sensitive to a given modality
overlap of individual receptor fields. Figure 1.14 versus another. In humans, the skeletal muscles
(left panel) shows one such sensory unit and its with the highest densities of sensory receptors
given receptive field. also elicit more sensitive motor control (see Sect.
The concept of spinal dermatomes is important 1.2.4). For example, we have fine motor control
to understand when, from a clinical/functional of the muscles in our hands, extraocular muscle,
perspective, one attempts to identify the underly- lips, and tongue. Another example is our ability
ing cause of a patient’s sensory or motor losses. In to discriminate pressure; this sensation is more
general, one can consider that the afferent nerves precise within our fingertips or lips than in the
from a specific region of the body join together to middle of our backs.
form the various peripheral and (eventual) spinal
nerves. It then follows that a loss of sensation
within a specific dermatome likely indicates that 1.2.4 Sensory Input into Motor
the function of a given peripheral nerve is being Systems
compromised, for example, impinged or dam-
aged (Fig. 1.15). However, to block the sensation The human body has incredible abilities to main-
of pain, regional anesthesia can be administered tain posture and perform complex motor tasks
within a given dermatome during surgery. These with little or no conscious effort: as well as per-
anesthetic agents reversibly block function (i.e., form highly complex voluntary motions. To do
the propagation of information) and thus the elic- so, large amounts of sensory inputs (afferents) are
itation of action potentials associated with the required at all levels of the CNS; thus, there is a
modality of pain. high degree of divergence of this sensory infor-

typical sensory unit and its Central
projected receptive field Terminals
(left). Note that if a
stimulus point falls within Cell A B C
the overlapping receptive Body
fields of three different Site of
Afferent Stimulation
neurons, then action
Nerve
potentials may be
Fiber
generated in each (right).
Yet note that the discharge
Receptive
rates would likely differ for
Action potential
each such neuronal Field Receptor

receptor Terminals
Neuron A Neuron B Neuron C

16 P. A. Iaizzo
Fig. 1.15 Various regions innervated by cutaneous affer- spinal cord. There the afferent information (primarily via
ents form cutaneous nerves. These afferent fibers congeal interneurons) travels upward in various regions within the
to form the dorsal rootlets that then continue and enter the white matter (spinal column) toward the sensory cortex
within the brain
mation. Two sense organs within skeletal muscle a contracting muscle (responsiveness), or even do
are foremost in these roles—muscle spindles and so before the skeletal muscle contracts in order to
the Golgi tendon organs. Therefore, a thorough increase their relative sensitivities or gain. These
appreciation of the properties and functions of sense organs also have different types of afferent
these receptors is essential for understanding nu- innervations which can provide varied length in-
merous principles of motor control. formation including both phasic and static behav-
Muscle spindles within the human body, al- iors. As noted above, muscle spindles can have
though quite varied and complex in forms, have high densities in muscles that require fine motor
several common features (Fig. 1.16). These sense control. The general features of muscle spindles
organs lie in parallel arrangements with the con- are listed as follows:
tractile muscle fibers (extrafusal) that compose a
given skeletal muscle. Importantly, these recep- • Muscle spindles are composed of intrafusal
tors themselves can contract or shorten (efferent skeletal muscle fibers; there are two types—
innervation) to maintain their functional roles in nuclear-chain and the larger nuclear-bag
Fig. 1.16 Shown are the two types of intrafusal muscle innervations within a typical muscle spindle. These multi-
fibers that commonly compose muscle spindles. There type innervations allow the spindle to have both static and
are two types of afferent and several types of efferent dynamic sensing properties
fibers—and the numbers of each can also

vary from muscle spindle to muscle spindle. 1.3 Somatovisceral Sensibility
• There are two types of afferent innervations
of muscle spindles: (1) type Ia afferent fibers The sensory modalities within the skin and
which are primary (annulospiral) endings that associated structures as a whole constitute the
innervate the central regions of each type of category known as somatovisceral sensibility.
intrafusal fiber and (2) type II afferents which These associated receptors and modalities include
are considered secondary sensory endings and mechanoreception, thermoreceptors, proprio-
are typically located on nuclear-chain fibers. ception, and nociception (pain sensitivity). A
• The relative lengths of the intrafusal fiber can common feature of all these modalities is that
shorten in relation to the lengths of the asso- the receptors are not grouped within discrete
ciated extrafusal fibers (efferent innervation). sense organs and the given densities can vary
Activation of gamma motor neurons specifi- throughout one’s skin.
cally causes these intrafusal fibers to contract,
whereas beta motor neurons innervate both
intra- and extrafusal fibers which will activate
1.3.1 Processing in the Central
these muscle fiber populations simultaneously.
Nervous System
• The number of spindles per gram of muscle is
Underlying these subjective sensations are objec-
highest within small muscles that participate in
tively measurable events in the nervous system.
fine movements. For instance, inferior rectus
Receptors transform stimuli into trains of nerve
muscle of the eye contains 130 spindles/g, and
impulses (action potentials), which are subject
the triceps of the arm has 1.4 spindles/g. As a
to modification in various ways and at several
general rule of thumb, there are approximately
successive levels, for example, by numerous ex-
50–80 Golgi tendon organs for every 100 mus-
citatory and inhibitory synaptic projections. Our
cle spindles.
conscious perceptions are only a small fraction of
• Golgi tendon organs (GTOs) are the proprio-
the total role and output of this integrating system.
ceptive sensory receptors that sense changes
In other words, the ascension of this afferent (or
within a given muscle tension. They lie within
sensory) information can be traced through sev-
both the origins and insertions of skeletal mus-
eral locations within the spinal cord, brain stem,
cle tendons. They can also be referred to as
thalamus, and cortex (Fig. 1.17).
neurotendinous sensory organs.
18 P. A. Iaizzo
Peripheral
Nervous Central Nervous System Behavior
System
Cortex Associative
a,b,c,d (a)
Somato Visceral
receptors Integrated
response
Thalamus Limbic
(b) based upon
a,b,c,d
cognitive,
affective,
Stimuli Motor motivational,
Brainstem
a,b,c,d (c) motor and
vegetative
inputs
Spinal
Cord Vegetative
c,d (d)
Environmental
and internal
signals
Fig. 1.17 Shown here are the complexities of the signal- within the human central nervous system and their related
ing pathways associated with somatovisceral sensibility. integrative and efferent (effector) systems are shown at a
Note the high degrees of both divergence and convergence high level. Note the neural pathways, associated nuclei,
of this afferent information. The brain regions/structure and reflex mechanisms can involve thousands of neurons
When one considers the generalized functions pathways (projecting to the medulla oblongata
of the human brain, four different systems can be and then to the thalamus) which sends afferent
globally defined: (1) the associative system, with projections to the sensory cortical regions (SI and
primary roles of cognitive functioning and con- SII) of the parietal lobes of the cortex; this is
scious recognition; (2) the limbic system which also known as the lemniscal system. In general,
controls emotions and feelings; (3) the motor the SI region receives afferent inputs from the
system which includes voluntary and involuntary contralateral side of the body, whereas the SII
movements and postural control; and (4) the veg- regions have directed projections from both sides.
etative system which is primarily the involuntary These main ascending pathways track through the
control of bodily functions governed by the au- dorsal column of spinal cord, to dorsal column
tonomic nervous system (see Sect. 1.8). It should nuclei in the medulla (first set of synapses), to
be noted that these axons from the complete set medial lemniscus, to ventrobasal nucleus of the
of cutaneous and visceral receptors from the pe- thalamus (second set of synapses), and then to
ripheral sensory surface project to the thalamus the areas SI and SII of the cortex (third set of
and ultimately to the cortex in a somatotopically synapses) (Fig. 1.18).
organized manner.
1.3.2.2 Nonspecific Pathways
1.3.2 Basic Anatomy Nonspecific sensory pathways make connections
of the Somatosensory System with nearly all regions of the cerebral cortex. This
system, known as the extra-lemniscal system, is
1.3.2.1 Specific Pathways thought to be important in (1) perception, (2)
The medial lemniscus tract can be considered one’s overall state of consciousness, and/or (3)
as one of the most important ascending sensory orientating responses (Fig. 1.18).

simplified overview of the
general features of
somatosensory projection
from the body to the brain.
Two major types of
projections can be
described—specific and
nonspecific pathways. One
can track these pathways
upward to the SI and SI
somatosensory cortical
regions, as well as to other
cortical areas
1.3.3 Somatosensory Projection 1.3.4 Mechanoreception

Areas in the Cortex
It is important to define the specific sensory abili-
As noted above, the SI cortical area is located ties of mechanoreception, which is comprised of a
on the postcentral gyrus, immediately posterior number of qualities such as pressure, touch, vibra-
to the central sulcus (a deep furrow lying trans- tion, and tickle. Our ability to identify subjective
versely on the cerebral hemispheres). The SII area measurable properties of mechanoreceptors (e.g.,
lies on the upper wall of the lateral sulcus, which when a bristle is used to touch the skin at different
separates the parietal and temporal brain lobes. locations) is not present throughout the whole
Interestingly, the SI region represents topographic skin but can be ascribed to certain cutaneous
organization via projection from the opposite side locations or points on the skin (touch points;
of the body, and electrical stimulation of a given Fig. 1.19). Further, one can define simultaneous
SI cortical region (e.g., with an electrode during spatial thresholds for mechanoreception due to
surgery or with superficial magnoelectric coil ac- variations in the density of these receptors. It
tivation) typically elicits known describable per- should be noted that these receptors have distinct
ceptions. Additionally, it is common to record histological structures and afferent innervations;
evoked electrical potentials from the skull supe- many mechanoreceptors are supplied by myeli-
rior to the SI area (i.e., using electroencephalo- nated afferent nerve fibers (e.g., Group II axons
graphic or EEG analyses). with diameters of 5–10 μm and conduction ve-
20 P. A. Iaizzo
Arm Head
Hand, Fingers Torso, Leg
Eye, Nose
Face
Foot
Lips
Jaw
Tongue
Fig. 1.19 The somatosensory cortex can be considered cortex (left). The areas with higher numbers of receptory
to be somatotopically organized. In other words, one can contributions are shown as larger body features (right). For
define a spatial representation of the human body surface example, our hands and faces have higher mechanorecep-
for mechanoreception on the postcentral gyrus of the tion abilities than one’s back
Fig. 1.20 The schematic

representation of the
relative structures and
positions of several types Horny layer
of mechanoreceptors that Epidermis
can be identified in both
glabrous (hairless) and Corium
hairy skin
Subuctaneous
tissue
Hairless skin Hairy Skin
Meissner’s Merkel’s Pacinian Hair-follicle Tactile Ruffini

corpuscle disks corpuscle receptor disk ending
locities of 30–70 m/s). Listed below are several • Merkel’s disks, which are slowly adapting in-
identified mechanoreceptors and their general lo- tensity detectors, primarily located in the low-
cations and relative qualities: ermost layers of the epidermis
• Ruffini corpuscles, also known to be slowly
• Pacinian corpuscles, which are rapidly adapt- adapting intradermal receptors, but they are
ing acceleration detectors located in the sub- typically located in deep layers of the dermis
cutaneous tissues, tendons, fascia, periosteum,
joint capsules, and/or mesenteries Finally, it should also be noted that some
• Meissner corpuscles, which are moderately mechanoreceptors are supplied by unmyelinated
rapid adapting velocity detectors, typically lo- afferents. These receptors more typically respond
cated in glabrous (hairless) skin but can also to low-intensity tactile stimuli, and they are
be positioned as hair follicle receptors located most commonly in hairy skin (Fig. 1.20)
1.4 General Anatomic Simplistically, the primary function of this

and Functional Features highest level can be considered to form complex
of the Motor System motor plans according to a person’s intention.
Subsequently, the middle level structures become
The following is a brief overview of some of the activated (Fig. 1.21, blue boxes) including
key components of the human motor system. To the sensorimotor cortex, the cerebellum, parts
help understand our complex neural abilities and of basal ganglia, and/or various brain stem
the vast networks of neurons that are involved in nuclei. Once activated, these areas perform the
controlling movements, it is of value to describe generalized functions of converting the complex
the human motor system in a functional, hierar- motor plans into a number of smaller motor
chical fashion. programs: this then determines the patterns of
neural activations required to perform the desired
movements. Then these programs are further
1.4.1 Motor Control Hierarchy for broken down into subprograms that determine
Voluntary Movements the movements of, for example, individual joints.
As such, these programs and subprograms are
When humans perform a voluntary movement, transmitted via the descending pathways to the
we utilize and coordinate the activities first at the lowest control level of the motor system (Fig.
highest level (Fig. 1.21, yellow box). Thus, corti- 1.21, white boxes). Hence, the structures involved
cal areas associated with memory and emotion, in these activities include all levels of the brain
the supplementary motor area, and association stem and spinal cord from which motor neurons
cortex will be involved accordingly. These brain exit. More specifically, the functioning at this
areas, in turn, receive and correlate inputs from level coordinates the specific output tensions
many other CNS structures. within involved muscles and also the resultant
angles of specific joints necessary to carry out
Association Coretx &

Sensorymotor Cortex
Other Higher Centers
(Descending (Highest)
Pathways)
Brainstem Nuclei &
Subcortical Nuclei
(e.q., basal qanqlia &
thalamus) Cerebellum
Spinal Cord
S keletal Muscle, Tendon, Joint,

(Low)
Muscles and Skin Receptors
Fig. 1.21 A block diagram representation of the relative hierarchical pathways associated with voluntary move-
hierarchical and/or functional organization of the motor ments, yet it should be noted that sensory input is needed
systems in the human body. The yellow box (containing at the highest level to develop an optimized motor plan
several cortical regions) is the highest level, the blue boxes prior to initiation and the subsequent response to potential
represent the middle level, and the white boxes are the perturbations
lowest functional level. This representation describes the
22 P. A. Iaizzo
Fig. 1.22 Shown here are

the relative anatomical
connections of the spinal
cord to the central nervous
system via the brainstem
which can be further
subdivided into the
medulla oblongata, pons,
and mesencephalon (also
referred to as the
midbrain). The brainstem
is composed of numerous
nuclei and functional
regions
the motor programs and subprograms transmitted 1.4.3 Brain Stem Components
from the middle control levels. The receptors
providing inputs for the control of various The spinal cord makes its neural connections to
motor programs include the muscle spindles, the CNS via its anatomical connections to the
Golgi tendon organs, cutaneous receptors, joint brain stem proper (Fig. 1.22). The brain stem
receptors, and/or free nerve endings. is composed of multiple nuclei, many of which
In the following paragraphs, the specific func- have unique roles relative to the overall motor
tioning or the major CNS components of the mo- control system. A number of different brain stem
tor system will be described, starting at the lower nuclei play important roles in the posture control
level and moving upward in these pathways. response of the human body. More specifically,
equilibrium is maintained and the body is kept
in the normal upright position in the earth’s field
1.4.2 Spinal Cord of gravity by reflexes, that is, with no need for
conscious intervention. It is important to note
The spinal cord is not only a conduit for that these postural motor functions are largely the
afferent and efferent pathways, but it also responsibilities of coordinated brain stem motor
serves numerous critical functions; it can be centers. A proper movement cannot be performed
considered as a local motor-control system. without (1) putting the body into a desired posi-
In part, this local control system is composed tion in space, (2) having prior knowledge of the
of various reflex pathways which have inputs body in space, and (3) then maintaining a proper
from muscle spindles for length control, Golgi posture during the performance of the desired
tendon organ for tension control, stretch reflexes, motor task. As such, the brain stem helps convert
flexor reflexes, withdrawal reflexes, crossed- the overall motor plan or goal of an action into
extensor reflexes, and/or intersegmental reflex programs that determine the specific muscle/limb
pathways. Furthermore, when an appropriate movements. Additionally, these centers provide
signal is received from the periphery or from important feedback to the motor cortex through
a higher area of the CNS, by way of segmental the thalamic pathways.
reflexes (same cord level for input and output) and The brain stem is composed of several dif-
intersegmental reflexes (from different regions of ferent regions—the medulla oblongata, the pons,
the spinal cord, dermatomes), the spinal cord is and the mesencephalon (Fig. 1.22). Each of these
capable of executing complex movements and regions contains various groups of commonly
adjusting them to one another. functioning neurons described either as nuclei,
centers, or formations. Listed below are several

examples of these grouped functional neural com-
plexes:
• The red nucleus lies primarily within the

mesencephalon; these nuclei give rise to the
rubrospinal tracts (axons that cross sides of
the human brain immediately). Typically,
excitation of these neurons excites alpha and
gamma flexor motor neurons via interneuron Fig. 1.23 The relative anatomical locations (middle level
connections (i.e., numerous synapses) and, at motor structures) of the brainstem, cerebellum, basal gan-
glia, and thalamus
same time, causes inhibition in the extensor
muscles.
• The lateral vestibular nucleus gives rise the control of posture and muscle tone; (2) pro-
to the vestibulospinal tracts (these axons viding for course corrections during slow goal-
descend uncrossed) which excites both directed movements and the coordination of these
alpha and gamma extensor motor neurons movements with the postural system; and (3)
(predominantly monosynaptically). Com- allowing for the unimpeded performance of rapid
monly, with activation of these nuclei, nearly goal-directed movements (those designed by the
simultaneously the flexor muscles are actively motor cortical areas, e.g., motor programs). It
inhibited. can be considered that the primary role of the
• The reticular formation gives rise to both the cerebellum is to supplement the activities within
medial and lateral reticulospinal tracts. The the other motor centers and to coordinate them.
medial tracts are uncrossed and arise in the In part, the afferent connections into the cere-
pontine portion of the brain stem; they excite bellum include (1) inputs from vestibular nerves
both alpha and gamma extensor motor neu- and nuclei; (2) ascending somatosensory inputs
rons. The lateral tracts arise in the medullary from the lower body, via the spinal cord; and (3)
reticular formation and predominantly excite descending inputs from the cerebral cortex. More
flexor motor neurons (these axonal pathways detailed information relative to the cellular and
are both crossed and uncrossed). molecular aspect of the cerebellar structures has
been described in the literature recently; however,
Importantly, these brain stem nuclei and their it is beyond the scope of this chapter. Briefly,
associated tracts, originating within the various the cerebellar cortex has three primary layers—
nuclei, are integral for generation of both the tonic the molecular, granular, and Purkinje cell layers.
and righting reflexes; in other words, they are Further, the cerebellum also has been described
necessary for the maintenance of upright posture. to have specific longitudinal zones including the
These motor centers in the brain stem also receive vermis, pars intermedia, and the hemispheres.
inputs from the motor cortex and, in turn, send
signals to the cortical regions (also known as
the cerebrum) via connections to and from the 1.4.5 Motor Cortex
cerebellum (an important feedback mechanism).
More specifically, such pathways serve to coordi- One of the primary brain regions to be defined
nate postural and goal-directed movements (Fig. within the highest motor level is the motor cor-
1.23). tex. This brain region has particular importance
for maintaining goal-directed movements. Like
1.4.4 Cerebellum the somatosensory cortex, the motor cortex is
somatotopically organized, and there are multiple
The middle level motor structure known as the representations of the periphery in several motor
cerebellum plays crucial roles in (1) aiding in areas (Fig. 1.23). One general area of the motor
24 P. A. Iaizzo
cortex includes the primary motor cortex in the

precentral gyrus (this is sometimes referred to
as Brodmann areas 4 and 6, that is, originally
defined and numbered by the German anatomist
Korbinian Brodmann, who based this classifi-
cation on the cytoarchitectural organization of
neurons that he observed in the cerebral cortex);
this motor area is needed for motor program
generation. One can also define premotor areas
which are involved in complicated motor func-
tions such as (1) required changes in output forces
or velocities; (2) the decision to change from one Fig. 1.24 A block diagram of the major pathways con-
task to another; (3) required motor response to a necting the higher brain centers to the ultimate pathways
visual or auditory input; (4) two-handed coordi- to induce muscle contractions, the motor neurons
nated movements; and (5) required high degrees
of postural support needed for some specified ter includes the following specific nuclei: stria-
detailed movements. These cortical motor areas tum (caudate nucleus and putamen), pallidum,
also receive inputs from both the parietal (spatial substantia nigra, and the subthalamic nucleus. In
input centers) and occipital cortical (visual cor- humans, damage to these neural tissues, that is,
tical centers) lobes. Finally, one can also define associated with these ganglia/nuclei, will typi-
a supplementary motor cortex area, from which cally elicit themselves as defined motor defects
one may record readiness potentials, and this area (see Sect. 1.7). Note that efferent synapses from
lies near the limbic (emotions) system. both the motor cortex and the basal ganglia are
found within the thalamus (Figs. 1.9 and 1.24).
These areas are considered important coordi-
1.4.6 Efferent Connections nating centers for goal-directed motor programs
from the Motor Cortex or patterned movements (Fig. 1.25).
Corticospinal and corticobulbar tracts (the ma-

jority which have been identified to be crossed)
leave the motor areas in each half of the brain 1.5 Maintenance of Upright
(note that such tracts are typically named so to Posture and Sense
define where they begin and where they termi- of Equilibrium
nate). They give off numerous collaterals to the
thalamus, red nucleus, pontine nuclei (which, in Generally, two types of motor functions can be
turn, send projections to the cerebellum), dorsal- distinguished—the maintenance of posture and
column nuclei, and the reticular formation. Pro- the initiation of intended body movements. In
jections that reach the cord predominantly end on practice, these two types of motor functions are
interneurons and are excitatory to flexor muscles. inextricably conjoined. As such, goal-directed
Corticorubral and corticoreticular tracts are the movements can be optimally performed only
main cortical efferent pathways to the brain stem. if the body and limbs to be moved are first
put into the appropriate positions. Additionally,
for body positions (postures) to be maintained,
1.4.7 Basal Ganglia and Thalamus it is necessary that any forces which disturb
these body orientations be counteracted by
These structures are considered important subcor- appropriate movements (muscle contractions).
tical centers which link the “associative” cerebral In other words, movement without postural
cortex to the motor cortex. The basal ganglia cen- control is as impossible as postural control
Fig. 1.25 Shown here is a schematic representation of and feedback during execution (for both movement and
the human motor system in which the roles from plan to posture); then this information can be utilized to update
program to execution have various describable functions. motor programs so to optimize the future execution of
Sensory information diverges throughout this system al- similar motor plans
lowing for feedback before a plan is to be developed
without varied muscle activations. In general, the lymph. There are two morphological subunits of
maintenance of posture and/or a person’s balance these vestibular organs: (1) the macular organs (or
is accomplished by means of complex interacting statolith organs) and (2) the semicircular canals.
postural reflexes. The afferent inputs/pathways The receptors within these two organs are hair
for the initiation and control of these reflexes cells, which contain both stereocilia (60–80 per
arise from the eyes, the vestibular apparatus, cell) and one kinocilium (Fig. 1.26). Importantly,
and sensory inputs from the proprioceptors. these receptors are defined as secondary sensory
Ultimately, these efferent pathways have their cells because they have no neural processes (ax-
terminal effects on the alpha motor neurons of ons) of their own but are innervated by afferent
our skeletal muscles, and the primary integrating fibers (their cell bodies lie within the vestibular
centers for these reflexes are within both the brain ganglion).
stem and spinal cord.
1.5.1.1 Macular Organs
The macular organs contain receptors that primar-
1.5.1 Sense of Equilibrium ily respond to translational (linear) accelerations
of the head (temporal bone). Further, the cilia of
One of the key sensory systems associated with the sensory cells within these organs project into
a human’s ability to maintain equilibrium is the adjacent otolith membranes, which are composed
vestibular organ. These organs form one part of in part of calcite crystals. There are two types
the membranous labyrinth lying within the skull’s of macular organs—the utriculi which, at rest
temporal bone, which constitutes the inner ear with the head erect, is positioned approximately
(the other part of the temporal structure is the horizontal and the sacculi which is roughly po-
organ of hearing). The membranous labyrinth is sitioned vertical when the head position is erect
filled with endolymph and surrounded by peri- (Fig. 1.27).
26 P. A. Iaizzo
Fig. 1.28 A schematic diagram of the left horizontal

semicircular canal which lies in the temporal bone. An
angular acceleration in the direction of the arrow deflects
Fig. 1.26 A diagram of the vestibular labyrinth. The the cupula as shown by the dashed lines. Such a deflection
lymph spaces are in communication with those of the of the cupula activates the secondary sensory hair cells
cochlear labyrinth which, in turn, alters activities in the innervating afferent
fibers. Because there is a resting firing rate, movement
in one direction leads to an increase in action potential
frequencies, whereas a movement in the opposite direction
causes a decrease (i.e., a bidirectional receptor system)
1.5.1.2 Semicircular Canals

These vestibular receptors primarily respond to
angular (rotational) accelerations of the head. In
these organs, the cilia of the receptor cells are not
embedded in mineral inclusions (Fig. 1.28). The
cupula in the canals has the exact same density
as the endolymph; hence, they do not respond
to linear acceleration. There are three separate
semicircular canals to receive input from all three
spatial axes of the human head (horizontal, ante-
rior, and posterior).
Fig. 1.27 A schematic representation of two common 1.5.1.3 Central Vestibular System
types of hair receptor cells that can be found within the The primary afferent nerve fibers, innervating the
sensory epithelium of the vestibular organ. These are hair cell in the vestibular organ, are collectively
secondary sense cells and thus are shown in their asso-
ciated innervating afferent nerves. When the bundle of
known as the vestibular nerves. They terminate
cilia (stereocilia) moves toward kinocilium, membrane de- chiefly in the region of the vestibular nuclei, lo-
polarization results, whereas stereocilia movement away cated in the medulla oblongata. There are four pri-
from kinocilium results in membrane hyperpolarization. A mary vestibular nuclei which have been identified
depolarization induces an increased release of neurotrans-
mitters and thus a subsequent increase in the discharge rate
on each side of the body including the superior,
(a higher action potential frequency) in the afferent nerves. medial, lateral, and inferior nuclei. Note that be-
Note the potential for efferent input on the sensory cells; cause the angle of the head (movable at the neck
thus, there is a way to modulate receptor responsiveness. joints) is independent of the trunk, the CNS needs
A relatively high resting activity can be recorded in the
vestibular nerves, i.e., there is the spontaneous generation
to determine the position of the head relative to
of action potentials the trunk. Therefore, the vestibular nuclei receive
additional inputs from neck receptors, as well as sequent movements. In such cases, nearly all of
those from the limbs. There are numerous neural the aforementioned vestibular organs are impor-
efferent pathways that leave these vestibular nu- tant for these reflex responses/controls. Examples
clei, including the following: of such dynamic reflex responses (movements)
include head-turning reactions, eye rotation reac-
• Vestibulospinal tract (primarily contains tions, and one’s ability for corrective body orien-
gamma motor neurons to extensors) tation during a free fall.
• Motor neurons of the cervical cord
• Connections to oculomotor nuclei (e.g., im-
portant for the control of gaze as the head
moves) 1.6 Complex Integrative
• Contralateral connections to other nuclei (al- Functions of the Motor
lowing for comparisons to better determine System
orientation and rates of head movement)
• Cerebellar connections (coordination of fine 1.6.1 The Complex Motor Function
head movements) of Speech
• Connections to the reticular formation
• Connections and tracts projecting to the thala- Prior to major recent advances in molecular
mus and postcentral gyrus biology and functional MRI, practically all of
• Connections to the hypothalamus (which can our knowledge related to the physiology of
be associated with motion sickness) speech was derived from clinical observations
(i.e., post-mortem neuropathological studies and
1.5.1.4 Vestibular Reflexes electrical stimulation experiments of exposed
In general, one can consider that equilibrium brains of awake patients). For example, from
is maintained via reflex mechanisms, without the therapeutic transection of commissural fibers
primary participation of consciousness. The (split-brain operations), it was shown that, in
vestibular receptors and somatosensory inputs, general, the left hemisphere in most individuals
especially those from proprioceptors in the contains the necessary centers for speech. Over a
neck, are paramount for these reflex activities. hundred years ago, a French surgeon named Paul
There are two general categories in which such Broca first observed that lesions of the lower part
reflexes can be grouped—static and statokinetic of the third frontal gyrus on the left side of the
reflexes. The vestibular inputs for static reflexes brain caused a failure of speech (aphasia). Yet,
are primarily the macular organs. These reflexes with this specific impairment, speech could be
govern one’s fixed positions of the individual understood, but the patients rarely said anything
limbs with respect to one another and/or of spontaneously; however, on command and with
the body in space. Yet, it should be noted that hesitation, short sentences could be spoken. This
initial movements are often needed to bring the is defined as a motor aphasia involving loss
body into these patterned positions (postural or of coordination of motor programs, and the
attitudinal reflexes). Postural reflexes include associated area of the brain is called Broca’s
tonic neck reflexes, tonic labyrinthine reflexes, speech region (Fig. 1.29).
and compensatory eye positioning responses. Later that century, a German neurologist, Carl
Examples of righting reflexes include the Wernicke, described another type of aphasia in
labyrinthine righting reflexes and neck righting which the understanding of language/speech was
reflexes. severely impaired, but spontaneous speaking in
The other types of vestibular reflexes are the such patients was fluent, though often distorted.
statokinetic reflexes. These reflexes become op- This is a sensory aphasia and is highly correlated
erational in response to given movement stimuli, to another area of regional damage within the left
which then in themselves take the form of sub- temporal lobe (Fig. 1.29).
28 P. A. Iaizzo
Fig. 1.29 In general, speech functions are lateralized to areas responsible for articulation and the execution of
one hemisphere (left panel). Also defined in the left panel speech (motor function) are bilateral, and each half of the
are Broca and Wernicke speech areas. Cortical speech face is represented bilaterally (unlike the rest of the body)
1.6.2 Motor Neuron Recruitment of motor neurons that becomes activated in an

orderly manner during a given motor act. A phys-
Motor neuron recruitment is another elaborate ex- iologically observed task group may or may not
ample of the functional efficiency with which our overlap with anatomically defined motor neuron
brains commonly operate. The primary principles pools, but within each task group, recruitment of
of motor neuron recruitment are such that muscle motor units occurs usually in an orderly fashion,
(or limb) forces are increased smoothly and at the from small to large. Hence, you can hold small
same time overall muscle fatigue is minimized. weights for long periods of time because you
In the 1950s, Henneman and coworkers described are primarily activating motor units composed
the size principle of motor neuron recruitment. of slow twitch fibers; as you add weights, first
In general, the excitation of motor neurons is an you can increase the firing frequencies to these
inverse function of cell size (largest ones last), units and then add additional smaller ones, but
and their order of deactivation is also a direct eventually when greater forces are needed, the
function of cell size (largest ones first). Recall intermediate and fatigable motor units will need
that a motor unit is composed of the alpha motor to be recruited. This example of the hierarchical
neuron and all the muscle fibers that it innervates organization of the motor control system can be
(an example of divergence). All fibers in a unit are considered, in part, responsible for optimizing
of the same skeletal muscle fiber type, thus either performance under a variety of conditions, that is,
slow oxidative (fatigue resistant), fast oxidative through the coordination of different effector sys-
(intermediate type), or fast fatigable. The larger tems and the anticipation of operating constraints.
motor neurons can activate up to 10,000 skeletal For example, reaching and grasping reflect the
muscle fibers and thus as many synaptic con- outputs of two independent, though temporally
nections (neuromuscular junctions), whereas the coupled, motor programs (task groups).
smaller motor neurons typically innervate hun- One can, in turn, define the development of
dreds (or fewer) of the slow twitch fiber type. A motor skill to include (1) components of spa-
motor task group can be defined as a population tiotemporal precision; (2) one’s adaptability to
perform a given task; and/or (3) the relative con- 1.7 Pathophysiology
sistency and functional optimization of a given of the Motor System
movement. In other words:
Much of what we previously learned about motor
• Motor skill involves activating the right mus- control came from the study of various human
cles at the right time. motor disorders and the subsequent observation
• Skilled activities are efficient (metabolic de- of experimental animals in which specific lesions
mand is minimized). of focal injuries have been placed in a given
• Skill develops through practice. motor tract, brain nuclei, or motor center. Yet, as
• Skill involves the creation of mature motor noted above, recent work related to the molecular
programs (i.e., allowing the movement to be- biology of disease origin (e.g., the use of trans-
come more or less automatic). genic animals), the implantation of deep brain
• With increased skill, there is a marked re- stimulation electrodes, and functional imaging of
duction of activity in the auxiliary muscles the brain all have provided many new insights and
(e.g., those originally used for stabilizing or treatments. The following text briefly introduces
assisting), while activity in the prime movers several specific examples of motor defects as an
remains constant. additional means to better understand the hierar-
chical organization of the motor system, as well
Sensory information received from the periph- as its high degree of functional interdependencies
ery is required and processed centrally to deter- between the levels of the hierarchy.
mine limb positions, muscle tensions, etc., so that
adjustments can be made in the brain’s motor
programming before a movement is activated. 1.7.1 Disorders of the Spinal Cord
Subsequent sensory feedback allows for error
detection, so one can make appropriate ongoing A transection of the spinal cord can induce a
corrections either during a given movement or devastating impairment of motor function below
prior to subsequent movements (i.e., by updat- the site of the lesion. Paraplegia, defined as paral-
ing higher motor programs). It should be noted ysis of the lower limbs, is a common occurrence
that a special class of movements, for example, when the spinal cord is severed or damaged in the
ballistic (high velocity movements), are consid- thoracic regions (T2 to T12). Further, quadriple-
ered to be more or less executed without one’s gia, paralysis of all four limbs, occurs when le-
ability to optimize them during the action. These sions (damage) to the cord are more proximal
are also known as “feed-forward” programmed (i.e., within the cervical regions of the cord).
movement; these occur without the active use of When an individual’s spinal cord is completely
sensory feedback during the action, but rather severed, two functional disasters become imme-
with obtained sensory information that the higher diately evident: (1) all voluntary movements in
CNS centers received during such, this informa- one’s body parts innervated by the isolated spinal
tion is then used to set adjustments in advance segments are permanently lost, and (2) all sen-
for subsequent movements (thus updating future sations from those body regions are abolished.
ballistic motor programs). It should also be noted Spinal shock is defined as a transient condition of
that the human brain also controls the limits of decreased synaptic excitability of neurons lying
a given movement. Sensory information from the distal to a transversed section of the spinal cord.
periphery is processed centrally to determine the In other words, initially post-injury, there is a
limits of limb position, muscle tension, etc., in period of minimal reflex activity typically lasting
order to limit both errors in movements and/or from 2 weeks to several months in humans. Next,
potential damage to the body if such movements there is development of flexor hyperactivity (e.g.,
were beyond the normal functional ranges. flexor withdrawal movements which dominate
30 P. A. Iaizzo
for several months). In some patients, mass flex- cord; as such, one can observe improved and ex-
ion reflexes are then possible; in these individu- tended motor abilities over a decerebrate individ-
als, even subtle tactile stimulation is a sufficient ual. However, this individual will (1) lack alpha-
trigger to provoke widespread flexion responses gamma rigidity, (2) have righting reflexes that are
of the limbs. As early as 6 months post-injury intact (labyrinthine and neck righting reflexes),
in humans, extensor activity and tendon reflexes and (3) elicit an improved postural control.
may become hyperactive, and clonus (a series
of involuntary rhythmic contractions and relax-
ations) may be evident. Interestingly, sometime 1.7.3 Disturbances Within
after these periods, spinal standing is possible in the Cerebellum
some patients. It should also be noted that reflex
flexion is still easily elicited in such individuals Alterations in cerebellar activities are manifested
by noxious plantar stimulation, that is, all reflex chiefly as disturbances of muscular coordination,
pathways become hyperexcitable. Much research both during movements and associated with rest-
is ongoing so to treat these aforementioned im- ing muscle tone. Several specific clinical motor
pairments due to spinal cord damage. symptoms can be defined, along with the effects
In contrast to such spinal cord lesions, periph- on an individual’s motor abilities. Asynergia (or
eral paralysis can result from the degeneration of dyssynergia) is defined as an inability to supply
efferent pathways leaving the cord or the motor the correct amounts of neural activities to the
neurons themselves (e.g., in polio). This results various muscles involved in a given movement.
in clinical flaccid paralyses, commonly charac- Therefore, in such individuals, one can observe
terized by (1) reduced muscle tones (hypotonia); that (1) there is decomposition of movement,
(2) muscle atrophy (muscle wasting, due to the such that movements are no longer simultaneous
loss of the trophic influence of nerve on muscle but appear to occur in succession (robotic like);
viability); (3) the diminution (paresis) or ablation (2) dysmetria may occur, which means that a
(paralysis) of the forces of gross movements; given desired movement may go too far or not far
(4) the impairment of fine movements; and/or enough (there can be subsequent overcompensa-
(5) weakening or abolishment of one’s stretch tion); (3) ataxia is common, which can be elicited
reflexes. as an abnormal gait (e.g., walking with feet wide
apart); and (4) adiadochokinesia may occur such
that these individuals are no longer able to carry
1.7.2 Disruption of Functions out rapid movements.
Within the Brain Stem Intention tremors may be present in such in-
dividuals with cerebellar defects, that is, they
A decerebrate individual is defined as one in elicit undesired motor oscillations (tremors) dur-
which the brain stem has been damaged or tran- ing goal-directed movements. Also, when gen-
sected at the level of the tentorium of the cere- eral muscle tone is too low (i.e., hypotonus is a
bellum; in such, the spinal cord becomes more or symptom), there are also associated muscle weak-
less isolated from the red nucleus and more rostral nesses and elicited rapid fatigue. Further, because
motor elements (Fig. 1.30). Decerebrate rigidity of the important role the cerebellum plays in the
soon develops, which is a marked increase in the control of one’s visual gaze, cerebellar defects
tone of the entire extensor musculature (appear- may lead to permanent or prolonged nystagmus
ance of normal standing); tonic neck reflexes are (the presence of abnormal eye movements). Simi-
present. larly, such individuals may also experience a con-
In a midbrain individual, the medulla oblon- tinual dizziness known as vertigo. Nevertheless,
gata, pons, and mesencephalon are all consid- as a general rule, over time cerebellar defects
ered to be left in communication with the spinal often become well compensated for by the CNS.
Fig. 1.30 Major motor centers within the brainstem and resulting in a high spinal individual. Also indicated are
their connections to the cerebellum and spinal cord. Le- excitatory and inhibitory centers. In studying the major
sions or focal damage that would induce differing motor excitatory and inhibitory mechanisms (pathways), one can
abilities include (1) lesion/injury 1, resulting in a so- begin to understand the differing motor abilities that result
called decerebrate human; (2) lesion/injury 2, resulting in as various control centers on the alpha motoneuron are
a midbrain individual; and (3) creation of lesion/injury 3, modulated
1.7.4 Disorders Within the Basal sends inhibitory signals to the corpus striatum.
Ganglia These control signals are normally transmitted
between the involved neurons via synapses uti-
Lesions within the basal ganglion associated with lizing the inhibitory neurotransmitter dopamine.
altered motor control can lead to various forms It should be noted that L-dopa, a dopamine pre-
of movement disturbances. One classic disorder cursor which can cross the blood-brain barrier, is
is Parkinson’s disease, named after the English often used as an early treatment because it locally
physician, James Parkinson, who clinically de- converts to dopamine and activates the terminals
scribed the condition in the early 1800s. There are in the corpus striatum. More recently as symp-
three primary associated symptoms in such pa- toms progress, implantable neurostimulation has
tients: (1) rigidity which can be either widespread been shown to be a potential effective therapy in
or localized; (2) tremors, typically described as selected Parkinson patients.
resting tremors and/or pill rolling hand tremors;
and (3) akinesia which is described as a gen- 1.7.5 Impairment Within the Motor
eral loss of involuntary and associated move- Cortex
ments. Briefly, the functional cause of Parkin-
son’s disease is considered to be a degenera- As noted above, damage of the motor cortex can
tion within the substantia nigra, which normally cause numerous defects in one’s motor abilities.
32 P. A. Iaizzo
One example is the condition known as capsular 1.8.1 Sympathetic System

hemiplegia, which can be caused by lesions in
the region of the motor cortex, which then leads Also known as the thoracolumbar system, the
to over-excitation (e.g., epileptic attacks) or de- sympathetic system arises within the thoracic seg-
ficiency syndromes (which are more rare). The ments and upper two or three lumbar segments
more commonly occurring cortical disorders are of the spinal cord (Fig. 1.32). Thus, the cell bod-
those caused by bleeding or thrombosis in the ies of the preganglionic sympathetic neurons lie
medial cerebral artery, hence those resulting in a within the lateral horn of the thoracic and lumbar
stroke. When a stroke occurs in an individual, it cord. These axons are very thin, but many are
initially leads to a shock stage characterized by myelinated and their average conduction veloc-
flaccid paralysis of the contralateral side of the ities range between 1 and 20 m/s. More specif-
body. Later spasticity in the antigravity muscle ically, the axons of these efferents leave via the
may be observed (extensors in legs and flexors in ventral roots and white rami and then enter paired
arms), commonly referred to as spastic hemiple- paravertebral ganglia; postganglionic axons are
gia. much longer and more variable in their lengths
than preganglionic axons. Note that there are mul-
tiple intrasegmental interconnections between the
1.8 The Autonomic Nervous sympathetic ganglia which, in turn, helps to coor-
System dinate ANS efferent functions/responses through-
out one’s body.
The autonomic nervous system (ANS) mediates
the neuronal regulation of the internal milieu of
the human body (e.g., organ systems) and thus 1.8.2 Parasympathetic System
governs/innervates the smooth musculature, the
heart, digestive organs, and various glands. The Also referred to as the craniosacral system, the
action of this system is generally considered not parasympathetic system encompasses cell bodies
under direct cognitive voluntary control, and fur- of preganglionic neurons, which are located both
ther, the afferent information is usually not acces- within the brain stem and the sacral portions of
sible to consciousness. Although its main func- the spinal cord (Fig. 1.32). More specifically, the
tion is to maintain homeostasis, the ANS also preganglionic axons innervate eye muscles and
controls functions not related to this goal (e.g., the various facial glands and leave the brain stem
control of the sexual organs and the intraocular in the spinal nerves. Some axons are myelinated
muscles). and the cell bodies of the postganglionic neurons
In the ANS, the groups of axons between the are near or actually within the effector organs.
CNS and the effector cells consist of a minimum
number of two neurons and one synapse. The
cell body of the first neuron lies within the CNS, 1.8.3 Neurotransmitters in the ANS
whereas the synapses between the two peripheral
effector neurons are outside the CNS, located in Importantly, many or most of the internal organs
a cell cluster called the autonomic ganglion (Fig. within our bodies receive both sympathetic and
1.31). parasympathetic innervations. Note that, in gen-
Anatomical and physiological differences eral, the physiological influences of these two
within the ANS are the basis for its further subdi- systems on a given organ system are antagonistic.
vision into the sympathetic and parasympathetic On the other hand, the overall control of the ANS
components, each having their origins at different can be thought to be functionally synergistic. This
levels of the neuraxis (i.e., the brain stem and combined ANS control is like a braking/acceler-
spinal cord). ating system in a car, which allows for greater
control (quicker accelerations and faster stops).
Fig. 1.31 Provided here is

a comparison of the major
synaptic connection in the
somatic (innervating
skeletal muscles) and
autonomic nervous systems
(ANS). Note the additional
synapse (with cell bodies
forming ganglion) in the
ANS. Hence, one can
define preganglionic and
postganglionic efferent
fibers in the ANS. CNS
central nervous system
Fig. 1.32 Shown here is the general arrangement of the peripheral component of the sympathetic part of the autonomic
nervous system. This system also innervates vessels, sweat glands, and piloerector muscle (associated with hair follicles)
34 P. A. Iaizzo
parasympathetic system predominates during

rest or sleep; it decreases both heart rate and
contractile forces of the heart, increases intestinal
motilities throughout, induces contractions of the
gallbladder, and reduces bronchi diameters.
1.8.4 The Adrenal Medulla
The adrenal medulla lies within the inner cores

of the paired adrenal glands, which lie above
each kidney. These endocrine glands are com-
Fig. 1.33 The general actions of the catecholamines monly described as modified sympathetic gan-
noradrenaline (NA or norepinephrine), adrenalin (A or
epinephrine), and isoproterenol (I) on the adrenergic re-
glion. The adrenal medulla secretes amine hor-
ceptors of a given smooth muscle mones (epinephrine and norepinephrine), yet typ-
ically higher levels of epinephrine are released
in humans. Thus, the adrenal cells release cat-
All preganglionic synapses in the ANS use acetyl- echolamines/hormones into their focal capillary
choline as their neurotransmitter (hence, they are beds which in turn enter into the general cir-
referred to as cholinergic synapses). However, culation. The circulating hormones, via binding
two subpopulations of cholinergic synapses can to receptors on individual cells, act to regulate
be distinguished in the ANS and are classified metabolic processes throughout one’s body (Fig.
as two types of macromolecular receptors which 1.34). We are typically aware when there has
respond preferentially to the presence of various been an increased release of catecholamines in
modulating molecules: (1) nicotinic, the recep- response to various stressful stimuli (“our hearts
tors located on the postsynaptic membranes of are racing”); similarly, catecholamines can be re-
postganglionic neurons, and (2) muscarinic, the leased from these glands in response to emotional
receptors located on effector cells. stresses.
Further, the parasympathetic postganglionic
synapses are mainly of the cholinergic type
(acetylcholine), whereas those in the sympathetic 1.8.5 Central Organization
system are composed of an additional macro- of the ANS
molecule, which utilizes norepinephrine as the
primary neurotransmitter—adrenergic synapses Neurogenic resting activities are a fundamental
(will also respond to other catecholamine property underlying the autonomic control of our
molecules; see Fig. 1.33). Importantly, more various organ functions. If there are fairly con-
than one type of adrenergic receptor has sistent resting discharge rates, system effects can
been identified within the sympathetic system be modulated by either increasing or decreasing
(e.g., alpha, beta 1, beta 2 receptors can be these neurogenic rates (i.e., discharge rates of
identified; hence, there are varied targets for action potentials). The resting discharge rates in
pharmacological therapies). the ANS, for both the sympathetic and parasym-
The general functional role of the sympathetic pathetic neurons, are approximately 2 Hz. Addi-
nervous system is to respond rapidly to stress tionally, it is these resting frequencies of action
and danger, hence diverting resources from the potentials through which smooth muscle tones are
viscera to the somatic musculature. Furthermore, maintained. Note that changes in these discharge
this so-called fight or flight reaction is backed activities are regulated by both afferent inputs
up by the secretion of catecholamines from the via reflex networks and efferent controls from
adrenal medulla (see below). In contrast, the the CNS higher centers. Segmental reflexes also
Fig. 1.34 A schematic of the widespread actions following the release of catecholamines from the adrenal medulla.
These responses would aid an individual’s “fight or flight” response to danger
exist within the ANS, and the minimal number of (e.g., extreme hot and cold temperatures). The
synapses in these autonomic reflex arcs is three, hypothalamus is a small region of the inferior
with one lying within the CNS. If an individual brain, which is considered as a neuronal con-
suffers a spinal cord injury, the ensuing spinal tinuum extending from the midbrain through to
shock also typically results in 1–6 months of ANS the basal regions of the telencephalon. Further,
hyporeflexia; clinically, the skin below the trauma the lateral hypothalamus can be thought to be
appears dry and rosy due to low sympathetic reciprocally connected with both the upper brain
tone (i.e., vasodilatation), and later hyperreflexia stem and the limbic system (these are the brain
(e.g., heavy uncontrolled sweating, high blood centers which control emotions, learning, etc.,
pressure) is often present. as described below). As such, the hypothalamus
receives primary sensory inputs from afferents
near the body surface as well as from internal
1.9 The Hypothalamus structures via the ascending spinobulboreticular
and Homeostasis pathways (Fig. 1.35).
In contrast, the medial hypothalamus receives
The hypothalamus is the so-called brain center its main inputs from the lateral hypothalamic
which governs all essential “homeostatic” func- regions. These medial regions of the hypothala-
tions of the human body. These integrative func- mus also contain specialized neurons important
tions include control over (1) the autonomic ner- for sensing the conditions of both the blood and
vous system, (2) various somatic pathways, and cerebrospinal fluid. In turn, the medial hypothala-
(3) the body’s hormonal systems. Briefly, home- mus makes numerous connections to the pituitary
ostasis can be defined as the control of the internal gland (or hypophysis). There are two main types
milieu which, in general, is kept nearly constant signaling connection between these structures:
within narrow limit, that is, despite potential se- (1) neuronal connections to the neurohypoph-
vere perturbations that our bodies can experience ysis (axonal) and (2) hormonal system to the
36 P. A. Iaizzo
Fig. 1.35 Shown here are the general afferent and ef-
ferent pathways/connections of the hypothalamus (me-
dial and lateral), the pituitary gland (adeno- and neuro-
hypophysis), the limbic system, the thalamus, and the
mesencephalon. Note that the medial hypothalamus, via
the neuroendocrine interface, controls the functions of the Fig. 1.36 There are multilevel feedback loops that are
pituitary gland employed to regulate both hormone levels and neural
responses. For example, the medial hypothalamus can
sense blood levels of releasing hormones, hormone levels
adenohypophysis (to its anterior region). Thus, released by the pituitary gland, and also those released by
these multimodal connections are often referred target endocrine glands
to as a neuroendocrine interface (Fig. 1.35). Also
commonly defined as the hypothalamo-pituitary patterned cardiovascular response (change in
system, the activities of most endocrine glands are function/activity; Fig. 1.37). Yet, as described
regulated by hormones released from the adeno- above, these hypothalamic effects on the
hypophysis (anterior pituitary). It should be noted cardiovascular responses are typically mediated
that the hypothalamus releases both stimulating by appropriate/synergistic parasympathetic and
and inhibitory releasing hormones that, in turn, sympathetic pathways (modulations in firing
can affect the pituitary responses. rates). Additionally, afferent inputs for this
The tight control of homeostatic functions control are many and include those from baro-,
that are modulated via the hormone system chemo-, and mechanoreceptors in the atria,
is accomplished by multilevel, multi-hormone ventricles, aorta, and elsewhere.
feedback mechanisms. For example, the blood It should be emphasized that the hypothalamus
levels of releasing hormones as well as the exerts neuronal control over various autoregu-
released hormones by the pituitary can both be lation systems within the human body that are
sensed by specialized neurons within the medial deemed critical for species survival (Fig. 1.38).
hypothalamus itself (Fig. 1.36). Interestingly, Several examples of functions that are modulated
electrical stimulation of nearly any neural region in this way include (1) our ability to thermoregu-
within the hypothalamus is likely to cause a late; (2) the regulation of food intake; (3) our drive
1.10 Regulation of Body

Temperature:
Thermoregulation
In this section, regulation of body temperature

will be reviewed as a more detailed example of
humans’ extraordinary abilities for homeostasis.
Thermoregulation is similar to many other phys-
iologic control systems, in that the CNS center
uses negative feedback to minimize perturbations
from some predetermined preset “normal” values
Fig. 1.37 The body’s cardiovascular responses are more
or less under involuntary control and thus are regulated (similar to how room temperatures are regulated
by the autonomic nervous system (ANS). For example, via a thermostatic control system; Fig. 1.39). As
stimulation of any region of the medial hypothalamus such, destruction of associated controls centers
will induce changes in cardiovascular responses. These
within the hypothalamus will, in turn, cause poor
patterned responses are commonly associated with innate
behavior responses that are also attributed to the hypotha- regulation of one’s body temperature. Yet it
lamus, such as feeding or defensive behaviors which, in should be noted that sites other than those in
turn, appropriately modulate other body systems under the hypothalamus are also considered important
ANS control
for contributing to our thermoregulatory abilities
(e.g., both higher brain centers and the spinal
for fluid intake (or thirst); and (4) one’s sexual cord). By definition, humans are homeothermic,
drive and associated behaviors (e.g., control of or capable of regulating their body temperatures
penile erection and ejaculation). In other words, within very narrow limits. In contrast, body
like the control of cardiovascular responses, stim- temperatures in a poikilothermic individual (e.g.,
ulation of small areas of the hypothalamus can frog or an anesthetized patient) are commonly
cause an animal to elicit these aforementioned slightly higher than ambient temperatures. Thus,
characteristic behaviors. Note that additional be- homeotherms are spared the slowdown of bodily
haviors under the control of the hypothalamus functions which occurs in poikilotherms when
can be even more complex in nature and thus there is a decrease in environmental temperatures.
simultaneously involve intersegmental somatic, However, the advantages obtained by higher body
autonomic, and hormonal components. Examples temperatures come with the greatly imposed need
of such behavior patterns include the initiation for regulation and input energy.
of (1) defense and fighting (facial expression, When one considers human thermoregulation,
assuming a desired posture, etc.); (2) eating and the concept of an optimal temperature needs in-
drinking (searching behaviors); (3) reproductive troduction. Virtually every cell in the human body
behavior (mating rituals); and (4) thermoregula- functions most efficiently at 37 ◦ C (98.6 ◦ F). Yet,
tory responses (see below for details). Further- heat is also produced within the body by each
more, anatomically defined structures (e.g., nu- individual cell regardless of its primary cellular
clei) have been identified for a number of home- function. Some of this heat energy is retained in
ostatic functions including satiety, hunger, ther- the body, and the rest is discharged in a variety of
moregulation, and sleep. It should be noted that ways. Nevertheless, if the human body gets ex-
numerous functional disturbances can therefore cessively warm or cold, bodily functions become
result from damage to the hypothalamus (i.e., impaired and when extreme, eventually death will
commonly caused by tumors, trauma, or inflam- ensue.
mation).
38 P. A. Iaizzo
Fig. 1.38 Diagram of the functional organization of hy- sponses, thus affecting the endocrine, autonomic nervous,
pothalamic behavior patterns. The limbic system has direct and somatic nervous systems appropriately and simultane-
inputs into the lateral hypothalamus which, in turn, can ously
activate fixed programs to regulate various behavior re-
Fig. 1.39 Basic components that make up the negative production which will be influenced by internal and/or
feedback control system of body temperature regulation. external parameters
Efferent mechanisms will either involve heat loss or heat
NORMAL
Range Range with
at Rest Exercise
34 35 36 37 38 39 40 41 42 43 44 45 ºC
Death
Heat IIIness and
HYPOTHERMIA FEVER
Hyperthermic Therapy
Death
Cold Exposure and HYPER THERMIA

Surgical Hypothermia
Fig. 1.40 The temperature profile associated with ther- moderate to intense exercise, and the rate of increase will
moregulation in humans. Shown are changes in core body be exaggerated when exercising in a hot environment.
temperatures (◦ C). At rest, a human’s normal body tem- Clinical hypothermia is typically defined as a core body
peratures can range over 1 ◦ C throughout the day. For temperature below 36.5 ◦ C, which typically occurs under
example, our core temperatures typically drop while we the influence of general anesthesia during surgery. In an
sleep and may increase after eating a large meal. It is uncontrolled situation, if a human’s body temperature gets
quite normal that core body temperature will increase with too cold or too warm, death will occur
1.10.1 Core Temperature nonthermal stimuli. Like other receptors, each of

these populations can be defined by a specific
An interesting feature of thermoregulation is the receptive area (warm and cold points on the skin),
fact that the central body (vital organs) or core and their densities vary throughout the human
temperature (brain, spinal cord, heart, lungs, and body, as do their neuronal contributions within
thorax) and the peripheral temperature (limbs and the somatosensory cortex (Fig. 1.19); each af-
skin) would ideally both be set at approximately ferent fiber usually supplies only one warm or
37 ◦ C. Yet, during our normal daily lives, the cold point (an area of 1 mm2 or less). Like other
peripheral components can vary greatly from this receptor populations for a given modality, the
set point, with minimal consequences to our over- qualities of the receptors may also elicit specified
all human performance, whereas slight changes ranges or have a varied temporal responsiveness.
in our core temperatures (1–2 ◦ C) could have More specifically, some thermal receptors can
profound consequences. In general, hypothermia be considered to elicit static temperature sen-
is defined as significant decreases in core temper- sations, that is, temperature changes are related
atures (1–2 ◦ C), and hyperthermia is a significant to alterations in the action potential discharge
elevation in core temperature (2–3 ◦ C) (Fig. 1.40). rate in the afferent fibers and the discharge rates
Note that a human’s core body temperature can are proportional to the skin temperatures. Other
vary throughout the day and that some degree of thermal receptors elicit more phasic properties (or
hyperthermia may be induced by normal exercise. adaptation), for example, their sensation of tem-
perature fades away after a short period of time.
The environmental temperature range in which
1.10.2 Cutaneous Thermoreception complete adaptation occurs is called the neutral
(comfort) zone; its limits are 36 ◦ C to 30 ◦ C. The
Related to thermoregulation in humans, there are dynamic temperature sensations experienced by
two objective and subjective demonstrable qual- an individual while his or her skin temperatures
ities that an individual can define—the sense of are changing are basically determined by three
cold and warmth. As such, there are both specific parameters: (1) the initial temperature of one’s
warm and cold receptors that are insensitive to skin, (2) the rate of regional temperature changes,
40 P. A. Iaizzo
Thermoregulatory System:
Warm responses
Hypothalamus
Hypothalamus Active vasodilation
Other parts of brain Sweating
Skin surface Interthreshold
Range Vasoconstriction
Spinal cord
Nonshivering
Thermogenesis
Deep central tissues Shivering
Cold responses
Fig. 1.41 Thermoregulatory responses are based on tance of various tissues. The inputs are integrated within
mean body temperature, which is a physiologically the hypothalamic centers for temperature control to then
weighted average reflecting the thermoregulatory impor- elicit the appropriate warm or cold effector responses
Table 1.1 Body effector responses to cold and warm crease metabolic heat production and/or alter heat
Cold Warm losses or gains to or from the environment. It
Behaviorala Behaviorala should be noted that behavioral responses are
Vasoconstrictiona Vasodilationa primary reactionary responses to an uncomfort-
Non-shivering thermogenesis a Sweatinga able thermal environment, for example, you wear
Shivering a Panting many layers of warm clothes if you plan to go
Piloerectiona Salivation outside on a cold day, or remove or minimize
Fluffing of feathers clothing in a hot environment. It should be noted
Decreasing body temperature that these behavior effector mechanisms for ther-
a Responses which typically occur in humans mal management in clinical situations (e.g., in
the operating room) are taken over by healthcare
providers.
and (3) the size of the skin areas affected by the In the healthy individual, the primary
stimuli. mechanisms for heat production include (1)
voluntary muscular activities and behavior
changes (e.g., running in place or moving your
1.10.3 Central Thermoregulation limbs around); (2) involuntary tonic (tensing)
and rhythmic skeletal muscle activities (shiver);
As noted above, body temperature is regulated and/or (3) non-shivering thermogenesis (not
by central structures (primarily by the hypotha- associated with muscular contraction), including
lamus) that compare integrated thermal inputs the increased releases of both epinephrine and
from the skin surface, the neuroaxis, and/or deep thyroxine. The primary mechanisms to conserve
tissues. If an integrated thermal input exceeds heat include vasoconstriction of peripheral blood
one of the threshold temperatures for heat or vessels (via sympathetics) and behaviorally
cold, appropriate effector responses to maintain minimizing one’s body surface area to conserve
adequate temperature are initiated (Fig. 1.41). heat (e.g., if you fall into cold water, it is
Table 1.1 lists the primary body effector re- recommended for survival that you assume the
sponses to thermal perturbations via a variety of HELP position—heat escape lessening position,
mechanisms that will, in turn, increase or de- tucking your arms and legs close to your body).
The primary mechanisms for heat loss include nucleus, the hypothalamus (including the mam-
the following: millary bodies), and the anterior nucleus of the
thalamus (Fig. 1.43).
• Radiation: Electromagnetic waves are both In general terms, the limbic system controls
emitted and absorbed by the body. emotional behaviors and thus numerous com-
• Conduction: The simple transfer of thermal plex internal factors which motivate human ac-
energy from atom to atom or molecule to tions. Changes in one’s limbic system will disrupt
molecule in contact with each other; heat flows emotional behavior patterns (e.g., an amygdalec-
down a concentration gradient. tomized animal is incapable of functioning as a
• Convection: The process whereby air (or wa- member of a social group). The expression of
ter) next to the body is heated, moves away emotions is considered based largely on inherited,
(convective currents) from the body, and is inborn reactions. A complex of neurons known as
replaced by cool air or water. the monoaminergic system appears to be of great
• Passive evaporation: Evaporation from the importance in the global regulation of such be-
skin and respiratory tract in the absence of haviors. This group of neurons includes dopamin-
sweat (600 ml/day). ergic, noradrenergic (norepinephrine), and sero-
• Active evaporation: Sweat is actively secreted tonergic neurons that originate in the brain stem
from sweat glands and extruded into ducts, and innervate practically all regions of the brain.
which lead to the skin surface. The sweat is Brain areas in which self-stimulation has been
pumped to the surface by periodic contrac- reported to occur are also the brain regions in
tions of cells resembling smooth muscle in which there is vast innervation or input from
the ducts. Production and delivery of sweat to catecholaminergic neurons.
the surface are stimulated by the sympathetic These responses differ from those described
nervous system. for the ascending reticular activating system
within the CNS. It has been described that diffuse
So if we now embellish upon the negative electrical stimulation in the mesencephalic
feedback system for thermoregulation to include and pontine portions of the reticular formation
the aforementioned mechanisms, such a diagram can cause immediate and marked activation
can grow in complexity (Fig. 1.42). of the cerebral cortex and will even cause a
sleeping individual to awaken instantaneously.
This system is considered to extend upward
1.11 The Limbic from the mesencephalic reticular formation in
and the Ascending Reticular multiple diffuse pathways, which then terminate
Activating Systems in almost all areas of both the diencephalon
and cerebrum (thalamus and cortical gray
The human limbic system is associated with an matter). In other words, they are considered
individual’s ability to change moods and innate as nonspecific projections, and it is important
incentives to action (a person’s motivational in- to distinguish these from the classical sensory
teractions and emotions) but is also important in specific projections. Additionally, one can
the processes of learning and memory (Fig. 1.43). subdivide this system into two main ascending
The concept of a limbic system has primarily de- pathways: (1) the first passes upward to the
veloped from comparative neuroanatomic studies intralaminar midline, to the reticular nuclei of the
as well as through neurophysiological investiga- thalamus, and then subsequently through relay
tions. The following regions, composed of gray pathways to essentially all parts of the cerebral
matter (neural cell bodies), are anatomically in- cortex and basal ganglia; and (2) the other which
cluded in this system: the limbic lobe (consisting passes upward through the subthalamus, to the
of the cingulate and parahippocampal gyri), the hypothalamus, and then to their adjacent areas.
hippocampal formation, parts of the amygdaloid
42 P. A. Iaizzo
Fig. 1.42
Thermoregulation is
similar to many other
physiologic control
systems, in that the central
nervous center uses
negative feedback to
minimize perturbations
from some predetermined
set point. The
hypothalamus, which is the
primary control center,
receives thermal inputs
from numerous body
structures as well as from
sensors within the central
nervous system itself (e.g.,
medial hypothalamus and
spinal cord) and then
integrates a broad range of
functions to maintain
thermal control. The
efferent mechanisms
include the autonomic
nervous system (e.g.,
vasoconstriction,
vasodilation, sweating),
various somatic pathways
(e.g., behavior responses,
shiver), and/or the body’s
hormonal systems (e.g., the
release of thyroxine and
epinephrine)
1.11.1 Function of the Various ing system itself is subject to stimulation (arousal)
Portions of the Reticular or inhibition (which can lead to sleep). For ex-
Activating System ample, sensory stimuli from almost any part of
the body can cause arousal or induced activations.
In general, one can define two brain regions that It should be noted that some signals are much
contain structures associated with the reticular more stimulatory than others, such as pain and
activating system: (1) one being located within proprioceptive somatic impulses. The reticular
the mesencephalic and brain stem regions and formation receives tremendous input through a
(2) the other lies within the thalamus (Fig. 1.44). number of signals via the spinoreticular tracts, the
The reticular formation of the mesencephalon spinothalamic tracts (collaterals), and the spino-
and upper pons acts as an intrinsic activating tectal tracts. Modulation of these reticular centers
center; damage or a lesion present above this can also be retrograde in nature; these activities
center will induce coma. In contrast, stimulations are mainly directed at the mesencephalic portion
of the reticular areas in the brain stem below the of the reticular formation. These latter pathways
midline of the pons can inhibit this activating include those from (1) the somatic sensory cortex,
system and cause sleep. Relative to the thalamic (2) the motor cortex, (3) the frontal cortex, (4)
portion of this system, it has been noted that the basal ganglia, (5) the hippocampus and other
selective stimulations of various regions cause limbic structures, and (6) the hypothalamus. Con-
specific activations of only certain areas of the sider that one can try to resist the drive to sleep
cerebral cortex. Furthermore, the reticular activat- by movement, eating behaviors, or focusing one’s
Fig. 1.43 Primary brain regions which have been asso- These numerous brain regions work together to control
ciated with the so-called limbic system within humans. emotional behavior and numerous complex internal fac-
tors related to one’s internal motivation
attention on a topic. The state of wakefulness 1.11.2 Brain Waves

can generally be defined as activities in the brain
directed into the appropriate channels to give a Electrical recordings from the surface of the brain
person a sense of conscious awareness. A coma or from the outer surface of the head demonstrate
is defined as the opposite of brain activation. A continuous electrical activities within various un-
coma can result from any factor that diminishes derlying regions of the cortex. Both the intensi-
or stops activities in the mesencephalic portions ties and patterns of these electrical activities are
of the reticular activating system, for example, determined to a great extent by the overall levels
(1) from a brain tumor compression of the brain of regional excitations which, as noted above, are
stem, (2) due to vascular lesions caused by death controlled by the reticular activating system. An
of neural tissue (e.g., due to hypoxia or toxins), electroencephalogram (EEG) can be simply de-
or (3) as the result of infectious processes in the fined as a record of the brain’s regional electrical
brain stem (e.g., encephalitis). A coma is distinct activities (changes in the brain’s electrical fields).
from sleep in that a person cannot be aroused from An EEG can be recorded as a set of surface
a coma. potentials by placing electrodes on the scalp.
We are all aware that we can direct our atten- The monitored signals can range between 0 and
tion toward certain of our mental activities indi- 300 mV, and their frequencies range from 0 to
vidually. This ability has been assigned to that of approximately 50 Hz. The characteristics of the
the thalamocortical system, which can apparently recorded waves, the EEG patterns, are highly
activate small areas of the cerebral cortex. Also dependent on the degree of activities within
recall that the thalamus is the entryway for all one’s cerebral cortex. The features of these waves
sensory nervous signals to the cerebral cortex, and change markedly between states of wakefulness,
the stimulation of the various thalamic nuclei can sleep, and coma.
cause transient increases in cortical activity.
44 P. A. Iaizzo
Fig. 1.45 Recorded surface potentials from an electroen-

cephalogram (EEG) obtained by an array of electrodes
placed on the scalp. In this case, abnormal activity, perhaps
Fig. 1.44 Brain regions and several of the specific neural due to a tumor, was identified between a given pair of
pathways associated with the reticular activating system. electrodes, hence also identifying the relative location of
These specific nuclei include (1) the gigantocellular nu- the abnormal function
cleus, neuronal bodies that lie in the medial portions of the
reticular formation in the mesencephalon and upper pons
and are the principal activator portion of the reticular sys-
tem (these neurons release acetylcholine, a normally ex-
citatory transmitter); (2) the substantia nigra, the nucleus
that lies in the anterior portion of the mesencephalon and
contains neuronal cell bodies that secrete dopamine at their
nerve endings (axons of these neurons make connections
to the basal ganglia, hypothalamus, and cerebral cortex);
(3) the locus ceruleus, which is a small area located
bilaterally and posteriorly at the junction point between the
pons and mesencephalon and contains nerves that secrete
norepinephrine with either excitatory or inhibitory effects,
depending on the synaptic receptors (this center is thought
to play a role in REM sleep); and (4) the raphe nuclei, Fig. 1.46 Types of normal EEG waves
several very thin nuclei located in the midline of the lower
pons and medulla that secrete serotonin and send fiber to
widespread areas in the diencephalon and spinal cord (the state (mainly recorded from occipital cortical
release of serotonin from these fibers plays an essential regions).
role in causing normal sleep) • Beta waves: 14–25 Hz; activation patterns of
the CNS that typically occur when a person is
under tension (mainly recorded from parietal
Even in a healthy individual, EEG patterns and frontal cortical regions).
are irregular much of the time, but under certain • Theta waves: 4–7 Hz; typically elicited dur-
conditions, distinct patterns do appear. Some are ing emotional stress, disappointment, and frus-
characteristic of certain pathological behaviors tration (associated with parietal and temporal
(e.g., epilepsy), and others occur in all normal lobes).
individuals (Fig. 1.45). When a regular pattern is • Delta waves: Less than 3.5 Hz; these typically
sustained, it is commonly defined as one of the occur during deep sleep, in infancy, and/or
following waveforms (Fig. 1.46): in cases of serious organic brain disease
(throughout cortex).
• Alpha waves: 8–13 Hz, 50 mV; typically oc-
curring during quiet wakefulness or a rested When an individual’s level of activities
changes, so will his or her EEG patterns. For
Fig. 1.47 Rapid transition in the EEG waveforms due to

the evoked response of simply opening one’s eyes
Fig. 1.49 Generalized sleep pattern that may occur dur-

ing an 8-hour stay in bed
As an individual passes from a state of wake-

fulness into sleep, one can often observe charac-
teristic changes in EEG patterns. These general
patterns are the following:
• Alert wakefulness: Typically beta waves are

elicited.
• Quiet wakefulness: Primarily composed of al-
pha waves.
• Slow wave sleep (or non-REM sleep): These
stages of sleep are associated with decreases in
Fig. 1.48 Typical EEG patterns in various stages of
wakefulness and sleep both peripheral vascular tones and many other
vegetative functions (blood pressure, respira-
tory rate, and basal metabolic rate). Dreaming
example, alpha waves can be elicited while also occurs during these stages of sleep, but
resting when your eyes are closed, but typically they are not thought to be often remembered.
not detected when they are open (Fig. 1.47). Slow wave sleep can be divided into four addi-
tional stages: (1) stage 1, very light sleep, elic-
iting low-voltage EEG with “sleep spindles”
1.11.3 Sleep (bursts of alpha waves 8–13 Hz); (2) stages 2
and 3, theta waves (4–7 Hz) are common; and
Sleep is defined as a state of unconsciousness (3) stage 4 sleep, in which the EEG frequen-
from which a person can be aroused by appropri- cies become progressively slower until delta
ate sensory or other stimuli. During each night, waves are often present (< 3.5 Hz). Note that
a healthy person goes through multiple stages of the elicitation of delta waves probably occurs
two different types of sleep that alternate with intrinsically within the cortex, that is, when
each other (Figs. 1.48 and 1.49): (1) slow wave the cortex is not being driven by the reticular
sleep, or low-frequency EEG waves, and (2) rapid activating system.
eye movement (REM) sleep, also known as para- • REM sleep: The EEG patterns in this stage
doxical sleep. It is during REM sleep that the eyes of sleep are similar to that of early stages of
undergo rapid movements despite the fact that the wakefulness. Bouts of REM sleep typically
person is asleep (25% of sleep in this form occurs last between 5 and 30 minutes; in the ex-
roughly every 90 minutes). This type of sleep is tremely physical tired individual, the bouts of
more often associated with dreams that can be REM can be very short or even absent. REM
recalled. sleep is usually associated with active dream-
ing. It has been reported that a person is more
46 P. A. Iaizzo
difficult to arouse by sensory stimuli during muscles go into a relaxed state, and the overall
REM sleep, yet, to the contrary, we usually basal metabolic rates of the body can fall by 10 to
awaken in the morning during a bout of REM. 30% (hence, core temperature will decrease).
Importantly, somatic muscle tone is depressed
during REM sleep due to strong inhibition of
the spinal projections from the reticular system 1.12 Pain
(thus, we normally cannot physically act out
our dreams). During REM sleep, both heart Pain is the sensory modality that one may
rate and respiration may become irregular. In consider being of greatest importance in human
general, the EEG patterns during REM sleep medicine, for it is the effects of noxious
appear typical of those during wakefulness. stimuli or associated influences that bring
It is considered that the brain is quite active patients to physicians. Recent work in the
during REM sleep, but proper brain channels field of neuroengineering has focused on
are not activated as to allow a person to be modulating pain. Unlike the other sensory
aware of their surroundings. modalities, pain contributes little or no sensory
information regarding one’s surrounding external
environment. Yet, the detection of pain can be
1.11.4 Mechanisms of Sleep considered as indispensable for a normal life,
that is, note that the protective functions of pain
In broad terms, sleep can be considered as an ac- receptors (nociceptors) are not available through
tive inhibitory process. Exact sleep mechanisms other sensory modalities. Interestingly, it has been
are in general not known, but certain factors have reported that females can withstand a specific
been described. Importantly, it is known that hu- pain longer, but that males can tolerate more
mans have an essential sleep requirement; in other intense transient pain. The qualities of pain can
words, for the well-being of bodily functions, be roughly categorized according to either the
sleep is absolutely necessary. Yet, one’s sleep re- site of origin or by the nature of the pain itself
quirement is known to change with age and/or the (Fig. 1.50).
rate of body development. A biochemical theory
of waking and sleep has been proposed that postu-
lates decreases in the amounts of serotonin in the 1.12.1 Intensity of Pain (Quantity)
raphe nuclei in the brain stem will result in insom-
nia. Further, the amount of REM sleep can be re- All tissue-damaging or “noxious” stimuli give
duced by decreasing concentrations of serotonin rise to pain, and no particular single adequate
within one’s brain. Additionally, a decrease in the stimulus for pain can be specified. Like other
concentration of norepinephrine (noradrenaline) receptor systems, nociceptors in the skin are not
within the neurons of the locus ceruleus will result equally distributed (Fig. 1.51). It is difficult to
in the disappearance of slow wave sleep only. quantify pain, and, importantly, it is one’s effec-
Nevertheless, it has been described that tive reactions to pain that are of greater concern
prolonged wakefulness is associated with pro- to both patients and physicians than the actual
gressive malfunction of the human mind and can physiological aspects. Humans have the ability
also cause abnormal behavioral activities of one’s to redirect their attention to or away from this
nervous system (e.g., irritability and eventual psy- modality; that is, by doing so, one can weaken
chosis). It is considered that during sleep, sym- the sensation of pain and, in extreme cases, can
pathetic nervous system activity decreases while abolish it.
parasympathetic activity increases. Therefore, as Numerous injurious stimuli can activate noci-
would be predicted, arterial blood pressure falls, ceptors and thus be perceived as pain, including
pulse rate decreases, skin vessels dilate, activ- (1) mechanical stimuli, (2) thermal stimuli (which
ity of the gastrointestinal tract increases, skeletal can also be in the form of radiation which avoids
placed in contact with the skin can induce pain re-

sponses, including acetylcholine, serotonin, his-
tamine, H+ ions, K+ ions, and various peptides
(e.g., bradykinin). Importantly, there is a lack
of adaptation to pain, for example, headaches
and toothaches can last for hours. Yet, like other
somatosensory modalities, there is a specificity
theory that applies to the modality of pain (i.e.,
the identification of pain points in various tissues;
Fig. 1.51).
Nociceptors or pain receptors can be of several
types: (1) pure, those that respond to one type
of noxious stimuli (i.e., only respond to a given
thermal, mechanical, and chemical stimulus); (2)
polymodal, or those receptors responding to more
Fig. 1.50 Classification scheme of the sensory modality than one type of noxious stimuli; or (3) free
of pain. Superficial somatic pain arises from the skin
nociceptors. It is felt as an initial or “flash” pain (i.e.,
nerve endings, with specific membrane receptors
needle piercing the skin; a readily localized pain that fades for molecular activators of pain (e.g., H+ , K+ ,
rapidly) or as a delayed pain, with a latency of 0.5–1.0 sec- and/or ATP). Like other receptor populations,
onds, which has a dull burning character and dies out some types of nociceptors may contain filters;
slowly and is more difficult to localize. Deep somatic pain
typically originates from connective tissue, bones, joints,
that is, they possess a corpuscular anatomy. The
and muscles. This pain is usually dull in nature, is poorly axons of these nociceptive afferent fibers, via
localized, and tends to radiate into the surrounding areas. interneurons, send ascending signals within the
Visceral pain originates from one’s internal organs: this ventral half of the spinal cord to higher CNS
pain also tends to be dull or diffuse in character and thus
resembles deep pain accompanied with severe autonomic
structures. This specific pathway is often referred
responses. For example, spasms or strong contractions of to as the ascending spinothalamic tract or the
smooth muscles are very painful, especially when they are anterolateral column. Hence, this nociceptive in-
associated with ischemia (e.g., kidney stones) formation from the spinal cord ascends upward to
forebrain projections. Nevertheless, these projec-
tions can be considered to travel to the thalamus
and cortex via multiple sensory pathways (Fig.
1.52).
Special forms of pain can be defined which
include (1) projected pain (Fig. 1.52); (2) neu-
ralgia, a continuous painful irritation; and (3)
referred pain, sensations produced by nociceptors
in the viscera that are felt as having occurred at
Fig. 1.51 Specificity of the sense of pain. Theory postu- a distant site (Fig. 1.53). The underlying mecha-
lates the existence of special pain receptors which respond nisms for this latter form of pain are via interneu-
only to high intensity stimuli. Pain points are specific sites
on the skin similar to those of mechano- and thermore- ronal connections of the same neurons of origin in
ceptors. It should be noted that the skin is not uniformly the spinothalamic tract (e.g., if you are suffering
sensitive to pain a heart attack, you may feel pain in your left arm
and shoulder, or if your esophagus gets too cold,
you detect it as a central headache).
mechanical contact), and (3) chemical stimuli,
Considerable clinical time and effort are in-
which will elicit pain when a given pain ago-
vested in managing an individual’s response to
nist is placed in direct contact with skin. There
noxious stimuli, injuries, and/or underlying dis-
is no one pain substance, rather several agents
ease process, and rightly so. Pain hurts and this in-
48 P. A. Iaizzo
Fig. 1.52 Schematic for projected pain. In this case, pain signaling pathway goes from the periphery, through the
is detected because the afferent pathway is activated, but anterolateral column in the spinal cord, and ultimately to
not normally through a receptor ending activation. This the sensory cortex
Fig. 1.53 Schematic

representation of referred
pain. In this case,
nociceptors in the heart
were activated and synapse
on interneurons which also
have connections to the
nociceptor afferents
associated with the left arm
and shoulder; hence, the
projected pathways that
were activated were then
perceived by an individual
as pain in the arm and
shoulder, even though they
originated in the heart
formation provides important insights regarding • Physical: Immobilization, cold or warm wrap-
the site(s) of pathologies and/or the degree(s) of pings, diathermy, massage, and/or exercises.
injury. Listed below are several available modes • Pharmacological: Anesthetic agents adminis-
often employed to control a given individual’s tered via an infusion pump, injection, or topi-
pain: cally.
• Neurosurgical: Surgical procedures are essen- 1.13.2 The Visual Focusing System
tially irreversible and normally reserved for
chronic pain conditions, for example, a chor- The lens and cornea of the eye are the primary
dotomy which is the transection of the antero- optical components that focus the image upon the
lateral column of the spinal cord or a leuko- retina. The cornea plays a larger role than the lens
tomy which employs the transection of the in focusing light rays because the rays are bent
pathways from the thalamus to the frontal lobe. more in passing from air into the cornea than they
• Neurostimulation: Use of implantable are when passing in and out of the lens. How-
electrode stimulation systems to modulate ever, the lens is the primary structure responsible
the activity within ascending pain pathways or for making nearly instantaneous adjustments for
within cortical sites themselves. viewing objects at various distances, that is, the
structures which allow for the visual accommo-
dation process. The shape of the lens is controlled
1.13 Vision by the active contractions and relaxations of the
ciliary muscles and thus the tensions they apply
Electromagnetic radiation with a wavelength to the zonular fibers, which attach to the lens (Fig.
spectrum between 400 and 750 nm can be 1.56). Furthermore, the amount of light entering
perceived, by humans, as light. The longer the eye is controlled by a ring-like pigmented
wavelength components (750–650 nm) appear (commonly, brown, blue, or green) muscle known
as red light, and the shortest components as blue- as the iris; the hole in the center of the iris is called
violet light (400–450 nm). All objects in our the pupil. Stimulation of sympathetic nerves to
surroundings reflect light to different degrees: the iris causes these muscles to contract, which
monitored as luminescence or luminous intensity then enlarges the pupil, whereas stimulation of
of which the units are candela or cd per unit the parasympathetic nerves causes the diameter
area. The luminance of various objects (or parts of the iris to get smaller.
of a given object) determines relative contrasts, There are a number of common disorders of
defined as C = (Ia − Ib )/(Ia + Ib ). Vision is the focusing system which have been defined,
based primarily on the perception of bright/dark including the following:
contrasts. Color contrast enables us to distinguish
surfaces with a C = 0, that is, if they reflect • Presbyopia: Increasing stiffness of the lens that
different portions of the visual spectrum (Fig. makes accommodation for near vision more
1.54). difficult (e.g., one may need reading glasses as
she or he ages).
• Cataracts: Changes in the lens color or opacity
(replacement surgeries are very common, but
1.13.1 Functional Anatomy afterward, one loses all accommodation abili-
ties).
As noted above, light waves are propagated in all • Myopia (nearsightedness): An individual’s
directions from every point of a visible object. eyeball is too long, and distal objects focus at
These divergent light waves must pass through a point in front of the retina.
the human optical system that focuses them back • Hyperopia (farsightedness): An individual’s
onto the sensory detecting region of the eyeball, eyeball is too short, and near objects are fo-
before an accurate image of the object is achieved. cused behind the retina.
In the eye, the image of the object being viewed is • Astigmatism: The lens and/or cornea do not
focused upon the retina, which contains the light- have a smooth spherical surface; thus, the im-
sensitive receptor cells (Fig. 1.55). ages to be focused on the retina are distorted
accordingly.
50 P. A. Iaizzo
Fig. 1.54 A schematic diagram of a horizontal-sectional so. Our brains then are programmed so that the object is
view through a human eye. The image of the red arrow detected in its correct orientation. The optic nerve is one’s
passes through the focusing system to be projected on the blind spot, for there are no retinal receptors in this region
fovea of the retina; note that it becomes inverted in doing in the eyeball
Fig. 1.55 Relative

refractory mechanism
involved in focusing an
image upon the retina. The
cornea induces a great bent
of the light rays relative to
the lens

the relative actions of the
ciliary muscle on altering
the shape of the lens. As
these muscles contract, the
elastic tension in the Lens Lens
zonular fibers is reduced
and the curvature of the
lens increases (right, near
accommodation), whereas
reduction of muscle tone Dialator Sphincter
allows the elastic tension Muscles Muscle
of the lens capsule to allow Zonal Fibers
the curvature to decrease Zonal Fibers
(left, far accommodation)
Ciliary Ciliary
Muscle Muscle
(relaxed) (contracted)
• Glaucoma: The aqueous humor is formed 1.13.3 Visual Receptor Cells

faster than it is removed, resulting in increased
pressure within the eye; this will distort an Within the retina, one can define two types of
eyeball shape and change its overall focusing photoreceptors (which are both secondary sense
abilities. cells), commonly known as rods and cones. There
are approximately 120 million rod photorecep-
Fig. 1.57 A schematic representation of the primary the neuronal network (horizontal, bipolar, amacrine, and
structures within the retina. The retinal receptors (rods ganglion cell layers) before reaching and activating the
and cones) are embedded into the pigment epithelium. receptors
Light directed from the focusing system passes through
tor cells in the human eye (Fig. 1.57). These It should be noted that there are more rod
cells have primary functional abilities when one cells than cone cells and they are not equally
is in dim light or during night conditions: they distributed in the human retina (Fig. 1.58). More
provide for scotopic vision. The peak absorption or less, one can consider that there is a lack of rod
in the rod’s cell is approximately 500 nm, and cells in the fovea, which makes this region of the
the photopigment (the molecules which absorb retina essential for daytime vision. This is also the
the photons) is known as rhodopsin. The cone region of the retina where the image of an object
photoreceptors, of which there are approximately of primary interest is being projected; hence, one
six million in humans, are the receptor popu- controls their gaze toward focusing on the fovea.
lations most utilized for daytime lighting con- The high density of cones in this retinal region
ditions: they provide for photopic vision. Three and the fact that only the outputs from few cones
different photopigments can be found in a normal converge onto a single associated ganglion cell
human’s population of cone cells, and each of allow for a much greater spatial resolution (visual
these will absorb light most effectively at a differ- acuity). However, visual acuity also depends on
ent part of the visible spectrum; that is, consider the presence of simultaneous contrasts. Further-
that there are red, green, and blue cone receptor more, color vision or its perception depends not
cells.
52 P. A. Iaizzo
Fig. 1.58 Relative distribution of the cones and rods over and there are no receptors where the optic nerve leaves the
the retina. The y-axis is the receptor density, and the x- eyeball, thus creating a blind spot. Our peripheral vision is
axis is the relative distance from the fovea. Note that the primarily due to rods; hence, we have minimal abilities to
highest density of cone receptors is located in the fovea detect colors in those areas
only on the stimulus and receptors but also on • A triggered decrease in cGMP levels within
processing within the higher CNS centers. the receptor’s cytoplasm.
• Subsequent decreases in the fluxes of Na+ and
Ca2+ across the plasmalemma.
1.13.4 The Receptor Transduction • A resultant membrane hyperpolarization due
Process to these altered ion fluxes.
• This hyperpolarization of the membrane po-
Upon exposure to light and absorption of tential causes a decrease in neurotransmitter
photons, the receptor photopigment changes its release.
shape, which eventually leads to the formation • Altered responses in postsynaptic neurons
of a receptor potential; this simultaneously (e.g., decreases action potential discharge
alters the release of neurotransmitters from the rates).
photoreceptor to the innervating ganglion cells.
The following are the primary steps one can It is interesting to note that the stimuli in
define in this underlying transduction process these receptors (impinging light) ultimately lead
which occurs in these visual photoreceptor to decreased transmitter releases within the inner-
cells: vating synapses.
Convergence and divergence of connections
• Changes within the photoprotein complex within the retina are the basis of the detectable
(with opsin). receptive field of a given retinal ganglion cell, that
Fig. 1.59 A diagrammatic

representation of the major
visual pathways in the
human brain. The efferent
connections between the
visual cortex and
subcortical structure
induce proper sorting of
the projected information
for optimal processing of
this visual information.
There is divergence of this
visual information to
various visual cortical
areas which have various
known functional abilities
is, the area of the retina which when activated arranged within columns. Note that some
by appropriate visual stimulus can either excite neurons in the deeper layers become active,
or inhibit any given ganglion cell. Subsequently, shortly before eye movements are even made.
the processing of this visual information within • Lateral geniculate: Axons (fibers) of the op-
the CNS has a topological organization; that is, tic nerves terminate in size layers within the
the image of the environment as projected on the lateral geniculate, three associated with the
retina is represented in a systematic way within ipsilateral eye and three with the contralat-
associated central structures (Fig. 1.59). The optic eral eye. Achromatic light/dark patterns reveal
nerve sends projections to the lateral geniculate two main classes of neurons—contrast neu-
(in the thalamus) which, in turn, sends projections rons (respond strongly to light/dark contours)
to the superior colliculus and eventually the vi- and light/dark neurons (activity depends on
sual cortex. This representation is not necessarily mean luminance). Color-specific neurons are
linear, for the neural information obtained by also present.
the receptors within the fovea projects onto a • Visual cortex: Neurons in several different ar-
much larger portion of the visual cortex than the eas of the cortex deal simultaneously with
extra-foveal areas of similar sizes. The following different aspects of visual information: color,
lists important primary structures and pathways contours, orientation, movement, movement
associated with the CNS processing of visual direction, etc. They provide for (1) binocular
information: vision, (2) analysis and reconstruction of the
stationary world from ever-changing retinal
• Superior colliculi: Some neurons in these images, (3) the control of gaze, and/or (4)
brain centers respond preferentially to moving initiation of sampling movements by the eyes.
visual patterns, and some neurons within are
activated only if the stimulus moves through The fact that humans have two eyes allows for
the receptive field in a particular direction; greater depth perception abilities (binocular vi-
these cells are considered to be primarily sion). More specifically, when an object is viewed
54 P. A. Iaizzo
Fig. 1.60 Represented

here is the value of
possessing binocular vision
and the relative
reconstruction of the
cyclopean eye image.
When an object is outside
the homopter, its image is
projected to the right of the
fovea in the left eye and to
the left of the fovea in the
right eye. In this case,
binocular viewing gives an
uncrossed double image,
the position of which can
be found by mapping the
retinas of the left and right
eyes onto the imagined
retina of the cyclopean eye
(our brains perform such
processing continuously)
monocularly versus binocularly in alteration, it an object. Saccades are defined as fast, small
appears basically the same, except that binocular jerking movements rapidly bringing the eye from
vision provides a greater impression of spatial one fixation point to another, thus allowing for
depth. Because eyes are located at different places a rapid sweeping search of the desired visual
on our heads, their geometrical and optical prop- field. The saccades move the visual image over
erties cause the images of one’s surroundings on the receptors, thereby minimizing adaptation. On
the two retinas to be different (Fig. 1.60). the other hand, slow eye movements are involved
in tracking visual objects as they move through
the visual field, as well as during compensation
1.13.5 Eye Movements of our gaze during movements of the head. It
should be noted that, in part, the control centers
The control of our eye movements is an extraor- for these compensatory movements obtain their
dinary motor system that involves six muscles required sensory information as to our relative
innervated by three nerves, to move each of two head movements from the vestibular system.
eyes in perfect coordination (Figs. 1.61 and 1.62).
Furthermore, the images of our external world
are constantly being recreated on the retina. For 1.14 Sound and Hearing
example, small (few minutes of arc in amplitude)
and fast (between 20 and 150 Hz) eye movements Sound is defined as an oscillation of the
are present even when we fixate our gaze on molecules in an elastic medium (e.g., air, water,
Fig. 1.61 Relative

positions of the extraocular
muscles as they are
attached to the right eyeball

the relative actions of the
extraocular muscles on
movement of an eyeball.
The arrows indicate the
movement of the middle of
the cornea via contractions
of each of the six muscles
in isolation. Yet, it should
be noted that in nearly
every eye movement, each
of the six muscles is
activated or relaxed in
concert; further, this is
coordinated between the
relative movements of our
two eyeballs to perfectly
coordinate our gaze
between them
etc.), which is propagated through these mediums of only a single frequency, it is defined as a
as longitudinal pressure waves. These oscillations tone (Fig. 1.63). Yet, typically, the sounds we
of molecules create zones in which the molecules listen to daily are mixtures of several frequencies,
are either densely or loosely packed. The relative for example, sounds that have a musical quality
amplitude of the pressure variation is called sound consist of several fundamental frequencies and
pressure; in acoustics, the sound pressure level is their harmonics. Noise is a sound comprised of
expressed in decibels (dB). many unrelated frequencies. Amazingly, humans
The auditory threshold will depend on the can distinguish approximately 400,000 different
given frequency of the sound waves, with the sounds. For example, the phone rings, you answer
human ear being most sensitive in the ranges from it and identify the caller as your mother (who may
2000 to 5000 Hz. When a sound is composed even have a stuffy nose).
56 P. A. Iaizzo
(stirrup) (Fig. 1.64). The malleus is firmly fused

to the tympanic membrane, and the so-called
footplate of the stapes fits into an opening in the
petrous bone called the oval window. Hence, the
function of these bones is to couple the motions
of the tympanic membrane to those of the inner
ear. Because of size differences, the force per unit
that is transmitted is 15 to 20 times greater at
the oval window. Yet, this transmission process
can be damped by the contraction of two unique
skeletal muscles that are located within the mid-
dle ear; when contracting, they alter the relative
tension of the tympanic membranes and thus the
movements of the stapes on the oval windows.
Fig. 1.63 Representative sound pressure waves associ-
ated with a single tone (blue), a musical sound which Without these contractions, the pressure waves
contains appropriate harmonics and relative fundamental are amplified and projected onto the oval window,
frequencies (green), and a signal that our brains may detect a membrane structure at the beginning of the inner
as noise (red)
ear. As noted above, the inner ear is embedded
in the petrous part of the temporal skull bone.
1.14.1 Functional Anatomy In other words, the inner ear is comprised of
the organs of hearing and equilibrium, of which
The first step in hearing is the entrance of sound there is a continuum within the perilymphatic and
waves into the ear canal. Thus, the pressure waves endolymphatic fluid spaces.
must enter into the external ear (Fig. 1.64). The It is because of its shape that the inner ear’s
shapes of the outer ear (the pinna) and the ear auditory organ is called the cochlea (Latin for
canal (meatus) help to amplify and direct the “shell”). The cochlea consists of three parallel
sound. Sound waves reverberate from the sides canals which are coiled together—the scala tym-
and end of the ear canal, filling it with the continu- pani, scala media, and scala vestibuli (Fig. 1.64).
ous vibrations of pressure waves (we can dampen Special features of this system are listed below:
this by the use of ear plugs).
Once the pressure waves are transmitted down • The scala tympani and scala vestibuli commu-
the ear canal, they will reach the tympanic mem- nicate with one another at the helicotrema and
brane (ear drum). This is a delicate membrane contain perilymph. This fluid resembles extra-
which closes off the external canal and forms a cellular fluid in its composition (i.e., contains
partition between it and the middle ear, which 140 mM Na+ ) and is considered an ultrafiltrate
is also filled with air (Fig. 1.64). Also enter- of plasma. At the oval window, the annular
ing the middle ear are the Eustachian (auditory) ligament seals off the space around the stapes
tubes. These tubes connect the middle ear with so that no perilymph can leak out.
the pharynx. The slit-like endings of these tubes • The scala media is filled with endolymph,
in the pharynx are normally closed, but muscle which is rich in K+ (145 mM) and thus resem-
movements open their passages during yawning, bles intracellular fluid.
swallowing, or sneezing. This, in turn, allows the • The scala vestibuli is separated from the scala
pressures in our middle ears to equilibrate with media by Reissner’s membrane.
atmospheric pressure. • The scala media is separated from the scala
Within the cavity of the middle ear is a chain of tympani by the basilar membrane.
three flexibly linked bones called the ossicles— • The organ of Corti runs along the basilar mem-
the malleus (hammer), incus (anvil), and stapes brane; within this thickened ridge is where
Fig. 1.64 A schematic representation of the middle ear membrane separates the external ear canal from the middle
and inner ear of the human. The inner ear sensory struc- ear, and the ossicles (a chain of three flexible linked bones)
tures are contained within the temporal bone, i.e., both the are attached to it (via the malleus), as well as last in the
vestibular apparatus and organs of hearing. The tympanic chain to the oval window of the inner ear (the stapes)
the receptor hair cells are located: these are or more. Importantly, the inner ear contains a
secondary sensory cells. tectorial membrane, a gelatinous mass attached to
• The stria vascularis is a highly vascular struc- the inner wall of the cochlea which lies over the
ture which plays an important nutritive role (it organ of Corti; the stereocilia of individual hair
can also be a means by which toxins enter the cells adhere to this membrane.
inner ear and deafness may result).
The receptors in the modality of hearing are 1.14.2 Auditory Sensations

similar to those in the vestibular apparatus—hair
cells. However, these cells have only stereocilia, Sound must exceed a certain sound pressure level
and spatially, there are inner and outer rows of in order to be heard. As noted above, auditory
hair cells. It should be noted that these receptors threshold is frequency dependent, the human
are also secondary sense cells, and the afferents ear being most sensitive in the range of 2000–
innervating these receptors come from the bipolar 5000 Hz, but not equally at each frequency.
cells of the spiral ganglion, which lies at the A tone at any frequency, once the threshold
center of the cochlea. There are approximately has been passed, is sensed and becomes louder
3500 inner hair cells in humans, which form as the sound pressure increases. Thus, the fir-
one to two rows. These receptors are, in turn, ing frequencies within the innervating afferents
innervated by 90% of the approximately 30,000 and/or the number of hair cells activated becomes
to 40,000 afferents which make up the auditory increased. It is considered that the inner and outer
nerve. The outer hair cells form rows of three rows allow for levels of sounds to be precisely
58 P. A. Iaizzo
determined. If the sound pressure level is greatly complexes (bilateral); note that it is at this brain
increased, the eventual result is a sensation of level that signals from each ear are compared.
pain in the ear. Intense sounds may also cause From the dorsal cochlear nucleus, a dorsal tract
reversible loss of hearing, yet if prolonged, these crosses to the opposite side and terminates in the
sounds may cause irreversible damage (i.e., dam- nucleus of the lateral lemniscus. After synaptic
age of sensory cells). relay, these auditory pathways proceed through
Airborne sound must be transmitted from the the inferior colliculus and further to the medial
air to the fluids of the inner ear. The tympanic- geniculate body and onward to the primary
ossicle apparatus effectively matches the acoustic auditory cortex (i.e., in the transverse temporal
impedances of air and the inner ear to one an- gyri). Note that it is considered that only ventral
other, reducing the reflection of sound that would cochlear neurons have tuning curves similar to
have normally occurred. Movement of the stapes those of the auditory fibers.
transmits sound energy to the perilymph in the The origin of sound in space can be localized
scala vestibuli. Because the fluids of the inner (specified) rather precisely. This capacity requires
ear do not compress, some structure in the inner information from both ears to be transmitted to
ear must compensate for the pressure changes; the auditory cortex, where there are population
this structure is the round window. The pres- neurons that are activated only if the sound source
sure changes also set up traveling waves along is at a particular location: as there are neurons
the endolymphatic duct to the helicotrema. The which respond exclusively to sounds specific to
liquid-filled inner ear channels soon attenuate vocal communications. The functioning of these
these waves before they reach the helicotrema. neurons also depends on the level of conscious-
Between the sites of origin and extinction of the ness of the individual. Interestingly, we also have
waves are locations where the wave amplitudes abilities for selective hearing such that we can
are maximal. These are at different positions for increase thresholds for some frequencies and de-
each frequency, with higher frequencies closer crease them for others: there are also efferent
to the stapes and lower frequencies closer to the pathways and controls which alter functioning of
helicotrema. This spatial relationship is the basis our inner ears. In general, sound orientation in
for the place theory for hearing. In short, the space is based on differences in signal conduc-
sensory cells are most excited at the site of the os- tion times and their differing relative intensities
cillation maximum, so that different frequencies (Fig. 1.66).
excite different populations of hair cells.
Because the cilia of the hair cells make firm
contact with the tectorial membrane, the resulting
shear forces bend them, which is then considered 1.15 Taste and Smell
to create an adequate stimulus. The ion currents
generated by the bending of the cilia cause the The sensory sensations of taste and smell are
release of transmitters at the base of the hair cells derived from a selective and highly sensitive re-
which, in turn, excites the associated/innervating action of specialized sense cells to the presence
afferent nerves (bipolar cells). of the molecules of certain compounds. Gustatory
(taste) and olfactory (smell) sense cells act as ex-
teroceptors, and their reactions provide important
1.14.3 The Central Auditory System information about external stimuli. Yet, taste and
smell can also immediately affect a person’s ap-
The primary afferent nerves typically are petite, the flow of saliva, gastric secretions, and/or
bifurcated and send their inputs into the ventral avoidance of harmful substances. These senses
cochlear nucleus and the others to the dorsal can be characterized and distinguished by mor-
cochlear nucleus (Fig. 1.65). From the ventral phological and physiological criteria. Table 1.2
nucleus, a ventral tract runs to the olivary characterizes these two chemical senses.
Fig. 1.65 A simplified

representation of the major
pathways/connections
associated with auditory
functions. Here only the
pathways of one side are
shown for simplification,
however note that there is
bilateral activation of
auditory nuclei as well as
projections of activated
pathways. Hence, the brain
readily compares
information obtained from
each ear to aid in
determining the
directionality of the
originating sound
In comparison to other senses, taste and smell (3) Modalities are similar sensory im-
both exhibit high degrees of adaptation (recall pressions that an individual can be
that there is little or no adaption or accommo- consciously or unconsciously aware of.
dation to pain). In other words, the excitation in (4) Sensations are used for the interpretation
the afferent pathways declines markedly during of a perception, and this is based upon an
maintained stimuli, and thus, perceptions are cor- individual genetic makeup.
respondingly diminished. (5) Pitch, sweetness, and lightness are all
examples of qualities.
2. Which of the following in not true of glial
Homework cells?
(1) They play essential roles in the regulation
1. Which of the following is not true concerning
and repair of neurons and neural path-
our sensory system?
ways after injury.
(1) A threshold stimulus is defined as
(2) They play little or no role in the plasticity
the smallest stimulus that produces a
of neural processing (e.g., synaptogene-
detectable response by the brain.
sis).
(2) There are two basic types of receptor
(3) Glial cells may function to form the
accommodation—slowly adapting and
myelin sheath of neuronal axons.
rapidly adapting (phasic) responses.
60 P. A. Iaizzo
Fig. 1.66 The brain’s

ability to perform accurate
calculations of
conduction-time
differences in auditory
inputs allows for
directional hearing
Table 1.2 Characteristics of taste and smell

Taste Smell
Receptors Secondary sense cells Primary sense cells; endings of cranial nerves V (IX
and X)
Position of receptors On the tongue Nose and throat
Afferent cranial nerves VII, IX I, V (IX, X)
Stations in central 1. Medulla oblongata 1. Olfactory bulb
nervous system 2. Ventral thalamus 2. Telencephalon (prepyriform area)
3. Cortex (postcentral gyrus) Connections to limbic system and hypothalamus
Connections to hypothalamus
Adequate stimulus Molecules of organic and Molecules of almost exclusively organic, volatile
inorganic substances, mostly compounds in gas form, becoming dissolved only at
nonvolatile. Stimulus source receptor. Stimulus source usually at a distance
near or in direct contact with
sense organ
Number of Small Very large (thousands), in many poorly defined
qualitatively 5 basic qualities: sweet, sour, quality classes
distinguishable stimuli salty, savory, and bitter
Absolute sensitivity Relatively low Very high to some substances (107 molecules per ml
At least 1016 or more air, as little as 102 or 103 in animals)
molecules/ml solution
Biological Contact sense Long-distance sense
characterization Used for testing food and Used to test environment (hygiene) and food, and by
control of food intake and animals in foraging, communication, and
processing (salivary reflexes) reproduction. Strong emotional weighting
(4) Some types of glial cells help to create (3) Reflexes, both monosynaptic (flexion re-
the blood-brain barrier. flex) and polysynaptic ones that exist
(5) They typically outnumber neurons in the within the spinal cord, typically play mi-
human brain. nor roles in the execution of movements.
3. In an attempt to increase your core temper- (4) One of the brain stem’s important func-
ature, involuntarily, the body would not ini- tions is the reflex control of posture.
tiate which of the following effector mecha- (5) The association motor cortex and sub-
nisms? cortical motivation areas are important in
(1) Initiate rhythmic muscle tensing. movement design or the development of
(2) Increase its production of thyroxine. the “plan” of a series of movement.
(3) Vasodilation of its peripheral blood ves- 6. Following a complete human spinal cord
sels. transection, which would likely not occur?
(4) An increase of activities within the pre- (1) There would be the development of
optic nuclei in the hypothalamus. flexor reflex activity below the lesion,
(5) The induction of non-shivering thermo- before there would be reflex behaviors
genesis. returned within the extensor muscles.
4. Which is not true regarding reflexes? (2) There would be little or no return of
(1) Reflexes bring information into the CNS autonomic system reflexes below the site
via afferent pathways and out via efferent of the lesion, such as the control of skin
pathways. blood flow.
(2) An example of a polysynaptic reflex is (3) There would be an initial period of mini-
the cough reflex, in which receptors in mal reflex activity, spinal shock, in body
the mucosa of the trachea and bronchi are areas below the lesion.
stimulated and then chest muscles, your (4) There would be a transient decrease in
diaphragm, and others contract automat- synaptic excitability to the interneurons
ically. lying distal to the lesion (e.g., loss of
(3) A neurologist can determine where dam- excitatory inputs from higher motor cen-
aged nerves may occur in spinal der- ters).
matomes by testing your reflex responses (5) Mass flexion reflexes could be triggered,
in various body parts. by even some subtle degree of tactile
(4) Reflex time is defined as the time it takes stimulation, below the level of the lesion
a beta motor neuron to stimulate a muscle (e.g., to cause a person to be thrown out
fiber. of a wheelchair).
(5) Vestibular reflexes maintain posture and 7. Which of the following statements regarding
equilibrium without primary participa- vision is not true?
tion of consciousness, yet if asked, you (1) Cataract results from an increase in opac-
can identify where your body is in space. ity (clouding) of the lens: this is like
5. Which is not true concerning the general looking through a fogged-up car’s wind-
functional features of the human motor con- shield.
trol system? (2) Glaucoma, which means that the aqueous
(1) An individual can continue to develop humor is formed slower than it is
new motor programs throughout their removed, results in decrease pressure
life. within the eye; this has minimal effect
(2) The motor cortex, thalamus, and basal on our visual systems’ overall focusing
ganglia are important for providing both abilities.
innate and continually learned motor (3) Contraction of ciliary muscles allows the
programs. tension zonal fibers to reduce and the
curvature of the lens to increase.
62 P. A. Iaizzo
(4) Presbyopia, increasing stiffness of the tives to action (a person’s motivational

lens, is a condition in which the lens interactions and emotions).
cannot accommodate adequately for near (5) The parasympathetic nervous system is
vision. minimally altered by intense changes in
(5) Membrane hyperpolarization occurs in your mood.
the rod and cone receptor cell when there 10. Which of the following statements is not true
is photon activation that then, in turn, concerning the human retina?
alters action potential frequencies within (1) There are approximately 120 million rod
the innervating bipolar cells. cells in the human eye, and these recep-
8. Which is not true of the hypothalamo- tors are needed for dim light or night
hypophyseal (hypothalamus-pituitary) conditions.
system? (2) Cone receptor cells are utilized for day-
(1) This system uses minimal feedback con- time lighting conditions (our photopic
trol, for example, via detection of blood vision).
concentrations of released hormones (3) There are no receptor cells located at
within specialized sensory cell which the site where the optic nerve leaves the
lie within the lateral hypothalamus. eyeball, our blind spot.
(2) The activities of most endocrine glands (4) In colored blindness, an individual is
are regulated by hormones formed in likely missing certain populations of rod
the adenohypophysis (anterior pituitary cells in the retina.
gland). (5) There are approximately 20 times more
(3) Efferent connections to the pituitary can rod than cone receptor cells in the eye,
be both neuronal and hormonal (e.g., us- but there is more divergence of neural
ing the portal circulation pathways). information from the cone receptors.
(4) The medial hypothalamus can release 11. You are taking off in an airplane, as there
both stimulation and inhibitory releasing is also change in cabin pressure, you yawn
hormones. to equalize the pressure within your middle
(5) A tumor within the pituitary gland may ear. Which of the following would likely not
lead to gigantism, that is, the over-release occur?
of growth hormones. (1) The auditory (Eustachian) tubes open as
9. Which of the following is not true relative you yawn.
to the general functioning of our limbic sys- (2) The activities within the macular organs
tem? within the temporal bone would become
(1) This system is associated with the altered with the gravitational changes.
process of both learning and memory: (3) The semicircular canals become
depression could affect one’s ability to activated because there are detected
learn. slight rotational accelerations.
(2) A lack of sleep may ultimately influence (4) The ossicle bones in the middle ear be-
the proper functioning of this system. come fixed by muscle contractions.
(3) Multiple brain centers/nuclei contribute (5) The vestibular ganglia elicit changes in
to this system, and thus, multiple neuro- synaptic activities.
transmitters are utilized within it: more 12. Processing of information by our human
specifically, the monoaminergic system brain can be divided into four globally
includes dopaminergic, noradrenergic defined or generalized systems, which of
(i.e., norepinephrine), and serotonergic the following is not one of these?
neurons. (1) The motor system: which governs both
(4) It is associated with an individual’s abil- voluntary and involuntary movements.
ity to change moods and innate incen-
(2) The associative system: which controls based on such facts about the brain, which of
cognitive functioning and conscious the following is not likely to be true?
recognition. (1) The brain can elicit a high degree of
(3) The sympathetic system: which controls plasticity, and thus, one can subsequently
“fight or flight responses.” elicit abilities to compensate for an oc-
(4) The vegetative system: which has invol- curred damage or lesion (e.g., a stroke,
untary control of bodily functions, home- head trauma).
ostasis. (2) Certain pharmacological therapies
(5) The limbic system: which governs an (therapeutic drugs) have their primary
individual’s emotions and feelings. beneficial effects by mimicking the
13. Which of the following in not true when one actions of certain neurotransmitters, yet
compares the senses of taste and smell: they may also induce unwanted side
(1) Both can affect one’s perceptions of a effects on other neural processes.
high-quality tasting food. (3) An imbalance in neurotransmitter release
(2) Only the sense of taste is considered in various brain regions may cause clini-
to have a very small number of cal symptoms.
distinguishable qualities. (4) There are only eight different receptor
(3) Like the sense of pain, the sense of molecules in the human brain’s postsy-
smell is not highly adaptable (i.e., naptic terminals, and these are rarely ac-
both modalities elicit relatively slow tivated by specific agonists.
accommodations to a given stimulus). (5) In the human brain, there is a high degree
(4) Only one of these senses employs of divergence of afferent sensory infor-
secondary sense cells for signal mation.
transduction. 16. An acute global damage within the left tem-
(5) The sense of taste requires larger poral lobe of an individual’s brain (in an
amounts of molecules of a given sub- otherwise healthy person) and this may cause
stance, so to elicit a threshold response. all of the following symptoms, except?
14. Which is not true concerning the sense of (1) A difficulty in finding the proper word
hearing? for a well-known object (e.g., like a
(1) We are able to better determine the direc- marker pen).
tion of a sound origin because we have (2) An impaired comprehension of language
two ears. or of a previously known foreign lan-
(2) The auditory threshold is dependent on guage.
the frequency of sound. (3) An impaired ability to speak desired sen-
(3) The same tone can be heard if a tuning tences; yet their motor abilities to do so
fork is held to either the ear or placed on have remained intact (your motor cortex
the temporal skull bone. was not affected).
(4) The shape of the outer ear is important (4) They elicit a decreased ability to read
for determining the direction and ampli- sentences and/or perform simple math
tude of a sound. equations.
(5) The round window, which lies between (5) They show a dramatic change in their in-
the middle ear and inner ear, does not nate abilities to quickly orient themselves
move. in space, that is, also identify which their
15. One-half of the 100,000 human genes con- body’s location in 3D space (e.g., where
tribute to building the brain; the adult brain you are in your chair relative to what
contains >10,000,000,000 neurons; the brain direction you home is).
utilizes over 100 different neurotransmitters; 17. Which of the following would not be con-
sidered a function of “feed-forward” pro-
grammed motor movements, that is, those
64 P. A. Iaizzo
that are normally aided by a proper function- isometrically scaled-up primate brain. J Comp Neurol
ing cerebellum? 513, 532–541 (2009)
3. O. Bandschapp, P.A. Iaizzo, Induction of therapeutic
(1) Adjustments for such movements are
hypothermia requires modulation of thermoregula-
made in advance of motor program tory defenses. Ther Hypothermia Temp Manag 1, 77–
execution based on current sensory 85 (2011)
inputs and past learned responses. 4. D. Barker, N.K. Chin, The number and distribution of
muscle-spindles in certain muscles of the cat. J Anat
(2) It allows for rapid (ballistic) movements,
94, 473–486 (1960)
such as a very rapid hitting of a tennis ball 5. D. Brooks, B. Hunt, Current concepts in concussion
with a racket. diagnosis and management in sports: a clinical re-
(3) If the cerebellum is damaged, such view. BC Med J 48, 453–445 (2006)
6. M. Chen, C.C. Cline, K.L. Frost, T.J. Kimberley,
movements might be initially impaired, S.T. Nemanich, B.T. Gillick, C.S. Albott, C.N. Pru-
but subsequently well compensated for. dente, L.O. Lim, B. He, Advances and challenges
(4) Co-activation of muscle spindles (beta in transcranial magnetic stimulation (TMS) research
motor neuron become stimulated) on motor systems, Chapter 11, in Engineering in
Medicine: Advances and Challenges, ed. by P. A.
may occur, such that one could detect
Iaizzo, (Elsevier, London, 2019), pp. 283–318
mismatches between actual and expected 7. W.C. Dement, C. Vaughn, The Promise of Sleep (Dell
movements. Publishing Inc, New York, 2000)
(5) This allows for ongoing error detections 8. W.J. Germann, C.L. Stanfield, Principles of Human
Physiology (Pearson Education, Inc./Benjamin Cum-
and corrections of an ongoing motor pro- mings, San Francisco, 2002)
grams during an elicitation of the actual 9. A.C. Guyton, J.E. Hall, Textbook of Medical Phys-
movement. iology, 10th edn. (W.B. Saunders Co., Philadelphia,
18. Which of the following is not an effector 2000)
10. E. Henneman, G. Somjen, D.O. Carpenter, Excitabil-
response following the activation of the sym-
ity and inhibitability of motoneurons of different
pathetic nervous system? sizes. J Neurophysiol 28, 599–562 (1965)
(1) A rapid increase in heart rate. 11. R.F. Schmidt, G. Thews, Human Physiology
(2) A decrease in intestine motility. (Springer-Verlag, New York, 1983)
12. N.A. Shaw, The neurophysiology of concussion. Prog
(3) Relaxation of the bronchi (dilation).
Neurobiol 67, 281–344 (2002)
(4) Relaxation of the sphincter muscle 13. G.J. Tortora, S.R. Grabowski, Principles of Anatomy
known as the iris (one’s pupil becomes and Physiology, 9th edn. (John Wiley & Sons, Inc.,
smaller). New York, 2000)
14. E. Widmaier, H. Raff, K. Strang, Vander’s Human
(5) Increased blood flow to skeletal muscles.
Physiology: The Mechanisms of Body Function, 15th
edn. (McGraw Hill, New York, 2018)
15. E.P. Widmaier, H. Raff, K.T. Strang (eds.), Vander’s
References/Additional Sources Human Physiology: The Mechanisms of Body Func-
tion, 12th edn. (McGraw Hill, New York, 2011)
1. Atlas of Human Cardiac Anatomy. http://
www.vhlab.umn.edu/atlas
2. F.A. Azevedo, L.R. Carvalho, L.T. Grinberg, J.M.
Farfel, R.E. Ferretti, R.E. Leite, W. Jacob Filho, R.
Lent, S. Herculano-Houzel, Equal numbers of neu-
ronal and nonneuronal cells make the human brain an
Biopotential Measurements and
Electrodes 2
Abraham Akinin, Akshay Paul, Jun Wang, Alessio Buccino,
and Gert Cauwenberghs
Abstract developments in electrode materials, interface

circuits, and embedded systems for neural
Neural biopotentials are electrical signals
interfaces with tailored instrumentation
generated by the cells of the nervous system.
solutions at a range of spatial and temporal
Recording and monitoring the aggregate
scales are driving advances toward future
or individual behavior of neurons yields
unprecedented medical therapies and neu-
information about the brain and the peripheral
roscience discoveries.
nervous system frequently used in clinical and
research settings. Many different modalities Keywords
of recording neural biopotentials have been
developed. These differ in spatial scale, Electrode-electrolyte interface ·
temporal resolution, and purpose. In order Electrophysiology · Local field potentials ·
to faithfully record and make use of these Volume conduction · Neural recording and
neural signals, we must understand the signal stimulation · Front-end amplifier · Dynamic
properties, the specific kind of electrodes range · Common-mode rejection · Active
required for measurement, and the most grounding · Charge balance · Current clamp
appropriate circuit architecture needed to and voltage clamp · Closed-loop
amplify and process these signals. Continued neurofeedback
Electronic Supplementary Material: The online version

of this chapter (https://doi.org/10.1007/978-3-030-43395- 2.1 Introduction
6_2) contains supplementary material, which is available
The ionic currents that traverse the cell mem-
A. Akinin brane of neurons give rise to biopotentials. These
Department of Bioengineering, UC San Diego, La Jolla, electrical signals can be recorded with special-
CA, USA
ized instrumentation in order to assess physiolog-
Nanovision Biosciences, La Jolla, CA, USA ical function, conduct neuroscience research, and
A. Paul · J. Wang · G. Cauwenberghs () even provide a novel medium of communication
Department of Bioengineering, UC San Diego, La Jolla, through brain-computer interfaces (BCI). Biopo-
CA, USA
tentials can be recorded from different locations
A. Buccino in the body, according to the organ or system
Department of Bioengineering, UC San Diego, La Jolla,
CA, USA
targeted for examination. Figure 2.1 shows the
location of various signals of interest to neural
Centre for Integrative Neuroplasticity, University of Oslo,
Oslo, Norway
engineering.

66 A. Akinin et al.
EEG
ERG
EMG
ENG
ExG Electrographic Modality Measurement Target
EEG Electroencephalography Brain
EGG ERG Electroretinography Retina
EMG Electromyography Skelletal Muscle
ENG Electroneurography Peripheral Nerve
EGG Electrogastrography Stomach & Intestines
Fig. 2.1 Anatomical regions of sources of biopotentials of various modalities measured in neural engineering
Aside from the diversity of locations and At these even smaller distances, μECoG can
anatomical structures that can be recorded, resolve even smaller and faster brain signals.
these signals also vary significantly in their The downside to decreasing the spacing of
spatiotemporal properties. Spatial resolution is the electrodes in order to fit more channels is
generally dependent on the distance between the increased difficulty in covering large areas of the
electrodes and the target measurement source, brain.
as well as the spatial diversity of average Following this trend toward finer, cellular-
electrical activity in the electrode’s vicinity. scale resolution, electrode arrays can be placed on
Electroencephalography (EEG), which registers the cortex with penetrating shanks that minimize
the collective dynamics of neuronal activity over the distance to cortical neurons. A complemen-
large regions in the brain as electrical fields tary realm of neural engineering pursues in
through volume conduction in extracellular vitro studies using benchtop experimental setups
space, is generally measured on and around with brain slices or neural cultures measuring
the scalp with electrode arrays that exceed electrical potentials at and below cellular scale
1 cm pitch. At these distances, biopotential through fine-pitch multielectrode arrays (MEAs).
signals from the brain have a bandwidth below Further, intracellular potentials can be recorded
100 Hz. Electrocorticography (ECoG) is a through glass pipette patch clamp electrodes
related modality to EEG which measures these inserted through the cell membrane. Figure 2.2
brain biopotentials directly on the surface of compares the scale, coverage, and temporal
the cerebral cortex. Although it is surgically resolution of these modalities. Ranging from
invasive, the decreased distance from the neural whole-body recordings of electromyography
sources allows ECoG to distinguish faster and (EMG), measuring the activation of whole
smaller nuclei of brain activity, at higher spatial muscle groups, to sub-cellular patch clamp
and temporal resolutions. Currently in transla- recordings of single-unit activity, biopotential
tional development, microelectrocortigography signals span many orders of magnitude in spatial
(μECoG) records brain biopotentials with sub- and temporal scale and thus require diverse
millimeter pitch, high density electrode arrays. specialized instrumentation.
2 Biopotential Measurements and Electrodes 67
Fig. 2.2 Spatiotemporal 0.1m Muscle

characteristics of neural Group
biopotential signals
EMG
10cm Whole
Brain
EEG
Anatomical Coverage
Spatial Resolution
1cm 10cm2
ECoG
1mm 1mm2
µECoG
Penetrating Microelectrodes 100

100µm
Neurons
Benchtop MEA
Patch Clamp Recording
10µm 1 Neuron
0.1ms 1ms 10ms 0.1s 1s
Temporal Resoluon
Fig. 2.3 Generic block Closed-Loop Neural Instrument

diagram of a wireless
closed-loop neural Power
instrument that performs Stimulator Harvesting/
biopotential recording and Body- Management
stimulation of electrically Electrode
Interface
active tissue
Analog DSP &
ADC Telemetry
Front End Control
The design methodology for a successful med- nal processing (DSP) may then be used to further
ical neuroengineering device is tightly coupled condition the signal or extract relevant physiolog-
to a set of well-defined design objectives. For ical information. The digital output data stream
example, the device may require monitoring the can then be logged for local storage or wire-
brain of a patient with epilepsy, during surgery, lessly transmitted for further external processing.
through a flexible electrode array to localize a Other relevant and indispensable blocks include
seizure inducing nucleus. Once the objectives power management including possible provisions
in the application setting are clearly defined, it for power harvesting directly from environmental
becomes relatively straightforward to determine sources such as RF incident power and body heat.
the specific engineering requirements. A system- Increasingly, neural instruments include neu-
level description can usually be formulated with rofeedback capabilities, in which the signals ob-
a block diagram describing the major compo- tained from the sensors are locally processed to
nents required toward realization in hardware. modulate neural activity and function. As such,
Figure 2.3 shows an example block diagram of the instrument will apply electrical or other mod-
a typical neural instrument. Neural biopotentials ulatory signals to biological tissue in order to
are sensed through a specialized electrode and an restore lost function or prevent a pathological
analog front-end (AFE), which contains ampli- condition. When a medical device applies electri-
fiers and analog signal processing circuits condi- cal excitation, or neuromodulation, depending on
tioning the signal for subsequent digitization by information derived from recorded biopotentials,
an analog-to-digital converter (ADC). Digital sig- it is known as a closed-loop neuroprosthesis.
68 A. Akinin et al.
Having determined a system architecture, it CD

becomes possible to consider the specific require-
CH
Electrode
ments of the component subsystems. In this chap- RS
Solution
ter, we will consider the electrode and AFE re-
quirements while keeping in mind their effect on
the rest of the system.
RD
(a)
2.2 Electrodes for Neural
Interfaces Cd
Electrode
Solution
An electrode is a component of an electrical RS
circuit that interfaces with non-metallic media.
They are the primary component in a biopotential
ZW EHC
recording or stimulation instrument. In a
recording application, the electrode couples
Rct
galvanically to capture the local field potential. In
(b)
a stimulating application, the electrode sources Fig. 2.4 Lumped-element circuit model of the electrode-
current through ionic transport to affect the electrolyte interface. (a) Polarizable, non-Faradaic elec-
local electric fields. Given their roles, the trode. (b) Non-polarizable, Faradaic electrode
dimensions, geometry, and composition are of
utmost importance to design requirements. Signal 2.2.1.1 Electrode-Electrolyte
degradation due to inferior electrode design Double-Layer Interface
or placement is unlikely to be ameliorated by The electrode-electrolyte interface serves as an
design improvements in blocks further down intermediary between the electronic charge trans-
the signal chain. In this section we discuss the port in metallic conductors and the ionic charge
properties and models of electrodes as they transport in the aqueous medium of the elec-
relate to biopotential measurement and current trolyte. On the electrolyte side, a double layer of
stimulation. ions forms in response to a buildup of electrical
potential. The first layer is composed of ions
that are chemically adsorbed onto the electrode
2.2.1 Electrode Properties and surface, while the following layer has free ions
Modeling electrostatically attracted to the surface charge.
This looser second layer is also influenced by
It is conceptually helpful to consider the thermal motion in the solution, and as such is
behavior of an electrode by representing its known as the diffuse layer. Figure 2.4 shows two
electrochemical function as an ensemble of main electrode-electrolyte models using lumped
classical lumped circuit elements. Through this elements. Although more refined models exist
abstraction, we may determine how they affect that take into account more detailed physics of
the signal as it transduces onto the measurement double layers, these two examples suffice to ex-
circuit. Standard linear circuit and signal analysis plain the general behavior of electrodes.
techniques can then be used for experimental
characterization of an electrode, including its 2.2.1.2 Impedance
polarization properties as a function of the Two main types of electrodes can be distin-
material and its impedance and noise properties guished, based on their intrinsic impedance
as a function of frequency. properties.
Polarizable, non-Faradaic electrodes have a equilibrium, complementary charge builds up on

mostly capacitive interface to the electrolyte as both sides of the interface. This space charge
shown in Fig. 2.4a. Here CH denotes the surface gives rise to a potential difference, termed
charge or the Helmholtz layer. As this capacitor the half-cell potential, which is specific to
is in series with the rest of the circuit, it blocks the the metallic element in the electrode being
flow of DC currents. In series, the diffuse layer reduced to its ionic equivalent in the electrolyte
behaves as a parallel capacitance and resistance (e.g., Ag and Ag+ ). In an electrochemical cell,
pair. The capacitance CD is related to how much the overall potential between electrodes in a
charge can be accumulated in the diffuse layer shared electrolyte medium is the total potential
and the resistance RD to the work required by resulting as the difference between two half-cell
the ions to move through the solution. Further, potentials. Hence for accurate and reproducible
the bulk of the solution also presents a series biopotential measurement, it is advisable to
resistance RS , dependent on geometry, size, and use the same electrode type for both the
distance of the electrodes. signal and the reference electrodes, eliminating
an important source of electrode voltage
Non-polarizable, Faradaic electrodes permit offset.
charge to flow from electronic currents in the
metallic section of the circuit to ionic currents 2.2.1.4 Noise
throughout the solution. Charge transfer between Thermal noise as stochastic fluctuations in ion
electrons and ions occurs as part of the chemical transport naturally arises from random-walk
processes of reduction and oxidation. An example interactions between the various electrochemical
of a non-polarizable electrode widely used for compounds at thermal equilibrium. The square
noninvasive electrophysiology directly over magnitude of thermal voltage noise in the
the skin is the Ag/AgCl (silver/silver chloride) electrode is proportional to temperature,
electrode, in which a layer of silver chloride electrode resistance, and spectral bandwidth:
over the silver bulk of the electrode provides a vn2 = 4kT RΔf . Other important sources of
buffer for direct charge transfer through redox- noise include 1/f noise, with square magnitude
based one-to-one exchange of electrons on the inversely proportional to frequency, primarily
electrode side and chloride ions (Cl− ) on the due to random fluctuations in mass transport at
electrolyte side of the AgCl layer. In Fig. 2.4b the the electrical double layer [1].
entire double-layer capacitance is represented by
Cd , while dissipative elements Rct and ZW occur
in parallel. Rct is the charge transfer resistance, 2.2.1.5 Water Window and Current
and it relates to the kinetics of redox reactions Transfer Capacity
at the electrode surface. ZW is the Warburg An important consideration in the electrode volt-
impedance which is a constant phase element age range for electrodes used in electrical stim-
(with phase +45◦ ) is related to the frequency ulation is the electrolysis breakdown of water
dependent diffusion of charged particles in molecules into oxygen and hydrogen gas above
the solution. In Faradaic electrodes, we must a critical voltage threshold, which depends on the
also consider the contribution from the bulk electrode material. These gases as by-products of
of the solution separating electrodes from each the electrolysis are detrimental to tissue survival
other. and electrode longevity. To avoid water break-
down, voltage limiters are usually included in
2.2.1.3 Half-Cell Potential the control circuits driving the electrodes. The
As the result of the exchange between ionic and water window puts a practical limit on the spe-
electronic charge at the electrode-electrolyte cific current transfer capacity of a given electrode
interface through the electrochemical redox depending on geometry, surface roughness, and
(reduction-oxidation) reactions at thermal material [2].
70 A. Akinin et al.
2.2.2 Volume Conduction for Importantly, Eq. 2.1 applies both to modeling the
Electrical Recording and electric potential measured on the electrodes
Stimulation due to currents emanating from electrically
active cells for recording and to modeling
The electric potentials induced by neural activity the effect of stimulating currents from the
and recorded by an electrode in the extracellular electrodes on extracellular fields surrounding the
space, or generated by the electrodes when used cells.
for stimulation can be, in a first approximation, When neurons are active, ionic currents flow
modeled analytically using volume conduction in and out of their membranes. The dynamics
theory [3]. Considering a quasistatic approxi- of a neuron can be computed using the cable
mation of Maxwell’s equations and assuming a equation [5,6]. From the solution of this equation,
conductive, isotropic, homogeneous, linear, and one can calculate the transmembrane currents of
infinite medium, the electric potential φex (ex different parts of the neuron. Owing to the linear
stands for extracellular) generated by a point nature of volume conduction in the extracellular
current source Is (t) (monopolar current source) medium, the electric potential generated by each
at a position rs can be computed at any point r of these currents Ii (t) is simply summed to arrive
(except for rs ) as [4]: at the electric potential at an electrode location re
(Fig. 2.5a):
Is (t)
φex (r, t) = (2.1) 1 Ii (t)
4π σ | r − rs | φex (re , t) = . (2.2)
4π σ i | re − ri |
where σ is the conductivity of the medium and the
ground reference (φex = 0) is assumed far away In the general case of bipolar recording, where
from the current. The extracellular conductivity a second electrode at position rref is used rather
of neural tissue is typically in the order of 0.3 S/m, than a distant electrode for the reference, the
but this varies depending on brain regions in differential potential between the electrodes be-
vivo or inslice or culture conditions in vitro. comes:

1 Ii (t) Ii (t)
φdiff = φex (re , t) − φex (rref , t) = − . (2.3)
4π σ i | re − ri | | rref − ri |
Conversely, the electrical potential that arises moment p as [3, 9]:

from electrical stimulation is derived from Eq. 2.1
by considering an electrode at position re with a p = Ie (r+ − r− ) = Ie d (2.5)
stimulating current Ie . The extracellular potential
at position r reads (Fig. 2.5b): where d is the vector between the positive and the
negative current locations. Defining the middle
Ie (t)
φex (r, t) = . (2.4) point of this vector as re (Fig. 2.5c), the extracel-
4π σ | r − re | lular potential generated by the current dipole can
Stimulating currents can be also applied in a bipo- be approximated as:
lar fashion to increase the selectivity of stimula-
p(t)(r − re ) Ie (t) d cos(θ )
tion. If two electrodes are used to deliver oppos- φex (r, t) ≈ =
ing currents, then they make up a dipolar current 4π σ | r − re | 3 4π σ | r − re |2
(2.6)
source. Considering the positive current source
+Ie at position r+ and the negative source −Ie where we defined d as the distance between the
at position r− , we can define the current dipole current sources and θ as the angle between p
Fig. 2.5 (a) Modeling electric potential generated by a Modeling electric potential from a dipolar current source.
neuron. The contribution of transmembrane currents of the (d) Modeling electric potential from a stimulating current
neuron is summed at the electrode location [7]. (b) Model- of a planar device using the Method of Images [8]
ing electric potential from a monopolar current source. (c)
and (r − re ). This approximation is valid when Eq. 2.8 differs from Eq. 2.1 only in the scalar that
the distance | r − re | is substantially more multiplies the denominator, making the potential
than 3d [3]. For closer distances, the two currents twice as large in the latter case, as current can only
should be summed separately as two opposing flow in the semi-space facing the electrode plane.
monopolar sources. The MoI can also be extended to account for sev-
So far we have assumed that the medium eral plane interfaces with different conductivities,
surrounding the electrodes is homogeneous for example, a brain slice mounted on a micro-
and isotropic, but these assumptions are not electrode array and placed in a saline solution, or
usually fully satisfied. Considering, for example, an epidural ECoG electrode facing the dura mater,
microelectrode arrays for in vitro recording the arachnoid, the CSF, and subsequently the pia
[10, 11], one can clearly observe that the and cortical tissue (see Fig. 2.9b).
assumption of homogeneity is not satisfied, as So far we have also considered the electrode as
cell cultures or brain slices are mounted on top of a single point in space (re ). However, electrodes
the electrodes plane. In this case, the potential are not points and they have a finite size. A simple
generated by a monopolar current source is way to include the spatial extent of the electrode
obtained directly by the Method of Images (MoI in the calculation of the electric field generated
– Fig. 2.5d) [8]: by a current is called the disk approximation [4].
Considering, for example, an electrode injecting
Ie (t) a current Ie (t) in the tissue, one can randomly
φex (r, t) = (2.7)
2π σ | r − re | draw N points belonging to the electrode surface
72 A. Akinin et al.
(rei ) and split the stimulating current into small determined, the requirements for the electrode,
contributions (owing to the linearity assumption). AFE, ADC, and DSP can be derived. Although
The electric potentials can then be approximated the electrode is the first component in the signal
as: path, and is of critical importance, system design
is not generally focused on optimizing electrode

N
Ie (t)/N geometry and composition. Custom electrode de-
φex (r, t) = . (2.8)
e=1
2π σ | r − rei | sign and optimization require access to special-
ized fabrication tools and facilities, and the de-
The same approach can be used when computing sign itself is largely constrained by the available
the electric potential generated by neural activity geometry of the physiological recording space.
at the electrode locations. In contrast, AFE design has many more design
The above-described formulations rely on sev- parameters that can be adjusted and specialized
eral assumptions. First of all the conductivity tools for simulation.
of the medium is assumed to be scalar, hence
neglecting capacitive properties of the tissue. This
assumption seems however to be well justified for 2.3.1 Analog Front-Ends
relevant frequencies in extracellular recordings
[3, 4]. Biopotential signals vary in their characteristics
Second, the medium is assumed to be across the neuroengineering modalities of inter-
isotropic, but this assumption is harder to relax. In est. Figure 2.2 details the spatiotemporal resolu-
the neural tissue, in fact, the presence of oriented tion of different neural interfaces. Additionally,
pyramidal cells makes conductivity anisotropic we may also consider the amplitude of these
[12]. Anisotropy in the tissue can be accounted signals to vary, as electrodes are located a finite
for with analytical solutions [8, 9]. distance away from biopotential sources and av-
Finally, the extracellular milieu is assumed erage all surrounding electrical activity. It thus
to be homogeneous (without discontinuities) and becomes a crucial part of the system’s design
infinite. This is clearly a stronger assumption, to use an AFE that is appropriate to the signal
considering that in order to measure the electric characteristics. The role of the AFE is to amplify
potentials generated by the neurons, we insert a the very small biopotential signals with low noise
probe in their vicinity. As mentioned above, for generation, while filtering out interference and
planar electrode arrays, one can use the Method of other irrelevant signals.
Images. For more complicated cases, numerical
solutions, such as finite element methods (FEM), Operational amplifiers (opamps) are high-
can be used. FEM approaches are popular to gain active circuits that can amplify the voltage
predict the response of stimulation in the spinal difference between two input terminals. When
cord [13, 14] or to study the effect of complex they are connected in negative feedback either
probe geometries [15]. directly or through some impedance network,
they are able to replicate any analog operation
(i.e., addition, subtraction, multiplication by a
2.3 Circuit Techniques for Neural constant, and even nonlinear transformations).
Interfaces Using the dynamic properties of capacitors
enables frequency domain filtering, such as
Electrical circuit theory provides a useful tool low-pass, band-pass, and high-pass filters to
to evaluate and design the systems that acquire selectively resolve low, intermediate, and high-
and transduce biopotential signals. Once the char- frequency content in the signal, respectively. A
acteristics of the biopotential signal are under- simplified analysis of circuits containing opamps
stood and the intended acquisition precision is with negative feedback can be accomplished by
making the following two assumptions:

1. The voltage drop across the input terminals is RF
zero: V + = V − ; and VO = 1 + VI (2.9)
RI
2. The opamp itself has infinite input impedance:
II N = 0. with positive gain strictly greater than unity. A
main advantage of this topology is the (ideally)
Armed with these two simplifying assumptions, infinite input impedance it presents, as current
and verifying the opamp is operating within its does not flow into the opamp input terminals. One
linear regime of design specifications (input/out- specific use of this circuit is as a unity gain buffer,
put voltage range, output current, bandwidth), the where RF is zero as a short and RI is omitted as
behavior of an AFE can be estimated by the stan- an open circuit, providing impedance buffering of
dard node analysis technique, specifying Kirch- a sensitive high-impedance voltage node.
hoff’s current law (KCL) at each voltage node
in the circuit except at the output of the opamp. The inverting amplifier is another widely used
The loss of this latter specification is however configuration for the opamp producing negative
compensated by the extra specifications of the amplification, where the signal input is connected
above simplifying opamp assumptions, ensuring to the negative input terminal through a resis-
an equal number of equations and unknowns in tor, while another resistor provides feedback to
the circuit analysis. the negative terminal. Analyzing the circuit in
Figure 2.6 shows some common configura- Fig. 2.6b, we obtain the output as:
tions of opamp circuits. Figure 2.6a and b both
implement pseudo-differential single-ended am- RF
VO = − VI (2.10)
plifiers with a single output referenced to a com- RI
mon ground, while Fig. 2.6c implements a fully
differential amplifier producing a differential out- with negative gain, the magnitude of which
put for a differential input signal. ranges anywhere between zero and infinity by
adjusting RF relative to RI . One drawback
A non-inverting operational amplifier is one of the inverting topology is that the input
type of configuration for the opamp providing impedance, ZI N = RI N , is considerably low
positive amplification, where the signal input is (kiloohms to megaohms) for any practically
connected to the positive input terminal and a realizable resistance at high-gain settings. This
resistive divider provides feedback to the negative is undesirable for use in a biopotential recording
terminal. Figure 2.6a depicts the non-inverting AFE as electrodes have high impedances and
amplifier. By applying the analysis technique de- may therefore attenuate the signal. The remedy
tailed above, we can derive the output as: commonly employed is to precede the inverting
RF RF
VI RI RI
VO VI VI+ VO-
VO
VI-
VO+
RI
RI RF
RF
(a) (b) (c)
Fig. 2.6 Types of operational amplifier configurations: (a) pseudo-differential non-inverting amplifier, (b) pseudo-
differential inverting amplifier, and (c) fully differential amplifier
74 A. Akinin et al.
amplified by a voltage buffering amplifier such are described in Sect. 2.4. Various strategies
as the non-inverting amplifier. for improving IAs have been devised. Many
of these are specifically for the purpose of
The fully differential amplifier relies on a dif- better recording biopotential signals. Figure 2.7
ferent kind of opamp which produces dynami- showcases several alternative solution strategies
cally balanced differential voltage outputs ampli- to recording sensitive differential signals.
fying the differential input. This is advantageous
to reject common-mode noise and power supply The difference amplifier as shown in Fig. 2.7a
interference. Figure 2.6c shows how the dual is the most basic circuit configuration to record a
differential outputs in this amplifier can provide differential biopotential. It combines attributes of
negative feedback to both input terminals and the inverting and non-inverting amplifier config-
thus present a balanced input impedance at both urations. With matched resistances, the output of
terminals. The differential mode gain of the fully this difference amplifier is:
differential amplifier is:
RF
VO = (V2 − V1 ). (2.12)
RF RI
VO = VO+ − VO− = (V + − VI− ) (2.11)
RI I
Despite its simplicity, the basic difference ampli-
Notice that although the output expression does fier suffers from several disadvantages that pre-
not contain an explicit negative sign, the input clude its practical use for biopotential recording.
and output polarities of the amplifier topology are Notably, having the input connect to the negative
flipped. Thus, we may consider this amplifier to feedback terminal results in low input impedance
be the differential form of the inverting amplifier on the order of RI . Since large voltage gain re-
in Fig. 2.6b. It is not possible to configure a quires relatively low values for RI leading to
fully differential amplifier in a non-inverting input impedances in the kiloohm range, this con-
mode equivalent to Fig. 2.6a. For this reason, figuration is almost never used directly to mea-
its input impedance is limited by the input sure biopotentials. Additionally, unless the four
resistors RI . On the other hand, as the signal is resistors in this circuit can be perfectly matched
encoded differentially between two terminals, (or in the correct ratio), this amplifier topology
the maximum signal output swing range is suffers from low CMRR.
doubled.
The 3-opamp instrumentation amplifier is the
Instrumentation amplifiers (IA) are used to prototypical architecture for instrumentation am-
measure small differential signals, while rejecting plifiers. It is also simply known as instrumen-
common mode levels. Another requirement of tation amplifier (IA) without any additional de-
IA is high input impedance, to avoid attenuating scription. In Fig. 2.7b, we can see it is indeed
signals from sensors with high output impedance. composed of 3 opamps in two stages. In the
This requirement is fundamental in biopotential first stage, 2 non-inverting opamps amplify the
recording as electrodes frequently present very difference V2 −V1 into a differential output signal.
large impedance due to their small size or In the second stage, a difference amplifier like the
imperfect contact. Aside from attenuating the one shown in Fig. 2.7a provides further amplifica-
signal of interest, low input impedance greatly tion and subtracts the common mode signal. Al-
decreases the system’s common-mode rejection though not typical, it is also possible to implement
ratio (CMRR), which is a measure of how this second stage with a fully differential opamp.
well the instrument is able to reject common- The transfer function of the classic IA is:
mode noise and interference, equal at both

terminals and hence zero by purely differential R1 R3
VO = 1 + 2 (V2 − V1 ). (2.13)
measurement. These and other important metrics RG R2
V1 R2 R3
RF
R1
RI
V1
VO RG VO
V2
RI
R1
RF
V2 R2 R3
(a) (b)
C2 S1
C1 S1 S2
C2
- C1
VI+ VO S1
VI+ VO-
- S2
VI
VO+ V I- VO+
C1 C2 S1
C1 S1
S2
Pseudo-resistor
C2 S1
(c) (d)
V I+
VI- VO
I1 x
VI+ +
R1 I1 R2 VO
VI- -
x
RF
RI
(e) (f)
Fig. 2.7 Architectures for differential and instrumenta- coupled amplifier, (d) switched capacitor instrumentation
tion amplifiers: (a) difference amplifier, (b) 3-opamp in- amplifier, (e) current balancing instrumentation amplifier,
strumentation amplifier, (c) fully differential capacitively and (f) differential difference amplifier
76 A. Akinin et al.
This architecture is favored for its high differ- put impedance may be a problem at high sig-
ential gain and high common-mode rejection. nal frequencies for some implementations where
Additionally, as the first stage is composed of the input capacitor C1 is very large, possibly
output-coupled non-inverting amplifiers, it offers tens of picofarads. Minimizing the size of C1 in
very high input impedance, allowing for biopo- this circuit precludes achieving high gain as the
tential measurements through non-contact and transfer function within the passband is directly
high impedance electrodes. One further advan- proportional to it:
tage of this topology, is that changing a sin-
gle resistor, RG , allows for tuning of the differ- C1 +
VO (j ωpass ) = VO+ − VO− = (V − VI− ).
ential gain of the circuit. Despite these advan- C2 I
tages, resistor matching is still critical to achiev- (2.14)
ing very high CMRR in excess of 100 dB, as
needed in highly sensitive application settings A switched-capacitor instrumentation amplifier
such as EEG recording on the scalp but difficult as shown in Fig. 2.7d is a discrete-time circuit
to accomplish in an integrated process without that works by sampling the instantaneous voltage
laser trimming. Moreover, the use of three sep- at periodic intervals, unlike the continuous-time
arate amplifiers results in extra noise and power amplification in the previous examples Fig. 2.7a–
consumption. c. In the sampling phase, the first part of the
interval, the S1 switches closes and charges the C1
A fully differential capacitively coupled ampli- capacitors to the input voltage. The second phase
fier is effectively the same topology shown in has the S1 switches open, while the S2 closes
Fig. 2.6c, except the passive elements used for setting the output to the product of the input and
gain ratioing are implemented with capacitors the above capacitor ratio. In order for the sampled
rather than resistors. Capacitive feedback ratio- voltage to completely settle in the relatively
ing, particularly when Fig. 2.7c is implemented short phase interval, the effective bandwidth
as an integrated circuit, ensures both accurate of the amplifier must be much faster than the
gain and lower power consumption for the IA. continuous-time signal bandwidth. This results
Achieving more accurate gain and better CMRR in increased power consumption and integrated
by accurate matching of pairs of capacitors like noise. Additionally, sampling a voltage onto a
C1 and C2 is relatively straightforward in custom- capacitor like C1 results in a phenomenon known
designed integrated circuits in standard semicon- as kT /C noise. As the name implies, the mean
ductor fabrication processes. Low power con- square value of this sampling error due to thermal
sumption is also more easily achievable owing noise is vn2 = kB T /C, where kB is the Boltzmann
to the ability to accurately integrate very small constant. Not having any preamplifier, the settling
capacitances, which in turn give rise to very large time of input capacitors is dependent on generally
impedances reducing the current draw of the am- high-impedance biopotential electrodes. Despite
plifier. In order for the amplifier to have a stable this architecture’s higher power consumption,
DC operating point despite the infinite impedance the time discretization it performs can replace
of the capacitive elements lacking feedback at the sample and hold circuit of the subsequent
zero frequency, Fig. 2.7c makes use of very high ADC.
resistance pseudo-resistors, which are leak ele-
ments composed of self-biased transistors with A current balancing instrumentation amplifier
ultrahigh resistance [16]. Without such pseudo- as conceptually demonstrated in Fig. 2.7e
resistors, implementing the large resistances in alleviates CMRR issues encountered in practical
the gigaohm-teraohm range required for the high- implementation of the above amplifier topologies
pass filter cutoff frequency of biopotential record- due to difficulties in matching resistors,
ing would use prohibitively large silicon area. capacitors, and even whole amplifiers. An
On the other hand, just like Fig. 2.7a, low in- open-loop transconductance amplifier first
stage converts a differential input voltage into brane proteins, these experiments can discover
a differential output current I1 , which flows the effect of various drugs, genetic manipulations,
through the balanced resistor R1 . The current and various pathologies on the basic building
I1 is then copied to flow through R2 resulting in blocks of the nervous system.
an amplified voltage which is then buffered by In order to record the cell membrane voltage,
a final circuit. This topology may include some a very different kind of microelectrode must be
kind of current feedback to the first stage’s output used. Commonly, a Ag/AgCl filament inside
to cancel common mode. As is evident, the lack a glass pipette filled with saline fluid and a
of duplicated resistors, capacitors, and amplifiers sharply tapering tip perforates the cell membrane
eliminates the CMRR losses due to matching. probing the intracellular space. A more advanced
Although Fig. 2.7e has much higher potential technique involves a patch clamp electrode which
CMRR and input impedance, low noise design has a flat tip that can form a seal around a patch
might incur significant power costs in copying of the cell membrane, through suction on the
currents and powering all the stages. electrode fluid. Additional manipulation can
either perforate the isolated membrane region,
A differential-difference amplifier is yet forming a longer-lasting intracellular interface
another type of instrumentation amplifier that than the sharp microelectrode or purposefully tear
seeks to maintain high input impedance and off a section of membrane to specifically study
high CMRR despite matching challenges. It its properties in isolation. Application of these
behaves like the fully differential amplifier electrodes to in vitro systems requires specialized
shown in Fig. 2.6c except without the inverting microscopy and mechanical micromanipulation
amplifier’s low impedance inputs. Within this tools. In vivo measurements of the intracellular
grouped amplifier, a first stage with two parallel potential require even more sophisticated
amplifiers converts voltage inputs into currents, optical and mechanical equipment. Beyond
which are summed together and amplified by simply recording the intracellular potential, the
a second stage. The key in maintaining high following techniques are applied in neuroscience
input impedance is using two isolated terminals experiments:
exclusively for the input signal and two separate
terminals exclusively for feedback. Voltage clamp is a configuration used to mea-
Other architectures and topologies are possible sure the behavior of ionic currents across the
and frequently featured in the scientific literature cell membrane while keeping the membrane po-
of biopotential amplifier design. Combining dif- tential constant. Practically, this is accomplished
ferent features presented in Fig. 2.7 may improve through feedback and operational amplifiers. A
the performance of a specific design targeting a potentiostat is a circuit that sets a potential dif-
particular application. ference between two nodes while measuring the
current required to maintain such potential. Fig-
ure 2.8a shows a simple voltage clamp potentio-
2.3.2 Intracellular Recording and stat circuit consisting of only one pipette elec-
Clamping Circuits trode and one return electrode in the solution.
Recalling the properties of opamps in negative
A different kind of neural instrumentation has feedback, the circuit will set the voltage at the
been developed for the acquisition of intracel- inverting terminal to the input control voltage
lular potentials. These measurements of mem- Vclamp , while the feedback across the amplifier
brane voltage provide the means to study in vivo through the transimpedance element RF results in
neural networks and the behavior of synapses an amplifier output:
and even characterize single ion-channel trans-
port proteins. Aside from characterizing the fun- VO = Vclamp + Imeas RF . (2.15)
damental physiological behavior of transmem-
78 A. Akinin et al.
RF
Iclamp
Vmeas
Imeas +
- VO
RA
RA
VO -
+ RF
Vm Vm
Vclamp
RI
Return Electrode Return Electrode
(a) (b)
Fig. 2.8 Basic instantiations of (a) voltage clamp and (b) current clamp instruments making use of a single intracellular
electrode
injection while monitoring the potential gener-

A second amplifier may be used to subtract the
ated is known as a galvanostat. Figure 2.8b shows
Vclamp and isolate the term directly proportional
a galvanostatic current clamp instrument imple-
to the membrane current. Unfortunately, due to
mented by simply measuring the potential on
RA , the access resistance of the microelectrode,
the intracellular electrode connected to a current
the membrane voltage is not exactly equal to the
source. A current that discharges or depolarizes
clamp voltage but rather
the membrane voltage eventually leads to an ac-
tion potential. This action potential manifests as
Vm = Vclamp − Imeas RA , (2.16) a sudden spike in the membrane voltage that is
recorded by the current clamp. Similar to the
which depending on the magnitude of RA can be voltage clamp, a current clamp composed of a
significantly different. One solution to this prob- single intracellular electrode cannot simultane-
lem is to have two different pipette electrodes: ously inject current and faithfully record the exact
one recording the intracellular voltage and one membrane voltage. Contrasting from the voltage
injecting current. As there is no current flowing clamp case, the clamp current setting Iclamp is
through the recording electrode (connected to a accurately set, and the current dependent term
different high input impedance amplifier), there in the voltage measurement can be eliminated
is no voltage difference between the command in post-processing of the data if the electrode
voltage Vclamp and Vm . This 3-electrode potentio- properties of RA are known:
stat, which requires probing the same cell with
two different pipettes, is difficult to use in small RF
VO = (Vm + Iclamp RA ) 1 + . (2.17)
neurons. A different approach toward construct- RI
ing a voltage clamp with a single penetrating
electrode involves time multiplexing the voltage
Dynamic clamp is an advanced rendition of the
sensing and current injection functions. Although
above voltage and current clamp techniques,
this method manages to record from the electrode
where the instrument can inject currents
when there is no current (therefore no voltage
generated with a prescribed algebraic dependence
drop), the settling time of this feedback control
on the membrane voltage, emulating a variable
system must be smaller than the time constant of
conductance on the electrode side of the interface.
the neuronal membrane.
The dependence of these currents on voltage can
take the mathematical expression of neurotrans-
Current clamp is used to investigate the ex-
mitter receptors and different ion channels that
citability of neurons. Keeping a certain current
exist in the cell membranes of neurons. Among directly affects available bandwidth and signal-
other applications, technique can enable investi- to-noise ratio, equally important as design
gations of neuronal responses to the uniquely considerations in ensuring sufficient signal
behaving ion channels, and even simulating quality. Therefore, judicious administration of
the complex dynamics of chemical synapses in a power budget among all the blocks of a neural
order to form hybrid biological-neuromorphic recording and stimulation instrument is of critical
neural networks. Dynamic clamps can be fully importance.
implemented with analog circuit control systems
that enforce the desired I-V relationship, or they
can be implemented through digitization and 2.4.2 Bandwidth
digital signal processing (DSP) in the loop.
As discussed in Sect. 2.1 and Fig. 2.2, the
temporal resolution of biopotential signals varies
2.4 Design Considerations and extensively across specific applications. The
Performance Metrics AFE used in a particular application must be
adapted or configured with sufficient bandwidth
Innovations in semiconductor technology to amplify the targeted biopotential signals,
and new circuit design topologies constantly avoiding aliasing and distortion degrading the
empower newer, more demanding applications signal. Because noise and interference from the
in neural engineering. Although these advances electrode and outside sources can decrease the
have enabled the impressive miniaturization of signal-to-noise ratio (SNR), amplifier systems are
modern technology, many functional aspects generally designed to limit undesirable content
of circuit design result in performance trade- outside the frequency band of interest. Even with-
offs. In this section we discuss some design out the need to reject out-of-band interference,
considerations, and decisions that must be made having excessive bandwidth in the AFE can
in order to optimize performance, and the metrics include more total integrated noise to the final
that define performance quality. digitized signal and needlessly consume more
power.
2.4.1 Power Consumption

2.4.3 Input Dynamic Range
One of the primary factors limiting design
choices in instrumentation is power consumption. Large gain amplifiers are required to magnify
Particularly in the case of implantable, portable, biopotentials from the lower- to mid-μV range
and wearable systems, gratuitous power to cover the full ADC input range in order to
consumption is detrimental. These systems are maximize precision in signal acquisition. This
generally limited by how much instantaneous in turn limits the maximum amplitude of a sig-
power they can harvest and how much total nal that can be received without saturating the
energy they can store in a battery. As such, output of the first stage AFE. Some types of
power autonomy is a critical consideration. instruments require great flexibility in configur-
Another consequence of high power consumption ing gain and bandwidth settings because they
is excessive heat generation which can cause receive different ranges of signals. For exam-
tissue damage and discomfort to users of ple, some MEA systems record intracellular ac-
biopotential recording equipment. The objective tion potentials with approximately 100 mV peak
is minimizing power conflicts with other to peak amplitude, as well as extracellular lo-
design requirements. The level of power used cal fields with μV-range amplitudes. Some com-
in the instrumentation, particularly the AFE, mon methods employed to increase input dy-
namic range (IDR) include: AC-coupling or high-
80 A. Akinin et al.
pass filtering DC-offsets, having programmable 2.4.5 Noise

or automatically adjusting gain and predictive
autoranging. Noise is a more fundamental problem in neural
One important reason for increasing IDR is the instrumentation, which needs to be managed
possibility of recording biopotentials simultane- through careful design considerations in the
ously with electrical stimulation, enabling appli- electrodes and the interface circuits. Noise limits
cations such as closed-loop deep brain stimulators the attainable precision in biopotential recording.
(DBS), closed-loop retina prostheses, and other Many design choices can determine how much
emerging therapies. Even in conventional clini- noise is added to the physiological signals in the
cally implemented applications, recording biopo- process of acquisition and digitization. Beyond
tentials in the presence of large interference sig- the thermal noise and 1/f noise inherent to
nals arising from other devices and surgical tools the electrode-electrolyte interface discussed in
is not uncommon. Sect. 2.2.1, additional noise is contributed by
circuit components in the electronics. The most
significant noise contribution takes place directly
2.4.4 Cross-Talk at the input stage of the AFE, where signal
amplitudes are smallest and most susceptible
Advances in neural instrumentation have allowed to the presence of additive noise. In contrast,
researchers and users of the technology to in- subsequent stages in the signal processing
crease the number of channels that they can si- pipeline operate at signal levels substantially
multaneously record. This increased throughput higher than the levels of additional noise sources
has been made possible due to increasing minia- present. It is useful to consider the effect of noise
turization of electrode arrays, connectors, and from each stage, as the equivalent input referred
interconnect traces. Cross talk is the presence noise by dividing the magnitude of noise by the
of interference signals from other channels in total accumulated gain from the AFE input to
a particular channel. Cross talk is likely to be the noise source origin. This way the effect of
caused by capacitive and other coupling between electronic circuit noise can be directly compared
electrodes or interconnect traces in the circuitry. by computing the signal-to-noise ratio (SNR),
Material selection in the insulation and encapsu- the ratio of the signal power over the noise
lation layers must be considered, as well as the power.
possibility these will degrade over the lifetime of
the device giving rise to even more cross talk. 2.4.5.1 Front-End Amplifier Noise
Beyond actual cross talk, designing electrode ar- Model
rays with overly fine pitch exceeding the spatial The noise contributed by active and passive
resolution of the signal of interest (Fig. 2.2) does resistive components in the AFE circuit can
not result in a significant increase of information, be minimized through systematic model-based
while giving rise to many of the problems that transistor-level circuit design. The transistors and
result in cross talk and noise. Although cross resistors that are inside an AFE generate two
talk is not primarily a circuit architecture issue, major types of noise: thermal noise and flicker or
the addition of electrode impedance monitoring 1/f noise. Like the thermal noise due to ionic
circuits to AFEs can accurately quantify the ex- motion at the electrode-electrolyte interface,
istence of cross talk. Most typically, the cross thermal noise generated in the AFE results
talk can be adequately compensated through DSP, from the random-walk thermal fluctuations of
and often the cross talk contributed by the instru- electrons or holes in semiconductors. The model
mentation is negligible to the amount of cross of thermal noise contributed by a single transistor
talk already present in the signal due to vol- in saturation and weak inversion depends on drain
ume conduction such as in EEG recorded on the current IDS as follows [17]:
scalp.
2
in,th = 2qIDS Δf (2.18) and offsets, is correlated double sampling (CDS)
[21], in which two samples of the amplifier output
where q is the charge of an electron and Δf is are collected in close succession: one measuring
the signal bandwidth. As evident from (2.18), the the signal and another measuring a reference such
average square noise is linearly proportional to as ground by bypassing the electrode input with
the average drain current, while the signal power an external reference or connecting to a separate
is proportional to the drain current square. Thus reference electrode. The premise of this technique
the SNR linearly improves with the magnitude of is that 1/f noise and other low-frequency
current. Consequently, increasing the current noise sources are highly correlated over short
while maintaining voltage at the same level time scales, so that periodic auto-zeroing at
directly increases power consumption. This sufficiently high rate eliminates most of it. The
results in a trade-off between thermal noise periodic auto-zeroing with the reference deci-
reduction and power consumption. Strategies mates the signal bandwidth or requires sampling
to reduce thermal noise without increasing power at higher frequency to maintain the same signal
consumption involve low voltage and higher bandwidth, which is worthwhile only if the de-
current circuit architectures. crease in flicker noise power is greater than the in-
Likewise, flicker noise, also known as 1/f crease in thermal noise power, when the 1/f noise
noise or pink noise, results primarily from trap- corner lies in the signal band, a condition met at
ping and release of charge carriers, at random higher amplifier bias levels maximizing signal-
time intervals, by lattice impurities at the Si/SiO2 to-noise ratio rather than minimizing power.
oxide interface surrounding the semiconductor
active element. which contributes a significant 2.4.5.2 Net Noise Contributions
source of noise at low frequency [18, 19]: The relative contributions between electrode
noise and circuit noise depend on electrode type
2
gm K 1 and geometry and on the available power budget
2
in,f = Δf (2.19) for signal amplification. In general, electrode
Cox W L f
noise is strongly correlated with the contact
where gm is transconductance, K is a process- impedance, but the actual level is significantly
dependent constant, W and L are width and higher than just the thermal noise from the
length of the MOS transistor, and Cox is the gate resistive portion of the impedance, especially
oxide capacitance. In some processes, PMOS for dry-contact electrodes that are gaining more
transistors are known to have less 1/f noise than widespread use than conventional wet-contact
NMOS transistors and therefore are often used in gel-based electrodes for their greater comfort and
the input differential pair of a front-end amplifier long-term endurance [22]. The aggregate sum of
for low-noise low-frequency applications in the electrode noise sources
√ can be quite large,
biosensing. Enlarging the MOS device size also on the order of μV / H z at 1 Hz, even for wet
decreases 1/f noise inversely proportional to electrodes. This far exceeds the noise contribution
area. of circuit components, illustrating the importance
An alternative approach to mitigate flicker of proper electrode selection. Due to integrated
noise in area-limited designs involves a high- current noise, both wet and dry electrodes have
frequency chopper that translates the input signal sharp 1/f 2 spectra, which show up as baseline
to a higher frequency for amplification and drifts in the time domain [22].
subsequently translates it back to the original Non-contact electrodes can pick up additional
frequency [20]. Expectedly, chopping is not noise from the insulating material between the
without trade-offs: increased power, decreased metal and skin [22]. In particular, acquiring
input impedance, and somewhat higher thermal signals through fabrics can be noisy due to
noise. Another frequently used method to miti- the intrinsic high resistance of the fabric (>100
gate flicker noise, as well as low-frequency drifts MΩ). This amounts to the equivalent of inserting
82 A. Akinin et al.
a large resistor in series with the amplifier periodically exchanging sides between pairs of
input and can add significant noise in the signal metal lines carrying signal and reference through
bandwidth. equally spaced via bridges.
To completely eliminate any common-mode
disturbances, it is critical that the AFE fully re-
2.4.6 Interference and jects them and purely amplifies the difference
Common-Mode Rejection in potential between the non-inverting VI+ and
inverting VI− inputs. An ideal differential AFE
In addition to the intrinsic noise sources that are outputs a voltage VO proportional only to this
fundamental to the operation of the electrodes difference VI+ − VI− ; practical limitations in the
and circuits, external noise and interference due circuit implementation may produce an additional
to parasitic electrical coupling from the environ- component in the output that depends on the
ment as well as biasing and supply variations may common mode (VI+ + VI− ) / 2:
also contaminate the signal. Unlike the intrinsic
noise, the extent of parasitic coupling from the 1
VO = Ad (VI+ −VI− )+Acm (VI+ +VI− ) (2.20)
environment, such as line noise at the 50/60 Hz 2
mains frequency, can be controlled through care-
ful design of the cabling connecting the electrodes where Ad and Acm are the differential gain and
and AFE, as well as the AFE circuits themselves. common-mode gain of the AFE, respectively.
In particular, common-mode noise sources, that AFEs with higher common-mode rejection ratio
couple nearly identically to the positive and neg- CMRR = Ad / Acm are proportionally more
ative leads of the AFE, can be completely elim- effective at suppressing common-mode noise rel-
inated with a properly designed truly differential ative to the differential signal. Most AFE designs
AFE. offer a CMRR greater than 80 dB; this implies
that common-mode disturbances at the input will
2.4.6.1 Differential Sensing Circuit be attenuated 10,000× more strongly than the
Techniques to Mitigate differential signal is being amplified. This is im-
Common-Mode Interference portant as 50/60 Hz mains line noise coupling to
The most thorough means to eliminate interfer- the electrodes, and wiring can easily exceed mV-
ence due to parasitic electrical coupling from the levels and otherwise inundate μV-level biopoten-
environment is to completely shield the wiring tial signals present between the electrodes.
between electrodes and AFE, such as by using
coaxial cabling in which the signal is carried 2.4.6.2 Input Impedance-Boosting
on the inner core surrounded by a solid ground Techniques
shield. This solution, adding substantial capaci- Even an AFE with perfect common-mode re-
tance on the signal line and incurring extra costs, jection (infinite CMRR) may still suffer from
is often impractical and unnecessary. A simpler common-mode leak-through in the presence of
solution is to ensure that the two wires carrying an imbalance in impedances between the signal
the signal and the reference are subject to the and reference paths feeding to the non-inverting
same interfere, which then appears as a common- and inverting AFE inputs. These imbalances are
mode additive disturbance to the differential sig- unavoidable despite careful design of the AFE cir-
nal between the wires. By physically bringing cuit, because the electrode-electrolyte/tissue in-
the two wires in close proximity along their en- terface impedance is highly variable and unpre-
tire length, any parasitic electrical coupling from dictable. Due to the finite input impedance into
the outside would be nearly identical to both either or both non-inverting and inverting input
of them. A practical means to realizing near- terminals to the AFE, these variations in interface
identical parasitic coupling is a twisted pair of impedances at the signal and reference electrodes
conductors. This strategy can be employed even cause a leakage of the common-mode voltage
on printed circuit boards or integrated circuits by from the electrodes, into a differential component
between the AFE inputs. This differential leakage VCM = (VI+ + VI− ) / 2 close to zero. This is
cannot be distinguished from the true differential accomplished by sensing the difference between
voltage between the electrodes by the AFE and the common-mode voltage VCM and ground and
hence passes through with full magnification. The feeding back the amplified difference with large
effect of this leakage is equivalent to an effective negative gain to an additional active ground
CMRR of the AFE: electrode in contact with body tissue. The
location for this electrode is typically far removed
|Zin (j ω)|
CMRReff (j ω) ≈ (2.21) from signal-carrying electrodes in order not to
|Zsig (j ω) − Zref (j ω)| interfere with the electrophysiological setup.
For electrocardiography (ECG) applications, this
where Zin is the AFE input impedance and
ground electrode is typically applied to the right
Zsig and Zref are the electrode-electrolyte/tissue
leg, hence the term “driven right leg” (DRL)
impedances for the signal and reference
commonly used to refer to the active grounding
electrodes, respectively. In addition to their
circuit, no matter where this electrode is applied.
effect in degrading CMRR, variations and
For EEG applications, the DRL electrode is
mismatch in electrode impedances also reduce
typically applied on the mastoid behind the ears.
signal amplitude and make the system more
Active grounding is much more effective than
susceptible to movement artifacts. Therefore, it
passive grounding by directly connecting the
is of paramount importance to mitigate all these
body to ground due to the impedance of the
effects by maximizing the AFE input impedance
ground electrode, causing voltage variations
well beyond the expected range and variation in
away from zero due displacement currents
electrode impedances.
induced by 50/60 Hz mains line noise and other
Although the AFE input resistance Rin is typ-
sources of common-mode noise acting on the
ically very high (in the teraohm range), the mag-
body. Active grounding with DRL accomplishes
nitude of its input impedance |Zin (j ω)| at higher
an effective grounding impedance that is smaller
frequencies can be substantially smaller due to
than the electrode-tissue impedance by a factor
AFE input capacitance, in addition to line capac-
1 + ADRL , where ADRL is the open-loop gain
itance in carrying the signals from the electrodes
of the DRL amplifier. Hence large reduction in
to the AFE. In many cases, the input impedance is
common-mode voltage can be obtained by large
limited by the parasitic switched-capacitor resis-
DRL gain.
tance of the input chopper or by the AC-coupled
In order to obtain large gain in the DRL circuit,
input capacitors. A positive feedback can boot-
an open-loop amplifier can be employed. How-
strap the AC-coupled input capacitors to boost the
ever, the feedback by the DRL circuit requires
input impedance, achieving input impedance on
careful design for stability. The DRL amplifier
the order of gigaohms. In order to further boost
is typically integrated with the AFE differential
the input impedance to teraohm levels, a unity-
amplifier on the same die and using the same de-
gain amplifier with active shielding can be used to
sign principles; for instance, capacitive feedback
bootstrap capacitance of the input transistor and
with pseudo-resistors around an OTA can realize
all other parasitic capacitance [23, 24].
a low-power DRL along with the capacitively
coupled AFE implementation in Fig. 2.7c. Typi-
2.4.6.3 Active Grounding: Driven Right
cally, large capacitance up to a few nF is required
Leg
to ensure stability due to variation in electrode
An alternative to techniques boosting CMMR by
impedances. A digitally assisted DRL circuit has
active boosting or active shielding of parasitic
the capability to have larger gain at the mains
input capacitance is to mitigate common-mode
frequency for higher rejection and lower gain
noise and interference directly through active
elsewhere for stability. In dry-electrode applica-
grounding. Rather than minimizing common-
tions, common-mode feedback to one of the dif-
mode gain Ad , active grounding operates by
ferential inputs in the front-end increases CMRR
dynamically driving the common-mode voltage
84 A. Akinin et al.
and ensures its stability independent of electrode signals because of the shear distance of the neuron
impedance variations. sources through the skull to the external elec-
For safety, a large (megaohm-range) resistance trodes. The obvious advantage of external elec-
is typically connected in series with the DRL out- trodes like those used for scalp EEG is that they
put, limiting its range of output current for short- enable a noninvasive neural interface. On the
circuit protection while leaving the DRL open- other hand, if greater biopotential quality and
loop gain unaffected for precise active grounding. neural signal features are desired and invasive
implants are acceptable, electrode sensors can
be placed very close or even through neurons
2.5 Survey of Neural in the brain. Large-area neural engineering tools
Engineering Applications for biopotential measurement include scalp elec-
troencephalography (EEG), magnetoencephalog-
The field of neural engineering has made sig- raphy (MEG), electrical impedance tomography
nificant progress toward useful and viable tech- (EIT), and epidural ECoG. Small-area neural en-
nologies for interfacing with the brain and body. gineering tools for biopotential measurement in-
Tools for neural recording have been developed clude subdural ECoG, cortex microelectrode ar-
to reliably measure everything from the fine de- rays, μECoG, and deep brain microelectrode ar-
tails of action potentials in vivo to wide, body- rays.
area electrophysiological signals. Advancements
in electrode sensor materials, high-performance 2.5.1.2 Temporal, Spatial, and Spectral
integrated circuits (ICs), and precision miniatur- Resolution
ization of complex systems have improved exist- Examples of neural engineering applications can
ing applications such as EEG and enabled new be categorized based on the resolution they are
applications such as μECoG and vision-restoring capable of achieving in the spatial, temporal,
retinal implants. When exploring neural engineer- and spectral domains. For applications involving
ing concepts for a specific application, consider- fast neural signals such as action potentials and
ation is necessary of the particular biopotential short-wave ripples, an amplifier and analog-to-
to be investigated (i.e., spikes, LFP, brain waves, digital converter (ADC) with sufficiently high
nerve impulses, etc.), the physiological source of sampling rate are required to capture the fine
the biopotential signal (i.e., the brain, brainstem, temporal features in the signal. Some signals,
or peripheral nerves), and the limitations of avail- such as those arising from the midbrain and
able recording technologies. hippocampus, need to be measured in close
proximity to the source and thus require high
spatial resolution from the recording apparatus.
2.5.1 Electrodes and High-density microelectrodes inserted in the
Instrumentation midbrain are currently required in this setting to
spatially resolve these signals. Perhaps the most
2.5.1.1 Scale and Invasiveness important element of distinction between neural
Sensing biopotentials with large devices external interfacing tools is spectral resolution. Certainly
to the human body produce very different sig- the distance between the source and sensor affects
nals than small implanted devices. In part, this the range of spectral measurable because of the
is because of a trade-off in scale between sur- inherent low-pass filtering nature of biological
face area coverage and location-specific access tissues, but other factors arguably play a greater
(Fig. 2.9). Gel electrodes placed on the surface role in determining the spectral resolution of
of the scalp connected to a biopotential ampli- the system. These factors can include the noise
fier, for example, could easily cover the entire floor of the acquisition circuitry, the 1/f noise
projected surface of the brain accessible on the of the amplifier, impedance of the electrode, and
head but would pick up only faint, low-frequency sampling frequency.
Fig. 2.9 Types and positioning of integrated electrode technologies for interfacing with the brain at varying spatial
scale and spectral bandwidth, at corresponding varying degrees of invasiveness [30]
2.5.1.3 Experiment Model 2.5.1.4 In-Ear Placement

Neural engineering tools to be used for a specific Applications of neural engineering involving dis-
application will also depend on the type of crete wearable sensors have continued to gain
experiment and biological model to be used. popularity. A particularly promising unobtrusive
To measure from the brain in vivo, one could use electrophysiology modality is Ear-EEG. Unlike
any of the above mentioned tools. To measure conventional EEG which has several electrodes
from the peripheral nervous system, implantable placed on the forehead and scalp, ear-EEG pro-
nerve cuff or nerve needle electrodes are needed vides a miniaturized and discrete platform for
instead. In experiments involving small animals electrode placement in the outer ear and in the
such as mice, a head-mounted sensing module ear canal [25–27]. Electrode sensors can be inte-
should be considered. Mouse EEG has small grated into existing personal audio devices such
screw-like electrodes that push up against the as hearing aids and wireless earphones. Signals
skull to measure brain activity, while allowing recorded from these sensors are comparable in
the animal to move freely. In other settings a quality to those measured from conventional EEG
sample of neural tissue such as a brain slice for certain event-related potentials owing, in part,
will need evaluation in vitro, in which case, a to the proximity of in-ear electrodes to major
microelectrode array with high-density recording auditory processing centers of the brain, such
units and liquid containment for culture media as the auditory cortex in the temporal lobe, the
is necessary. Finally, in cases where neurons are brainstem, and the auditory nerve fibers [25–
being grown in cell culture from either explants 27]. Furthermore, the ear canal has been demon-
or iPSCs, a multielectrode array with sharp points strated to contain useful biomarkers of overall
or nanowires will enable intracellular recordings, health and physiology. These biomarkers include
in addition to extracellular and intercellular electrodermal activity (EDA), a biomarker for
recordings. overall excitement or stress levels, sodium-sweat
concentration, a representative measure of hydra-
tion, and cerumen conductance, an indicator of
86 A. Akinin et al.
sebum production and lipid transport [28, 29]. acquisition chip, places small microchips across
These unique attributes of the in-ear environment the cortical surface of the brain to create a net-
play a role in the dynamics of electrode-skin work of sensors covering a broad spatial area
impedance, the understanding of which is impor- and recording individually at high spatiotemporal
tant for high-quality biopotential measurement resolution [30]. Like μECoG, this new form of in-
and consistency in offset between trails, differ- terfacing being termed modular-ECoG (mECoG)
ent subjects, and, in extended-period, continuous has a small footprint in regard to both physical im-
health monitoring. planted space and power consumption (Fig. 2.12).
2.5.3 Neurotechnologies for

2.5.2 Minimally Invasive
Penetrating Electrodes
Electrocorticography
The above technologies provide for measurement
EEG recording off the scalp is noninvasive
of neural activity in noninvasive (EEG, in-ear
and relatively low-cost, but limited in its
EEG) and minimally invasive (ECoG, μECoG)
spatiotemporal resolution. To achieve higher
manner. However, many applications require to
spatiotemporal resolution and spatial coverage in
penetrate neural tissue in order to measure the ac-
interfacing with the brain, the distance between
tivity of single neurons or the low-frequency os-
the electrodes and neural tissue must be reduced.
cillations of neural populations as local field po-
This implies a need to cross from the regime
tentials (LFP). Penetrating electrodes serve this
of noninvasive modalities to invasive, implanted
purpose.
technologies. Significant demand by academia,
One of the most commonly used designs is
government funding agencies, and even the
the Utah array (Fig. 2.13a) available from Black-
private sector for advanced brain research has
rock (https://www.blackrockmicro.com/electrode-types/
driven the development of such chronically
utah-array/). It includes 100 passive penetrating
implantable neural interfaces that leverage
electrodes that measure the electric potential at
precision material fabrication techniques and
their tips. Utah arrays are an excellent source
high-performance amplifier and wireless IC
of recordings for brain-machine interface (BMI)
technologies. As elaborated in the following
applications. BMI applications using Utah arrays
section, penetrating electrode systems tend to
implanted in motor cortex were used successfully
be of very high resolution, expensive, and very
to enable tetraplegic patients to accurately control
invasive, causing long-term tissue damage. An
3D robotic arms [50]. The Utah array design
excellent middle ground, electrocorticography
makes it particularly suitable for BMI applica-
(ECoG), offers superior spatial and temporal
tions, as electrodes cover a relatively large area
recording resolution compared to EEG, lower
(2 mm2 ), which makes it likely to find neural
cost of fabrication, and is suitable for chronically
activity tuned to the task of interest (e.g., neurons
implantable use for practical long-term brain
tuned to arm/hand movements).
research, brain-computer interface (BCI), and
More recently, there has been a large inter-
cognitive rehabilitation (Figs. 2.10 and 2.11).
national effort in designing new neural probes
A number of technologies are enabling next-
to advance research in neuroscience. The design
generation fully implantable high-density ECoG
principle of these probes is essentially different
systems, including PEDOT electrodes [31],
than the Utah array design, as neuroscientists
signal amplifying and filtering front-end ICs
are usually interested in recording the simultane-
[32], drivers for voltage and current stimulation
ous activity of different brain regions at different
[33], and wireless antennas for power and
depths. Moreover, a higher density of the elec-
communication [34].
trodes is desired, as it facilitates the identification
One such next-generation ECoG tool known
of single neuron activity via spike sorting [51].
as ENIAC, or encapsulated neural interfacing
Fig. 2.10 ECoG integrated neural interface technologies fabricated from thin polyimide foil substrate for implanted
and applications. (a) Subdural ECoG array with an elec- ECoG [35]. (c) μECoG electrode array with 3 different
trode diameter of 2 mm and electrode pitch of 1 cm. The electrode diameters and a total of 124 recording sites
radiograph image shows the position of the ECoG array [36]. (d) Electrode array with low impedance electrodes
implanted in the subject on the cortex surface, below fabricated for biopotential recording from PEDOT-carbon
the skull [32]. (b) Flexible 252-channel electrode array nanotube (CNT) composite coatings [31]
(a) (b) (c)

50mm
Titanium
housing
Feedthrough
Antennae
(d) (e)
Fig. 2.11 Clinically available neural interface systems cuff electrode for biopotential measurement of peripheral
and applications. (a) NeuroVista seizure advisory system or spinal nerves [39]. (d) BrainCon’s BCI system for
monitors biopotentials in the brain [37]. (b) Neuropace general-purpose medical neural interfacing [40, 41]. (e)
RNS system monitor biopotential activity of the brain WIMAGINE wireless implantable multi-channel neural
leading up to and preventing seizures [38]. (c) Spiral nerve interface [42]
88 A. Akinin et al.
Nanowire Device
Retina Tissue
Light Transparent MEA

(c) (d)
(a)
Wireless recording
(battery)
Micro wire bundle
Wireless 3 mm
stim.
(battery) Sciatic nerve
Cuff electrodes
(b) (e)
3 mm
Fig. 2.12 Emerging technologies for next-generation and acquisition chip (ENIAC) is a completely on-chip
neural interfaces and applications. (a) High-resolution integrated system for ECoG recording, stimulation, and
retinal prostheses with optical addressing and inductive data transmission [30, 33, 45]. (d) The Neuralink Neu-
telemetry, toward retinamorophic vision restoration in pa- ralace implantable BCI platform that offers thousands of
tients with degenerated photoreceptors in the retina [43]. biopotential recording channels [46]. (e) Thin film, high-
(b) An implantable nerve cuff for biopotential recordings density peripheral nerve cuffs for biopotential recording of
of the peripheral nervous system in freely moving animal the injured nerves used pre- and postoperatively [47]
subjects [44]. (c) The encapsulated neural interfacing
Fig. 2.13 Penetrating MEA devices. (a) Utah array. (From [48]). (b) Neuropixels. (From [52]). (c) SiNAPS multi-shank
probe. (From [49])
The Neuropixels probe [52] (Fig. 2.13b – https:// 28 μm spacing which can be record simultane-
www.neuropixels.org/) has a single 1 mm-long shank ously at 25 kHz. A newer version of the SiNAPS
with 960 closely spaced metal electrodes. The probe has multiple shanks (Fig. 2.13c) to measure
electrodes are around 20 μm apart from each more brain regions simultaneously [49].
other. Up to 384 simultaneous channels can be These newly developed neural probes are rev-
recorded, and the user can choose from which olutionizing the field of systems neuroscience,
electrodes to record from. The SiNAPS probe enabling high-yield experiments with thousands
[53] has a similar design, with 512 channels with of recorded neurons across different regions that
were unimaginable only a few years back.
Homework
biological origin. Usually such amplifiers are
1. Cochlear implant system block diagram: in the form of voltage amplifiers, because they
Draw a block diagram for a cochlear implant are capable of increasing the voltage level of
and describe, in words, the function of each of a weak biopotential signal picked up by an
the blocks. electrode in contact with surrounding tissue.
2. EMG dipole traveling wave: The patellar Design a biopotential amplifier for measuring
reflex causes the quadriceps muscle to contract neural signals that takes in voltage as its input
in response to the patellar tendon being struck. and has an input impedance greater than 10
You are interested in measuring the magnitude GΩ, so as to avoid any loading of the signal
of the reflex response using EMG. Consider being measured. The output impedance of the
the propagation of an action potential along a amplifier should be sufficiently low to drive
muscle fiber bundle in the thigh. The action an external 1 kΩ load with minimal distortion.
potential travels at a velocity v = 10 m/s. The voltage gain should be greater than 100
Model the current entering the muscle from the over the signal frequency band in order to
extracellular medium at the action potential resolve low-amplitude biopotential signals.
onset as a current monopole −I traveling at The signal frequency band ranges from 0.1 Hz
v and a second current exiting the muscle to 1 kHz, and the voltage gain should attenuate
at repolarization as a current monopole +I outside of this range in order to suppress noise
following at a distance d = 1 cm. An electrode and interference from unwanted signals and
on the arm surface at a distance D = 5 cm motion artifacts. You have a 3.3V battery
from the muscle measures the EMG signal available and can use any number of opamps,
relative to body ground. Assume a volume resistors, and capacitors.
conductivity σ = 0.1 Ω −1 m−1 . You measure Specify all component values and other
an action potential magnitude of Vmax = 5 nV. parameters to quantify your design, and
(a) What value of current I would you ex- explain your reasoning behind design choices
pect for the current monopoles to yield the based on the specifications.
given action potential magnitude? 4. Electroporation: Often it is necessary to in-
(b) Plot the EMG signal as a function of time sert genetic material into a cell, crossing its
as the action potential goes by. membrane. Electroporation is one means to
3. Bioamplifier design to specifications: open the membrane for insertion through the
Biopotential amplifiers are typically used in application of a high voltage. Here we study
electrophysiological experiments to increase single cell electroporation, by injection of cur-
the amplitude of weak electrical signals of rents +I and −I through two nearby elec-
90 A. Akinin et al.
trodes into the extracellular space, as shown impedance of your amplifier should be
in the figure below. The voltage across A and near-infinite.
B, on both sides of the cell, should not ex- (e) 60 Hz line noise present on the electrode
ceed 400 mV; otherwise the cell may die. The wires should be reduced in the amplifier
conductivity of the extracellular medium is output.
σ = 1 Ω −1 m−1 , the inter-electrode distance is (f) The frequency range of interest in the volt-
Xi = 250 μm, the cell is midway between the age signal is 100 Hz to 10 kHz.
electrodes centered at distance Xm = 125 μm (g) [Bonus]: Design a voltage clamp circuit
from either electrode, and the cell diameter is to the same above specifications for the
d = 20 μm. Determine the maximum ampli- current clamp, except the signal you are
tude of the current I you can safely inject. measuring is now current into the record-
Hint: Express the voltage VA − VB across the ing electrode for a fixed voltage across the
cell as a difference between two biopotentials electrodes, and the current signal ranges
generated by the same current dipole. between −250 and +250 nA, where the
Xm
d
+I A B -I
Xi
5. Intracellular recording: In electrophysi- corresponding output voltage should range

ology there are several techniques used to from −1 to 1 V.
measure various aspects of electric activity 6. Electrode model: A simplified equivalent cir-
in single cells. One such technique is the cuit model diagram of the electrochemical in-
current clamp, which injects a current into the terface between an implanted electrode and
cell and measures the resulting membrane the surrounding tissue is shown below, with
voltage of the cell. Based on the design half-cell potential Ehc , double-layer capaci-
specifications below, design a circuit which tance Cd , double-layer resistance Rd , and elec-
accomplishes a current clamp for intracellular trolytic series resistance Rs .
voltage recording.
(a) You are given a variable current source Rd
that is set to the desired current clamp Rs
value. ELECTRODE TISSUE
(b) The reference electrode is located in the
Ehc Cd
extracellular space, and the recording elec-
trode reaches inside the cell (through a
glass pipette penetrating the cell mem- (a) Write the impedance of the electrode-
brane). tissue interface Z(j ω), and find its
(c) The signal you are trying to measure at magnitude |Z| as a function of radial
the recording electrode is in the range of frequency ω. What are the minimum
−80 to 40 mV. and maximum of this magnitude over the
(d) Amplify the signal so an external frequency range, and at what frequencies
voltmeter measures 1.2 V at 40 mV and are the minimum and maximum attained?
0 V at −80 mV. The output voltage should With what time constant does the transient
vary linearly with input voltage. The input in the current settle for a voltage step
across the electrode-tissue interface?
(b) Identify the parameters in the electrode- Assume the activities of X2+ and Cl− are
tissue model diagram from the follow- unity, and the electrochemical cell is main-
ing experimental observations on the tained at room temperature. What are the
electrode relative to another reference chemical reactions that take place at each
electrode in contact with the same tissue in electrode?
close vicinity, which has known half-cell (f) The two electrodes are connected
potential −200 mV, double-layer capaci- to a voltmeter with infinite input
tance 1 nF, double-layer resistance 20 kΩ, impedance, reading 0.99 V. Although
and electrolytic series resistance 1 kΩ: you are not sure about the polarity
(i) The voltage measured by a voltmeter of the voltmeter terminals, what can
between the electrode and the you say about the half-cell potential of
reference electrode is +300 mV. electrode X?
(ii) The impedance measured by a (g) You find out that X is a very good re-
multimeter between the two electrodes ducing agent. Can you now guess what
at 0.1 Hz is 125 kΩ. material is X? Hint: a table of standard
(iii) The impedance between the two electrode (half-cell) potentials will come
electrodes now measured at 100 MHz handy.
is 5 kΩ. (h) The two electrodes are connected with an
(iv) The transient in the current measured electrical wire. What direction does cur-
by an ammeter between the two rent flow through the wire? Does the XCl2
electrodes, for a 10 mV step applied by concentration in solution increase or de-
a voltage source across the electrodes, crease?
settles with two time constants: the (i) After several seconds, the current
shortest one is 10 μs, and the longer subsides. Describe what happened, and
one is 100 μs. estimate the total charge that was delivered
7. Electrodes, redox reactions, and half- over the electrical wire.
potentials: A pure Ag electrode with 1 cm2 8. Skin-electrode model: Consider the simpli-
surface area is immersed in a bleach solution. fied circuit model of the skin-electrode inter-
An electrical current of 5 mA is injected face below:
through the electrode into the grounded (a) Find the expression for the impedance of
solution for 1 minute. the circuit when the subject is not sweat-
(a) What is the resulting change in mass of the ing, that is, disregarding the sweat glands
electrode? and ducts contribution.
(b) What has changed in the properties of the (b) Find the expression for the impedance of
electrode? the circuit when the subject is sweating,
(c) Show the equivalent circuit model for the that is, including the sweat glands and
electrode and indicate approximate values ducts contribution.
for all parameters from the literature. Cite (c) Using the following parameter values,
all sources. sketch or plot the magnitude of the
(d) Sketch a Bode plot for the electrode impedance as a function of frequency,
impedance. from 0.1 to 100 Hz on a log-log scale.
(e) This new modified electrode and another Make sure to properly label your plot with
electrode of unknown material X are im- values and units.
mersed in 1 liter of 1 M XCl2 solution.
Eel = 200 mV Rgel = 1 k Cel = 1 pF Csk = 10 pF Csw = 0

Esk = 430 mV RB = 100 k Rel = 1 M Rsk = 10 M Rsw = ∞
92 A. Akinin et al.
Eel
Rel Cel
Electrode
Conductive Gel Rgel
Esk Esw
Skin
Rsk Csk Rsw Csw

Body
RB
9. Sweat glands model of skin-electrode (c) For Cel = 20 nF, Rel = 50 kΩ,
impedance: Again consider the above skin- Rgel = 2 kΩ, Csk = 10 pF, and
electrode model. Rsk = RB = 500 Ω, show the Bode
(a) What is Eel when the electrode is made plot (amplitude and phase as a function
of silver coated with a thin layer of silver of frequency) of the contact impedance
chloride? for your reduced model, superimposed
(b) Ignoring the effect of sweat glands and with that for the full model in the absence
ducts in this model, and considering that of sweat. How good is the reduced
the time constant Rsk Csk of the skin-body model at frequencies of physiological
interface internal to the skin is practically interest?
zero on any time scales of interest, (d) What do you expect the effect of sweat to
reduce this model to an approximate be? Does the contact impedance increase,
standard electrolyte-electrode form with a or decrease, and why?
single half-cell potential, a single parallel 10. Multi-channel biopotential amplification
combination of double-layer resistance with common-mode compensation and
and capacitance, and a single series DRL: Consider the four-electrode bioam-
resistance. plifier system below:
IN1 Ri OU T1
R1 R2 VCM
Rf
IN2 Ri OU T2
Ro
R1 R2 RL
IN3 Ri OU T3
R1 R2
VCM
EEG amplifiers DRL amplifier
(a) Derive the voltage outputs OUT1 , OUT2 , 11. Amplifier common-mode rejection: Con-
and OUT3 in terms of the electrode volt- sider the following bioinstrumentation
ages IN1 , IN2 , and IN3 , respectively. What amplifier, where R1 = 1.000 kΩ, R1 =
are the voltage gains on each of these 0.999 kΩ, R2 = 1.000 MΩ, and R2 =
leads? 1.001 MΩ.
(b) Derive the driven right leg voltage output
RL in terms of the common mode of IN1 ,
IN2 , and IN3 . What is the common mode
voltage gain driving the right leg?
(c) Find the effective resistance from body
to ground in terms of the RL electrode
impedance RRL and the resistances of the
circuit.
(d) What purpose do resistors Ri and Ro serve
in this circuit?
94 A. Akinin et al.
(a) Calculate the differential gain, the 2

(b) Find an expression for iout in terms of the
common-mode gain, and the common noise generators for each transistor and
mode rejection ratio (CMRR) in dB for necessary small signal parameters. Can
this amplifier. we neglect the noise contribution from
(b) For an input differential signal Vsig RMS M2 and why?
and a total output referred noise from the
amplifier and resistors of Vno,AMP RMS
added to Vo , find the input referred noise,
References
and the signal-to-noise ratio (SNR).
(c) Now consider that the signals VA and 1. A. Hassibi, R. Navid, R.W. Dutton, T.H. Lee, Com-
VB in the body are picked up by elec- prehensive study of noise processes in electrode elec-
trodes with impedances ZA = 900 kΩ, trolyte interfaces. J. Appl. Phys. 96(2), 1074–1082
and ZB = 1.1 MΩ respectively. How (2004)
2. S.F. Cogan, Neural stimulation and recording elec-
does this affect the differential gain and trodes. Annu. Rev. Biomed. Eng. 10, 275–309 (2008)
CMRR in (a) and the output SNR in (b)? 3. P.L. Nunez, R. Srinivasan et al., Electric Fields of the
How would you remedy this situation Brain: The Neurophysics of EEG (Oxford University
in the design of the bioinstrumentation Press, New York, 2006)
4. H. Lindén, E. Hagen, S. Leski et al., LFPy: a tool
amplifier? for biophysical simulation of extracellular potentials
12. Current source cascoding: Consider the generated by detailed model neurons. Front. Neuroin-
cascode transistor shown below. The circuit form. 7, 41 (2014)
is normally used as a high-impedance current 5. G.R. Holt, C. Koch, Electrical interactions via the
extracellular potential near cell bodies. J. Comput.
source in analog circuits. In this problem, Neurosci. 6(2), 169–184 (1999)
we will analyze the effect of adding a 6. D. Sterratt, B. Graham, A. Gillies, D. Willshaw, Prin-
cascode transistor to the output noise of the ciples of Computational Modelling in Neuroscience
circuit. (Cambridge University Press, Cambridge/New York,
2011)
7. F. Claudi, Pyramidal neuron. www.scidraw.io
8. T.V. Ness, C. Chintaluri, J. Potworowski, S. Łe˛ski,
i2out
H. Gła˛bska, D.K. Wójcik, G.T. Einevoll, Modelling
and analysis of electrical potentials recorded in mi-
croelectrode arrays (MEAs). Neuroinformatics 13(4),
403–426 (2015)
+ VOUT 9. E. Hagen, S. Næss, T.V. Ness, G.T. Einevoll, Multi-
M2 –
modal modeling of neural network activity: comput-
+ ing LFP, ECoG, EEG, and MEG signals with LFPy
V2 2.0. Front. Neuroinform. 12, 92 (2018)
–
10. L. Berdondini, A. Bosca, T. Nieus, A. Maccione,
Active pixel sensor multielectrode array for high spa-
tiotemporal resolution, in Nanotechnology and Neu-
M1 roscience: Nano-electronic, Photonic and Mechani-
+ cal Neuronal Interfacing (Springer, New York, 2014),
V1 pp. 207–238
–
11. U. Frey, U. Egert, F. Heer, S. Hafizovic, A. Hierle-
mann, Microelectronic system for high-resolution
mapping of extracellular electric fields applied to
brain slices. Biosens. Bioelectron. 24(7), 2191–2198
(2009)
(a) Draw a small-signal model of this circuit, 12. T. Goto, R. Hatanaka, T. Ogawa, A. Sumiyoshi, J. Ri-
era, R. Kawashima, An evaluation of the conductivity
including all noise generators. Do not profile in the somatosensory barrel cortex of Wistar
neglect transistor output impedances and rats. J. Neurophysiol. 104(6), 3388–3412 (2010)
the body effect for this question. 13. M. Capogrosso, N. Wenger, S. Raspopovic,
P. Musienko, J. Beauparlant, L.B. Luciani,
G. Courtine, S. Micera, A computational model for
epidural electrical stimulation of spinal sensorimotor 28. X. Zhou, Q. Li, S. Kilsgaard, F. Moradi, S.L. Kappel,
circuits. J. Neurosci. 33(49), 19326–19340 (2013) P. Kidmose, A wearable ear-EEG recording system
14. C.H. Lubba, Y. Le Guen, S. Jarvis, N.S. Jones, S.C. based on dry-contact active electrodes, in 2016 IEEE
Cork, A. Eftekhar, S.R. Schultz, PyPNS: multiscale Symposium on VLSI Circuits (VLSI-Circuits), 2016,
simulation of a peripheral nerve in python. Neuroin- pp. 1–2
formatics. 17(1) 63–81 (2019) doi https://doi.org/10. 29. A. Paul, S.R. Deiss, D. Tourtelotte, M. Kleffner,
1007/s12021-018-9383-z T. Zhang, G. Cauwenberghs, Electrode-skin
15. A.P. Buccino, M. Kuchta, K.H. Jæger, T.V. Ness, impedance characterization of in-ear electrophysiol-
P. Berthet, K.-A. Mardal, G. Cauwenberghs, ogy accounting for cerumen and electrodermal re-
A. Tveito, How does the presence of neural probes sponse, in 2019 9th International IEEE/EMBS Con-
affect extracellular potentials? J. Neural Eng. 16(2), ference on Neural Engineering (NER), 2019, pp 855–
026030 (2019) 858
16. R.R. Harrison, C. Charles, A low-power low-noise 30. S. Ha, A. Akinin, J. Park, C. Kim, H. Wang, C. Maier,
CMOS amplifier for neural recording applications. P.P. Mercier, G. Cauwenberghs, Silicon-integrated
IEEE J. Solid-State Circuits 38(6), 958–965 (2003) high-density electrocortical interfaces. Proc. IEEE
17. R. Sarpeshkar, T. Delbruck, C.A. Mead, White noise 105(1), 11–33 (2017)
in MOS transistors and resistors. IEEE Circuits 31. E. Castagnola, L. Maiolo, E. Maggiolini, A. Minotti,
Devices Mag. 9(6), 23–29 (1993) M. Marrani, F. Maita, A. Pecora, G.N. Angotzi,
18. A.L. Mcwhorter, 1/f noise and related surface effects A. Ansaldo, M. Boffini, L. Fadiga, G. Fortunato,
in germanium. PhD thesis (1955) D. Ricci, PEDOT-CNT-coated low-impedance, ultra-
19. A. van der Ziel, Unified presentation of 1/f noise flexible, and brain-conformable micro-ECoG arrays.
in electron devices: fundamental 1/f noise sources. IEEE Trans. Neural Syst. Rehabil. Eng. 23(3), 342–
Proc. IEEE 76(3), 233–258 (1988) 350 (2014)
20. C.C. Enz, E.A. Vittoz, F. Krummenacher, A CMOS 32. E.C. Leuthardt, G. Schalk, J.R. Wolpaw, J.G. Oje-
chopper amplifier. IEEE J. Solid-State Circuits 22(3), mann, D.W. Moran, A brain–computer interface
335–342 (1987) using electrocorticographic signals in humans. J.
21. C.C. Enz, G.C. Temes, Circuit techniques for reduc- Neural Eng. 1(2), 63 (2004)
ing the effects of op-amp imperfections: autozeroing, 33. S. Ha, A. Akinin, J. Park, C. Kim, H. Wang, C. Maier,
correlated double sampling, and chopper stabiliza- G. Cauwenberghs, P.P. Mercier, A 16-channel wire-
tion. Proc. IEEE 84(11), 1584–1614 (1996) less neural interfacing SoC with RF-powered energy-
22. Y.M. Chi, T.-P. Jung, G. Cauwenberghs, Dry-contact replenishing adiabatic stimulation, in 2015 Sympo-
and noncontact biopotential electrodes: methodolog- sium on VLSI Circuits (VLSI Circuits) (IEEE, 2015),
ical review. IEEE Rev. Biomed. Eng. 3, 106–119 pp. C106–C107
(2010) 34. J. Park, C. Kim, A. Akinin, S. Ha, G. Cauwenberghs,
23. Y.M. Chi, C. Maier, G. Cauwenberghs, Ultra-high P.P. Mercier, Wireless powering of mm-scale fully-
input impedance, low noise integrated amplifier for on-chip neural interfaces, in 2017 IEEE Biomedical
noncontact biopotential sensing. IEEE J. Emerging Circuits and Systems Conference (BioCAS) (IEEE,
Sel. Top. Circuits Syst. 1(4), 526–535 (2011) 2017), pp. 1–4
24. S. Joshi, C. Kim, G. Cauwenberghs, A 6.5-μW/MH z 35. B. Rubehn, C. Bosman, R. Oostenveld, P. Fries,
charge buffer with 7-ff input capacitance in 65-nm T. Stieglitz, A MEMS-based flexible multichannel
CMOS for noncontact electropotential sensing. IEEE ECoG-electrode array. J. Neural Eng. 6(3), 036003
Trans. Circuits Syst. II: Express Briefs 63(12), 1161– (2009)
1165 (2016) 36. E. Tolstosheeva, V. Gordillo-González, V. Biefeld,
25. V. Goverdovsky, W. von Rosenberg, T. Nakamura, L. Kempen, S. Mandon, A. Kreiter, W. Lang, A
D. Looney, D.J. Sharp, C. Papavassiliou, M.J. Mor- multi-channel, flex-rigid ECoG microelectrode array
rell, D.P. Mandic, Hearables: multimodal physiolog- for visual cortical interfacing. Sensors 15(1), 832–
ical in-ear sensing. Sci. Rep. 7(1), 6948 (2017) 854 (2015)
26. D. Looney, C. Park, P. Kidmose, M.L. Rank, 37. M.J. Cook, T.J. O’Brien, S.F. Berkovic, M. Murphy,
M. Ungstrup, K. Rosenkranz, D.P. Mandic, An in- A. Morokoff, G. Fabinyi, W. D’Souza, R. Yerra,
the-ear platform for recording electroencephalogram, J. Archer, L. Litewka et al., Prediction of seizure like-
in 2011 Annual International Conference of the IEEE lihood with a long-term, implanted seizure advisory
Engineering in Medicine and Biology Society, 2011, system in patients with drug-resistant epilepsy: a first-
pp. 6882–6885 in-man study. Lancet Neurol. 12(6), 563–571 (2013)
27. A. Paul, A. Akinin, M.S. Lee, M. Kleffner, S.R. 38. F.T. Sun, M.J. Morrell, and R.E. Wharen Jr., Respon-
Deiss, G. Cauwenberghs, Integrated in-ear device sive cortical stimulation for the treatment of epilepsy.
for auditory health assessment, in 2019 41st Annual Neurotherapeutics 5(1), 68–74 (2008) http://dx.doi.
International Conference of the IEEE Engineering org/10.1016/j.nurt.2007.10.069
in Medicine and Biology Society (EMBC), 2019, 39. Ardiem Medical, Inc. photograph of standard spi-
pp. 56–59 ral nerve cuff electrode with four electrodes ar-
96 A. Akinin et al.
rayed radially around the cuff diameter http://www. 47. A. Hess-Dunning, D. Tyler, Thin film, high-
ardiemmedical.com/neural-cuff/ density peripheral nerve cuffs Advanced Platform
40. J.D. Fischer, The Braincon Platform Software-A Technology Center, US Department of Veteran
Closed-Loop Brain-Computer Interface Software for Affairs. https://www.aptcenter.research.va.gov/
Research and Medical Applications. PhD thesis, programs/neural/peripheral-nerve-cuffs/
University of Freiburg (2015) 48. S.-J. Kim, S.C. Manyam, D.J. Warren, R.A. Nor-
41. M. Schuettler, F. Kohler, J.S. Ordonez, T. Stieglitz, mann, Electrophysiological mapping of cat primary
Hermetic electronic packaging of an implantable auditory cortex with multielectrode arrays. Ann.
brain-machine-interface with transcutaneous optical Biomed. Eng. 34(2), 300–309 (2006)
data communication, in 2012 Annual International 49. F. Boi, N. Perentos, A. Lecomte, G. Schwesig, S. Zor-
Conference of the IEEE Engineering in Medicine and dan, A. Sirota, L. Berdondini, G.N. Angotzi, Multi-
Biology Society (IEEE, 2012), pp. 3886–3889 shanks SiNAPS active pixel sensor CMOSprobe:
42. C.S. Mestais, G. Charvet, F. Sauter-Starace, M. Foer- 1024 simultaneously recording channels for high-
ster, D. Ratel, A.L. Benabid, Wimagine: wireless 64- density intracortical brain mapping. bioRxiv (2019),
channel ECoG recording implant for long term clin- p. 749911
ical applications. IEEE Trans. Neural Syst. Rehabil. 50. L.R. Hochberg, D. Bacher, B. Jarosiewicz, N.Y.
Eng. 23(1), 10–21 (2014) Masse, J.D. Simeral, J. Vogel, S. Haddadin, J. Liu,
43. S. Ha, M.L. Khraiche, A. Akinin, Y. Jing, S. Damle, S.S. Cash, P. Van Der Smagt et al., Reach and grasp
Y. Kuang, S. Bauchner, Y.-H. Lo, W.R. Freeman, G.A. by people with tetraplegia using a neurally controlled
Silva, G. Cauwenberghs, Towards high-resolution robotic arm. Nature 485(7398), 372 (2012)
retinal prostheses with direct optical addressing and 51. A.P. Buccino, M. Kordovan, T.V. Ness, B. Merkt, P.D.
inductive telemetry. J. Neural Eng. 13(5), 056008 Häfliger, M. Fyhn, G. Cauwenberghs, S. Rotter, G.T.
(2016) Einevoll, Combining biophysical modeling and deep
44. B. Lee, M.K. Koripalli, Y. Jia, J. Acosta, M. Sendi, learning for multielectrode array neuron localization
Y. Choi, M. Ghovanloo, An implantable peripheral and classification. J. Neurophysiol. 120(3), 1212–
nerve recording and stimulation system for experi- 1232 (2018)
ments on freely moving animal subjects. Sci. Rep. 52. J.J. Jun, N.A. Steinmetz, J.H. Siegle, D.J. Denman,
8(1), 6115 (2018) M. Bauza, B. Barbarits, A.K. Lee, C.A. Anastassiou,
45. C. Kim, S. Ha, J. Park, A. Akinin, P.P. Mercier, A. Andrei, Ç. Aydın et al., Fully integrated silicon
G. Cauwenberghs, A 144mhz integrated resonant probes for high-density recording of neural activity.
regulating rectifier with hybrid pulse modulation, in Nature 551(7679), 232 (2017)
2015 Symposium on VLSI Circuits (VLSI Circuits) 53. G.N. Angotzi, F. Boi, A. Lecomte, E. Miele,
(IEEE, 2015), pp. C284–C285 M. Malerba, S. Zucca, A. Casile, L. Berdondini,
46. E. Musk, Neuralink, An integrated brain-machine in- SiNAPS: an implantable active pixel sensor CMOS-
terface platform with thousands of channels. bioRxiv probe for simultaneous large-scale neural recordings.
(2019) Biosens. Bioelectron. 126, 355–364 (2019)
EEG Signal Processing: Theory
and Applications 3
David L. Sherman and Nitish V. Thakor
Abstract iments demonstrate the importance of spectral

methods to analyze the EEG frequency and
The electroencephalogram or EEG is intro-
amplitude variability assessed through the IQ
duced in this chapter. Properties of the EEG
measure and TEO as a tool to detect the burst
time series are discussed as well. These in-
suppression events in the experimental mod-
clude individual frequency band descriptions,
els of cardiac arrest. Our review of the EEG
and their critical functional properties are dis-
methods and the principled discoveries com-
cussed. A variety of measurement tools are
ing out of our experiments provide a general
introduced to assist in the frequency-based in-
introduction to the basic properties of the EEG
tensity measure. These include the traditional
data interpretation and clinical translation.
strategies of power spectrum and time-domain
analysis for continuous EEG signals, and other
Keywords
strategies for capturing the power frequency
information about the sporadic events through Electroencephalogram · Spectral Analysis ·
the Teager energy operator (TEO). For the Parametric · Autoregressive (AR) · Fast
analysis of wave features, we also consider Fourier transform (FFT) · Cardiac arrest ·
additional time-frequency methodologies, par- Wavelets · Entropy
ticularly wavelets. Lastly, apparent random-
ness of the EEG signals lends itself to entropy
or information-theoretic analysis. We discuss
an entropy-based model known as information
3.1 Introduction: EEG
quantity or IQ which is shown to reflect the
Generalities
changes in EEG from healthy, to injury, to
recovering states. As a case study, we examine
The electroencephalogram (EEG) is a dynamic
the use of EEG signal processing methods as
non-invasive and relatively inexpensive technique
a diagnostic tool in the recovery of the brain
used to monitor the state of the brain. A
after cardiac arrest which causes global is-
recent quote from an editorial describing new
chemic brain injury. The corresponding exper-
IFCN (International Federation of Clinical
Neurophysiology) standards reads “despite the
tremendous progress in structural and functional
of this chapter (https://doi.org/10.1007/978-3-030-43395- brain imaging of the last decades, scalp EEG
6_3) contains supplementary material, which is available has remained an indispensable diagnostic
to authorized users. tool for studying physiologic and pathologic
D. L. Sherman () · N. V. Thakor cerebral activity. EEG recordings offer a direct
Johns Hopkins University, Baltimore, MD, USA measurement of cortical activity with very high

98 D. L. Sherman and N. V. Thakor
temporal resolution within the range of studied stimuli can be observed in EEG and acquired
cognitive and epileptic processes (milliseconds)” as evoked or event-related potentials occurring
[1]. EEG has a number of clinical uses that range within fractions of a second of the stimulus.
from monitoring normal wakefulness or arousal EEG provides excellent temporal resolution,
states to complex clinical situations involving significantly better than imaging methods such as
seizure or coma. The brain signals contain unique magnetic resonance imaging (MRI) and positron
information from different regions at any given emission tomography (PET). On the other hand,
time. An EEG signal recorded with electrodes since only a fixed array of scalp electrodes can be
placed on the scalp consists of many waves with used, EEG provides less spatial resolution than
different characteristics. To capture the whole MRI and PET. For better localization of signal
brain activity, arrays of EEG recording electrodes sources within the brain, EEG is often combined
are distributed over the entire scalp. The large with MRI scans.
amount of data recorded from even a single EEG
electrode pair presents a difficult interpretation
challenge. Signal processing methods are needed 3.1.1 Traditional EEG Bands
to automate signal analysis and interpret the
signal phenomena. Normal EEG is not perfectly sinusoidal, but
The greatest advantage of EEG is that it is an waxes and wanes and shifts dominant frequency
instantaneous and continuous indicator of brain’s constantly. The fact that autoregressive models
function. It is possible to record EEG signals are used often to model EEG of all types
continuously for longer than 24 hours. From shows that the signals may be modeled by
many channels; depending on the application, variable bandwidth signals. The autoregressive
electrode counts can range from single/dual frequency domain method models signals
channels to the 10–20 clinical system using with varieties of individual bandwidth. EEG
a montage of electrodes (Fig. 3.1a). We also waves are normally viewed as irregular with
show the newly adopted 25 channel montage no decipherable pattern. That precipitated an
for high-density EEG recordings. Response to interest in both nonparametric (e.g., fast Fourier
Fig. 3.1 (a) The layout of common electrodes or mon- fixed distances across the skull. (b) We also include the
tage. This is called the 10–20 system of standardized elec- newly adopted 10–10 system for high-resolution EEG.
trode locations. Electrodes are located at individualized (Reproduced with permission from [2])
3 EEG Signal Processing: Theory and Applications 99
transform (FFT)-based spectral and wavelets) closed. The waves tend to disappear in sleep-
and parametric (e.g., autoregressive modeling). ing or attentive patients. Fig. 3.2 shows the
The most common approach is the traditional effect of visual stimulation on waveform shape
power spectral analysis that divides EEG into and spectral peak sharpness. When eyes are
five spectral bands: delta, theta, alpha, beta, and closed and visual stimulation ceases, the alpha
gamma bands as follows: wave assumes a virtual sinusoidal shape as in
2(a). The accompanying spectrum in 2(b) is
• Delta (0–4 Hz): Delta waves are the lowest sharply peaked at 10–12 Hz. When the eyes re-
frequency component and include all the open, the sinusoidal character disappears. This
waves in the EEG below 4 Hz. Delta waves phenomenon is called alpha desynchronization
are usually seen during sleep, in infancy, or in as it is believed that neural ensembles are
serious organic brain diseases. Dominance firing in synchrony during this prior rhythm.
of delta waves in animals that have had As the eyes open and visual stimulation and
subcortical transections producing a functional processing ensue, the synchrony among the
separation of cerebral cortex from deeper brain neural ensembles is lost, and the alpha rhythm
regions suggests that these waves originate assumes a more complex shape indicative of
solely within the cortex, independent of multiple frequency components. A sub-band
any activity in the deeper brain regions. of alpha, the so-called mu-band (10–13 Hz),
Intermittent rhythmic delta activity with a is affected by imagery, such as imagined limb
frontal emphasis is generally associated with movements. The mu-band modulation is used
destructive or compressive lesions involving in building brain-machine interfaces, e.g., for
the diencephalon and upper midbrain, with control of prosthesis [6], for example, by using
deep frontal lesions, and with acute metabolite the modulation of mu-band power to open or
and electrolyte disturbances [3]. Animals are close a prosthetic hand.
known to have more widespread activity in • Beta (13–30 Hz): EEG activity between 13
this range. and 30 Hz (and sometimes up to 50 Hz) is
• Theta (4–7 Hz): Theta waves have frequencies classified as beta waves. Beta waves tend to be
between 4 and 7 Hz. These waves occur mainly recorded from the frontal and parietal lobes. It
in parietal and temporal regions of children’s can be further classified into two subclasses:
brains. In healthy and alert adults, such slow beta I (13–20 Hz) and beta II (20–50 Hz). Beta
activity is generally inconspicuous or absent, I, almost twice the alpha wave in frequency,
but it does appear during emotional stress (dur- appears together with alpha wave. It is affected
ing periods of disappointment or frustration) by mental activity in much the same way as
or during certain stages of sleep [4, 5]. Dif- the alpha wave. Beta II appears only during
fuse theta and other slower activity are com- intense mental activity and tension. Thus, one
monly encountered shortly after a generalized type of beta activity is elicited by mental ac-
seizure, as well as in patients with metabolic tivity, whereas the other is inhibited by it [7].
disorders, white matter encephalopathy, or ex- Figure 3.3 illustrates that the beta band magni-
tensive lesions of the upper brain stem. tude is quite small relative to the alpha wave
• Alpha (8–13 Hz): Alpha waves are rhythmic amplitude. We examine two subjects’ power
waves occurring at frequencies between 8 and spectral density measurements from an occipi-
13 Hz. Brain activity in this frequency range tal derivation. One subject has a high beta peak
is often recorded from the occipital region and the other person has a low beta peak
(and sometimes from the parietal and frontal • Gamma (30–150 Hz): EEG activity in the rela-
regions as well) during consciousness and is tively higher frequencies beyond the beta band
attenuated by visual and other sensory stim- is classified as gamma. Sometimes it is bro-
ulus. Alpha waves are typically seen in an ken down into the gamma (30–80) and high
awake but relaxed person and when eyes are gamma band (>80 Hz). The gamma frequency
Fig. 3.2 The alpha wave: the phenomenon of desynchro- theta range near 4 Hz, as these sidebands are clearly
nization. (a) The EEG wave and (b) the associated spec- visible around the central peak. (c) The EEG wave and
trum. The signals are recorded from posterior electrode (d) spectrum of desynchronized alpha rhythm examples
O1 during an “eyes closed” session. The alpha wave is from the same subject as in a and b. Now with eyes open,
very distinctive in this picture, and the spectral energy is desynchronization occurs and the subject shows spectral
concentrated at a single prominent frequency at around dispersion in the EEG. Spectral dispersion means there are
12 Hz. The alpha wave has a strongly sinusoidal shape several peaks in the spectrum spread out over a wider range
but with waxing and waning amplitude. It seems to be of frequencies
modulated by a low frequency signal perhaps in the low
Fig. 3.3 Comparative 20

spectra of two
subjects—one with high
beta during alpha 10
synchronization and one
with low beta. Though 0 high beta
alpha and beta are low beta
considered independent -10
phenomena, beta is often a
dB
harmonic of the alpha

wave. Some people simply -20
have low beta that is not
seen above the spectral -30
roll-off of the EEG which
maybe artifact of the
spectral analysis
-40
methodology
10 20 30 40 50 60
Frequency (in Hz)
power in EEG signals is generally low, and are used as indicators of cognitive function
filtering to remove noise may further attenuate and arousal. Signal power or modulation at
the gamma power. However, in select research these frequencies is also used in brain-machine
and clinical applications, gamma frequencies interfaces as well as cognitive studies [8–10].
3.1.2 Paroxysmal Discharges a mono-rhythmic pattern with little reactivity,

and EEG Shapes (2) the burst suppression pattern, and (3) dif-
fuse and local slowing and complete slowing,
EEG signals also have shape characteristics that and (4) generalized periodic discharge [12].
uniquely characterize neurological disorders.
They belong to one of these three categories:
3.1.3 Survey of EEG Applications
(a) Nonperiodic (occasional spikes, bursts, and
random noise)—We can track the revival of A few primary applications of EEG recording and
the organism through its brain wave signa- analysis are briefly mentioned below:
ture waveforms. Early EEG recovers as burst-
ing or burst suppression events (intermittent Epilepsy Monitoring The most common clin-
bursts and isoelectricity). ical reason for getting an EEG by a referring
(b) Sinusoidal (alpha rhythm, sleep delta, sleep physician is to detect a suspected seizure disorder.
spindles, halothane rhythm). The EEG can confirm the diagnosis of epilepsy
(c) Nonsinusoidal and periodic (spike and wave and depending on the particular pattern of seizure
epilepsy)—see Fig. 3.4 taken from animal assist in the particular seizure type that is evident
epilepsy experiments—which examines [13–16]. Beyond the seizure, the EEG is also
spike-wave epilepsy recordings from the useful for assessing a variety of other cerebral
cortex as well as two thalamic relay zones disorders. In disorders of altered consciousness
as well. and potential encephalopathies, the EEG can offer
(d) Symmetry of alpha activity within hemi- convincing evidence of the degree of the disorder
spheres can be monitored. In cases of re- and indicate whether it is a local process with
stricted lesions such as tumors, hemorrhages, focal effects or something more widespread. The
and thrombosis, it is usual for the cortex prognosis can often be determined from the EEG
to generate lower frequencies. EEG signal itself. EEG recording and analysis has found ap-
distortion can be manifested by reduction in plications in basic research on origins and local-
amplitude, a decrease of dominant frequen- ization of seizure origin [17–19], testing epilepsy
cies beyond the normal limit, and production drug effects [20], and assisting in experimental
of spikes or special patterns. Epileptic cortical excision of epileptic focus [15, 21–25].
conditions produce stimulation of the cortex An active area of intense research is the area of
and the appearance of high-voltage waves (up seizure prediction with some important monitor-
to 1000 μV) referred to as “spikes” or “spike ing revisions having been made throughout [26,
and wave” phenomena. EEG patterns have 27]. Some seizure recordings from cortical and
been shown to be modified by a wide range of subcortical (thalamic) sources in a chemocon-
variables, including biochemical, metabolic, vulsant animal model of epilepsy are shown in
circulatory, hormonal, neuroelectric, and Fig. 3.4.
behavioral factors [11]. By tracking changes
of electric activity during such drug abuse- Sleep Studies The EEG is sensitive to a range of
related phenomena as euphoria and craving, states spanning from different levels of vigilance
brain areas and patterns of activity that mark states: stress state, alertness to resting state, hyp-
these phenomena can be determined. nosis, and sleep. The area of sleep studies is one of
(e) Useful prognostic information may also be the success stories of EEG. Sleep staging is very
gleaned from generalized characteristics of clearly reflected in a very reactive EEG. During
the EEG such as variability, reactivity, the normal state of wakefulness with open eyes, beta
variable sleep and wakefulness states, etc. waves are dominant. In relaxation or drowsiness,
Patterns that are usually signs of a poor out- alpha activity rises and if sleep appears, power of
come are (1) alpha coma which ostensibly is lower frequency bands increases. Sleep is gener-
Fig. 3.4 Typical

waveforms recorded during
seizures in an animal
model of absence epilepsy
caused by the
chemoconvulsant
pentylenetetrazol (PTZ).
The anterior thalamus is a
subcortical element that
has shown to have strong
association with cortex
during these seizures. The
spike and wave shape is
clearly evident in anterior
thalamic and cortical
derivations
ally divided into two broad types: (1) nonrapid and can be used for user’s command recognition,
eye movement (NREM) sleep and (2) REM sleep. e.g., desynchronization of the motor associated
NREM and REM occur in alternating cycles. mu waves (brain waves of alpha range frequency
NREM is further divided into stage I, stage II, associated with physical movements or intention
stage III, and stage IV [28]. The last two stages to move), changes in beta or high gamma bands,
correspond to a deeper sleep, where slow delta or presence or alteration of certain event-related
waves show up in higher proportions. With these potentials (ERPs) [32, 33].
slower dominant frequencies, responsiveness to
stimuli decreases, and so these are considered EEG Biofeedback Biofeedback machines are
indicative of deep sleep. Stage I sleep is typified devices for creation of different mind states (e.g.,
by slowing, disintegration into varying or increas- relaxation, top performance) by practical manip-
ing irregularities. Thus, EEG monitoring finds ulation of the brain waves into desired frequency
extensive use to investigate sleep disorders and bands by repetitive visual and audio stimuli. For
physiology [11, 29]. making the training more effective, biofeedback
methods can be involved. Originally, changes in
Brain-Computer Interface As the EEG pro- finger skin resistance or temperature were mon-
cedure is noninvasive and painless, it is being itored. EEG biofeedback or neurofeedback uses
widely used to study the brain organization of EEG signals for feedback input. It is suggested
cognitive processes such as perception, mem- that this learning procedure may help a subject
ory, attention, language, and emotion in normal to modify his or her brainwave activity. One of
adults and children. The brain-computer interface the methods involved in neurofeedback training is
(BCI) is a communication system that recognizes the so-called frequency-following response. Neu-
a user’s command only from his or her brain- rofeedback involves actively facilitating changes
waves and reacts according to them [30, 31]. For in the functioning of the brain in desired way,
this purpose, the intervening computer algorithm e.g., to increase the alpha activity, for achieving
and/or subject is trained. Simple tasks can consist behavioral goals such as increased relaxation or
of desired motion of a cursor or a pointer dis- attention (Fig. 3.5).
played on the screen only through the subject’s
imaging of the motion of his or her left or right Figure 3.6 shows reactive μ-rhythm desyn-
hand. As a consequence of imaging process, cer- chronization or loss of power with the presenta-
tain characteristics of the brainwaves are altered tion of selected motor imagery, namely, a hand,
Fig. 3.5 Absence status epilepticus with generalized, frontally predominant, 3-Hz spike- and slow-wave discharges.
Reprinted with permission from [11]
as a cue for reactive hand grasping. These elec- • Monitor human and animal brain development
trodes are located over the sensorimotor regions [46].
in both hemispheres as shown by Fig. 3.6. Certain • Test drugs for antiepileptic seizure effects [47].
subjects have a high-amplitude alteration in mu- • Monitor the neonatal electroencephalogram
rhythm after stimulus presentation, while others (EEG) [48, 49].
have a reduced change in power in this band [34]. • Monitor psychophysiological variables [50].
There are numerous other applications of EEG
recording and analysis that range from basic sci- Thus, EEG signal is both complex and non-
ence of brain organization, development, and cog- stationary (varying with time) and rich with in-
nition to clinical science and applications to man- formation. It is clearly seen as useful in terms of
age patient disease states, drug, and surgical treat- interpreting brain states and disease conditions.
ments: Therefore, signal analysis methods are needed to
interpret the rhythms, and they are deployed in
• Observe vigilance states including alertness, the instrumentation used in the laboratory and the
coma, and brain death [35]. clinic. Signal analysis refers to digitizing the EEG
• Locate areas of brain damage following head signal and then using computational and math-
injury, stroke, tumor, etc. [36, 37]. ematical tools to analyze and interpret the EEG
• Test afferent pathways (by evoked potentials) signals. Broadly speaking, EEG analysis methods
[38, 39]. can be divided into two signal categories: time-
• Monitor cognitive engagement (alpha and domain methods and frequency or spectral meth-
gamma rhythm) [40, 41]. ods. Time-domain measurements include mea-
• Produce biofeedback situations, alpha, etc. surements of the raw signal characteristics. Fre-
[42, 43]. quency or spectral domain methods are a broad
• Monitor and potentially manage anesthesia range of methods that decompose the EEG sig-
depth (e.g., the bispectral index for certain nals into their basis set, commonly spectral or
anesthetic agents such as propofol) [44, 45]. wavelet methods, and then study the components
Right Cluster
Left Cluster
High-Aptitude : LEFT MI High-Aptitude : RIGHT MI

1 1
0.5 0.5
Power (8-12Hz)
Power (8-12Hz)
0 0
-0.5 -0.5
-1 -1
-1.5 -1.5
-2 -2
-1 0 1 2 3 4 5 -1 0 1 2 3 4 5
Time[s] Time[s]
Low-Aptitude : LEFT MI Low-Aptitude : RIGHT MI

1 1
0.5 0.5
Power (8-12Hz)
Power (8-12Hz)
0 0
-0.5 -0.5
-1 -1
-1.5 -1.5
-2 -2
-1 0 1 2 3 4 5 -1 0 1 2 3 4 5
Time[s] Time[s]
Fig. 3.6 The μ-rhythm suppression during motor im- (on the left) and right (on the right) hand motor imagery.
agery over two sensorimotor regions. This figure illus- Power changes in the contralateral cluster with respect to
trates the mean power spectrum (±standard error across the imaged hand show a clear μ-rhythm suppression or
subjects) in the μ-band (8–12 Hz) over two groups of stronger in the high-aptitude (upper plots) as compared
electrodes with locations over sensorimotor regions in the to the low-aptitude (lower plots) group. (Reproduced with
left (yellow) and the right (purple) hemisphere during left permission from [34])
of the signals. Lastly, joint-time frequency meth- its respective mean, variance, and higher-order
ods, which also include wavelet methods, have moments [53]. It is typically used to detect and
also found considerable utility in EEG analysis decipher sleep states. Period or interval analysis
[51, 52]. counts the number of incidences where the EEG
crosses the zero voltage line [54, 55]. This tech-
nique is extremely sensitive and nonspecific [56].
3.2 Time-Domain Despite its limitations, it is routinely used in in-
Representation and Methods traoperative and depth of anesthesia monitoring.
Traces of some waveforms commonly seen in the
The most prevalent time-domain method is sim- EEG are shown below in Fig. 3.7. One method of
ply the visual inspection. Amplitude distribution depth of anesthesia monitoring uses the detrended
looks at the distribution of the EEG amplitude and fluctuation analysis [58]. Detrended fluctuation
Fig. 3.7 Typical EEG

waveforms from humans
during the progressive
steps in anesthesia. Awake,
sedated, fully anesthetized,
and burst suppression
EEG. (Reproduced with
permission from [57])
analysis (DFA) is a measure of variability in domain analysis is often used for detecting epilep-
a signal. It looks at the average variance of a tic spikes, seizures, bursting, and burst suppres-
detrended time series within different sized time sion, which are all complex features of EEG sig-
apertures [59]. It is a wholly temporal feature that nals indicative of abnormal brain injury or disease
looks at variability of a time series with linear states.
trend removal. The variability of the time series is
measured as a function of window size [59]. For
each of the different waveforms shown below, we 3.2.1 The Teager-Kaiser Energy
see a different DFA parameter. The bispectrum is Algorithm: Theory
another method for discerning anesthesia depth
[45]. The Teager energy operator (TEO) is a versatile
Modified versions like the period-amplitude tool for measuring instantaneous changes in sinu-
model are also employed for sleep and surgery soidal energy [60, 61]. Based on the dynamics of
[54, 55]. Segmentation analysis divides the EEG an oscillating system, it accurately tracks changes
into quasi-stationary segments. The segments can in frequency-dependent energy. Further modifi-
be of varying length. The analysis is useful for re- cations of this algorithm allow for the detection
ducing long EEG recordings into smaller data sets of modulations of this sinusoidal signal [60, 62]
that can be thoroughly analyzed. There are several such as the formant frequencies of speech. The
methods that are used to directly parameterize the TEO belongs to a class of generalized energy op-
EEG signal in the time domain. These include erators possessing optimal time-frequency prop-
slope, amplitude, and second derivative measure- erties with low distortion as seen by [62, 63].
ments. Other measures include moments such as Energy operators are the output of the second-
mean, variance, kurtosis, and skewness, calcu- order Volterra operations.
lated from the signal in the time domain itself. The TEO [61] is actually a Volterra filter [64]
Time-domain techniques are usually much faster, or
i.e., responsive to instantaneous signal analysis,
than frequency domain techniques. Another set of Ψ (n) = x 2 (n) − x (n − j ) x (n + j ) ≈ A2 ω2
time-domain measures that has been used for over (3.1)
a quarter century is the Hjorth descriptors [51,
52]. These methods are discussed in the section where A and ∗ are the amplitude and radian fre-
on seizure detection methods. These are quick quency of an input sinusoid in a pure noise, w(n).
assessments of immediately accessible parame- In [60], a class of quadratic detectors is defined
ters that are available in the raw signal. Time- as
Qkm (x(n)) ≡ x(n)x (n + k) humans usually ranges in duration from 100 to

1000 msecs, but bursts of shorter or greater length
− x (n − m) x (n + k + m) are also commonplace. They typically range in
(3.2) voltage from 100 to 1000 μV. BS range from
benign—induced by anesthetics or sedatives—to
Energy operators isolate a slowly varying severe life-threatening anoxic BS. BS patterns
baseband signal proportional to the amplitude are the rule with deep levels of anesthesia
and frequency of the input signal squared. The stemming from use of ether, barbiturates, or other
TEO represents the energy of the signal within anesthetics [35]. BS may also result from global
a typical frequency band [60]. If x(n) consists ischemic brain injury, where they may occur
of two components such as a signal, s(t), and following anoxic state and subsequent recovery
noise, w(n), then the standard TEO output has [67]. Energy operators such as the TEO have
cross-terms according to [61] been utilized for burst suppression detection [68,
69].
Ψ (s(n)+w(n))=Ψ (s(n)) +Ψ (w(n))

− s (n+1) w (n−1) −2s(n)w(n)
3.3 Frequency Domain Methods
+s (n−1) w (n+1)
(3.3) 3.3.1 Nonparametric Spectral
Methods
We assume that the input noise is second order
white or uncorrelated so that its autocorrelation is Spectral Analysis Spectral analysis method is
by far the most prominent form of EEG analysis
Rww (k) = E {w(n)w (n + k)} = 0, for k =
0 method [70]. Contemporary spectral analysis is
(3.4) the classic and most used nonparametric method.
It takes the EEG and converts it into a power spec-
Accordingly, for zero mean signal and noise trum. The power spectrum includes the subdivid-
input signals, the expected value of the TEO is ing of the waveform into delta, theta, alpha, and
simply beta bands. The next section covers this topic in
greater depth. Power at these different bands can
E {Ψ (s(n) + w(n))} = ω2 A2 + σ 2 (3.5) correlate with normal bodily function. The power
spectrum is most commonly denoted indirectly as
One application example of the time-domain the Fourier transform of the autocorrelation func-
method considered here is burst suppression tion. The power spectrum expression becomes
(BS) detection. Other time-domain methods are
Hjorth descriptors and the slope detector methods T
deployed for seizure detection. Alternating Pxx (f ) = rxx (τ ) exp (−j 2πf τ ) dτ (3.6)
patterns of bursts and suppression in the EEG 0
are clinically known as burst suppression or
BS. The history of BS began with Derbyshire’s
T
where rxx (τ ) = x (τ + t) x (τ ) dt is the defini-
observations during his anesthetized cat 0
experiments [65]. Chatrian first described BS tion of the autocorrelation function. The discrete
as a “pattern characterized by theta and/or delta form of the autocorrelation function is defined as
waves, at times intermixed with faster waves, and
intervening periods of relative quiescence” [66].
N
Niedermeyer revised this definition to include rxx (n) = x (τ ) x (τ + t) (3.7)

i=1
single spikes, grouped spikes, and poly-spikes
mixed with delta and theta activity [66]. BS in
The direct method involves taking the FFT parametric methods that have been developed to
without an intervening step of first calculating the analyze EEG signals.
autocorrelation. It is directly related to the fact The autoregression (AR) model will be cov-
that the sum of the mean square value of the signal ered in great depth in the next section. It uses past
is equivalent to its energy or values of a data set and their respective weighting
factors to predict future values. It is advantageous
∞ since it allows higher resolution, variable fre-
E= |x(t)|2 dt quency settings, and a variable bandwidth. The
−∞ autoregressive moving average (ARMA) model
is a generalized form of the AR model. It offers
By an extension of Parseval’s theorem, it is a more efficient and quicker way of representing
known that the power spectrum of a particular EEG seg-
∞ ∞ ment. Other parametric models include inverse
AR filtering, which attempts to inverse filter a
|x(t)| dt =
2
|X(f )|2 df (3.8)
signal to obtain the generating white noise or
−∞ −∞
residues. This is useful for detecting transient
So, the energy density over frequency is equiv- non-stationarities present in epileptiform EEG.
alent to |X(f )|2 . This is referred to as the energy The AR model is appropriate for characteriz-
spectral density or power spectral density. The ing short EEG signal segments. The AR model
power spectrum is calculated as the magnitude yields a modified spectrum, which can then be
squared of the Fourier transform or used to identify its characteristic spectral peaks
(see Fig. 3.8). Dominant frequency analysis is
P xx(f ) = |X(f )|2 (3.9) based on mathematical framework of autoregres-
sive (AR) modeling [71]. Using a fairly stable
This is the standard periodogram estimator. EEG time series and the correct model order,
Written out in standard format, it is written as dominant frequency analysis allows a spectral
breakdown of the EEG from the data itself. From
N 2 the peaks of the AR spectra, dominant frequen-
1
cies, or the frequencies in which the power in
PPer (f ) = x(n) exp (−j 2πf n) (3.10)
N n=0 EEG signal is concentrated, are identified. AR
analysis has several advantages. It has higher
There are several methods to do smoothing resolution than the fast Fourier transform (FFT),
in the frequency domain to curb large variabil- variable frequency settings, and variable band-
ity through averaging and windowing methods width capabilities. Unlike methods employing the
[71]. fast Fourier transform (FFT), such as the peri-
odogram, AR methods are strictly parsimonious
and compact, requiring as few variables as nec-
3.3.2 Parametric (Modeling) essary for adequately representing a spectrum.
Methods In Fig. 3.8, a comparison of the AR and FFT-
based spectra is shown. Clearly, the AR spectrum
Parametric analysis tries to describe the EEG is visibly free of any artifact caused by spurious
based on some parameters derived from the data peaks in the FFT method [72].
itself, or model of a biophysical process (like
alpha rhythms, delta waves, etc.). The drawback Mathematical Formulation of AR Modeling
of most nonparametric methods is that it requires Mathematically, an autoregressive or AR model
a large data set. Parametric methods build on a set represents the time series x(n) as the weighted
of standard values and are adapted with a much sum of previous values of itself, offset by an error
smaller data set. Below are some examples of in the prediction factor, w(n):
30
30
20
Power/Frequency (dB/Hz) 20
Power/Frequency (dB/Hz)
10
10
0
0
-10
-10
-20
-20
-30
-30
-40
-40
-50
0 5 10 15 20 25 30 0 5 10 15 20 25 30
Frequency (Hz) Frequency (Hz)
Fig. 3.8 Comparative contours of (a) a periodogram us- seizure data from a neonatal EEG database. The contour
ing the FFT-based power spectrum and (b) the autore- of the AR spectrum is a lot smoother than the periodogram
gressive (AR) spectrum of the same data. Data was EEG
∗
x(n) = w(n) − a(1)x (n − 1) ap (n) = ap−1 (n) + Kp ap−1 (p − n) (3.12)
− a(2)x (n − 2) · · · − a(p)x (n − p)
In the above equation, ap (n) are the parameters
The weights are given by the AR parameters for model order p, and ap-1 (n) are the parameters
a(i), i = 1 . . . p, where p represents the model for order p-1. Kp is the reflection coefficient for
order. Thus, AR equation can be broken into two order p found by minimizing the arithmetic mean
parts, one that is the estimated value: of the forward and backward linear prediction
error power [71]. A more detailed explanation
x̂(n) = −a(1)x (n − 1) and derivation of these equations can be found in
[74].
− a(2)x (n − 2) · · · − a(p)x (p − 2)
(3.11) Obtaining the Power Spectrum and Dominant
Frequency Peaks Once the AR parameters are
and the other which is the error in prediction, obtained, they can be used to create a power
which is obtained when the actual and estimated spectrum of the stationary time series. The z-
value are subtracted: transform of the original AR equation yields
w(n) = x(n) − x̂(n) W (z)

X(z) =
1+a(1)z−1 +a(2)z−2 + · · · + a(p)z−p
When appropriate model orders and parame-
ters are chosen, w(n) reduces to zero mean white Here, X(z) and W(z) are the z-transforms of
noise. x(n) and w(n). The power spectrum is given by
2
Obtaining the AR Parameters Several algo- W (z)

P (z)=
rithms have been developed to generate the AR 1+a(1)z +a(2)z +. . . . +a(p)z
−1 −2 −p
parameters for a given data sequence and model (3.13)

order. One method uses the Burg algorithm [73].
It is one of the earliest and the best-known AR The poles of X(z) are given by the roots of the
algorithms. The method is based on the Levin- denominator of the X(z):
son recursive solution to Yule-Walker equations
which relates the order p parameters to the order zp + a(1)zp−1 . . . a(p)
(p-1) parameters:
Evaluation of the above expression at the unit referring to baseline low, baseline medium, and
circle (z = exp(jω)) results in the following: baseline high band powers, respectively. During
the experiment, AR transformed EEG data is
exp (pj ω) + a(1) exp ((p − 1) j ω) . . . a(p) averaged at subsequent one-minute intervals.
These power levels are then normalized relative
Factoring of the above result yields to their baseline levels to produce three power
band measurements as:
(exp (j ω) −P1 ) · (exp (j ω) −P2 )
(3.14) Pnlf = Pl /Pbl
. . . . . . (exp (j ω) −P3 ) Pnmf = Pm /Pbm (3.16a-b-c)
Pnhf = Ph /Pbh
Here, the Pi ’s are the poles of X(z). Thus, the
values ω (real) for which exp(jω) matches the Now, the information can be reduced to one
phase of Pi represent the real pole of X(z). This quantitative number, the normalized separation
frequency ω is where the dominant frequency of [78]. This study showed that the hypoxic–
this particular data set exists: ischemic (HI) injury causes a dispersion
or redistribution of power in the dominant
ωdominant = of roots frequencies. NS monitors the rate of recovery
for each band with respect to baseline. A high NS
And the analog frequencies of the spectral represents a disproportionate recovery of power,
roots are given by and vice versa. A high NS implies a poor recovery
of the electrical function.
Fsampling
Fdominant = of roots
2π |Plf − Pmf | + |Pmf − Phf | + |Plf − Phf |
NS =
Plf + Pmf + Phf
Once the dominant frequencies are obtained,
(3.17)
the powers at these frequencies need to be tab-
ulated. There are a number of solutions to this
issue. One way is to integrate the power spectrum
between desired frequencies [75].
3.3.3 Parametric Methods of Signal
Power (fk ) = 2 · Re [PAR (z) |ARroots ] Processing: The MUSIC
PAR (z) = X(z) · X∗ (1/z∗ ) Algorithm
(3.15)
MUSIC (MUltiple SIgnal Classification) is a very
3.3.2.1 Diagnostic Power popular spectral analysis technique that requires
of the Autoregressive Method the estimation of the autocorrelation matrix of a
Is Used as a Dominant random process assumed to be composed of a
Frequency Method to Calculate known number of complex sinusoidal signals im-
Normalized Separation mersed in white noise [79]. The key to improved
For our experimental work, we used the AR performance of these methods is the division of
power spectrum to develop a new index of EEG information in the autocorrelation matrix into two
recovery—the normalized separation (NS) [76, vector subspaces, the signal subspace and the
77]. This examines the separation in the power in orthogonal (or noise) subspace. They also provide
different bands that are normalized by baseline high-resolution estimates of the sinusoidal fre-
band power. In our conception of normalized quencies in conventional spectral estimation. The
separation, one minute’s worth of baseline data is dimensionality of the signal subspace is deter-
averaged to establish baseline power levels: Pbl , mined by the number of assumed complex sinu-
Pbm , and Pbh , where bl, bm, and bh are subscripts soids immersed in the noise sequence in question.
Eigenanalysis allows for removal of extraneous matrix will have exactly 2M nonzero eigenvalues.
noise contributions for a much higher signal to The eigenvalue/eigenvector decomposition of the
noise ratio in the ultimate calculation of the fre- autocorrelation matrix, Rp , is
quencies of the embedded sinusoids. A compari-
son of the seizure EEG to background in these
p

2M

RP = λi v i ⊗ v H
i + ρ w vi ⊗ vH
i

M i=1 i=p+1
R(K) = 1/2 cos (μi kT ) + ρi δ(k) (3.18) (3.22)
i=1
The remaining p + 1–2 M eigenvectors of
where ρ i is the noise variance and δ(k)is the the signal matrix span the noise subspace. The
Kronecker delta. The (p + 1) × (p + 1) autocor- spectral estimator makes use of the orthogonal-
relation matrix where p > 2M can subsequently ity between both subspaces. The noise subspace
be expressed as the sum of corresponding signal eigenvectors will be orthogonal to any vectors
and noise autocorrelation matrices: spanning the signal subspace, e.g., the sine waves
comprising the signal. This property is exploited
Rp = Sp + Wp (3.19) by the noise subspace estimator known as the
MUSIC algorithm. The frequency estimator is
The signal matrix can be written in terms of
a sum of rank one outer products. The outer 1
products are of the form P (f ) = (3.23)
sH (ω) Nv s (ω)

M
Pi where s(ω) = [1 exp (−jω) exp (−j2ω) . . .
Sp = si ⊗ sH
i + si ∗ ⊗si
T
(3.20) exp (−jpω)]T and the matrix of p + 1-2M eigen-
i=1
2
vector is
where si = [1 exp (−jωi ) exp (−j2ωi ) . . .
2M
exp (−jpωi )]T is the sinusoidal signal vector for Nv = vi ⊗ vH
i (3.24)
frequency ωi . The signal matrix has a complete i=p+1
Vandermonde structure.
Likewise, the noise matrix Wp = ρI, where There have been several MUSIC applications
ρ is the added white noise. Here, SP and Wp are for the EEG. These have been mainly used in
the signal and noise autocorrelation matrices, re- the area of source localization and electric field
spectively. The signal matrix will have the eigen- specification in the brain [80, 81].
decomposition

p+1
3.3.4 Wavelets
SP = λi v i ⊗ v H
i (3.21)
i=1 3.3.4.1 The Wavelet Transform:
Variable Time and Frequency
where λ denotes the ith eigenvalue and vi , the re- Resolution. The Continuous
spective eigenvector and λ1 ≤ λ2 ≤ λ2 ≤ . . . λ2M . Wavelet Transform (CWT)
It is clear that for (p + 1) eigenvalues of the signal A decomposition of a signal based on a wider
matrix, SP , there are M nonzero eigenvalues. The frequency mapping and consequently better time
corresponding M eigenvectors known as the prin- resolution is possible with the wavelet transform.
cipal eigenvectors will span the same subspace as The continuous wavelet transform (CWT) [82] is
the M real sinusoids comprising the signal portion defined thusly for a continuous signal, x(t),
of the time series. It can be shown that the signal

1 t −τ or
CWTx (τ, a) = √ x(at)g ∗ dt
a a
f 1 t−τ
(3.25) CWTx τ, a= =√ x(t)g ∗ dt
fo f/f o f/fo
or with a change of variable as (3.27)

√ τ This is the equivalent to logarithmic scaling
CWTx (τ, a) = a x(at)g ∗ t − dt of the filter bandwidth or octave scaling of filter
a
(3.26) bandwidth for power-of-two growth in center fre-
quencies. Larger center frequency entails a larger
where g(t) is the mother or basic wavelet, ∗ de- bandwidth, and vice versa.
notes complex conjugate, a is scale factor, and The complex-valued Morlet wavelet is often
τ is a time shift. Typically, g(t) is a bandpass selected as the choice for signal analysis using the
function centered around some center frequency, CWT. The Morlet wavelet [82] is defined as
fo. Scale a allows the compression or expansion of
t2
g(t) [83]. A larger-scale factor generates the same g(t) = ej 2πfo t e− 2 (3.28)
function compressed in time, whereas a smaller-
scale factor generates the opposite. When the an- with its scaled version written as
alyzing signal is contracted n times, similar signal
features or changes that occur over a smaller t t2
= ej a t e− 2a2
2πfo
g (3.29)
time window can be studied. For the wavelet a
transform, the same basic wavelet is employed
with only alterations in this signal arising from The Morlet wavelet insures that the time-scale
scale changes. Likewise, a smaller-scale function representation can be viewed as a time-frequency
enables larger time translations or delays in the one as in (30). This wavelet has the best repre-
basic signal. sentation in both time and frequency because it
The notion of scale is a critical feature of the is based on the Gaussian window. The Gaussian
wavelet transform because of time and frequency function guarantees a minimum time-bandwidth
domain reciprocity. When the scale factor, a, is product, providing for maximum concentration in
enlarged, the effect on frequency is compression both time and frequency domains [87]. This is the
as the analysis window in the frequency domain best compromise for a simultaneous localization
is contracted by the amount 1/a [84]. This equal in both time and frequency as the Gaussian func-
and opposite frequency domain scaling effect can tion’s Fourier transform is simply a scaled version
be put to advantageous use for frequency localiza- of its time-domain function.
tion. Since we are using bandpass filter functions, The example of the scaled versions of the Mor-
a center frequency change at a given scale yields let wavelet is shown in Fig. 3.9a. The effect of the
wider or narrow frequency response changes de- scaling factor, a, is illustrated in the time domain
pending on size of the center frequency. This is by the changes in width and scale in upper Fig.
the same in analog or digital filtering theory as 3.9a. A smaller scale factor such as 0.25 or 0.5
“constant-Q or quality” factor analysis [85–87]. results in a wavelet with smaller width in the time
At a given Q or scale factor, frequency translates domain with more concentrated energy in the
are accompanied by proportional bandwidth or center of the wavelet. In turn, the accompanying
resolution changes. In this regard, wavelet trans- frequency domain content is wider and at higher
forms are often written with the scale factor ren- frequencies for this smaller wavelet scale factor.
dered as The large wavelet scale factor results reciprocally
in a shorter frequency extent with a wider time
f span.
a=
f0
Fig. 3.9 (a) A mother wavelet (a = 1) with one dilation constant. The figure below (b) shows the respective time
(a = 2) and two contractions (a = 0.25 and 0.5). Note the (duration) and frequency ranges covered for the different
amplitude changes that keep the area under the wavelet dilation factors. (Reproduced with permission from [84])
Also the Morlet wavelet is defined by an ex- Once again time-frequency (T-F) reciprocity
plicit function and leads to a quasi-continuous determines the degree of resolution available in
discrete version [86]. A modified version of the time and frequency domains. Choosing a small
Morlet wavelet leads to fixed center frequency, fo , window size σ in the time domain yields poor fre-
with width parameter, σ , quency resolution while offering excellent time
resolution, and vice versa. To satisfy the require-
t2
g (σ, t) = ej 2πfo t e− 2σ 2 (3.30) ment for admissibility and G(0) = 0, a correction
term must be added. For ω > 5, this correc-
tion term becomes negligibly small and can be
omitted. The requirements for the wavelet to be From (30.33), it can be seen that for low fre-
analytic and of zero mean is best satisfied for quencies ω (larger scales a), the width ω of the
ω0 = 5.3 [82]. Gaussian is smaller and vice versa. In fact, the
Should the CWT cover a wide frequency ratio ω/ω is constant [92], i.e., Morlet wavelets
range, a computational problem would arise. may be considered filter banks of constant Q-
For example, if we wish to display the CWT factor. Figure 3.9a and b shows the principle of
over ten octaves (a change by one octave scaling with associated dilation and contraction
corresponds to changing the frequency by a in the time domain and their concomitant effects
factor of 2), the computational complexity (size in the frequency domain.
of the summation) increases by a factor of Based on Eqs. (30.32 and 30.34a,b), the
210 = 1024. The algorithm by Holschneider et al. wavelet transform can be implemented in the
[88] [89] solves this problem for certain classes frequency domain. At each scale, the Fourier
of wavelets by replacing the need to resample image of the signal can be computed as
the wavelet with a recursive application of an
interpolating filter. Since scale is a multiplicative Y (ω, a) = S (ω) • Gm (ω, a) (3.33)
rather than an additive parameter, another way
of reducing computational complexity would be with S(ω) being the Fourier transform of the
by introducing levels between octaves (voices). signal, Gm (ω, a) being the scaled Fourier image
Voices are defined to be the scale levels between of the Morlet wavelet at scale a, and • standing
successive octaves, uniformly distributed in a for element-by-element multiplication (window-
multiplicative sense [90, 91]. Thus, the ratio ing in frequency domain). The signal at each
between two successive voices is constant. For scale, a, will finally be obtained by applying the
example, if one wishes to have ten voices per inverse Fourier transform:
octave, then the ratio between successive voices
is 21/10 . The distance between two levels ten CWT (τ, a) = {F F T }−1 Y (ω, a) (3.34)
voices apart is an octave.
The CWT can also be implemented in fre- This approach has the advantage of avoiding
quency domain. Eq. (3.25) may be formulated in computationally intensive convolution of time-
the frequency domain as domain signals by using multiplication in the fre-
quency domain, as well as the need of resampling
√ the mother wavelet in time domain [93, 94].
CWT (τ, a) = a S (ω) G ∗ (aω) ej τ ω dω
Note that the CWT is in the general case of
(3.31) a complex-valued transformation. In addition to
its magnitude, its phase often contains valuable
where S(ω) and G(ω) denote the Fourier trans- information pertinent to the signal being ana-
formed s(t) and g(t), and j = (−1)1/2 . The analyzed, particularly in instants of transients [84].
lyzing wavelet g(t) has generally the following Sometimes, the T-F distribution of the nonstation-
Fourier transform: ary signal is much more important. This may be
√ obtained by means of real-valued wavelets. Alter-
Gτ,a (ω) = a G (aω) ej ωτ (3.32a) natives to the complex-valued Morlet wavelet are
simpler, real-valued wavelets that may be utilized
The Morlet wavelet (3.29 and 3.30) in fre- for the purpose of the CWT. For example, the
quency domain is a Gaussian function: early Morlet wavelet, as used for seismic signal
analysis [95], had the following form:
1
Gm (ω) = √ e−(ω−ω0 ) /2
2
g(t) = cos(5t)e−t
2
(3.32b) /2
(3.35)
2ω
It is represented by a few cycles of a sine separate signal constituents. Linearity means that
wave tapered by a Gaussian envelope. Though cross-terms are not generated in applying either
computationally attractive, this idea contradicts the linear T-F or time-scale operations. Aside
the requirement for an analytic wavelet, i.e., its from linear TFRs, there are quadratic T-F rep-
Fourier transform G(ω) = 0 for ω < 0. An ana- resentations which are quite useful in display-
lytic function is generally complex valued in time ing energy and correlation domain information.
domain and has its real and imaginary parts as These techniques, also described elsewhere in
Hilbert transforms of each other [95, 96]. This this volume, include the Wigner-Ville distribu-
guarantees only positive-frequency components tion (WVD), smoothed WVD, the reduced infer-
of the analyzing signal. ence distribution (RID), etc. One example of the
The short-time Fourier transform (STFT) has smoothed Wigner-Ville distribution is
the same time-frequency resolution regardless of
frequency translations. The STFT can be written
1
as W (t, f ) = s ∗ t − τ e−j τ 2πf
2
∞ (3.37)
−2πjf t 1 τ
STFT (τ, f ) = x(t)g ∗ (t−τ ) e dt × s ∗t + τ h dτ
2 2
−∞
(3.36) where h(t) is a smoothing function. In this case,
the smoothing kernel for the generalized or Co-
where g(t) is the time window that selects the hen’s class of TFRs is
time interval for analysis or otherwise known
τ
as the spectrum localized in time. The STFT is φ (t, τ ) = h δ(t) (3.38)
often thought to be analogous to a bank of band- 2
pass filters each shifted by a certain modulation
These methods display joint T-F information
frequency, fo . In fact the Fourier transform of a
in such a fashion as to display rapid changes
signal can be interpreted as passing the signal
of energy over the entire frequency spectrum.
through a multiple bandpass filters with impulse
They are not subject to variations due to window
response, g(t)ej2π ft , and then using complex de-
selection as in case of the STFT. A problematic
modulation to downshift the filter output. Ulti-
area for these cases is the elimination of those
mately, the STFT as a bandpass filter rendition
cross-terms that are the result of the embedded
simply translates the same low pass filter function
correlation.
through the operation of modulation. The char-
It is to be noted that the scalogram or scaled
acteristics of the filter stay the same though the
energy representation for wavelets can be repre-
frequency is shifted.
sented as a Wigner-Ville distribution as [87]
Unlike the STFT, the wavelet transform imple-
mentation is not frequency independent so that
higher frequencies are studied with analysis fil- |CWTx (τ, a)| =2
Wx (u, n) Wg
ters with wider bandwidth. Scale changes are not
equivalent to varying modulation frequencies that u−t
∗ , an dudn
the STFT uses. The dilations and contractions of a
the basis function allow for variation of time and (3.39a)
frequency resolution instead of uniform resolu-
tion of the Fourier transform. where
Both the wavelet and Fourier transform are
1 −j τ 2πf 1
linear time-frequency representations (TFRs) for Wx (t, f ) = x ∗ t− τ e x ∗t+ τ dτ
2 2
which the rules of superposition or linearity apply
(3.39b)
[97]. This is advantageous in cases of two or more
3.3.4.2 The Discrete Wavelet Transform Because of the scaling factor a in the denominator
In the discrete TFRs, both time and scale changes of the argument of the wavelet, the wavelet had to
are discrete. Scaling for the discrete wavelet be resampled at a sampling interval Ts /a for each
transform involves sampling rate changes. A scale a.
larger scale corresponds to subsampling the
signal. For a given number of samples, a larger 3.3.4.3 Application of Wavelets
time swath is covered for a larger scale. This is and Entropy: The Definition
the basis of signal compression schemes as well of IQ—Information Quantity
[98]. Typically, a dyadic or binary scaling system From the perspective of the information theory,
is employed so that given a discrete wavelet the amount of information contained in a signal
function ψ(x) is scaled by values that are binary. can be physically quantified by calculating the
Thus, entropy [99]. The classical Shannon entropy is
defined mathematically as

ψ2j (t) = 2j ψ 2j t (3.40)

M
where j is the scaling index and j = 0, 1, 2, SE = − p(m)log2 p(m) (3.42)

m=1
3 . . . . In a dyadic scheme, subsampling is always
decimation in time by a power of 2. Translations where p(m) is the probability of finding the
in time will be proportionally larger as well as for system in the mth microstate with 0 ≤ p(m) ≤ 1
a more sizable scale.
M
It is for discrete time signals that scale and res- and p(m) = 1. To analyze nonstationary
m=1
olution are related. When the scale is increased, signals, the temporal evolution of SE must
resolution is lowered. Resolution is strongly re- be determined. To do so, an alternative time-
lated to frequency. Subsampling means lowered dependent SE measure based on a sliding tem-
frequency content. Rioul and Vetterli [87] use the poral window technique is applied [100]. Letting
microscope analogy to point out that smaller scale {s(i) : i = 1, . . . , N} denote the raw sampled sig-
(higher resolution) helps us to explore fine details nal, we define a sliding temporal window as the
of a signal. This higher resolution is apparent with set W(n; w; ) = {s(i), i = 1 + n, . . . , w + n}
samples taken at smaller time intervals. of length w ≤ N. Here, ≤ w is the sliding step,
Following the definition in (26–27), the dis- and n = 0, 1, . . . , [n/] − w + 1, where [x]
crete implementation of the CWT in time domain denotes the integer part of x.
is a set of bandpass filters with complex-valued By dividing signals into the predictable part
coefficients, derived by dilating the basic wavelet and the uncertain parts using wavelet transform
by the scale factor a for each analyzing frequency. (WT), the measure calculates the entropy from
The discrete form of the filters for each a is the the uncertain parts. The measure is called
convolution: information quantity (IQ) [101]. Through the use
k+ n
of wavelet and subband entropy, we pinpoint and
1 2 i−k localize events in a time-frequency entropy space,
S (k, a)= √ s(i) gm ∗
a n a which offer temporary diversions from general
i=k− 2
entropic trends. To calculate the probability,
n

1 2
i pn (m) within each window W(n; w; ), we
=√ s (k − i) gm ∗ introduce intervals such that
a n a
i=− 2
M
(3.41) W (n; w; Δ) = ∪ Im (3.43)
m=1
with k = τ / Ts , where Ts is the sampling interval. Then, the probability pn (m) that the sampled
The summation is over a number of terms n. signal belongs to the interval Im is the ratio be-
tween the number of the signals found within matose in the immediate post-resuscitative period
interval Im and the total number of signals in [105]. Half of patients survive the hospitaliza-
W(n; w; ). Using pn (m), SE(n) is defined as tion, but less than half of those recover with-
out significant neurologic deficits [104]. Among

M
survivors, neurological complications represent
SE(n) = − pn (m)log2 pn (m) (3.44) the leading cause of disability [106, 107]. Major
m=1
advances have been made to improve the care
Based on the above arguments, we can define of these patients. The importance of early defib-
the information quantity (IQ). First, the DWT rillation has been clearly established, and high-
coefficients within each window are obtained as risk patients now frequently receive implantable
cardioverter defibrillators (ICDs). Additionally,
WC (r; n; w; Δ) = DWT [W (n; w; Δ)] (3.45) the general population has benefited from the in-
creasing public availability of automated external
To calculate pn wc (m) within each transformed defibrillators (AEDs). In short, early resuscitation
window WC(r; n; w; ), we define intervals Im wc and defibrillation have increased survival from
in W(n; w; ) such that cardiac arrest [108].
However, none of the advances in resuscitation
M practice have improved neurological functional
WC (r; n; w; Δ) = ∪ Im wc (3.46) outcome. In fact, an increase in the number of
m=1
severe neurological injury cases among cardiac
As withpn (m) in SE, the probability pn wc (m), arrest survivors has been noted [109]. Neurologi-
within each window WC(r; n; w; ), is calcu- cal injury remains the leading cause of morbidity
lated. Finally, IQ is defined as and disability among survivors [106, 107, 110–
113]. Thus, it is not surprising that the American

M
Heart Association noted in its 2000 Guidelines for
IQ(n) = − pn wc (m)log2 pn wc (m) (3.47) Cardiopulmonary Resuscitation and Emergency
m=1
Cardiovascular Care:
Thus, we can explore the IQ evolution of the Although the importance of Cardiopulmonary Re-
whole raw EEG signal, {s(i) : i = 1, . . . , N}. This suscitation (CPR) and Basic Life Support (BLS)
is done in the next section along with exploring is undisputed, the efficacy of CPR in prolonged
arrest is modest at best. When CPR and defibril-
ways of improving outcome after cardiac arrest. lation are delayed or when definitive care is not
closely followed, the Chain of Survival is broken.
The cerebral cortex, the tissue most susceptible
to hypoxia, is irreversibly damaged, resulting in
3.4 An Application of EEG: death or severe neurological damage. The need
Detecting Brain Injury After to preserve cerebral viability must be stressed in
Cardiac Arrest research endeavors and in practical interventions.
The term cardiopulmonary-cerebral resuscitation
has been used to further emphasize this need. [114]
Cardiac arrest (CA) is a major health problem
in both developed and developing countries. In Several large multi-institutional groups have
the United States alone, it claims over 1000 pro- been assembled to attack the problem of achiev-
ductive lives per day [102]. Worldwide there are ing meaningful survival from cardiac arrest.
over 130 million deaths due to cardiac arrest, and These include the NHLBI Post-resuscitative and
cardiac arrest remains the major cause of death initial Utility in Life Saving Efforts (PULSE)
in the United States [103]. In the United States Initiative and the Resuscitation Outcomes
of the initial 5–8% out-of-hospital CA survivors, Consortium (ROC) [102, 115] (https://roc.uwctc.
approximately 40,000 patients are admitted to an org/tiki/tiki-index.php) supported by the National
intensive care unit [104], where 80% remain co- Institutes of Health (NIH), National Heart,
Lung, and Blood Institute (NHLBI), National period of 2 hours, hypoxia was induced for a
Institute of Neurological Disorders and Stroke 30-minute period by ventilating the animals
(NINDS), and other related agencies. These with a gas mixture with an FiO2 of 0.1 (10%
efforts underscore the importance of neurological oxygen in nitrogen). Hypoxia was followed by
monitoring in cardiac arrest, enhancing the focus a 5-minute period of room air. The airway was
on improving outcomes, and highlighting the then occluded for 7 minutes to produce asphyxia.
importance of early interventions. At 7 minutes of asphyxia, CPR was provided
Recent clinical trials demonstrated that ther- by reinstitution of ventilation (100% oxygen)
apeutic hypothermia after CA can improve sur- and sternal chest compression performed at a
vival and functional outcomes compared to nor- rate of 100/min with a 50% duty cycle using a
mothermic controls [116–118]. As a result, the In- pneumatically driven thumper (Life Aid—Cardio
ternational Liaison Committee on Resuscitation Pulmonary Resuscitator, Model 1018, Michigan
made the recommendation to cool unconscious Instr., Grand Rapids, MI). EEG was monitored
patients resuscitated from out-of-hospital arrest continuously.
with an initial rhythm of ventricular fibrillation To generate the NS index, digitized data
to 32–34 ◦ C for 12–24 hours [119]. After adult were divided into overlapping segments that
cardiac arrest and infant asphyxic episodes, the were 3.3 seconds long and had an overlap of
loss of EEG and its particular pattern of recov- 2.55 seconds. Each one-minute record was
ery prove the need for advanced technologies preprocessed for noise reduction. First, the
to furnish information about the outcome of the mean of the segment (DC) was subtracted.
subject. With this diagnostic imperative or need Then a second-order polynomial was fitted to
comes the requirement for proving that a partic- the remainder and consequentially subtracted.
ular technology or signal processing strategy can Finally, the data was bandpass filtered with a sixth
capture all of nuances of the EEG signal. There order Butterworth filter with cutoff frequencies
are a variety of EEG phenomena. Tracing the evo- of 0.24 and 26 Hz.
lution of the EEG starts with preclinical animal This led us to the following question: does an
models. Traditionally, animal experiments pro- altered EEG immediately after injury as measured
vide the input to the critically important bench-to- by high neurological deficit score, NS, predict
bedside pipeline. Experiments done on the bench poor neurological outcome at 24 hours after in-
are highly controlled and tightly structured so that jury? To answer this question, a standardized
dependent variables are not influenced unduly by neurological examination was developed to de-
extraneous or external variables. In the following termine neurological outcome. The examination
section, we describe our investigation to exper- included assessment of consciousness, brainstem
imental and preclinical work with animal mod- function, behavior (motoric, orientation, and ac-
els. Armed with the information gleaned from tivity), and incidence of seizure. From these mea-
the bench and animal experiments, we transition surements, a neurological deficit score (NDS)
these efforts to the “real” or clinical world. was tabulated, which created an index of the
animal’s outcome against which the EEG results
could be compared. The neurodeficit score and its
3.4.1 Experimental Methods for component subscores are shown in [120].
Hypoxic-Asphyxic Cardiac The NS shows the progression of recovery of
Arrest and the Use the EEG. We found that NS holds a strong corre-
of Normalized Separation lation with the neurological deficit score. Figure
3.10 confirms that an animal with a high NS
One-week-old piglets were anesthetized with (greater spectral dispersion) also has a low neu-
sodium pentobarbital. The trachea was intubated rological deficit score (poor neurological func-
and lungs were ventilated to maintain normal tion), and vice versa. We see that in the first
blood gases. After a post-surgery stabilization case, all three frequencies recover in parallel;
Fig. 3.10 (a) Balanced A 1.2

recovery in the dominant
frequency, NS and 1.0
outcome recovery of the
relative power in the three 0.8
Relative Power
dominant frequency bands
for two animals. : 0.6
1–5.5 Hz, : 9–14 Hz,
and : 18–21 Hz. Left: 0.4
A uniform spectral
recovery resulting in a low 0.2
NS confirmed by a high
NDS (good outcome). (b) 0.0
Spectral recovery for an
15 45 80 120 160 200 240
animal with a high NS
indicating spectral Time since ROSC (in min)
dispersion or unequal
recovery of different Power-low freq. Power-mid freq. Power-high freq.
frequency bands. The
neurological deficit score
B 1.2
(NDS) of this animal is
low, indicating a bad
outcome. (Reproduced 1.0
with permission from [72])
0.8
Relative Power
0.6
0.4
0.2
0.0
15 45 80 120 160 200 240
Time since ROSC (in min)
Power-low freq. Power-mid freq. Power-high freq.
this constitutes a balanced recovery and a good termined by the consciousness and behavioral
overall outcome. In the second case, the recovery neurological deficit score (NDS) at a later time
is unbalanced and leads to a poor outcome. (24 hours). The specific time epochs identified in
The recovery of EEG power for all the animals Fig. 3.11 suggest that the EEG recovery spectral
in the two groups of low and high (bad and measure, NS, could be used to differentiate the
good outcome at 24 hours, respectively) NDS- animals into two subgroups, with good and bad
behavioral subscore groups is shown in Fig. 3.11. outcomes.
Univariate t-tests place significant epochs at 45,
60, 80, and 220 min (p < 0.01, p < 0.005, p < 0.02,
and p < 0.05, respectively). There were significant 3.4.2 Detecting and Counting
differences between the high and the low behav- Bursts
ioral subscore groups across all of the epochs
(p < 0.01) and across all subjects (p < 0.0001). Another very significant component of our in-
Thus, this study shows a high statistical corre- vestigation was the observation throughout our
lation between the initial quantitative EEG mea- experiments that the EEG recovery was punctu-
sure, NS, and the outcome of the animal as de- ated by periods of electrical silence and bursts in
Fig. 3.11 NS for groups 1.4

having low and high
1.2
Normalized Separation
neurological deficit scores **
** ***
in the behavioral subscore 1.0
category. See text for *
details on the results from 0.8
statistical tests. Results of 0.6
univariate tests of
significance are indicated: 0.4
∗ , p < 0.05; ∗∗ , p < 0.02; 0.2
∗∗∗ , p < 0.005; ∗∗∗∗ ,
p < 0.0001. (Reproduced 0.0
with permission from [72]) 15 45 80 120 160 200 240
Time since ROSC (in min)
High Behavioral Subscore Low Behavioral Subscore
energy, commonly known as burst suppression or juncture, we have an underlying, unifying theme
simply bursting [65]. Initial experiments focused of bursting in the EEG after cardiac arrest and
on the prognostic value of this bursting and devel- wish to quantitate the bursting phenomenon as
opment of a computational burst counting algo- an indicator of injury and recovery. The vari-
rithm. This development led to the employment ous measures that we use, such as burst counts,
of the Teager energy operator (TEO), which we have a common umbrella or consistent interpre-
consequent evaluated versus a clinical “gold stan- tive framework that focuses on the volatility or
dard” and a more conventional energy operator how unpredictable the EEG signal is. Each and
[61]. every measure that we have been using can be
After confirming the prognostic value of burst- reinterpreted within the basis of characteristic en-
ing, our next step was to develop a burst detection tropy. In this fashion, both temporal and spectral
device. A burst is an abrupt change in energy, indicators have generated a unique tapestry when
which can be detected by a clear, sharp rapid understood as instances of altered entropy dis-
increase in energy at its beginning. We currently plays by the brain after injury and recovery states.
employ the Teager energy operator (TEO) to de- Examining the bursting phenomenon, we can see
tect burst activity in the EEG. The TEO is a low that accompanying highly periodic bursting, en-
distortion method of finding energy of signals. tropy is lowered. Bursting that occurs at ran-
The TEO provides low interference as compared dom is less certain and entropy increases. Spiky
to traditional methods of finding energy such bursts display widely varying amplitude levels
as the square law detector (SLD). The Teager transitioning rapidly from baseline to the spike’s
energy operator removes much of second-order peak. In this case, residual entropy is higher.
harmonic distortion that the SLD leaves behind Bursts that resemble multilevel continuous EEG
[77, 121, 122]. display an amplitude diversity that increases en-
tropy formally. Examples of bursting are shown
in Fig. 3.12. All of our spectral and temporal
3.4.3 EEG and Entropy: A Novel evidence accumulated thus far points to entropy
Approach to Brain Injury as a unifying concept that incorporates all of
Monitoring the seemingly diverse elements of EEG parox-
ysms under the same umbrella. Large increases
Our past efforts have rewarded us with powerful in entropy deviating from monotonous and mori-
measures of assessing the EEG during recovery bund, immediate post-ictal EEG are evident in
from cardiac arrest. We realized that what we a healthy resumption of normal EEG. Through
need now is a better-all-encompassing vision of the use of wavelet, multiscale, and subband en-
the evolving EEG prior to burst fusion. At this tropy, we will be able to pinpoint and localize
Fig. 3.12 Recovering (a)

EEG after 5-min asphyxia 100
causing cardiac arrest in
microvots
50
experimental animals. This
0
is an example of EEG with
an animal with poor –50
outcome recovery. Thirty –100
minutes after CPR and 0 5 10 15
return of spontaneous
circulation (ROSC), EEG (b)
exhibits asynchronous 100
bursting in panels a and b. 50
microvots
This is shown with low 0
amplitude bursting. One
hour later, the animal still –50
exhibits bursting with –100
0 5 10 15
higher amplitude bursts at
Time(sec)
90 min after ROSC
(c)
100
microvots
50
0
–50
–100
0 5 10 15
(d)
100
microvots
50
0
–50
–100
0 5 10 15
Time(sec)
events in a time-frequency entropy space, which component of EEG results in a higher level of
offer temporary diversions from general entropic statistical uncertainty and as a result higher en-
trends [101, 123–125]. tropy. On the other hand, bursting that occurs after
We have an underlying, unifying framework postischemic recovery is less random and more
for EEG evolution after cardiac arrest that focuses predictable, and entropy increases. Bursts that
on the volatility or how unpredictable the EEG resemble multilevel continuous EEG display an
signal is. This unpredictability can be reinter- amplitude diversity that increases entropy. Large
preted within the basis of entropy. The approach increases in entropy deviating from monotonous
to qEEG analysis is based on the hypothesis that and moribund, immediate post-ictal EEG are ev-
brain injury results in a reduction in the informa- ident in a healthy resumption of normal EEG.
tion content of the brain rhythm. From the per- Figure 3.13 shows EEG evolution before and
spective of the information theory, the amount of after CA brain injury and quantitative analysis us-
information can be quantified by calculating the ing IQ as well as several example EEG segments
entropy [99]. As a preliminary study, we used the from different periods several minutes after return
information quantity (IQ) measure to study EEG of spontaneous circulation (ROSC).
during the recovery of brain function from CA Figure 3.14 shows the nature of the EEG signal
[101, 127]. Complex, random, and unpredictable (insets) and the qEEG trends, as measured by
Baseline 7th min
Information Quantity (IQ)

1
After injury
0.8 5 min. injury
After injury 37th min 0.6
0.4
100 mV
127th min 0.2 7 min. injury
0
0 50 100 150
A B Time (minutes)
Fig. 3.13 (a) EEG segments extracted from the 4-hr con- relative severity of the CA injury and consequent effects
tinuous recording of an ischemic insult experiment with 5- on EEG. This establishes our model and method for qEEG
min asphyxia. (b) The qEEG analysis is shown calculating analysis to characterize graded levels of injury. (Repro-
information quantity (IQ) in terms of entropy. Rats receive duced with permission from [126])
5 and 7 min of CA insult. IQ clearly distinguishes the
Neuroloaic Deficit Score (NDS): 46 Neuroloaic Deficit Score (NDS): 74

IQ IQ
Information Quantity (IO)
36 36 Isoelectric
34 31 phase
2 Fast
5 increase
14
0.50
17
0.22 Slow
increase
0.067 0.064
0 50 100 150 50 100
Time (min) Time (min)
Fig. 3.14 IQ comparisons of good (NDS: 74) and poor namely, isoelectric phase just after cardiac arrest, fast
(NDS: 46) outcome animals. The small figure inside each increase phase, and slow increase phase. (Reproduced
plot is compressed EEG. We quantify IQ evolution from with permission from [126])
various perspectives, mainly in three different phases,
the IQ levels, for two subjects: one has a poorer 3.4.4 Enhancing Recovery
outcome (NDS of 46), and another has a better from Cardiac Arrest: The Use
outcome (NDS of 74). What we discovered is of Orexin
that the recovery patterns are quite distinctive,
with periods of isoelectricity, fast progression, Orexin is a hypothalamic neuropeptide that en-
and slow progression. In addition, in the poor hances arousal via scattered synapses in thala-
outcome case, there is a period of spiking and mus, cortex, and ascending brainstem networks.
bursting, while in the good outcome case, there Orexin deficiency has been shown to be present
is a rapid progression to a fused, more continuous after global ischemia, and orexin receptors are
EEG. The entropy-based analysis, and the derived upregulated in ischemic brain [128, 129]. In past
measure IQ, captures the trends in EEG evolu- research, intraventricular injection of orexin-A in
tion after brain injury. Thus, these derived signal rats with anesthetic-induced burst suppression re-
measures can serve as monitoring tools as well sulted in rapid desynchronization of EEG activity
as hold the potential for prognosticating outcome and interruption of burst suppression [130–132].
after brain injury. Our initial studies using orexin used an intraven-
Fig. 3.15 Strong IQ measures through 90 min in the depicts time. Using a General Linear Model to analyze
orexin-treated animals. Mean normalized IQ values with for repeated measures, IQ was significantly higher in the
standard deviation bars in orexin-A and control groups orexin-A group compared to controls (p = 0.008) during
at baseline, during CA, and at serial intervals after return the first 120 minutes. IQ values subsequently converged in
of spontaneous circulation (ROSC). The y-axis depicts IQ the two groups. (Reproduced with permission from [133])
values normalized to baseline (range 0–1), and the x-axis
tricular injection of orexin to enable enhancement imal’s nostrils. At 30 minutes post-ROSC, the rats
or in improved recovery focused on checking were randomized to receive saline (vehicle), low
the NDS and IQ measurement of the recovering (10 μM) ORXA, or high dose (50 μM) of ORXA
EEG. intranasally. Each animal received 10 μl × 3 in
The orexin-treated animals also exhibited a each nostril for a 30-second interval (60 μl total).
higher IQ levels as Fig. 3.15 illustrates. Using re- Noteworthy we can show a statistical improve-
peated measures ANOVA on baseline-normalized ment in NDS with the high-dose ORXA group.
IQ levels, we find that IQ levels in the treated Fig. 3.15 shows that there was an average increase
group are higher than the controls for the first in NDS due to ORXA (p ≤ 0.025) with high dose
2 hours after recovery begins for these animals. in comparison to both the low-dose and saline
Later, the scores are virtually the same statisti- (control) cases.
cally. The efficacy of using orexin to help boost The primary EEG effect of the ORXA dosing
recovery in the near term after cardiac arrest is is the increase in power of the gamma rhythm
seen in the animals’ recovery. (30–50 Hz). We chart the gamma increase after
We also examined a novel route for orexin drug administration in Fig. 3.16. We describe the
administration. Intranasal dosing would allow for increase with the use of the gamma fraction which
medical intervention in the unconscious patient is the proportion of gamma power in the total
who has had a cardiac arrest and allow for ad- EEG power. After drug administration, the high-
ministration without the need for interventional dose ORXA case shows a statistically significant
surgery. In the next set of experiments, orexin difference in gamma fraction when compared to
was administered through an atomizer in each an- the saline controls.
Fig. 3.16 A distinct increase in NDS with the high-dose (control) and low-dose ORXA NDS averages. (p ≤ 0.025)
ORXA group which averages above 55. The EEG gamma as the asterisk and pound sign flags indicate. Furthermore,
fraction is plotted for the three phases: (a) baseline, (b) as we compare gamma fraction levels, we see that there
ROSC, and (c) application of the drug. The effect of the is a significant difference after drug administration which
drug is noteworthy as it exceeds averages for the saline occurs 35 min after ROSC. (Reproduced with permission
from [92])
processing methods reviewed here, phenomena

3.5 Conclusion such as bursting and burst suppression, postis-
chemic recovery, and correlation with neurolog-
EEG signals offer a window into brain’s electrical ical deficits and outcomes were established. The
activity. Their clinical utilization is facilitated by key goal is to adopt a set of standards for moni-
the development of a standard for EEG electrodes toring and clinical use of the EEG. The recently
and availability of high-quality clinical instru- held ASET Workshop on Neurodiagnostics that
mentation. However, the signals have complex was held in 2018 [134] indicates that questions
presentation that does not make it easy to read will continue to be asked about standards of these
and interpret the signals. Therefore, signal pro- EEG automated systems. Slowly the community
cessing methods have been developed to facilitate is beginning to adopt system-wide criteria for
their interpretation and analysis in a quantitative testing and evaluation dealing with the clinical
manner. The simplest approach is to analyze and application of these systems. The future should
interpret the signals in time domain, looking for be interesting indeed for automated EEG.
features such as seizure spikes or bursts and burst
suppression events. The complexity and the non-
stationarity of EEG signals benefit from paramet-
ric and nonparametric modeling methods, partic- Homework
ularly in research. A range of methods, from well-
known EEG frequency bands to more advanced 1. What are the chief uses of the EEG? Di-
nonparametric methods such as wavelets, and vide these into clinical/diagnostic as well as
parametric modeling and entropy or information purely functional or cognitive and behav-
analysis methods, are used to analyze and inter- ioral.
pret the trends in EEG in many experimental and 2. What are the different bands of the EEG
clinical situations. Thus, EEG signal analysis and and what are their primary uses? Why is the
monitoring has found wide acceptance in research power spectrum such an important tool for
and in clinical studies. This chapter illustrated the discovering the state of the EEG at any one
application of EEG signal processing methods by time?
applying these to the recordings from animal and 3. Why is the EEG so effective for detecting
clinical models of global ischemic brain injury epileptic seizures in living beings? What is
after cardiac arrest. With the help of the signal the primary characteristic of the seizure sig-
nal that we measure in different anatomically neighbor classification. Technol. Health Care 26,
connected areas of the brain? 509–519 (2018)
6. W. Yi, S. Qiu, K. Wang, H. Qi, F. He, P. Zhou,
4. What are the differences between parametric
L. Zhang, D. Ming, EEG oscillatory patterns and
(model based, e.g., autoregressive) and non- classification of sequential compound limb motor
parametric (e.g., FFT-based) spectral meth- imagery. J. Neuroeng. Rehabil. 13, 11 (2016)
ods? 7. D. Trubutschek, S. Marti, H. Ueberschar, S. De-
haene, Probing the limits of activity-silent non-
5. What are the chief characteristics of the MU-
conscious working memory. Proc. Natl. Acad. Sci.
SIC method of spectral analysis? U. S. A. 116, 14358–14367 (2019)
6. Define the normalized separation of the 8. D.R. Kramer, M.F. Barbaro, M. Lee, T. Peng, G.
EEG? What spectral method is used to Nune, C.Y. Liu, S. Kellis, B. Lee, Electrocortico-
graphic changes in field potentials following natural
calculate it? How is it calculated? What is
somatosensory percepts in humans. Exp. Brain Res.
the optimum normalized separation? 237, 1155–1167 (2019)
7. Define IQ or the information quantity? How 9. M.E.M. Mashat, C.T. Lin, D. Zhang, Effects of task
does IQ reflect the total entropy in the EEG? complexity on motor imagery-based brain-computer
Interface. IEEE Trans. Neural Syst. Rehabil. Eng.
How does IQ magnitude prognosticate out-
27, 2178–2185 (2019)
come after cardiac arrest in rats? 10. N.R. Wilson, D. Sarma, J.D. Wander, K.E. Weaver,
8. What are uses of the measurement index that J.G. Ojemann, R.P.N. Rao, Cortical topography
we call wavelets? Is there any advantage that of error-related high-frequency potentials during
erroneous control in a continuous control brain-
we glean from using wavelets from indepen-
computer Interface. Front. Neurosci. 13, 502
dent sinusoidal signals? (2019)
9. Why is it so important to monitor the brain 11. B.J. Fisch, Fisch & Spehlmann’s EEG Primer: Basic
after cardiac arrest occurs? Does the EEG Principles of Digital and Analog EEG (Elsevier,
Amsterdam, 1999)
offer any benefits for monitoring the brain
12. A.J. Rowan, E. Tolunsky, Primer of EEG
after cardiac arrest? (Butterworth-Heinemann, Philadelphia, 2002)
10. What are primary effects of the stimulant 13. O. Bennett-Back, S. Uliel-Siboni, U. Kramer, The
and neuropeptide, orexin, on the EEG? What yield of video-EEG telemetry evaluation for non-
surgical candidate children. Eur. J. Paediatr. Neurol.
measures have been utilized to categorize the
20, 848–854 (2016)
EEG after orexin treatment? 14. D.K. Chen, S. Majmudar, A. Ram, H.C. Rutherford,
M. Fadipe, C.B. Dunn, R.L. Collins, Change in ill-
ness perception is associated with short-term seizure
References burden outcome following video-EEG confirmation
of psychogenic nonepileptic seizures. Epilepsy Be-
hav. 83, 186–191 (2018)
1. L. Maillard, G. Ramantani, New recommenda-
15. H. Chen, M.Z. Koubeissi, Electroencephalography
tions of the IFCN: From scalp EEG to electrical
in epilepsy evaluation. Continuum (Minneap Minn)
brain imaging. Clin. Neurophysiol. 128, 2068–2069
25, 431–453 (2019)
(2017)
16. R.S. Fisher, H.E. Scharfman, M. deCurtis, How can
2. M. Seeck, L. Koessler, T. Bast, F. Leijten, C. Michel,
we identify ictal and interictal abnormal activity?
C. Baumgartner, B. He, S. Beniczky, The standard-
Adv. Exp. Med. Biol. 813, 3–23 (2014)
ized EEG electrode array of the IFCN. Clin. Neuro-
17. C. Liu, R. Zhang, G. Zhang, T. Yu, J. Tai, W. Du,
physiol. 128, 2070–2077 (2017)
L. Li, Y. Wang, High frequency oscillations for
3. T. Cerrahoglu Sirin, P. Bekdik Sirinocak, B.N.
lateralizing suspected bitemporal epilepsy. Epilepsy
Arkali, T. Akinci, S.N. Yeni, Electroencephalo-
Res. 127, 233–240 (2016)
graphic features associated with intermittent rhyth-
18. S. Rose, J.S. Ebersole, Advances in spike localiza-
mic delta activity. Neurophysiol. Clin. 49, 227–234
tion with EEG dipole modeling. Clin. EEG Neu-
(2019)
rosci. 40, 281–287 (2009)
4. T. Lees, T. Chalmers, D. Burton, E. Zilberg, T.
19. E.H. Smith, C.A. Schevon, Toward a mechanistic
Penzel, S. Lal, S. Lal, Electroencephalography as
understanding of epileptic networks. Curr. Neurol.
a predictor of self-report fatigue/sleepiness during
Neurosci. Rep. 16, 97 (2016)
monotonous driving in train drivers. Physiol. Meas.
20. D. Sherman, N. Zhang, S. Garg, N.V. Thakor, M.A.
39, 105012 (2018)
Mirski, M.A. White, M.J. Hinich, Detection of non-
5. M. Li, H. Xu, X. Liu, S. Lu, Emotion recogni-
linear interactions of EEG alpha waves in the brain
tion from multichannel EEG signals using K-nearest
by a new coherence measure and its application
to epilepsy and anti-epileptic drug therapy. Int. J. 33. C. Guger, R. Spataro, F. Pellas, B.Z. Allison, A.
Neural Syst. 21, 115–126 (2011) Heilinger, R. Ortner, W. Cho, R. Xu, V. La Bella,
21. A.D. Bhimani, A.N. Selner, D.R. Esfahani, R.G. G. Edlinger, J. Annen, G. Mandala, C. Chatelle, S.
Chiu, C.L. Rosinski, D. Rosenberg, A. Mudreac, Laureys, Assessing command-following and com-
R.J. Diamond, Z. Almadidy, A.I. Mehta, Intracranial munication with vibro-tactile P300 brain-computer
electrode placement for seizures before temporal Interface tools in patients with unresponsive wake-
lobectomy: A risk-benefit analysis. World Neuro- fulness syndrome. Front. Neurosci. 12, 423 (2018)
surg. 121, e215–e222 (2019) 34. S. Marchesotti, M. Bassolino, A. Serino, H. Bleuler,
22. P. Sharma, M. Scherg, L.H. Pinborg, M. Fabri- O. Blanke, Quantifying the role of motor imagery in
cius, G. Rubboli, B. Pedersen, A.M. Leffers, P. Ul- brain-machine interfaces. Sci. Rep. 6, 24076 (2016)
dall, B. Jespersen, J. Brennum, O.M. Henriksen, S. 35. B.P. Shortal, L.B. Hickman, R.A. Mak-McCully, W.
Beniczky, Ictal and interictal electric source imaging Wang, C. Brennan, H. Ung, B. Litt, V. Tarnal, E.
in pre-surgical evaluation: A prospective study. Eur. Janke, P. Picton, S. Blain-Moraes, H.R. Maybrier,
J. Neurol. 25, 1154–1160 (2018) M.R. Muench, N. Lin, M.S. Avidan, G.A. Mashour,
23. P. Nemtsas, G. Birot, F. Pittau, C.M. Michel, K. A.R. McKinstry-Wu, M.B. Kelz, B.J. Palanca, A.
Schaller, S. Vulliemoz, V.K. Kimiskidis, M. Seeck, Proekt, C.S.G. Re, Duration of EEG suppression
Source localization of ictal epileptic activity based does not predict recovery time or degree of cogni-
on high-density scalp EEG data. Epilepsia 58, 1027– tive impairment after general anaesthesia in human
1036 (2017) volunteers. Br. J. Anaesth. 123, 206–218 (2019)
24. J. Zhang, W. Liu, H. Chen, H. Xia, Z. Zhou, S. Mei, 36. D.A. Turner, Enhanced functional outcome from
Q. Liu, Y. Li, Multimodal neuroimaging in presurgi- traumatic brain injury with brain-machine interface
cal evaluation of drug-resistant epilepsy. Neuroim- neuromodulation: Neuroprosthetic scaling in rela-
age Clin. 4, 35–44 (2014) tion to injury severity, in Translational Research in
25. L. Martinkovic, H. Hecimovic, V. Sulc, R. Marecek, Traumatic Brain Injury, ed. by D. Laskowitz, G.
P. Marusic, Modern techniques of epileptic focus lo- Grant, (CRC Press, Boca Raton, 2016)
calization. Int. Rev. Neurobiol. 114, 245–278 (2014) 37. C. Wang, M.E. Costanzo, P.E. Rapp, D. Darmon,
26. A. Aarabi, B. He, Seizure prediction in patients D.E. Nathan, K. Bashirelahi, D.L. Pham, M.J. Roy,
with focal hippocampal epilepsy. Clin. Neurophys- D.O. Keyser, Disrupted gamma synchrony after mild
iol. 128, 1299–1307 (2017) traumatic brain injury and its correlation with white
27. A. Aarabi, B. He, Seizure prediction in hippocampal matter abnormality. Front. Neurol. 8, 571 (2017)
and neocortical epilepsy using a model-based ap- 38. D. Akhmetshina, A. Nasretdinov, A. Zakharov, G.
proach. Clin. Neurophysiol. 125, 930–940 (2014) Valeeva, R. Khazipov, The nature of the sensory
28. B.S. Chang, D.L. Schomer, E. Niedermeyer, Nor- input to the neonatal rat barrel cortex. J. Neurosci.
mal EEG and sleep: Adults and elderly, in Nieder- 36, 9922–9932 (2016)
meyer’s Electroencephalography: Basic Principles, 39. S. van der Lely, M. Stefanovic, M.R. Schmidhalter,
Clinical Applications, and Related Fields, ed. by M. Pittavino, R. Furrer, M.D. Liechti, M. Schubert,
D. L. Schomer, F. L. da SIlva, (Wolters Kluwer T.M. Kessler, U. Mehnert, Protocol for a prospec-
Health/Lippincott Williams & Wilkins, Philadel- tive, randomized study on neurophysiological as-
phia, 2011) sessment of lower urinary tract function in a healthy
29. J.J. Riviello Jr., D.R. Nordli Jr., E. Nieder- cohort. BMC Urol. 16, 69 (2016)
meyer, Normal EEG and sleep: Infants to adoles- 40. R. Arya, C. Roth, J.L. Leach, D. Middeler, J.A.
cents, in Niedermeyer’s Electroencephalography: Wilson, J. Vannest, L. Rozhkov, H.M. Greiner, J.
Basic Principles, Clinical Applications, and Re- Buroker, C. Scholle, H. Fujiwara, P.S. Horn, D.F.
lated Fields, ed. by D. Schomer, F. L. da SIlva, Rose, N.E. Crone, F.T. Mangano, A.W. Byars, K.D.
(Wolters Kluwer Health/Lippincott Williams & Holland, Neuropsychological outcomes after resec-
Wilkins, Philadephia, 2011) tion of cortical sites with visual naming associ-
30. K. Casimo, K.E. Weaver, J. Wander, J.G. Ojemann, ated electrocorticographic high-gamma modulation.
BCI use and its relation to adaptation in cortical Epilepsy Res. 151, 17–23 (2019)
networks. IEEE Trans. Neural Syst. Rehabil. Eng. 41. R. Arya, J.A. Wilson, H. Fujiwara, J. Vannest, A.W.
25, 1697–1704 (2017) Byars, L. Rozhkov, J.L. Leach, H.M. Greiner, J.
31. M.J. Khan, U. Ghafoor, K.S. Hong, Early detection Buroker, C. Scholle, P.S. Horn, N.E. Crone, D.F.
of hemodynamic responses using EEG: A hybrid Rose, F.T. Mangano, K.D. Holland, Electrocortico-
EEG-fNIRS study. Front. Hum. Neurosci. 12, 479 graphic high-gamma modulation with passive listen-
(2018) ing paradigm for pediatric extraoperative language
32. C. Guger, R. Spataro, B.Z. Allison, A. Heilinger, R. mapping. Epilepsia 59, 792–801 (2018)
Ortner, W. Cho, V. La Bella, Complete locked-in and 42. N. Braun, S. Debener, A. Solle, C. Kranczioch, H.
locked-in patients: Command following assessment Hildebrandt, Biofeedback-based self-alert training
and communication with vibro-tactile P300 and mo- reduces alpha activity and stabilizes accuracy in the
tor imagery brain-computer interface tools. Front. sustained attention to response task. J. Clin. Exp.
Neurosci. 11, 251 (2017) Neuropsychol. 37, 16–26 (2015)
43. I. Dziembowska, P. Izdebski, A. Rasmus, J. Brudny, 55. J.C. Ehlen, F. Jefferson, A.J. Brager, M. Benveniste,
M. Grzelczak, P. Cysewski, Effects of heart rate K.N. Paul, Period-amplitude analysis reveals wake-
variability biofeedback on EEG alpha asymmetry dependent changes in the electroencephalogram dur-
and anxiety symptoms in male athletes: A pilot ing sleep deprivation. Sleep 36, 1723–1735 (2013)
study. Appl. Psychophysiol. Biofeedback 41, 141– 56. L.L. Gustafsson, W.F. Ebling, E. Osaki, D.R. Stan-
150 (2016) ski, Quantitation of depth of thiopental anesthesia in
44. L. Kranaster, C. Janke, C. Hoyer, A. Sartorius, the rat. Anesthesiology 84, 415–427 (1996)
Management of severe postictal agitation after elec- 57. M. Jospin, P. Caminal, E.W. Jensen, H. Litvan,
troconvulsive therapy with bispectrum electroen- M. Vallverdu, M.M. Struys, H.E. Vereecke, D.T.
cephalogram index monitoring: A case report. J. Kaplan, Detrended fluctuation analysis of EEG as
ECT 28, e9–e10 (2012) a measure of depth of anesthesia. I.E.E.E. Trans.
45. E.A. Mukamel, K.F. Wong, M.J. Prerau, E.N. Biomed. Eng. 54, 840–846 (2007)
Brown, P.L. Purdon, Phase-based measures of cross- 58. A. Shalbaf, M. Saffar, J.W. Sleigh, R. Shalbaf, Mon-
frequency coupling in brain electrical dynamics un- itoring the depth of anesthesia using a new adaptive
der general anesthesia. Conf. Proc. IEEE Eng. Med. Neurofuzzy system. IEEE J. Biomed. Health Inform.
Biol. Soc. 2011, 1981–1984 (2011) 22, 671–677 (2018)
46. W. Xie, B.M. Mallin, J.E. Richards, Development 59. J.S. Paul, C.B. Patel, H. Al-Nashash, N. Zhang,
of brain functional connectivity and its relation to W.C. Ziai, M.A. Mirski, D.L. Sherman, Prediction of
infant sustained attention in the first year of life. Dev. PTZ-induced seizures using wavelet-based residual
Sci. 22, e12703 (2019) entropy of cortical and subcortical field potentials.
47. C.S. Ouyang, C.T. Chiang, R.C. Yang, R.C. Wu, I.E.E.E. Trans. Biomed. Eng. 50, 640–648 (2003)
H.C. Wu, L.C. Lin, Quantitative EEG findings and 60. P. Maragos, J.F. Kaiser, T.F. Quatieri, Energy sep-
response to treatment with antiepileptic medications aration in signal modulations with application to
in children with epilepsy. Brain Dev. 40, 26–35 speech analysis. IEEE Trans. Signal Process. 41,
(2018) 3024–3051 (1993)
48. S.L. Massey, H. Shou, R. Clancy, M. DiGiovine, 61. J.F. Kaiser, On a Simple Algorithm to Calculate the
M.P. Fitzgerald, F.W. Fung, J. Farrar, N.S. Abend, ‘Energy’ of a Signal (IEEE, 1990)
Interrater and Intrarater agreement in neonatal elec- 62. P. Maragos, A. Potamianos, Higher order differential
troencephalogram background scoring. J. Clin. Neu- energy operators. IEEE Signal Process. Lett. 2, 152–
rophysiol. 36, 1–8 (2019) 154 (1995)
49. F. Pisani, E. Pavlidis, The role of electroencephalo- 63. J. Fang, L. Atlas, Quadratic detectors for energy
gram in neonatal seizure detection. Expert. Rev. estimation. IEEE Trans. Signal Process. 43, 2582–
Neurother. 18, 95–100 (2018) 2594 (1995)
50. J.P. Fuentes, S. Villafaina, D. Collado-Mateo, R. 64. C.L. Nikias, A.P. Petropulu, Higher Order Spectral
de la Vega, N. Gusi, V.J. Clemente-Suarez, Use Analysis: A Nonlinear Signal Processing Frame-
of biotechnological devices in the quantification of work (Prentice Hall, Englewood Cliffs, 1993)
psychophysiological workload of professional chess 65. E. Niedermeyer, D. Sherman, R. Geocadin, The
players. J. Med. Syst. 42, 40 (2018) burst suppression electroencephalogram. Clin. Elec-
51. I.A. Akbar, A.M. Rumagit, M. Utsunomiya, T. troencephalogr. 30, 99–105 (1999)
Morie, T. Igasaki, Three drowsiness categories as- 66. G. Chatrian, L. Bergamini, M. Dondey, D. Klass, M.
sessment by electroencephalogram in driving sim- Lennox-Butchthal, I. Petersen, A glossary of terms
ulator environment. Conf. Proc. IEEE Eng. Med. most commonly used by clinical electroencephalo-
Biol. Soc. 2017, 2904–2907 (2017) graphers. Electroencephalogr. Clin. Neurophysiol.
52. L.J. Herrera, C.M. Fernandes, A.M. Mora, D. Mig- 37, 538–548 (1974)
otina, R. Largo, A. Guillen, A.C. Rosa, Combination 67. J. Fuzik, L. Gellert, G. Olah, J. Heredi, K. Kocsis, L.
of heterogeneous EEG feature extraction methods Knapp, D. Nagy, Z.T. Kincses, Z. Kis, T. Farkas, J.
and stacked sequential learning for sleep stage clas- Toldi, Fundamental interstrain differences in cortical
sification. Int. J. Neural Syst. 23, 1350012 (2013) activity between Wistar and Sprague-Dawley rats
53. T. Limpiti, B.D. Van Veen, H.T. Attias, S.S. Na- during global ischemia. Neuroscience 228, 371–381
garajan, A spatiotemporal framework for estimat- (2013)
ing trial-to-trial amplitude variation in event-related 68. Z. Liang, Y. Wang, Y. Ren, D. Li, L. Voss, J. Sleigh,
MEG/EEG. I.E.E.E. Trans. Biomed. Eng. 56, 633– X. Li, Detection of burst suppression patterns in
645 (2009) EEG using recurrence rate. ScientificWorldJournal
54. D.T. Plante, M.R. Goldstein, J.D. Cook, R. Smith, 2014, 295070 (2014)
B.A. Riedner, M.E. Rumble, L. Jelenchick, A. Roth, 69. M. Sarkela, S. Mustola, T. Seppanen, M. Koskinen,
G. Tononi, R.M. Benca, M.J. Peterson, Effects of P. Lepola, K. Suominen, T. Juvonen, H. Tolvanen-
partial sleep deprivation on slow waves during non- Laakso, V. Jantti, Automatic analysis and monitor-
rapid eye movement sleep: A high density EEG ing of burst suppression in anesthesia. J. Clin. Monit.
investigation. Clin. Neurophysiol. 127, 1436–1444 Comput. 17, 125–134 (2002)
(2016)
70. F.L. da Silva, EEG analysis: Theory and practice, in 85. F. Hlawatsch, G.F. Boudreaux-Bartels, Linear and
Electroencephalography: Basic Principles, Clinical quadratic time-frequency signal representations.
Applications and Related Fields, ed. by E. Nieder- IEEE Signal Proc. Mag. 9, 21–67 (1992)
meyer, F. L. da Silva, (Williams & Wilkins, Balti- 86. O. Meste, H. Rix, P. Jane, P. Caminal, N.V. Thakor,
more, 2011) Detection of late potentials by means of wavelet
71. S.L. Marple, Digital Spectral Analysis, 2nd edn. transform. I.E.E.E. Trans. Biomed. Eng. 41, 625–
(Dover Publications, Inc., Mineola, 2019) 634 (1994)
72. D.L. Sherman, M.K. Atit, R.G. Geocadin, S. 87. O. Rioul, M. Vetterli, Wavelet theory: Mapping sig-
Venkatesha, D.F. Hanley, A.L. Natarajan, N.V. nal to a time-scale plane. IEEE Signal Process. Mag.
Thakor, Diagnostic instrumentation for neural in- 8, 14–39 (1991)
jury. IEEE Instrum. Meas. 5, 28–35 (2002) 88. M. Holschneider, Wavelets: An Analysis Tool
73. J.P. Burg, Maximum entropy spectral analysis, 37th (Clarendon Press, Oxford, 1995)
meeting of the Society of Exploration Geophysi- 89. M. Holschneider, R. Kronland-Martinet, P.
cists, 1967 Tchamitchian, A real-time algorithm for signal
74. V. Goel, A Novel Technique for EEG Analysis: analysis with the help of the wavelet transform,
Application to Neonatal Hypoxia-Asphyxia, BME in Wavelets: Time-Frequency Methods and Phase
(Johns Hopkins University, Baltimore, 1995) Space, ed. by J. Combes, A. Grossmann, P.
75. S.M. Kay, Modern Spectral Estimation: Theory and Tchamitchian, (Springer, New York, 1989)
Application (Prentice Hall, Englewood Cliffs, 1988) 90. A.H. Najmi, J. Sadowsky, The continuous wavelet
76. V. Goel, A.M. Brambrink, A. Baykal, D.F. Hanley, transform and variable resolution time-frequency
N.V. Thakor, Dominant frequency analysis of EEG analysis. Johns Hopkins APL Technical Digest 18,
reveals brain’s response during injury and recovery. 134–140 (1994)
IEEE Trans. Biomed. Eng. 43, 1083–1092 (1996) 91. J. Sadowsky, The continuous wavelet transform:
77. D.L. Sherman, A.M. Brambrink, R.N. Ichord, V.K. A tool for signal investigation and understanding.
Dasika, R.C. Koehler, R.J. Traystman, D.F. Han- Johns Hopkins APL Technical Digest 15, 306–318
ley, N.V. Thakor, Quantitative EEG during early (1994)
recovery from hypoxic-ischemic injury in immature 92. H.R. Modi, Q. Wang, S. Gd, D. Sherman, E. Green-
piglets: Burst occurrence and duration. Clin. Elec- wald, A.V. Savonenko, R.G. Geocadin, N.V. Thakor,
troencephalogr. 30, 175–183 (1999) Intranasal post-cardiac arrest treatment with orexin-
78. V. Goel, A. Brambrink, D. Hanley, R. Koehler, A facilitates arousal from coma and ameliorates
N.V. Thakor, Dominant frequency analysis reveals neuroinflammation. PLoS One 12, e0182707 (2017)
Brain’s response to injury and recovery. IEEE Trans. 93. D.L. Jones, R.G. Baraniuk, Efficient approximation
Biomed. Eng. 43, 1083–1092 (1996) of continuous wavelet transforms. Electron. Letters
79. R.O. Schmidt, Multiple emitter location and signal 27, 748–750 (1991)
parameter estimation. IEEE Trans. Antennas Propa- 94. M. Vetterli, J. Kovacevic, Wavelets and Subband
gation AP-34, 276–280 (1986) Coding (Prentice Hall, Englewood Cliffs, 1995)
80. N. Makela, M. Stenroos, J. Sarvas, R.J. Ilmoniemi, 95. P. Goupillaud, A. Grossmann, J. Morlet, Cycle-
Truncated RAP-MUSIC (TRAP-MUSIC) for MEG octave and related transforms in seismic signal anal-
and EEG source localization. NeuroImage 167, 73– ysis. Geoexploration 23, 85–102 (1984)
83 (2018) 96. A.V. Oppenheim, R.W. Schaffer, Discrete Time Sig-
81. K.G. Mideksa, A. Singh, N. Hoogenboom, H. Hell- nal Processing (Prentice Hall, Englewood Cliffs,
riegel, H. Krause, A. Schnitzler, G. Deuschl, J. 1989)
Raethjen, G. Schmidt, M. Muthuraman, Comparison 97. F. Hlawatsch, G. Boudreaux-Bartels, Linear and
of imaging modalities and source-localization algo- quadratic time-frequency signal representations.
rithms in locating the induced activity during deep IEEE Signal Process. Mag. 9, 21–62 (1992)
brain stimulation of the STN. Conf. Proc. IEEE Eng. 98. N.V. Thakor, D. Sherman, Wavelet (time-scale) anal-
Med. Biol. Soc. 2016, 105–108 (2016) ysis in biomedical signal processing, in Biomedical
82. R. Kronland-Martinet, J. Morlet, A. Grossmann, Engineering Handbook, ed. by J. D. Bronzino, (CRC
Analysis of sound patterns through wavelet trans- Press, Boca Raton, 1995)
forms. Intern. J. Pattern Rec. Artificial Intell. 1, 273– 99. C.E. Shannon, A mathematical theory of communi-
302 (1987) cation. Bell Syst. Tech. J. 27, 623–656 (1948)
83. C.S. Burrus, R.A. Gopinath, H. Guo, Introduction to 100. A. Bezerianos, S. Tong, N.V. Thakor, Time-
Wavelets and Wavelet Transforms: A Primer (Pren- dependent entropy estimation of EEG rhythm
tice Hall, Upper Saddle River, 1998) changes following brain ischemia Ann. Biomed.
84. J. Semmlow, B. Griffel, Wavelet Analysis Biosignal Eng. 31, 221–232 (2003)
and Medical Image Processing (CRC Press, Boca 101. H.C. Shin, S. Tong, S. Yamashita, X. Jia, R.G. Geo-
Raton, 2014), pp. 217–246 cadin, N.V. Thakor, Quantitative EEG and effect of
hypothermia on brain recovery after cardiac arrest.
I.E.E.E. Trans. Biomed. Eng. 53, 1016–1023 (2006)
102. M.H. Weil, L. Becker, T. Budinger, K. Kern, G. 115. L.B. Becker, M.L. Weisfeldt, M.H. Weil, T.
Nichol, I. Shechter, R. Traystman, H. Wiedemann, Budinger, J. Carrico, K. Kern, G. Nichol, I.
R. Wise, M. Weisfeldt, G. Sopko, Workshop exec- Shechter, R. Traystman, C. Webb, H. Wiedemann,
utive summary report: Post-resuscitative and initial R. Wise, G. Sopko, The PULSE initiative: Scientific
utility in life saving efforts (PULSE): June 29-30, priorities and strategic planning for resuscitation
2000; Lansdowne Resort and Conference Center; research and life saving therapies. Circulation 105,
Leesburg, VA. Circulation 103, 1182–1184 (2001) 2562–2570 (2002)
103. P.A. Meaney, B.J. Bobrow, M.E. Mancini, J. Chris- 116. L. Arvidsson, S. Lindgren, L. Martinell, S. Lundin,
tenson, A.R. de Caen, F. Bhanji, B.S. Abella, M.E. C. Rylander, Target temperature 34 vs. 36 degrees C
Kleinman, D.P. Edelson, R.A. Berg, T.P. Aufder- after out-of-hospital cardiac arrest – A retrospective
heide, V. Menon, M. Leary, the American Heart observational study. Acta Anaesthesiol. Scand. 61,
Association Emergency Cardiovascular Care Com- 1176–1183 (2017)
mittee, Cpr Quality Summit Investigators, C.C.P. the 117. P.Y.K. Pang, G.H.L. Wee, M.J. Huang, A.E.E. Hoo,
Council on Cardiopulmonary, Resuscitation, Car- I.M. Tahir Sheriff, S.L. Lim, T.E. Tan, Y.J. Loh,
diopulmonary resuscitation quality: [corrected] im- V.T.T. Chao, J.L. Soon, K.L. Kerk, Z.H. Abdul
proving cardiac resuscitation outcomes both inside Salam, Y.K. Sin, C.H. Lim, Therapeutic hypother-
and outside the hospital: a consensus statement from mia may improve neurological outcomes in extra-
the American Heart Association. Circulation 128, corporeal life support for adult cardiac arrest. Heart
417–435 (2013) Lung Circ. 26, 817–824 (2017)
104. M.S. Eisenberg, T.J. Mengert, Cardiac resuscitation. 118. B.R. Scholefield, F.S. Silverstein, R. Telford, R.
N. Engl. J. Med. 344, 1304–1313 (2001) Holubkov, B.S. Slomine, K.L. Meert, J.R. Chris-
105. P. Safar, Cerebral resuscitation after cardiac arrest: tensen, V.M. Nadkarni, J.M. Dean, F.W. Moler,
A review. Circulation 74, IV138–IV153 (1986) Therapeutic hypothermia after paediatric cardiac ar-
106. P. Vaagenes, M. Ginsberg, U. Ebmeyer, L. Ernster, rest: Pooled randomized controlled trials. Resuscita-
M. Fischer, S.E. Gisvold, A. Gurvitch, K.A. Hoss- tion 133, 101–107 (2018)
mann, E.M. Nemoto, A. Radovsky, J.W. Severing- 119. J.P. Nolan, P.T. Morley, T.L. Hoek, R.W. Hickey,
haus, P. Safar, R. Schlichtig, F. Sterz, T. Tonnessen, Therapeutic hypothermia after cardiac arrest. An
R.J. White, F. Xiao, Y. Zhou, Cerebral resuscitation advisory statement by the advancement life support
from cardiac arrest: Pathophysiologic mechanisms. task force of the international liaison committee on
Crit. Care Med. 24, S57–S68 (1996) resuscitation. Resuscitation 57, 231–235 (2003)
107. K. Berek, M. Jeschow, F. Aichner, The prognostica- 120. X. Jia, M.A. Koenig, R. Nickl, G. Zhen, N.V. Thakor,
tion of cerebral hypoxia after out of hospital cardiac R.G. Geocadin, Early electrophysiologic markers
arrest in adults. Eur. Neurol. 37, 135–145 (1997) predict functional outcome associated with temper-
108. I.G. Stiell, G.A. Wells, B.J. Field, D.W. Spaite, ature manipulation after cardiac arrest in rats. Crit.
V.J. De Maio, R. Ward, D.P. Munkley, M.B. Lyver, Care Med. 36, 1909–1916 (2008)
L.G. Luinstra, T. Campeau, J. Maloney, E. Dagnone, 121. D. Sherman, A. Brambrink, R. Ichord, V. Dasika, R.
Improved out-of-hospital cardiac arrest survival Koehler, R. Traystman, D. Hanley, N. Thakor, Quan-
through the inexpensive optimization of an exist- titative EEG during early recovery from hypoxic-
ing defibrillation program: OPALS study phase II. ischemia injury in immature piglets: Burst oc-
Ontario Prehospital Advanced Life Support. JAMA curence and duration. Clin. Electroenceph, Ac-
281, 1175–1181 (1999) cepted (1999)
109. N.R. Grubb, Managing out-of-hospital cardiac arrest 122. D.L. Sherman, M.J. Hinich, N.V. Thakor, The higher
survivors: 1. Neurological perspective. Heart 85, 6– order statistics of energy operators with applications
8 (2001) to neurological signals, 1998 IEEE symposum on
110. S.A. Mills, Risk factors for cerebral injury and car- time-frequency and time-scale, Pittsburgh, 1998
diac surgery. Ann. Thorac. Surg. 59, 1296–1299 123. H.A. Al-Nashash, J.S. Paul, W.C. Ziai, D.F. Hanley,
(1995) N.V. Thakor, Wavelet entropy for subband segmen-
111. B.C. White, L.I. Grossman, B.J. O’Neil, D.J. DeGar- tation of EEG during injury and recovery. Ann.
cia, R.W. Neumar, J.A. Raafols, G.S. Krause, Global Biomed. Eng. 31, 653–658 (2003)
brain ischemia and reperfusion. Ann. Emerg. Med. 124. H.A. Al-Nashash, N.V. Thakor, Monitoring of
27, 588–594 (1996) global cerebral ischemia using wavelet entropy rate
112. W. Longstreth, T. Inui, L. Cobb, M. Copass, Neu- of change. I.E.E.E. Trans. Biomed. Eng. 52, 2119–
rologic recovery after out of hospital cardiac arrest. 2122 (2005)
Ann. Int. Med. 98, 588–592 (1983) 125. X. Kang, X. Jia, R.G. Geocadin, N.V. Thakor, A.
113. D. Levy, D. Bate, J. Carrona, et al., Prognosis in Maybhate, Multiscale entropy analysis of EEG for
nontraumatic coma. Ann. Int. Med. 94, 293–301 assessment of post-cardiac arrest neurological re-
(1981) covery under hypothermia in rats. I.E.E.E. Trans.
114. AHA, Guidelines 2000 for Cardiopulmonary Resus- Biomed. Eng. 56, 1023–1031 (2009)
citation and Emergancy Cardiovascular Care. Circu-
lation 102(suppl I), I-1-I-384 (2000)
126. H.C. Shin, S. Tong, S. Yamashita, X. Jia, R.G. 130. Y. Yasuda, A. Takeda, S. Fukuda, H. Suzuki, M.
Geocadin, N.V. Thakor, Quantitative EEG Assess- Ishimoto, Y. Mori, H. Eguchi, R. Saitoh, H. Fujihara,
ment of Brain Injury and Hypothermic Neuro- K. Honda, T. Higuchi, Orexin a elicits arousal elec-
protection after Cardiac Arrest, EMBS Annual troencephalography without sympathetic cardio-
Intl. Conference, IEEE, New York City, 2006, pp. vascular activation in isoflurane-anesthetized rats.
6229–6232 Anesth. Analg. 97, 1663–1666 (2003)
127. X. Jia, M.A. Koenig, H.C. Shin, G. Zhen, S. Ya- 131. I. Sato-Suzuki, I. Kita, Y. Seki, M. Oguri, H. Arita,
mashita, N.V. Thakor, R.G. Geocadin, Quantitative Cortical arousal induced by microinjection of orex-
EEG and neurological recovery with therapeutic ins into the paraventricular nucleus of the rat. Behav.
hypothermia after asphyxial cardiac arrest in rats. Brain Res. 128, 169–177 (2002)
Brain Res. 1111, 166–175 (2006) 132. H.L. Dong, S. Fukuda, E. Murata, Z. Zhu, T.
128. T. Nakamachi, S. Endo, H. Ohtaki, L. Yin, D. Kenji, Higuchi, Orexins increase cortical acetylcholine
Y. Kudo, H. Funahashi, K. Matsuda, S. Shioda, release and electroencephalographic activation
Orexin-1 receptor expression after global ischemia through orexin-1 receptor in the rat basal forebrain
in mice. Regul. Pept. 126, 49–54 (2005) during isoflurane anesthesia. Anesthesiology 104,
129. E.A. Irving, D.C. Harrison, A.J. Babbs, A.C. Mayes, 1023–1032 (2006)
C.A. Campbell, A.J. Hunter, N. Upton, A.A. Par- 133. M.A. Koenig, X. Jia, X. Kang, A. Velasquez, N.V.
sons, Increased cortical expression of the orexin-1 Thakor, R.G. Geocadin, Intraventricular orexin-A
receptor following permanent middle cerebral artery improves arousal and early EEG entropy in rats after
occlusion in the rat. Neurosci. Lett. 324, 53–56 cardiac arrest. Brain Res. 1255, 153–161 (2009)
(2002) 134. ASET 59th annual conference proceedings. Neuro-
diagn. J. 58, 235–256 (2018)
Brain–Computer Interfaces
4
Bin He, Han Yuan, Jianjun Meng, and Shangkai Gao
Abstract translation, to device control, and various

applications. The performance assessment
Brain–computer interfaces (BCIs) have
and challenges of BCIs are also discussed.
emerged as a novel technology that bridges
Examples of noninvasive BCIs are discussed
the brain with external devices. BCIs
to aid readers for an in-depth understanding
have been developed to decode human’s
of the noninvasive BCI technology, although
intention, leading to direct brain control of
this chapter is aimed at providing a general
a computer or device without going through
introduction to brain–computer interfaces.
the neuromuscular pathway. Bidirectional
brain–computer interfaces not only allow
Keywords
brain control but also open the door for
modulating the central nervous system Brain–computer interface · Brain–machine
through neural interfacing. We review the interface · BCI · BMI · Neural interface ·
concepts, principles, and various building Mind control · Neuroprosthesis ·
blocks of BCIs, from signal acquisition, Neurorobotics · EEG
signal processing, feature extraction, feature
4.1 Introduction
of this chapter (https://doi.org/10.1007/978-3-030-43395-
6_4) contains supplementary material, which is available Brain–computer interfaces are a new technology
to authorized users. that could help to restore useful function to people
B. He () severely disabled by a wide variety of devas-
Department of Biomedical Engineering, Carnegie Mellon
tating neuromuscular disorders and to enhance
University, Pittsburgh, PA, USA
e-mail: [email protected] functions in healthy individuals. The first demon-
strations of brain–computer interface (BCI) tech-
H. Yuan
Stephenson School of Biomedical Engineering, nology occurred in the 1960s when Grey Wal-
University of Oklahoma, Norman, OK, USA ter used the scalp-recorded electroencephalogram
e-mail: [email protected] (EEG) to control a slide projector in 1964 [1] and
J. Meng when Eberhard Fetz taught monkeys to control
School of Mechanical Engineering, Shanghai Jiao Tong a meter needle (and thereby earn food rewards)
University, Shanghai, China
by changing the firing rate of a single cortical
e-mail: [email protected]
neuron [2]. In the 1970s, Jacques Vidal devel-
S. Gao
oped a system that used the scalp-recorded visual
Department of Biomedical Engineering, Tsinghua
University, Beijing, China evoked potential (VEP) to determine the eye gaze
e-mail: [email protected] direction (i.e., the visual fixation point) in humans

132 B. He et al.
and thus to determine the direction in which a expanding or enhancing media access, computer
person wanted to move a computer cursor [3, gaming, or artistic expression. Furthermore, BCI
4]. At that time, Vidal coined the term brain– technology has recently begun to be explored as
computer interface. Since then and into the early a means to assist in the rehabilitation of people
1990s, BCI research studies continued to appear disabled by stroke and other acute events. This
only every few years. In 1980, Elbert et al. showed chapter provides an introduction to the underlying
that people could learn to control slow cortical concepts and principles as well as the applications
potentials (SCPs) in scalp-recorded EEG activity of BCIs.
and could use that control to adjust the vertical
position of a rocket image moving across a TV
screen [5]. In 1988, Farwell and Donchin [6] re- 4.2 BCI Definition and Structure
ported that people could use scalp-recorded P300
event-related potentials (ERPs) to spell words on 4.2.1 What Is a BCI?
a computer screen. Wolpaw and his colleagues
trained people to control the amplitude of mu and According to present understanding, the role of
beta rhythms (i.e., sensorimotor rhythms) in the the central nervous system (CNS) is to respond
EEG and showed that the subjects could use this to occurrences in the environment or in the body
control to move a computer cursor [7]. by producing appropriate outputs. The natural
The pace and breadth of BCI research began outputs of the CNS are either neuromuscular or
to increase rapidly in the mid-1990s, and this hormonal. Correspondingly, the natural inputs
growth has continued almost exponentially into of the CNS are from different sensory organs,
the present. The work over the past 20 years has peripheral nerves, internal hormones, etc. A
included a broad range of studies in all the areas brain–computer interface (BCI), which could
relevant to BCI research and development, in- interact with the CNS bidirectionally, gives the
cluding basic and applied neuroscience, biomed- CNS new output that is not neuromuscular or
ical engineering, materials science, electrical hormonal or provides new inputs to the CNS,
engineering, signal processing, machine learning, which could be direct stimulations to the CNS
computer science, assistive technology, clinical by injecting physical energy, such as deep brain
rehabilitation, and human factors engineering stimulation (DBS), transcranial electrical stimu-
[8–10]. lation (TES), transcranial magnetic stimulation
The central goal of BCI research and devel- (TMS), transcranial focused ultrasound (tFUS),
opment is the realization of powerful new assis- or other forms of brain signal modulation. A
tive communication and control technology for BCI is a system that measures CNS activity and
people severely disabled by neuromuscular disor- converts it into artificial output that replaces,
ders such as amyotrophic lateral sclerosis (ALS), restores, enhances, supplements, or improves
stroke, spinal cord injury, cerebral palsy, mul- natural CNS output; it can also be considered as
tiple sclerosis, and muscular dystrophies. This a system to influence CNS activity and behavioral
emphasis has been encouraged and strengthened performance by injecting physical energy such
by increased societal appreciation of the needs as TES, TMS, tFUS, or direct brain signal
of people with severe disabilities, as well as by modulation and thereby changes the ongoing
greater realization of their ability to live enjoyable interactions between the CNS and its external or
and productive lives if they can be provided with internal environment.
effective assistive technology. In addition, in re- To understand this definition, one needs to
cent years a number of investigators have begun understand each of its key terms, starting with
to explore possibilities for developing BCIs for CNS. The CNS is composed of the brain and the
the general population. These include systems spinal cord and is differentiated from the periph-
for enhancing or supplementing human perfor- eral nervous system (PNS), which is composed of
mance in demanding tasks such as image analysis the peripheral nerves and ganglia and the sensory
or continuous attention, as well as systems for receptors. The unique features of CNS structures
4 Brain–Computer Interfaces 133
are their location within the meningeal coverings ticular measures (or features) from them, and con-
(i.e., meninges), their distinctive cell types and verts (or translates) the features into new artificial
histology, and their role in integrating the numer- outputs that act on the environment or on the
ous different sensory inputs to produce effective body itself. Alternatively, a BCI system could
motor outputs. In contrast, the PNS is not inside also deliver physical energy directly to the brain
the meninges, does not have the unique CNS through transcranial electrical, magnetic, acoustic
histology, and serves primarily to bring sensory stimulation or direct-current stimulation to the
inputs to the CNS and to carry motor outputs brain (e.g., DBS or direct cortical stimulation),
from it. to modulate the CNS to change the information-
CNS activity comprises electrophysiological, processing patterns within the brain and affect
neurochemical, and metabolic phenomena (such human behaviors.
as neuronal action potentials, synaptic potentials, Figure 4.1 illustrates the concepts of
neurotransmitter releases, and oxygen consump- bidirectional BCIs, either controlling a device by
tion) that occur continually in the CNS. These the brain bypassing the common neuromuscular
phenomena can be monitored by measuring elec- pathways or modulating and affecting the brain
tric or magnetic fields, hemoglobin oxygenation, by injecting external physical energy.
or other parameters employing sensors on the A BCI output could replace natural output that
scalp, on the surface of the brain, or within the has been lost to injury or disease. Thus, someone
brain. A BCI records brain signals, extracts par- who cannot speak could use a BCI to spell words
Fig. 4.1 Schematics of bidirectional brain–computer in- that correlate with the user’s intent. These features are then
terface (BCI) systems. For a brain-to-device BCI, signals translated into commands that control application devices
produced by brain activity are recorded from the scalp, that replace, restore, enhance, supplement, or improve
from the cortical surface, or from within the brain. These natural CNS outputs. For a device-to-brain BCI, neuro-
signals are analyzed to extract signal features (e.g., ampli- modulation can be exerted on the brain through physical
tudes of EEG rhythms or firing rates of individual neurons) energy to modulate the CNS activity
134 B. He et al.
that are then spoken by a speech synthesizer [11] ing interactions between the CNS and its exter-
or someone who has lost limb control could use nal or internal environment. The CNS interacts
a BCI to operate a powered wheelchair [12] or constantly with the environment and the body.
control a robotic arm [13, 14]. These interactions comprise its outgoing motor
A BCI output could restore lost natural output. outputs along with its incoming sensory inputs.
Thus, someone with a spinal cord injury whose By monitoring CNS activity and translating it into
arms and hands are paralyzed could use a BCI artificial outputs that act on the environment or
to control stimulation of the paralyzed muscles the body, BCIs modify both CNS motor outputs
with implanted/attached electrodes so that the and sensory inputs (i.e., feedback). Devices that
muscles move the limbs [15, 16] or someone who only monitor brain activity and do not employ it
has lost bladder function from multiple sclerosis to modify the continuing interactions of the CNS
could use a BCI to stimulate the peripheral nerves with its environment are not considered BCIs.
controlling the bladder so as to produce urination. In addition to interacting with and control-
A BCI output could enhance natural CNS out- ling the environment by the brain, a BCI might
put. Thus, someone engaged in a task that needs modulate brain signals through direct physical
continuous attention over a long time (e.g., driv- stimulation such as TES, TMS, tFUS, and DBS
ing a car or performing sentry duty) could employ or through neurofeedback trainings. Convention-
a BCI to detect the brain activity preceding breaks ally, such device-to-brain interfacing systems are
in attention and then produce an output (such as referred to as neuromodulation approaches (see
a sound) that alerts the person and restores atten- Fig. 4.2 for the illustration of device-to-brain BCI
tion [17]. By preventing the periodic attentional approaches) and will be treated comprehensively
breaks that normally compromise natural CNS in Chaps. 6, 7, and 8 for deep brain stimulation,
output, the BCI enhances the natural output. transcranial magnetic stimulation, and transcra-
A BCI output could supplement natural CNS nial electrical stimulation. In this chapter, we will
output. Thus, someone controlling cursor position mainly focus on brain-to-device interfacing and
with a standard joystick might employ a BCI to control.
choose items that the cursor reaches [18]. Or a
person could use a BCI to control a third (i.e., 4.2.2 Alternative or Related Terms
robotic) arm and hand [19]. In these examples, the
BCI supplements natural neuromuscular output BCIs are also called brain–machine interfaces or
with another artificial output. BMIs. The choice between these two synony-
Lastly, a BCI output might possibly improve mous terms is essentially a matter of personal
natural CNS output. For example, a person whose preference. One reason for using BCI rather than
arm movements have been compromised by a BMI is that the word “machine” in BMI implies
stroke damaging sensorimotor cortex might em- a fixed translation of brain signals into output
ploy a BCI that measures signals from the dam- commands, which does not match the reality that
aged areas and then excites muscles or controls a computer and the brain are essentially partners
an orthosis that improves arm movement [20]. in the interactive adaptive control that is required
Because this BCI application enables the produc- for successful BCI, or BMI, function.
tion of more normal movements, its continued use The terms dependent BCI and independent
might induce activity-dependent CNS plasticity BCI appeared in 2002 [10]. In accord with the
that improves the natural CNS output and thus definition of a BCI, both employ brain signals to
helps to restore more normal arm control. control applications; however, they differ in how
The first two kinds of BCI application, re- they depend on natural CNS output. A dependent
placement or restoration of lost natural outputs, BCI employs brain signals that depend on mus-
are the focus of most present-day BCI research cle activity. The BCI developed by Vidal [3, 4]
and development. At the same time, the other used a VEP that depended on gaze direction and
three types of applications are drawing increasing therefore on the muscles that controlled gaze. A
attention. Furthermore, a BCI changes the ongo- dependent BCI is basically an alternative way to
Fig. 4.2 A summary of invasive and noninvasive device- ulation (tDCS) via scalp sponge electrodes, or transcranial
to-brain BCI technologies (also called neuromodulation). focused ultrasound stimulation (tFUS) using pulsed ultra-
Invasive techniques include DBS, in which a lead is im- sound from a transducer on the scalp. These neuromodu-
planted into a deep brain structure, and cortical stimulation approaches impact the brain by injecting physical
lation, in which electrodes are placed on the brain sur- energy to modulate the neural activation and connectivity
face. Noninvasive techniques include transcranial mag- within the brain. (From Edelman et al. [35], licensed under
netic stimulation (TMS), transcranial direct-current stim- CC BY 4.0)
detect messages conveyed by natural CNS out- has four components: 1) signal acquisition, 2) fea-
puts. Thus, it does not give the brain a new output ture extraction, 3) feature translation, and 4) de-
independent of natural outputs. Nevertheless, it vice output commands or neurofeedback training
can still be very useful. paradigm. Note that, besides these four traditional
Contrastingly, an independent BCI does not BCI components, a direct physical energy might
depend on natural CNS output; muscle activity is be injected to interact with or affect the CNS (also
not needed to generate the crucial brain signals. an approach called neuromodulation). A BCI also
Thus, in BCIs that measure EEG sensorimotor has an operating protocol that specifies how the
rhythms, the user typically employs mental im- onset and timing of operation or physical energy
agery to modulate sensorimotor rhythms in order injection is controlled; how the feature translation
to produce the BCI output. For those who are process is parameterized, the nature of the com-
severely disabled by neuromuscular disorders, mands that the BCI produces, the neurofeedback
independent BCIs are likely to be more effective. training that the BCI induces; and how errors in
The recent term hybrid BCI is used in two translation are handled. A successful operating
ways [21]. It can be applied to a BCI that employs protocol enables the BCI system to be flexible and
two different types of brain signals (e.g., VEPs to serve the particular needs of each of its users.
and sensorimotor rhythms) to produce its outputs, The signal acquisition component measures
or it can be applied to a system that combines a brain signals using a particular kind of sensor
BCI output and a natural muscle-based output. In (e.g., scalp or intracranial electrodes for electro-
this second usage, the BCI output supplements a physiological activity, functional magnetic res-
natural CNS output (as Fig. 4.1 illustrates). onance imaging for metabolic activity, etc.). It
amplifies the signals to enable subsequent pro-
4.2.3 The Components of a BCI cessing, and it may also filter them to remove
noise such as 60-Hz (or 50-Hz) power line inter-
A BCI detects and measures features of brain sig- ference. The amplified signals are digitized and
nals that reveal the user’s intentions and translates transmitted to a computer.
these features in real time into commands that The feature extraction component analyzes the
achieve the user’s intent or affect the user’s brain digitized signals to isolate signal features (e.g.,
state (Fig. 4.1). In order to do this, a BCI system power in specific EEG frequency bands or fir-
136 B. He et al.
ing rates of individual cortical neurons) and ex- CNS entirely new artificial outputs derived from
presses them in a compact form suitable for trans- brain signals. In essence, they ask the CNS, which
lation into output commands. Effective features has evolved to produce muscular and hormonal
need to have strong correlations with the user’s outputs, to produce entirely new kinds of out-
intent. Since much of the most relevant (i.e., puts. Thus, for example, the sensorimotor cor-
most strongly correlated) brain activity is tran- tical areas, which normally act in combination
sient or oscillatory, the signal features most com- with subcortical and spinal areas to control mus-
monly extracted by present-day BCIs are EEG cles, are now required instead to control specific
or electrocorticogram (ECoG) response ampli- brain signals (such as neuronal firing patterns or
tudes, power in particular EEG or ECoG fre- EEG rhythms). The fundamental implications of
quency bands, or firing rates of single cortical this requirement become evident when BCI use
neurons. To ensure the accurate measurement of is considered in terms of two basic principles
the chosen signal features, artifacts such as elec- that govern how the CNS produces its natural
tromyogram (EMG) from cranial muscles need to outputs.
be avoided or eliminated. First, the task of producing natural outputs is
The signal features are provided to the feature distributed throughout the CNS, from the cerebral
translation algorithm, which converts them into cortex to the spinal cord. No one area is entirely
commands for the output device, that is, into responsible for a natural output. Actions such
commands that achieve the user’s intent. Thus, as speaking, walking, or playing the piano are
a decrease in power in a specific EEG frequency produced by the integrated activity of cortical
band might be translated into an upward displace- areas, basal ganglia, thalamic nuclei, cerebellum,
ment of a computer cursor, or a particular evoked brain stem nuclei, and spinal cord interneurons
potential measure might be translated into the se- and motoneurons. Thus, while the cortex usu-
lection of a letter to be added to a document being ally initiates walking and monitors its course,
composed. The translation algorithm should be the rhythmic rapid sensorimotor interactions that
able to accommodate and adapt to spontaneous underlie effective walking are handled primarily
or learned changes in the user’s signal features by circuits in the spinal cord [22]. The final re-
in order to ensure that the user’s possible range sult of this highly distributed CNS activity is the
of feature values covers the full range of device proper excitation of the spinal (or brain stem)
control and also to make control as effective and motoneurons that activate muscles and thereby
efficient as possible. produce actions. In addition, while activity in the
The commands that the feature translation al- different CNS areas that are participating gener-
gorithm produces are the output of the BCI or ally correlates with the action, the activity in a
the input of the brain, which has to be modulated particular area may vary considerably from one
internally [17]. They go to the application and performance of the action to the next. At the
there produce functions such as letter selection same time, the coordinated activity in the many
[17], cursor control [18], robotic arm operation areas involved ensures that the action itself is
[13, 14], wheelchair movement [12], etc. The stable.
operation of the device provides feedback for the Second, natural CNS outputs (such as speak-
user and thereby closes the control loop. ing, walking, or playing a musical instrument) are
acquired initially and maintained in the long term
4.2.4 The Unique Challenge of BCI by adaptive changes in the many CNS areas that
Research and Development contribute to them. Throughout life, CNS neurons
and synapses change continually to master new
As noted earlier, the natural CNS function is to skills and to maintain those already learned [23,
produce muscular and hormonal outputs that act 24]. Referred to as activity-dependent plasticity,
on the outside world or the body. BCIs give the this continuing change underlies the acquisition
and preservation of both common skills (e.g., 4.2.5 BCI Operation Depends
walking and talking) and special skills (e.g., ath- on the Interaction of Two
letics, singing); and it is guided by its results. Adaptive Controllers
For example, as muscle strength and body size and the User Interface
and weight change during life, CNS areas change
appropriately to maintain these skills. In addition, Muscle-based CNS outputs are optimized to serve
the basic CNS features (i.e., its anatomy, phys- the goals of the organism, and the adaptation re-
iology, and plasticity mechanisms) that support sponsible for this optimization takes place mainly
this ongoing adaptation are the results of evolu- in the CNS. In contrast, BCI outputs can be op-
tion shaped by the need to produce appropriate timized by adaptations in the CNS and/or in the
muscle-based actions. BCI itself. Thus, a BCI may adapt to the am-
Given these two principles that numerous CNS plitudes, frequencies, and other basic character-
areas participate in natural outputs and that adap- istics of the user’s brain signals; it may adapt to
tive plasticity occurs continually in all these ar- improve the fidelity with which its output com-
eas, BCI use presents a unique challenge for the mands match the user’s intentions; and it may
CNS, which has evolved and is continually adapt- adapt to improve the effectiveness of CNS adap-
ing to optimize its natural outputs. In contrast tations and perhaps to guide the CNS adaptive
to natural CNS outputs, which are produced by processes.
spinal motoneurons and the muscles they control, In sum, a BCI introduces a second adaptive
BCI-based CNS outputs are produced by signals controller that can also change to ensure that
reflecting activity in another CNS area, such as the user’s goals are achieved. Thus, BCI usage
the motor cortex. Activity in the motor cortex requires successful interaction between two adap-
is normally one of the multiple contributors to tive controllers, the user’s CNS and the BCI. The
natural CNS output. But when its signals control management of the complex interactions between
a BCI, this activity becomes the CNS output. In the concurrent adaptations of CNS and BCI is one
sum, the cortex is given the role normally per- of the most difficult problems in BCI research. In
formed by spinal motoneurons; that is, it produces the past two decades, a majority of studies have
the final product, the output, of the CNS. How focused on either training subjects’ brain while
well the cortex performs this new unnatural role fixating the decoding algorithm after each ses-
depends on how effectively the multiple CNS sion’s calibration or adapting the machine learn-
areas that normally combine to control spinal ing algorithm in real time within each session
motoneurons (which are downstream in natural while minimizing subjects’ learning effort [25].
CNS function) can instead adapt to control the Until recently, studies of both invasive and nonin-
relevant cortical neurons and synapses (which are vasive BCI [26, 27] showed a piece of converged
largely upstream in natural CNS function). evidence that subjects’ learning curve probably
The available evidence indicates that the benefits most from collaboration, adapting both
adaptations needed to control activity in the CNS controllers, that is, the brain and the decoder
areas that produce the signals used by BCIs are algorithm. Theoretical analysis also indicates that
possible but as yet very imperfect. As a rule, BCI adaptation of the BCI system should be at an ap-
outputs are much less smooth, rapid, and accurate propriate rate, not too slow nor too frequent [28].
than natural muscle-based CNS outputs, and their Studies showed day-to-day variability in perfor-
moment-to-moment and day-to-day variability is mance using daily retrained decoder and non-
disturbingly high. These problems (especially stable neural ensembles when tracking subjects’
poor reliability) and the different approaches to performance from weeks to months [26]. Ors-
solving them represent major challenges in BCI born et al. showed that beneficial neuroplasticity
research. could occur alongside mild and gradual decoder
138 B. He et al.
adaptation, yielding performance improvements, 4.2.6 Choosing Signals and Brain

skill retention, and resistance to interference from Areas for BCIs
native motor networks [26]. Similarly, in their
study, Perdikis and colleagues only recalibrated Brain signals acquired by a number of different
the decoder of participants twice during the train- electrophysiological and metabolic methods can
ing periods in multiple months. Sufficient time be used as BCI inputs for brain-to-device con-
was provided to the subject to adapt their brain trol. These signals differ in topographical res-
rhythms to the fixed decoder. Besides the two olution, frequency content, area of origin, and
controllers, Perdikis et al. even argue for and eval- technical needs. The major electrophysiological
uate the importance of the application interface, methods as applied to BCIs are illustrated in
which is one of the three pillars of a successful Fig. 4.3. They range from EEG with its centime-
BCI system besides the subject and the machine ter resolution, to ECoG with its few millime-
learning algorithm. The effect of application inter resolution, to neuronal action potentials with
terface on BCI performance was rarely investi- their tens-of-microns resolution. Each of these
gated previously. Some of the previous studies electrophysiological methods has been used by
might assume that using more attractive or more BCIs and deserves continued evaluation, as do the
natural application interface would cause better metabolic methods such as functional magnetic
engagement of participants [14, 29–31], which resonance imaging (fMRI) and functional near-
implicitly showed a similar idea along this line infrared imaging (fNIRs). Each has distinctive
[32]. Future investigations should consider the advantages and disadvantages, while electrophys-
application interface as an important factor to the iological signals have gained wide adoption due
BCI performance. Various application interfaces to its high temporal resolution and portability.
including control of physical apparatus [14, 33], The role of neuronal action potentials (spikes)
immersion of virtual reality [34], or switching as basic units of communication between neurons
the stereotype of center-out trial-based task to suggests that spikes recorded from many neurons
continuous tracking task [13] should be further could provide multiple degrees of freedom and
explored. might therefore be the optimum signals for BCIs
to employ. In addition, the clear relationships
between cortical neuronal activity and normal
motor control provide logical starting points for
Fig. 4.3 Schematic of a

brain-to-device
brain–computer interface.
Signals are acquired from
the brain through the use of
internal or external stimuli.
A computer then decodes
these signals to interpret
the user’s goal and
translates the result into an
action of the output device.
Subjects can often observe
such effects and modulate
their brain signals to
accomplish the desired
task. (From Edelman et al.
[35], licensed under CC
BY 4.0)
BCI-based control of applications such as robotic lobe [37]. Due to the unique characteristics and
arms. On the other hand, the importance of CNS complexity of producing human languages, it is
adaptation for all BCIs and the evidence that not possible to do the experiments in animal
appropriate training can elicit multiple degrees of models. ECoG, which is vastly used in the clinical
freedom from even EEG signals suggest that the setting, has a high temporal and spatial resolution.
difference between the BCI performance possible The most common type of intractable epilepsy
with single neurons and that possible with EEG or is usually caused by the pathological change of
ECoG may not be nearly as large as the difference temporal lobe; however, a good number of these
in their respective topographical resolutions. patients with focal epilepsy in the temporal lobe
The most important point is that questions still preserve intact speech ability. Thus, ECoG-
about signal selection are empirical questions that based speech BCI could be developed and vali-
can be answered only by experimental evidence, dated in this population [41]. The advancement
not by a priori assumptions about the fundamental of speech BCI may benefit patients undergoing
superiority of one kind of signal or another. For ECoG recordings who cannot speak due to, for
BCI usage, the crucial issue is which signals can example, brain stem stroke and cerebral palsy
best indicate the user’s intent and serve the pur- [42]. Recent advancement of deep learning neural
pose of applications, that is, which signals are the network and its application in speech decoding
best language for communicating to the BCI the produce significant progress in decoding the flu-
output that the user wants, to achieve the purpose ent speech directly from the brain signals [11, 36,
such a BCI is aimed at. 41]. The quick development of speech BCI may
The choice of the optimum brain areas from be a vital option in clinical treatment for those
which to obtain the signals is also an empirical who have language disabilities.
question at the time. The work to date has fo-
cused largely on signals from sensorimotor au-
ditory, and visual areas of cortex. The BCI ca- 4.3 Signal Acquisition
pacities of signals from other cortical or sub-
cortical areas are just beginning to be investi- As discussed earlier, translation of intent into
gated. This is an important aspect of BCI reaction is dependent on the expression of the in-
search, particularly because the sensorimotor cor- tent in the form of a measurable signal. Proper
tices of many possible BCI users have been com- acquisition of this signal is important for the
promised by disease or injury, and/or their vision functioning of any BCI. The goal of signal ac-
may be impaired. Different brain areas may differ quisition methods is to detect the voluntary neural
in their adaptive capabilities and in other factors activity generated by the user, whether the signals
that could affect their capacity to function as the are acquired invasively or noninvasively. Each
sources of BCI output commands. For example, method of signal acquisition is associated with
reconstructing speech from the neural responses an inherent spatial and temporal signal resolution.
recorded from the human auditory cortex opens The choice of the appropriate method to use in
up the possibility of a speech BCI to restore a particular circumstance depends on striking a
speech in severely paralyzed patients [36, 37]. balance between the feasibility of acquiring the
This new speech BCI is different from the conven- signal in the operating environment and the reso-
tional P300 speller or SSVEP-based virtual key- lution required for proper translation.
board which translates users’ visual attention into
characters, words, and sentences via special vi-
sual stimulus pattern [38–40]. These conventional 4.3.1 Invasive Techniques
BCI spellers mainly decode brain signals from the
visual occipital cortex. However, the nascent field The invasive acquisition of brain signals for use in
of speech BCI directly decodes the brain signals BCIs is primarily accomplished by electrophys-
from the speech production areas in the temporal iologic recordings from electrodes that are neu-
140 B. He et al.
rosurgically implanting either inside the user’s cortical column can sum to yield a detectable
brain or over the surface of the brain. The motor signal. Subdural electrodes are closer to neuronal
cortex has been the preferred site for implanting structures in superficial cortical layers than EEG
electrodes since it is more easily accessible and electrodes placed on the scalp, and therefore, the
has large pyramidal cells, which produce measur- signals that they record have higher amplitude
able signals that can be generated through simple (as well as a broader frequency bandwidth).
tasks such as actual or imaginary motor move- Whereas scalp electrode recordings represent
ments. Other brain areas such as the supplemen- synchronized activity from a large number of
tary motor cortex, parietal cortex, and subcortical neurons and synapses over extended regions of
motor areas can also serve as candidate sites for cortex [48], subdural recordings are sensitive
electrode implantation. Information from com- to smaller sources of synchronized neuronal
plementary imaging techniques such as fMRI can activity. Subdural recordings also have a higher
help determine potential target areas for a specific signal-to-noise ratio than scalp recordings and
subject [43]. fMRI measurement of the blood- have increased ability to record and study gamma
oxygenation level dependent (BOLD) response activity (i.e., activity >30 Hz). Since gamma
has facilitated the determination of cortical areas activity has been shown to be well correlated
useful for the recording of brain activity and has with the surrounding single-unit activity recorded
also been shown to provide reliable BCI control by penetrating microelectrodes [49], ECoG
across several cortical areas using different cog- can yield an effective representation of the
nitive tasks. underlying cortical electrical activity with less
invasiveness and more stability than penetrating
4.3.1.1 Intracortical microelectrodes, albeit still invasive.
With chronic recording using implanted micro- The standard clinical electrodes used for
electrode arrays, the key factors for successful ECoG monitoring in epilepsy patients typically
recording are the spatial/temporal resolution of have diameters on the order of a few millimeters.
the desired signal, the number and placement Although finer than scalp electrodes, this
of electrodes, and the functional lifetime of the dimension is still much larger than that of a
device. A growing number of electrode technolo- typical cortical column. Therefore, most studies
gies have been developed to meet these require- involving subdural ECoG use gross motor
ments. Significant advancement has been wit- movements to determine tuning parameters.
nessed in intracortical BCIs research over the It was shown that overt movements as well
past two decades, demonstrating brain-controlled as motor imageries are accompanied not only
robotic arms in nonhuman primates [44, 45] and by relatively widespread mu and beta event-
human subjects [46, 47]. For a comprehensive related desynchronization (ERD), but also by
coverage of intracortical BCIs, see Chapter 5 in a more focused event-related synchronization
this book. (ERS) in the gamma frequency band [50]. In
the first closed-loop ECoG-based BCI, study
4.3.1.2 Cortical Surface subjects quickly learned to modulate high-
A less-invasive approach, though still requiring frequency gamma rhythms in motor cortical
surgical implantation of the recording device, areas and in Broca’s speech area to control a
is ECoG. This technique, in which an electrode one-dimensional computer cursor in real time.
array is implanted subdurally over cortex, has Subsequent studies achieved two-dimensional
been used mainly in preparation for surgery in control of a computer cursor using the upper arm
people with epilepsy. As is the case for EEG region of motor cortex for one dimension and
recording, this technique takes advantage of the hand region of motor cortex for the other
the fact that most large cortical neurons are dimension [51]. Other investigators explored
orientated perpendicular to the cortical surface distinctly human traits such as speech and
and that locally synchronized activity within a language processing that cannot be analyzed
in an animal model and have had success using BCI control with several degrees of freedom can
gamma activity from a speech network to control be achieved with just a few electrodes [18, 29].
a cursor in one dimension [52]. The subjects Over the past few decades, EEG-based BCIs
used self-selected imagery to modulate gamma- have been widely investigated in healthy human
band activity at one or more specific electrodes. subjects, as well as in people with amyotrophic
This represents a new approach in ECoG-based lateral sclerosis (ALS) and in those with severe
BCIs. CNS damage from spinal cord injuries and stroke,
resulting in substantial deficits in communication
and motor function.
4.3.2 Noninvasive Techniques Compared with invasive BCIs, EEG-based
BCI methods have the advantage of no surgical
There are many methods of measuring brain ac- risk, signal stability, and low cost. However,
tivity through noninvasive means. Noninvasive since EEG represents scalp manifestation of
techniques reduce risk for users since they do brain electrical activity from a distance, it
not require surgery or permanent attachment to has a lower signal-to-noise ratio than many
the device. Techniques such as EEG, magnetoen- invasive methods. The spatial resolution of
cephalography (MEG), fMRI, and fNIRS have EEG is also reduced by the volume-conduction
been used in noninvasive BCIs. effect [48]. Many noninvasive BCIs are based
on classification of different mental states
4.3.2.1 EEG rather than decoding kinematic parameters as
EEG is the most prevalent method of signal acqui- is typically done in invasive BCIs. Various
sition for BCIs. EEG recording has high temporal mental strategies exploiting motor, sensory, and
resolution: it is capable of measuring changes in cognitive activity detectable by EEG have been
brain activity that occur within a few millisec- used to build communication systems. In these
onds. The spatial resolution of EEG is not as systems, typically one mental state corresponds
good as that of implanted methods, but signals to one direction of control and four independent
from up to 256 electrodes can be measured at the mental states are generally required for full two-
same time [53]. EEG is easy to set up, portable, dimensional control. Therefore, a substantial
and inexpensive and has a rich literature of past period of training is typically required for users
performance. The practicality of EEG in the lab- to develop the skill to maintain and manipulate
oratory and the real-world setting is unsurpassed. various mental states to enable the control. This
EEG recording equipment is portable, and the can be quite demanding for users, especially
electrodes can be easily placed on the subject’s disabled users. Other investigators attempted
scalp by simply donning a cap. In addition, since to directly decode the kinematic information
EEG systems have been widely used in numerous related to movement or motor imagery and
fields since their inception more than 90 years have reported success in revealing information
ago, the methods and technology of signal acqui- about the (imagined) movement direction and
sition with this modality have been standardized. speed from the spatiotemporal profiles of EEG
Finally, and most important, the method is nonin- signals [54–56]. In a closed-loop experiment by
vasive. Bradberry et al. [57] using the direct decoding
Many EEG-based BCI systems use an elec- of kinematic information, subjects were able
trode placement strategy based on the Interna- to attain two-dimensional control after a short
tional 10/20 system as detailed in Fig. 4.4. For training (∼40 minutes).
better spatial resolution, it is also common to use a It will also be important to develop better
variant of the 10/20 system that fills in the spaces understanding of the mechanisms of information
between the electrodes of the 10/20 system with encoding in EEG signals. It has been demon-
additional electrodes. Nevertheless, EEG-based strated that detailed kinematic information, not
simply gross mental states, is represented in the
142 B. He et al.
Fig. 4.4 Placement of electrodes for noninvasive signal lobe; P: Parietal lobe; O: Occipital lobe). The number
acquisition using EEG. This standardized arrangement of or second letter identifies its hemispherical location (Z:
electrodes over the scalp is known as the International denotes line zero and refers to an electrode placed along
10/20 system and ensures ample coverage over all parts of the cerebrum’s midline; even numbers represent the right
the head. The exact positions for the electrodes are at the hemisphere; odd numbers represent the left hemisphere;
intersections of the lines calculated from measurements the numbers are in ascending order with increasing dis-
between standard skull landmarks. The letter at each elec- tance from the midline). (From [197], http://www.bem.fi/
trode identifies the particular sub-cranial lobe (FP: Pre- book/, with permission)
frontal lobe; F: Frontal lobe; T: Temporal lobe; C: Central
distributed EEG signals [54–56]. Interestingly, EEG [48]. The rationale behind this approach
brain signals recorded on the scalp surface is the linear relationship between current source
and those recorded intracranially reveal similar strength and the voltage recorded at the scalp.
encoding models [58], suggesting that knowledge Thus, one may estimate equivalent current
gleaned from invasive BCIs could be transferred density representations in regions of interest
to the understanding of EEG-based BCI signals. from noninvasive EEG or MEG recordings. He
This might further advance noninvasive BCI and colleagues proposed to use such EEG-based
technology and thereby possibly achieve source signals to classify motor imagery states for
high degrees of control and reduce training BCI purposes [59]. Such source imaging–based
requirements. approach has shown promising results based on
Source analysis has been widely used to motor imagery paradigm [43, 60–63].
estimate the sources of the brain activity that The use of source estimation in BCI applica-
produces noninvasively recorded signals such as tions involves increased computational cost due
to the need to solve the inverse problem. On the originating from the medial aspect of the poste-
other hand, such source analysis transforms sig- rior parietal cortex (PPC) was synchronized and
nals from sensor space back to source space and direction-sensitive [66]. These results in human
can lead to enhanced performance due to the use subjects are compatible with the functional orga-
of a priori information in the source estimation nization of monkey PPC derived from intracranial
procedure [13]. recordings. From the viewpoint of BCI research,
these findings may suggest new approaches for
4.3.2.2 MEG developing control signals utilizing such high-
MEG measures the magnetic induction produced frequency components in MEG, or in EEG as well
by electrical activity in neural cell assemblies. [67].
The magnetic signal outside of the head is on A merit of using MEG is that magnetic fields
the order of a few femtoteslas, one part in 109 are less distorted by the skull layer than are elec-
or 108 of the earth’s geomagnetic field. MEG is tric fields. However, studies so far have shown
commonly recorded using the SQUID (supercon- that the performance and training times for EEG-
ducting quantum interference device), in which it and MEG-based BCIs are comparable [68]. In ad-
is also necessary to provide shielding from exter- dition, the instrumentation necessary for MEG is
nal magnetic signals, including the earth’s mag- more sophisticated and more expensive than that
netic field. The SQUID MEG recording requires for EEG. These factors have tended to discourage
a laboratory setting. A modern MEG system is BCI research using MEG recording so far.
equipped with an array of up to ∼300 gradiome-
ters evenly distributed in a helmet shape with an 4.3.2.3 fMRI
average distance between sensors of 1∼2 cm. Re- Functional magnetic resonance imaging or func-
cently the feasibility of a wearable MEG system tional MRI (fMRI) [69–71] measures changes in
was reported for human use [64], although it is the blood flow (i.e., the hemodynamic response)
a technology that is still under development and related to neural activity in the brain. It sam-
currently quite expensive. ples very large numbers of spatial locations span-
MEG has similarities to EEG. MEG and EEG ning the whole brain and provides an ongoing
are, respectively, magnetic and electric fields pro- stream of information from the many measure-
duced by neuronal and synaptic activity. Both ment points at the same time. Compared to prior
methods sense synchronized brain activity. MEG methods for acquiring brain signals, fMRI there-
detects only the tangential components of a neural fore provides measurements that are highly dis-
current source, whereas EEG is sensitive to both tributed and highly parallel, on the order of mil-
tangential and radial components. Importantly, limeter resolution. For example, a modern MRI
like EEG, MEG is also a noninvasive record- scanner can currently sample from ∼216 spatial
ing technology. Studies using electrophysiologi- locations per second, each location (i.e., each
cal source imaging techniques have located com- voxel) with a dimension on the order of 3x3x3
mon cortical sources underlying the control pro- mm. In fMRI, the same volume is sampled re-
vided by the EEG- and MEG-based BCIs [63, peatedly at short, regular intervals (e.g., once
65]. Meanwhile, other investigators reported that per second) using an imaging contrast, such as
kinematic parameters are similarly represented in the blood-oxygen-level-dependent (BOLD) con-
MEG and EEG recordings, since the key informa- trast [72], that is sensitive to the hemodynamic
tion is embedded in the lower frequency ranges response. The intensities of BOLD contrast are
[55]. Nonetheless, the high-frequency informa- related to the changes in the deoxyhemoglobin
tion in MEG signals is being actively investigated concentration in the brain tissue. When neurons
for neural encoding. Notably, it was found that are activated, increases in blood flow are associ-
in human subjects who are planning a reaching
movement, the 70–90 Hz gamma-band activity
144 B. He et al.
ated with increases in local glucose metabolism 4.3.2.4 NIRS

and increases in local oxygen consumption. The Functional near-infrared spectroscopy (fNIRS) is
changes in local deoxyhemoglobin concentration another noninvasive technique. It utilizes light in
are reflected in the brightness of the MRI image the near-infrared range (700 to 1000 nm) to deter-
voxels at each time point. It has also been re- mine the oxygenation, blood flow, and metabolic
ported that a strong colocalization of fMRI acti- status of localized cortical regions. It is similar
vation and electrophysiological sources exist dur- to BOLD-fMRI in terms of the imaging contrast;
ing hand movement and motor imagery [43, 73]. that is, it measures the hemodynamic response.
fMRI imaging is thought to be quite safe. It does It can produce relatively well-localized signals
not use an exogenous contrast agent. Typically, with a spatial resolution on the order of cen-
it does not involve any invasive procedure, injec- timeters, and it provides information related to
tions, drugs, radioactive substances, or X-rays. It neural activity. However, since the images rely on
requires an instrument that provides a strong ex- the shallow-penetrating photons, NIRS operates
ternal magnetic field and radio-frequency energy effectively only for brain structures that are on
pulses. or near the brain surface. NIRS is also inherently
fMRI images can be processed in real time limited in its imaging contrast (i.e., hemodynamic
as they are collected, namely, as real-time fMRI responses), which results in a temporal resolution
(rtfMRI) [74] so that the resulting information on the order of seconds and a delay of several sec-
is immediately available and can thus be used onds for feedback. Thus, in terms of information
for feedback purpose. For example, the mental transfer rate, fNIRS-based BCIs are likely to be
states inferred from the rtfMRI can be used to less effective than BCIs based on electromagnetic
guide a person’s cognitive process or a clinician’s signals. Compared to fMRI, it stands as a com-
interventions in the case of psychiatric disorders. promise between imaging capability and practical
The advantage of using fMRI for neurofeedback usability (i.e., fNIRS is inexpensive and portable).
is the high spatial resolution and deep penetration. Its flexibility of use, portability, and affordability
The direct sampling of three-dimensional volume make NIRS a viable alternative for clinical studies
information in small voxels enables the detection and possibly for practical use.
of activity in all areas of the brain, including
deep structures such as the amygdala. In con-
trast, EEG/MEG measurements near the surface 4.3.3 Neural Signals Used by BCIs
of the head are made far from these locations
and the spatial resolution for EEG/MEG source 4.3.3.1 Sensorimotor Rhythms
imaging of deep brain activity is relatively lim- Electromagnetic recording from the brain at rest
ited. However, recent studies have suggested the exhibits endogenous oscillatory activity that is
possibility of detecting deep brain activity from widespread across the entire brain. As shown in
EEG and MEG as validated from intracranial Fig. 4.5, this activity can be split into several
recordings (see Chapter 13). bands. This spontaneous activity consists mainly
On the other hand, an essential limit of rtfMRI of oscillations in the alpha-frequency band (8–
or fMRI lies in its underlying mechanism: it mea- 13 Hz), which is called the mu rhythm when
sures changes in blood flow rather than neuronal focused over the sensorimotor cortex and the
activity. The technique is therefore inherently in- visual alpha rhythm when focused over the visual
direct and noisy. Most importantly, there is an cortex. This idling oscillation is thought to be
intrinsic delay of several seconds in the response caused by complex thalamocortical networks of
of fMRI, no matter how fast the images can be neurons that create feedback loops. The synchro-
obtained. This means that the feedback given to nized firing of the neurons in these feedback
a subject is delayed by several seconds. This loops generates observable oscillations. The fre-
could affect the usefulness of rtfMRI in many BCI quency of oscillations decreases as the number
applications. of synchronized neurons increases. The under-
Fig. 4.5 Various signal bands present in the EEG signal. in the alpha band (8–13 Hz) and the beta band (14–30 Hz).
The delta band ranges from 0.5 to 3 Hz and the theta band The gamma band, which is just beginning to be applied in
ranges from 4 to 7 Hz. Most BCI systems use components BCI, is >30 Hz
lying membrane properties of neurons, the dy- using motor imagery paradigms. Studies have
namics of synaptic processes, the strength and demonstrated that people can learn to increase
complexity of connections in the neuronal net- and decrease sensorimotor rhythm amplitude
work, and influences from multiple neurotrans- over one hemisphere using motor imagery
mitter systems also play a role in determining the strategies and thereby control physical or virtual
oscillations. devices [13, 14, 18, 29–31, 63, 81, 82].
Other oscillations detected over the sensori-
motor cortex occur in the beta frequency band 4.3.3.2 Slow Cortical Potentials
(14–30 Hz) and in the gamma band (>30 Hz). A completely different type of signal measured by
Together with the mu rhythm, these oscillations EEG is the slow cortical potential (SCP) (see Fig.
recorded over sensorimotor cortex are called 4.7) that is caused by shifts in the depolarization
sensorimotor rhythms (SMRs). They originate levels of pyramidal neurons in cortex. Negative
in sensorimotor cortex and change with motor SCP generally reflects cortical activation, while
and somatosensory function. These oscillations positive SCP generally reflects reduced activa-
occur continually during “idling” or rest. During tion. SCP occurs from 0.5 to 10 seconds after the
nonidling periods, however, these oscillations onset of an internal event and is thus considered
change in amplitude and/or frequency, and these a slow cortical potential [83]. People can learn to
changes are evident in the EEG or MEG. Task- control SCPs and use them to operate a simple
related modulation in sensorimotor rhythms is BCI [84].
usually manifested as an amplitude decrease in
the low-frequency components (alpha/beta band) 4.3.3.3 The P300 Event-Related
(also known as event-related desynchronization Potential
(ERD) [75]). In contrast, an amplitude increase The P300 is an endogenous event-related poten-
in a frequency band is known as event-related tial (ERP) component in the EEG and occurs in
synchronization (ERS) [75]. For example, it the context of the “oddball paradigm” [85]. In this
has been found that the planning and execution paradigm, users are subject to events that can be
of movement lead to predictable decreases in categorized into two distinct categories. Events
the alpha and beta frequency bands [75]. Also, in one of the two categories occur only rarely.
as illustrated in Fig. 4.6, many studies have The user is presented with a task that can be ac-
demonstrated that motor imagery can cause complished only by categorizing each event into
ERD (and often ERS) in primary sensorimotor one of the two categories. When an event from
areas [75, 77–80]. Such characteristic changes the rare category is presented, it elicits a P300
in EEG rhythms can be used to classify brain response in the EEG. As shown in Fig. 4.8, this
states relating to the planning/imagining of is a large positive wave that occurs approximately
different types of limb movement. This is the 300 msec after event onset. The amplitude of the
basis of neural control in EEG-based BCIs P300 component that is inversely proportional to
146 B. He et al.
Fig. 4.6 Event-related desynchronization (ERD) and and it can be identified by a decrease in signal amplitude.
event-related synchronization (ERS) phenomena before ERS is the result of an increase in the synchronization of
and after movement onset. ERD/ERS is a time-locked neurons, which causes an increase of power in specific
event-related potential (ERP) associated with sensory frequency bands; and it can be identified by an increase
stimulation or mental imagery tasks. ERD is the result in signal amplitude. (From Pfurtscheller and Neuper [76],
of a decrease in the synchronization of neurons, which with permission, © 2001 IEEE)
causes a decrease of power in specific frequency bands;
Fig. 4.7 Slow cortical –10

potential (SCP) signals to –7 PV
convey different intents.
SCPs are caused by shifts
in the dendritic 0 top
amplitude (PV)
depolarization levels of
certain cortical neurons. target
They occur from 0.5 to +7 PV
10 seconds after the onset 10
of an internal event and are
thus considered a slow
cortical potential. (From
bottom
Kübler et al. [83], with target
20
permission)
0 0.5 1.0 1.5 2.0

time (s)
the frequency of the rare event is presented. This 4.3.3.4 Event-Related Potentials
ERP component is a natural response and thus Exogenous event-related potentials (ERPs) are
especially useful in cases where either sufficient responses that occur in the EEG at a fixed time
training time is not available or the user cannot be after a particular visual, auditory, or somatosen-
easily trained. sory stimulus. The most common way to derive
ERP from EEG recording is aligning the signals
VOLTAGE or phase. This method does not require averaging

(a/d u) and thus can be applied to single trials. Therefore,
–50 it is useful for BCI control (although it is still
Pz
other subject to the limitations of its signal-to-noise
0 choices ratio).
desired The ERP most commonly used in BCIs is the
choices visual evoked potential (VEP), which occurs in
50
response to a visual stimulus. One frequently used
VEP is the steady-state visual evoked potential
100 (SSVEP). SSVEPs and other VEPs depend on the
user’s gaze direction and thus require muscular
TIME control. To produce such signals, the user looks at
150
0 200 400 (ms) one of the several objects on a screen that flicker
–100 100 300 500
at different frequencies in the alpha or beta bands.
Frequency analysis of the SSVEP shows a peak
Fig. 4.8 P300 ERP component. When the user sees ob-
jects randomly flashed on a screen, the P300 response at the frequency of the object at which the user is
occurs when the user sees the flash of the object the user looking. Thus, a BCI can use the frequency of this
is looking for (or wishes to select), while the flashes of peak to determine which object the user wants to
the other objects do not elicit this response. The amplitude
select [86, 87].
of the P300 component is inversely proportional to the
rate at which the desired object is presented and occurs
approximately 300 msec after the object is displayed. It 4.3.3.5 Spikes and Local Field
is a natural response and requires no user training. (From Potentials
Kubler et al. [83], with permission)
Both spikes and local field potentials are acquired
from microelectrodes implanted through invasive
according to the stimulus onset and then averag- techniques. Spikes reflect the action potentials
ing them. The number of stimuli averaged typi- of individual neurons. Since the CNS appears
cally range from a few (e.g., in BCI applications) to encode information in the firing rates of
to hundreds or thousands in other neuroscience neurons, recording spiking activity may be
research. ERPs are sometimes characterized as highly useful. Local field potentials (LFPs)
“exogenous” or “endogenous.” In general, exoge- represent mainly synchronized events (largely
nous ERPs are shorter latency and are determined in the frequency range of <300 Hz) in neural
almost entirely by the evoking stimulus, while populations. The major sources of LFPs are
endogenous ERPs are longer latency and are de- synaptic potentials (which are also the major
termined to a considerable extent by concurrent sources for EEG/MEG/ECoG). Other integrative
brain activity (e.g., the nature of the task in which somadendritic processes, including voltage-
the BCI user is engaged). dependent membrane oscillations and after-
ERPs are related to the ERD/ERS described potentials following somadendritic spikes, can
above. ERPs reflect in large part activity in the contribute to LFPs. LFPs and their different
ongoing EEG that is phase-locked by the stimuli. band-limited components (e.g., theta (4–7 Hz),
Typically, after averaging, the ERP contains in- alpha, beta, gamma) are tightly related to
formation about very low-frequency components cortical processing. Gamma-band LFP activity
(i.e., <1 Hz). Other components are canceled out is especially tightly coupled to spiking activity.
in the process of averaging across repetitions, and Because LFPs reflect signals from many neurons,
the information above 1 Hz is poorly represented. their spatial resolution (and possibly their
An alternative way to characterize task-related functional specificity) is lower than that of
EEG signals is to examine the rhythmic activity spiking activity. See next chapter for BCIs using
before averaging, in terms of power (ERD/ERS) intracortical recordings.
148 B. He et al.
4.4 Signal Processing generated by non-neural sources such as muscular

movements, particularly of the facial muscles.
The goal of BCI signal processing is to extract This type of noise in EEG is especially important
features from the acquired signals and translate as signals generated by muscular movements may
them into logical control commands for BCI ap- have much higher amplitudes and can easily be
plications. A feature in a signal can be viewed as mistaken for actual EEG activity. The problem
a reflection of a specific aspect of the physiology is further complicated when the frequencies and
and anatomy of the nervous system. Based on this scalp locations of the non-neural noise and the
definition, the goal of feature extraction for BCI chosen EEG features are similar.
applications is to obtain features that accurately Typically non-CNS artifacts are the result
and reliably reflect the intent of the BCI user. of unwanted potentials from eye movements,
EMG, and other non-neural sources. They are
often more prominent in the EEG than brain
4.4.1 Feature Extraction signals. Simple instructions to the user to not use
facial muscles can help and trials that contain
The goal of all processing and extraction tech- such artifacts can be disregarded, but these
niques is to characterize an item (i.e., the desired approaches are not always adequate to remove
user selection) by discernible measures whose such noise. Mathematical operations such as
values are similar for those in the same category linear transformations and component analyses
but different for items in another category. Such are also used for artifact removal.
characterization is accomplished by choosing rel- After artifact removal, spatial filtering tech-
evant features from the numerous choices avail- niques are useful for enhancing features with a
able. This selection process is necessary since specific spatial distribution. In BCI systems that
unrelated features can cause the translation al- use mu or alpha rhythms, the selection of spatial
gorithms to have poor generalization, increase filters can greatly affect the signal-to-noise ratio
the complexity of calculations, and require more [88]. A high-pass spatial filter such as the bipolar
training samples to attain a specific level of accu- derivation calculates the first spatial derivative
racy. and emphasizes the difference in the voltage gra-
In addition, even though a BCI user is able dient in a particular direction. The surface Lapla-
to generate detectable signals that convey her or cian [89, 90] also acts as a high-pass filter and can
his intent, signal acquisition methods also capture be approximated by subtracting the average of the
noise generated by other unrelated activity in or signal at four surrounding nodes from the signal at
outside of the brain. Thus, it is important that the node of interest. It is the second derivative of
feature extraction maximize the signal-to-noise the spatial voltage distribution and thus is effec-
ratio. tively a spatial high-pass filter that emphasizes the
contributions from the neural areas closest to the
4.4.1.1 Artifact/Noise Removal recording electrode (node of interest) [91]. Spline
and Signal Enhancement functions can be used to more accurately estimate
Artifact or noise removal plays an important the surface Laplacian from EEG recordings [92],
role in EEG-based BCIs. Since signals are often but in most BCI applications finite difference
captured across several electrodes over a series estimates are used from EEG recordings in a few
of points in time, existing methods concentrate electrodes due to computational efficiency.
on either spatial-domain processing or temporal- Temporal-domain processing techniques are
domain processing or both. To minimize noise also useful in maximizing the signal-to-noise ra-
in the signal, it is important to understand its tio. These methods work by analyzing the signal
sources. First, noise can be captured from neural across a period of time. Some temporal-domain
sources when brain signals not related to the processing methods such as Fourier analysis re-
target signal are recorded. Noise can also be quire significantly long signal segments, while
others such as band-pass filtering or autoregres- 4.10), or a TFR method based on wavelets (Fig.
sive analysis can work on shorter time segments. 4.11). Parametric approaches are also commonly
Though all temporal-domain processing methods used to estimate the time/frequency features, such
work well during offline BCI analysis, some of as autoregressive (AR) modeling for stationary
them are not as useful as spatial-domain process- signals and adaptive autoregressive modeling for
ing methods during online analysis because of the nonstationary signals, which are widely imple-
rapid responses required. mented in online BCI systems due to their com-
putational efficiency. However, it is worth noting
4.4.1.2 Feature Extraction Methods that such parametric modeling approaches usu-
The methods for extracting features depend ally require predetermined parameters, such as
largely on the type of neural signals used in the model order [95], which can influence BCI
the BCI and the characteristics associated with performance.
the underlying neural process. For neural signals Neural network, especially deep neural net-
representing mass responses of a large number of work (or deep learning), is attracting more and
neurons (EEG/MEG/ECoG), defining features by more attention for feature extraction and feature
spatial location is as important as defining them translation. An early effort was to use neural net-
by temporal/spectral characteristics. In order to works for classifying motor imagery tasks [96].
optimize the spatial information, the channels Several studies using deep learning approaches
used for BCI control are usually a selected subset showed moderate success on offline analysis of
of a few channels. These can be selected with existed public BCI data sets [97, 98]; however,
methods such as principal components analysis the effectiveness has to be further validated by
(PCA), common spatial pattern analysis (CSP) more extensive online experiments. On the con-
[93], and independent component analysis (ICA) trary to the moderate success in conventional
[94], or based on a priori knowledge of the BCI applications, the neural network approach
functional organization of the relevant cortical seems to be more successful in the speech BCI.
area(s). Electrophysiological source imaging Angrick et al. designed a densely connected 3D
(ESI) methods have also been proposed as a convolutional neural networks to reconstruct the
spatial deconvolution approach to extracting spoken words from ECoG signals in the auditory
spatial information about the features used in cortex and obtained relatively high-quality speech
a BCI [13, 59–63]. [41]. Akbari et al. used a deep neural network
In order to define the temporal/spectral param- to estimate the parameters of a speech vocoder
eters of the chosen features, the neural signals directly and achieve relatively high performance
are usually subjected to time-frequency analysis. on a digit recognition task [36]. Instead of directly
Frequency-based features have been widely used decoding the parameters of a speech synthesizer
in signal processing because of their ease of ap- from the ECoG signals, Anumanchipalli and col-
plication, computational efficiency, and straight- leagues used a two-stage approach to solve the
forward interpretation. Because these features do problem. They first decoded the articulatory kine-
not provide time-domain information, they are matic features from the continuous ECoG sig-
not sensitive to the nonstationary nature of EEG nals by training a recurrent neural network. Then
signals. Thus, mixed time–frequency representa- they translated the kinematic features into the
tions (TFRs) that map a one-dimensional signal acoustic sound via a general model, which map
into a two-dimensional function of time and fre- the recorded speech into the movements of the
quency can be used to analyze the time-varying vocal-tract articulators via a recurrent neural net-
spectral content of the signals. A typical example work by their previously accumulated data [11].
is the extraction of the ERD feature in senso- They showed successful reconstruction efficacy
rimotor rhythms, which can be obtained using in closed vocabulary tests, and human listeners
a traditional moving-average method (as shown could identify and transcribe the reconstructed
in Fig. 4.9), an envelope-extraction method (Fig. speech. Further investigation is needed to delin-
150 B. He et al.
Fig. 4.9 Techniques used to extract ERD and ERS from samples are averaged across all trials. Finally, variability is
raw EEG signals. First, the raw EEG signal from each reduced and the graph is smoothed by averaging over time
trial is band-pass filtered. Second, the amplitude samples samples. (From Pfurtscheller and Lopes da Silva [75], with
are squared to obtain the power samples. Third, the power permission from Elsevier)
eate the sources of these successes, that is, due to method. The feature selection procedure adds or
the deep learning algorithms or due to the use of removes features to counteract prediction errors
invasive ECoG signals (vs. EEG). as new training data are introduced. Embedded
algorithms, however, are of little use when there
4.4.1.3 Feature Selection is a high level of interaction among relevant
and Dimensionality Reduction features.
Feature selection algorithms are used in BCI In the second category, filter algorithms, spe-
designs to find the most informative features cific features are selected prior to, and indepen-
for determining the user’s intent. This approach dent of, the translation process. These algorithms
is especially useful for BCI designs with work by removing irrelevant features (those pro-
high-dimensional input data, as it reduces the viding redundant data or contaminated by noise)
dimension of the feature space. Since a feature prior to training the translation technique. One
selection block reduces the complexity of the approach to filtering involves calculating each
translation problem, higher translation accuracies feature’s correlation with the user’s intent and
(i.e., higher accuracies of determining the user’s then selecting a fixed number of features with the
intent) can be achieved. highest scores. Another filtering approach derives
As discussed by Blum and Langley [99], higher-order features based on features from the
feature selection techniques can be divided into raw data, sorts these higher-order features based
three major categories. In the first category, called on the amount of variance they explain, and then
embedded algorithms, the feature selection is a selects a fixed number of the highest-scoring fea-
part of the translation (also called classification) tures.
Raw EEG For certain situations, existing signals are not

sufficient for high accuracy feature extraction.
Some methods introduce more signals to cap-
ture additional information about the state of the
brain (e.g., by using 56 electrodes where only 2
were previously used). For example, the increased
Laplacian EEG spatial data can be processed to derive common
spatial patterns. This is achieved by projecting
the high-dimensional spatiotemporal signal onto
spatial filters that are designed such that the most
discriminative information is inherent in the vari-
ances of the resulting signals [100].
12Hz EEG and Envelope
4.4.2 Feature Translation
Translation techniques are algorithms developed

with the goal of converting the input features
(independent variable) into device control
Averaged Envelopes
commands (dependent variables) that achieve
the user’s intent [10]. Translation techniques
used widely in other areas of signal processing
are adapted to BCI technology. Ideally, the
translation algorithm will convert the chosen
–1 0 1 2 3 features into output commands that achieve the
Time (Second) user’s intent accurately and reliably. Furthermore,
an effective translation algorithm will adapt so as
Fig. 4.10 Steps of feature extraction for sensorimotor to adjust for spontaneous changes in the features
rhythms. It is difficult to detect a coherent component in
the raw EEG signal depicted in the top frame because there and will also encourage and facilitate the user’s
is a lot of noise in the signal. The second frame shows the acquisition of better control over the features.
signal after being processed through a surface Laplacian There are numerous types of feature transla-
filter that focuses on EEG components in a specific spatial tion algorithms. Some use simple characteristics
frequency range. As shown in the third frame, the signal
is then band-pass filtered to isolate the frequencies of such as amplitude or frequency, and some use
interest. The features become evident in the fourth frame single features. Some advanced algorithms utilize
as they are extracted by using a grand averaging method a combination of spatial and temporal features
over a fixed bin or window size produced by one or more physiological processes.
Algorithms currently in use include, but are not
The final category consists of wrapper algo- limited to, linear classifiers, Fisher discriminants,
rithms. Wrapper algorithms select features by Mahalanobis distance-based classifiers, neural
using the translation algorithms to rate the vi- networks (NN), support vector machines (SVM),
ability or quality of a feature set. Rather than hidden Markov models, and Bayesian classifiers.
selecting a feature set based on the results of A thorough literature review for classification
the translation, these algorithms use the trans- algorithms of EEG-based BCI has recently
lation algorithm as a subroutine to estimate the been carried out [101]. Lotte et al. summarized
accuracy of a particular subset of features. This the newly developed feature translation or
type of algorithm is unique to a translation algo- classification methods including the adaptive
rithm and particularly useful with limited training classifier, matrix and tensor classifier, transfer
data. learning and deep learning besides the previously
152 B. He et al.
Hz Hz
10 10
20 20
C3
C3
1
30 30
10 10
C4 0
C4
20 20
30 30
–2 –1 0 1 2 3s –2 –1 0 1 2 3s
Fig. 4.11 Time-frequency representations (TFRs) of sen- were normalized to be 2 s (solid line). (From Yuan et al.
sorimotor rhythms during motor imagery. TFRs were re- [63], with permission from IEEE, © 2008 IEEE)
aligned at time = 0 s (dashed line) and the target times
commonly used linear classifier, nonlinear of BCI paradigms including motor imagery,
Bayesian classifier, classifier combinations, ERPs, and SSVEP-based BCI. The covariance
etc. [101]. The adaptive classifier seems to matrix of EEG signals during the BCI task
provide superior performance to static ones contains abundant task-related information.
in general. This is intuitive since the EEG The Riemannian geometry–based methods map
signals are nonstationary signals and adaptive the covariance matrix of EEG trials into the
approaches are better at tracking the changes in geometrical space and the computation is in a
the dynamic process than the static approaches. Riemannian manifold, which is a non-Euclidean
However, since BCI systems are a two-learners space [103]. The covariance matrix of EEG
system, that is, the human and the machine, the signals could be treated as the notion of the
adaptation frequency might be critical. Either traditional basic data points. Thus, the ideas of
too fast adaptation or too infrequent adaptation the center of mass and nearest neighbors could
might be detrimental to the BCI system [26– be applied intuitively in the geometrical space.
28]. A good amount of comparisons between The previous research result of the Riemannian
adaptive and static classifiers in the literature is approach showed good robustness to noise [104].
offline analysis or comparison within a single Further investigations and especially under real-
session. Thus, the superiority of using adaptive time experimental settings are warranted to
classifiers in many studies probably does not validate the efficacy.
account for the learning process of subjects Whatever translation algorithm is used, the
[102]. More careful investigation has to be outcomes of translation can be control commands
conducted to clarify the conditions further when in two ways: continuous or discrete. The follow-
adaptive classifiers improve both the subject’s ing section details the difference between these
learning and the system’s performance. Transfer two ways of translation.
learning and deep learning methods also show
improvements in certain cases, but their benefits 4.4.2.1 Continuous Feature Translation
remain uncertain yet. Transfer learning might be In continuous feature translation, consecutive
good when building a general model from a large output commands are generated continually
population of participants. It might decrease or based on the features. Examples of this translation
eliminate the tedious or costly training period. are the kinematic parameters (arm position,
Deep learning showed remarkable success in velocity, etc.) that control a prosthetic arm.
the speech BCI recently; however, whether The features are usually derived from short-time
it provides superior performance in more windowed signals and are then continuously fed
general applications needs further investigation. into the translation algorithm so that dynamic
Particularly, the Riemannian geometry–based outcomes are obtained for BCI control. A fixed
method seems to work very well in a variety translation algorithm can be used for continuous
feature translation. Algorithms that adapt can build an alternative control channel that may be
often yield better performance. Due to the faster and more accurate than current BCIs driven
demands of processing the features in consecutive by neural signals. Thus, BCIs are particularly
short-time windows, the choice of feature needed for users who lack all muscle control
extraction methods and translation methods or whose remaining control is easily fatigued
should favor those with less computational load, or otherwise unreliable. These people include
which may not be those algorithms that perform those who are nearly totally paralyzed but retain
best in offline testing. However, the advantage cognitive function (e.g., people with advanced
of using continuous translation is that it allows ALS) and those who have movement disorders
the users to adjust their strategies in the course of that abolish useful muscle control (e.g., people
control. This is beneficial for learning by the user with severe cerebral palsy). Although people with
as well as by the BCI. these disorders may have lost the ability to control
any muscle movement, their cognitive function
4.4.2.2 Discrete Feature Translation may still be intact and they may therefore have
In contrast, discrete feature translation produces the potential to control a BCI and use it to com-
periodic commands at fixed intervals. An exam- municate. For these locked-in people, conven-
ple of this type of translation is a BCI that uses tional communication methods based on muscle
a P300 signal. A P300-based BCI will typically activity may have little to offer them so that
issue a command every several seconds. Thus, even the simplest BCI-based communication, like
it is particularly suited for applications such as the ability to say yes or no, can be extremely
word processing, which requires discrete letter valuable.
selections, and less suited for applications such Thus far, most current BCI research has been
as multidimensional robotic arm control, which is carried out in healthy subjects. A few studies have
best implemented by a continuous series of output been conducted to test the feasibility of BCI com-
commands. munication in severely disabled people in labora-
tory settings or even in their homes. The transfer
of current BCI communication systems into use
4.5 Major BCI Applications by severely disabled people for useful purposes
faces several challenges. First, the disease states
4.5.1 Replacing Lost that abolish voluntary muscle control may also
Communication impair user control of the signal features used by
a BCI. For example, ALS may lead to loss of
An important application for BCI technology is cortical neurons, which might conceivably affect
providing a new method for communication so generation or control of the sensorimotor rhythms
that a person who has lost normal means of com- or evoked potentials used for BCI-based commu-
munication can interact with his or her exter- nication. Thus, it may be important to develop
nal environment. Current BCIs are suitable for diverse BCI systems that are based on various
environmental control (e.g., temperature, lights, types of neural signals so that more options can be
television), for answering yes/no questions, and provided for different types of brain impairments.
for simple word processing or e-mailing. Furthermore, damage to prefrontal cortex (e.g., in
While such communication can be provided multiple sclerosis, Parkinson’s disease, or ALS)
through brain control, there are alternative op- can impair attention and thereby adversely affect
tions not involving neural signals. Those who BCI use. For these users, a long-duration training
retain the control of only a single muscle can protocol may be problematic. Thus, for these
often use this for communication. For example, users, BCI systems that require minimal train-
the electric activity associated with finger mus- ing, such as SSVEP-based systems, may be most
cles, eyebrows, or the diaphragm can be used to suitable.
154 B. He et al.
4.5.2 Replacing Lost Motor for and using BCIs can lead to changes in neu-
Function and Promoting ral activity that facilitate the use of prosthetic
Neuroplasticity to Improve devices, especially when combined with func-
Defective Function tional electric stimulation (FES) [106, 107]. Such
learning-related changes are especially important
Perhaps the highest degrees of control achieved so for people with brain injuries, such as those who
far in BCI development is with neuroprostheses have suffered from stroke [20, 108]. In a study us-
developed for restoring motor function. The state- ing MEG recordings, patients with chronic hand
of-the-art in movement control is multidimen- hemiplegia after stroke successfully learned to
sional and point-to-point (and continuous) control use motor imagery to control their sensorimotor
of a robotic arm. In humans, sensorimotor rhythm rhythms, and they were able to use a BCI to
modulation based on noninvasive EEG record- control an orthotic device that opened and closed
ings has demonstrated three-dimensional control their paralyzed hands [109]. As shown in Fig.
of a computer cursor [81, 105] or continuous 4.12, subjects’ performances steadily improved
real-time flight control of a virtual helicopter as they learned to use the device. Comparison
[29, 31] or physical quadcopter [30], or real- between the early and late training stages re-
time operation of a powered wheelchair [12], or vealed enhanced sensorimotor rhythms in the ip-
continuous control of a robotic arm [13, 14]. A silesional hemisphere, which was the hemisphere
direct decoding of three-dimensional movement used to control the device. Several randomized
trajectory from human EEGs has also been re- controlled studies have indicated that assisting
ported [54]. Such replacement of motor function movement with FES coupled to BCI use can
could be valuable for patients who suffer from substantially improve upper-limb function in in-
various degrees of paralysis. It is estimated that dividuals who have been mildly to moderately
there are currently over two million people in [110] or severely [20, 108, 111] impaired by
the United States alone suffering from paralysis. stroke. Studies with both invasive and nonin-
Additionally, every year there are approximately vasive BCIs also indicate that learning-related
12,000 new cases of spinal cord injury in the changes can occur over days to months [26, 102].
United States. The list of causes of paralysis Interestingly, once users have learned to operate
is extensive and includes stroke, cerebral palsy, a neuroprosthesis with a BCI, they retain this
ALS, multiple sclerosis, muscular dystrophies, skill months later without intervening use [18],
trauma, and other neurodegenerative conditions. suggesting a long-term learning-related change
Many individuals suffer from permanent loss of in neural circuits. Thus, BCIs might be used to
motor function. A neuroprosthesis, therefore, of- help actually restore motor function by promot-
fers an opportunity to get back a useful substitute ing beneficial neuroplasticity in neuromuscular
for normal motor control. While conventional pathways.
options based on limited muscle activity may
also provide such function, BCI-operated neuro-
prostheses could provide an embodied prosthetic 4.5.3 Supplementing Normal
control that is directly related to the user’s in- Function
tention. For example, when users want to move
their arms, they could instead move a robotic arm BCI technology may also be used to supplement
by communicating with the BCI their intention to normal neuromuscular function. This is particu-
move their own arms. They would not have to use larly true when considering BCI applications for
different muscle activity, such as eye-blinking, to use in the daily life of healthy individuals for the
move a robotic arm. purpose of enhancing quality of life or functional-
Another exciting possible application of BCI ity. One potential application is to aid navigation
technology is promoting neuroplasticity to restore by means of BCI use. Controlling a computer
lost function. Studies have shown that training cursor represents one such application aimed not
(a) 0.9 (b) (c) (d)

0.4
0.8
sucessful trials
Proportion of
0.7
0.6 R2
0.5
0.4
0 0.0
0.9
0.8
sucessful trials
Proportion of
0.7
0.6
0.5
0.4
0
0.9
0.8
sucessful trials
Proportion of
0.7
0.6
0.5
0.4
0
0 10 20
Training day
Fig. 4.12 Patients with chronic hand hemiplegia after correlations between sensorimotor mu rhythm amplitudes
stroke were trained to move a cursor on a screen via mod- from signals recorded from sensors above the ipsilesional
ulation of ipsilesional sensorimotor mu rhythm recorded primary motor cortex, and successful performance at b
by MEG. Successful trials with the BCI resulted in the (early) or c (late) training time points demonstrate mod-
opening or closing of the patient’s paralyzed hand via a ulation of sensorimotor rhythms with BCI training. Red
mechanized orthosis. This figure shows the results from and yellow colors identify areas where there was a high
three patients. (a) The performance of these patients degree of correlation. (d) Single axial MRI scans obtained
across sessions indicates that the proportion of successful for each patient. Each patient’s lesion is highlighted in red.
trials increased over time. The statistical maps for the (From Dimyan and Cohen [198], with permission, © 2011
Nature)
only at helping disabled people to gain control tual campus [29, 31] or physical campus [30], and
of external devices, but also serving as a means recently, real-time controlling of a robotic arm
for healthy individuals to control external devices [13, 14].
without using normal neuromuscular channels. A challenge in using BCI technology to sup-
Studies have shown promise in accomplishing plement normal function is the limited informa-
navigation in a virtual world, including moving tion transfer rate compared with that of normal
a computer cursor [18, 81], walking in a virtual muscular control. A healthy subject will prefer
world [112], continuous real-time controlling of manual typing over BCI use to accomplish that
flight of a helicopter in a three-dimensional vir- task. BCI might provide an additional degree of
156 B. He et al.
freedom, such as a third arm control [19]. In 4.5.5 Providing Neurofeedback

some certain cases, BCI might support some tasks
that need more than two hands and the accuracy Neurofeedback could be dated back with
is not a critical issue; thus, it might be benefi- experiments showing that humans could self-
cial to the healthy population. Nevertheless, BCI control electroencephalographic signals in real
technology controls may meet the need for cases time [119]. An essential part of a typical BCI
in which high information transfer rate is not system is providing neurofeedback, which
an essential factor and nonmuscular control is is then translated into a control command
desirable. interacting with a peripheral device such as a
computer cursor [18], a quadcopter [30], or
a robotic arm [14]. As a progenitor of BCI
4.5.4 Augmenting/Virtualizing technology, providing neurofeedback could be
Reality with BCI used for self-modulating the psychophysiological
signals in the brain for self-regulation instead of
The development of virtual reality (VR) and aug- commanding peripheral devices [120, 121]. In
mented reality (AR) gives researchers better tools the research field of adaptive neurofeedback, the
in the end-user interaction [34, 113, 114]. The brain activation is treated as the independent
combination of BCI with VR/AR might result in variable and the behavior and thought are
better users’ embodiment and engagement. Espe- treated as dependent variables. It could open an
cially in certain conditions such as stroke reha- exciting field of innovative treatment for patients
bilitation, VR/AR may play a unique and vital with psychopathological conditions such as
role [115]. Patients who lose their ability to move attention deficit disorder [17, 122], etc. The BCI
might struggle to perform motor imagination like technology might enhance the cognitive function
healthy participants [116]. In VR, an avatar is of the aging population [123] or provide novel
easily created and the avatar might induce a per- approaches to improve the sustained attention
ception illusion of the body ownership in certain status, for example, providing a more sensitive
conditions [117]. This included perception of im- feedback signal such that users can learn to sense
mersion might be facilitated to the neural rehabil- upcoming attentional lapses earlier and prevent
itation since this change of perception alters the them from manifesting in behavior [17].
underlying cognitive process. Bermudaz and col- The long-term effect of neurofeedback and the
leagues used a first-person perspective VR in their transfer benefits in clinical treatment are still un-
BCI system, and they combined a personalized known. Furthermore, the causal brain–behavior
training in the virtual environment as well [118]. relationship, which might help to understand the
Their data showed users’ enjoyment and engage- underlying neural mechanism of neurofeedback,
ment for the BCI-VR system in a group of healthy is needed. Thus, further investigations of these
subjects, although Coogan et al. [34] did not ob- questions using a more rigorous experimental
serve improved performance in a group of healthy design, for example, excluding the placebo effect,
subjects with VR setting as compared to a tradi- should be performed [121].
tional setting. In their studies, Johnson et al. [115]
showed a substantial improvement of behavior in
motor recovery when using BCI and VR in stroke 4.6 Examples of EEG-Based BCI
subjects. Although the combination of BCI and Systems
VR seems promising in some applications such
as stroke rehabilitation, due to the few numbers of With the growing kinds and combinations of sig-
subjects in previous literature, further studies with nals, feature extraction methods, and translation
a larger scale of the subject population need to be techniques, the number and variety of different
performed. BCI systems are increasing rapidly [124]. Basic
research typically starts using offline analyses, arise in trying to integrate hardware and software
where signal acquisition is followed by feature from different sources. As more BCI paradigms
extraction and translation as a separate step. This are proposed, it is very useful to have a general
type of BCI simulation allows researchers to re- software platform for comprehensive evaluation
fine and test extraction and translation algorithms of different BCI methodologies.
before testing them in actual online use. On the Such a general platform should readily support
other hand, ultimately, any new BCI technique different BCI methodologies and facilitate the
needs to be tested online to assess its perfor- interchange of data and experimental protocols
mance. [126].
A useful categorization of BCI systems is
external versus internal. External BCI systems, BCI2000 Perhaps the most widely used general-
also known as exogenous BCI systems, classify purpose software platform for BCI research is
based on a fixed temporal context in regard BCI2000 (http://www.bci2000.org/). BCI2000
to an external stimulus not under the user’s was developed and is being maintained by
control. These systems use brain signals evoked the BCI laboratory at the Wadsworth Center,
by external stimuli, such as VEPs. These BCI New York State Department of Health, Albany,
systems do not require extensive training but do New York, USA, in collaboration with the
require a controlled environment and stimulus. University of Tübingen in Germany [127]. Figure
Internal BCI systems, also known as endogenous 4.13 shows the overall structure of BCI2000.
BCI systems, on the other hand, classify based on It consists of four modules (Source, Signal
a fixed temporal context with regard to an internal Processing, User Application, and Operator
event. These systems use brain signals evoked by Interface) that communicate with each other.
tasks such as motor imagery and usually require BCI2000 supports the incorporation of different
significant user training. data acquisition hardware, signal-processing
In another widely accepted BCI categorization routines, and experimental paradigms. BCI
as proposed by Zander et al. [125], the BCIs researchers can use it to start their research
are categorized as active, reactive, and passive. quickly and effectively. The se of BCI2000 is
An active BCI is a BCI that derives its outputs free for academic and research institutions. A
from brain activity that is directly consciously detailed description of the BCI2000 software
controlled by the user, independently from exter- platform and its practical applications can be
nal events; a reactive BCI is a BCI that derives found in Schalk et al. [127].
its outputs from brain activity arising in reaction
to external stimulation, the user indirectly mod- OpenViBE It is another popular open-source
ulates that; a passive BCI is a BCI that derives BCI platform that has grown fast in recent years
its outputs from arbitrary brain activity without [128]. OpenViBE is a C++ based software
the purpose of voluntary control, for enriching a platform designed for real-time processing
human–computer interaction with implicit infor- of biosignal data. The key features of the
mation [125]. platform are (i) modularity and reusability. The
platform consists of a set of software modules
devoted to data acquisition, signal processing,
4.6.1 General-Purpose Software and visualization, as well as to the interaction
Platforms for BCI Research with virtual reality (VR). (ii) The platform is
designed for different types of users, including
With the advances in BCI research and devel- BCI researchers, clinicians, VR developers, etc.
opment that have taken place during the past (iii) The platform operates independently from
decades, the number of laboratories conducting different software targets and hardware devices.
BCI research has grown substantially. However, (iv) The platform can be integrated with high-
when building new BCI systems, problems often end VR applications. Meanwhile, its graphical
158 B. He et al.
Fig. 4.13 BCI2000 design. BCI2000 consists of four During operation, information (i.e., signals, parameters, or
modules: Operator, Source, Signal Processing, and Appli- event markers) is communicated from the Source module
cation. The Operator module acts as a central relay for sys- to the Signal Processing to the User Application module
tem configuration and online presentation of results to the and back to the Source module. (From Schalk et al. [127],
investigator. It also defines onset and offset of operation. with permission)
language for designing signal-processing chains tion (LVQ) method. LVQ is a vector quantiza-
is attractive [129]. tion method in which the high-dimensional input
space is divided into different regions with each
region having a reference vector and a class label
4.6.2 BCIs Based on Sensorimotor attached. During feature translation, an unknown
Rhythms input vector is classified by assigning it to the
class label of the reference vector to which it is
Wolpaw and coworkers developed a BCI system closest [131].
that allows users to control to move a computer He and colleagues investigated the possibil-
cursor in one, two, or three dimensions. The EEG ity of using BCI control based on sensorimotor
is recorded as the users actively controlled mu rhythms for continuous navigation of an object
and/or beta rhythm power (amplitude squared) at in a virtual three-dimensional world [29, 31],
one or several specific electrode locations over or physical world [13, 14, 30]. Control signals
sensorimotor cortex. The EEG power spectra are were derived from motor imagery tasks, and in-
calculated by an autoregressive method to gen- telligent control strategies were used to improve
erate the feature vector [18, 81]. This method the performance of navigation. By using a con-
provides multidimensional control that is compa- stant forward flying velocity, three-dimensional
rable in speed and accuracy to that achieved to navigation was reduced to two-dimensional nav-
date in humans with microelectrodes implanted igation, which allowed human subjects to fly
in cortex [130]. a virtual helicopter to any point in the three-
Pfurtscheller and coworkers developed a BCI dimensional space [31]. Further studies have en-
system that used mu rhythm EEG recordings mea- abled human subjects to perform fast, accurate,
sured over sensorimotor cortex. The raw EEG sig- and continuous control of a virtual helicopter in
nals were filtered to yield the mu band (8–12 Hz) three-dimensional space [29]. In this BCI sys-
and then squared to estimate the instantaneous tem, the virtual helicopter’s forward-backward
mu power. Five consecutive mu-power estimates translation and elevation controls were actuated
during ERD were combined to create a five- through the modulation of sensorimotor rhythms
dimensional feature vector that was classified us- that were converted to forces applied to the virtual
ing one-nearest neighbor classifier with reference helicopter at every simulation time step, and the
vectors generated by a learning vector quantiza- helicopter’s angle of left or right rotation was
linearly mapped, with higher resolution, from Using the combination of two sequential low
sensorimotor rhythms associated with other mo- dimensional controls, efficient control of a
tor imaginations. These different resolutions of robotic arm for performing tasks requiring
control allow for interplay between general intent multiple degrees of freedom was achieved.
actuation and fine control as is seen in the gross Additionally, the participants maintained their
and fine movements of the arm and hand. Subjects ability to modulate their brain rhythms to
controlled the helicopter with the goal of fly- control the robotic arm over multiple months.
ing through rings (targets) randomly positioned It showed the potential of human operation of
and oriented in a three-dimensional space. After prosthetic limbs using noninvasive EEG-based
establishing the technique, He and colleagues BCI technology. Later on, Edelman et al. [13]
further demonstrated that human subjects could presented a noninvasive framework using EEG
fly a physical quadcopter to any point in a 3-D to achieve the continuous control of a robotic
real world using control of EEG signals recorded arm for random target tracking. Their continuous
from scalp [30]. Figure 4.14 illustrates the study pursuit task and associated training paradigm
design where a sitting subject performs multidi- promoted the participant’s engagement; this
mensional control of the flight of a quadcopter to enhanced engagement demonstrated nearly 60%
fully explore an unconstrained 3-D space to any of behavioral improvement for traditional center-
target point in the 3-D space. out tasks and more than 500% improvement in the
In another study, Meng et al. demonstrated proposed continuous pursuit task. Additionally,
that healthy human subjects could operate a the noninvasive electrophysiological source
robotic arm to reach and grasp objects in a imaging approach further improved the BCI
complex 3-D environment using only their control compared to the traditional technique in
thoughts through motor imagination [14]. sensor space. Such advances in the noninvasive
Fig. 4.14 A diagrammatic representation of an EEG- by the control signal, which is sent regularly through
based BCI system for control of a quadcopter. The bio- WiFi. At the same time, a camera that is mounted in the
electric signal generated from motor imaginations of the quadcopter sends the video images to the computer as
hands is represented in the background of the figure. The well. The subject adjusts control and adapts to the control
signal is acquired through the amplifiers in the subjects’ parameter of the system based on the visual feedback
workstation where it is then digitized and passed to the from the video. Restoration of autonomy and the ability
computer system. The raw signal is processed in real time to freely explore the world are the driving factors for the
in the computer. The movement of the quadcopter is driven development of the system. (From LaFleur et al. [30],
licensed under CC BY 3.0)
160 B. He et al.
Fig. 4.15 EEG BCI control of a robotic arm in humans. Comparing BCI performance of robotic arm and virtual
By integrating both the user and machine learning aspects cursor control demonstrated the ease of translating neural
of BCI technology, continuous control of a robotic arm control of a virtual object to a realistic assistive device
has been demonstrated using EEG source imaged signals. useful for clinical applications. (From Edelman et al. [13]
with Permission)
robotic arm control promise major impacts on by the target stimulus, the computer is able, after
the eventual development and implementation of a sufficient number of repetitions, to identify the
neuroprosthetic limbs. Figure 4.15 illustrates the row and column that evoke a P300 response. The
BCI control of the robotic arm for continuous item at the intersection of this row and column
tracking of a computer cursor from EEG source is recognized as the target item, that is, the item
imaged signals in human subjects. desired by the user.
P300-based BCIs have been tested in severely
disabled people [132]. Current research focuses
4.6.3 BCIs Based on P300 on improving system performance such as speed,
accuracy, consistency, and user comfort [133–
The P300-BCI has now become one of the widely 136]. Hong et al. [137] proposed a new type of
used and successful BCI paradigms. The P300 is BCI speller (i.e., the N200-speller) that uses a
a positive deflection in the ERP, with a latency motion-onset visual ERP component. This sys-
of 200 to 700 ms after stimulus onset (see Fig. tem has the advantage of lower luminance and
4.8). The response is elicited when subjects attend contrast thresholds and thus reduces the discom-
to a sequence of stimulus events, including an fort of bright stimuli.
infrequently presented target (i.e., the “oddball”)
event. The P300 response is typically recorded
over central-parietal areas. 4.6.4 BCIs Based on Visual Evoked
Most P300-BCIs use the visual P300 ERP with Potentials
the row/column paradigm (RCP) [6, 38]. In the
RCP, a matrix (e.g., 6 × 6 cells) containing the Among noninvasive EEG-based BCIs, systems
alphabet, numbers, and other items is presented based on visual evoked potentials (VEPs) have
to the user for selection. The rows and columns been studied extensively. VEPs recorded over oc-
of the matrix flash in a random order (see Fig. cipital areas are triggered by the sensory stimula-
4.16). The subject attends to the desired item tion of a subject’s visual field. VEPs reflect visual
letter and counts how many times the row and information-processing mechanisms in the brain.
column containing it flashes. Since P300 poten- The stimulation of the central visual field evokes
tials are prominent only in the responses elicited larger VEPs than does peripheral stimulation. A
Fig. 4.16 Classical visual P300-based BCI: the row/col- column containing the item the BCI user wishes to select)
umn paradigm. The rows and columns of the matrix flash has a 1/6 probability of appearing
in random order. The infrequent event (i.e., the row or
VEP-based BCI is a tool that can identify a target which is frequency locked to the flickering
on which a user is visually fixated via the analysis target. As such, f-VEP BCIs are often referred
of concurrently recorded EEG. In a VEP-based to as SSVEP BCIs. Target identification can be
BCI, each target is coded by a unique stimulus achieved through power spectral analysis. In past
sequence, which in turn evokes a unique VEP decades, the robustness of f-VEP BCI systems
pattern. To ensure reliable identification, VEPs has been convincingly demonstrated in many
derived from different stimulus sequences should laboratory and clinical tests. The advantages of an
be orthogonal, or near orthogonal, to each other f-VEP BCI include simple system configuration,
in some transform domain (e.g., the frequency little or no user training, and high information
domain). transfer rate (ITR) (30–60 bits/min).
Stimulus sequence design is an important As shown in Fig. 4.17b [144], in time-
consideration for an SSVEP-based BCI. modulated VEP (t-VEP) BCIs, the flash
Depending on the specific stimulus sequence sequences of different targets are mutually
(i.e., the modulation approach) used, current independent. This may be achieved by requiring
SSVEP-based BCIs fall into four categories: that flash sequences for different targets are
frequency-modulated VEP (f-VEP) BCIs [138, strictly nonoverlapping, or by randomizing the
139]; time-modulated VEP (t-VEP) BCIs [140, duration of ON and OFF states in each target’s
141]; code-modulated VEP (c-VEP) BCIs [142]; flash sequence. The briefly flashed stimuli
and phase-modulated VEP BCIs (p-VEP) [87, elicit visual evoked potentials, which have short
143]. latencies and durations.
As shown in Fig. 4.17a [144], each target In a t-VEP BCI, a synchronous signal must be
in a frequency-modulated (f-VEP) BCI flickers given to the EEG amplifier for marking the flash
at a different frequency. This generates a onset of each target. t-VEPs are time-locked and
periodic visual evoked response with the same phase-locked to visual stimulus onset. Thus, since
fundamental frequency as that of the flickering the flash sequences for all targets are mutually
stimulus, as well as its harmonics. Because the independent, averaging over several short epochs
flicker frequency of f-VEP BCIs is usually higher synchronized according to the flash onset time of
than 6 Hz, the evoked responses from consecutive each possible target will produce VEPs for each
flashes of the target overlap with each other. possible target. Since foveal (i.e., fixation point)
This generates a periodic sequence of VEPs—a VEPs are larger than peripheral VEPs, the target
steady-state visual evoked potential (SSVEP)— producing the largest average peak-to-valley VEP
162 B. He et al.
Fig. 4.17 Examples of stimulations of VEP BCIs. (a) lus sequences of a c-VEP-based BCI. Right: A sample of
Left: The stimulus sequences of an f-VEP-based BCI. time course of the evoked response. (d) Left: The stimulus
Targets flash at different frequencies. Right: The power sequences of a p-VEP-based BCI. The phase difference
spectrum of the VEP derived from a target flickering at between adjacent targets is 60 degree. Right: The phase
10 Hz. (b) Left: The stimulus sequences of a t-VEP-based distribution of response signals from stimuli with different
BCI. Target flashes are mutually independent. Right: The phases. (Revised from Bin et al. [144] and Wang et al. [40]
evoked response to a single stimulus. (c) Left: The stimu- with permission)
amplitude can be identified as the fixation target. neuromuscular disabilities may lack) and visual
An accurate target identification in a t-VEP BCI fatigue from prolonged fixation.
requires averaging over many epochs. Further- The most significant progress in an SSVEP-
more, to prevent the overlap of two consecutive based BCI is the improvement of information
VEPs, t-VEP BCIs usually have low stimulus transfer rate (ITR) of the systems. Chen et al.
rates (4 Hz). Thus t-VEP BCIs have a relatively developed a new joint frequency-phase modula-
low information transfer rate (30 bits/min). tion method in their SSVEP-based BCI speller
In a code-modulated (c-VEP) BCI, pseudo- (see Fig. 4.18) to enhance the discriminability
random stimulus sequences are used. The most between SSVEPs with a very narrow frequency
commonly used pseudorandom sequence in c- range. The system obtained an impressive high
VEP BCIs is the m-sequence. M-sequences have ITR of 5.32bits/s or 319.2bits/min [145]. Nakan-
an autocorrelation functions that are a very close ishi et al. recently presented a novel data-driven
approximation to a unit impulse function and spatial filtering approach for SSVEP detection.
are nearly orthogonal to its time lag sequence. The ITR in this system was as high as 325 bit-
Thus, in c-VEP BCIs, an m-sequence and its time s/min [146].
lag sequence can be used for different stimulus
targets. Sample stimulation sequences and their
time course of evoked potentials are shown in Fig. 4.6.5 BCIs Based on Auditory
4.17c [144]. At the beginning of each stimulation Evoked Potentials
cycle, a synchronous signal, which provides a
trigger for target identification, is given to the BCIs that use visual stimuli have been shown
EEG amplifier. The template matching method is to be effective as we discussed earlier. However,
generally used for target identification. some severely disabled people may have diffi-
A c-VEP-based BCI system was developed culty using a BCI that requires good vision, due
by Sutter in 1984. Bin et al. [142] described a to compromised vision or loss of eye movement
PC-based c-VEP BCI and tested it in five sub- control. Nevertheless, even in severely paralyzed
jects. The average information transfer rate (ITR) patients, such as those suffering from ALS, hear-
reached 108 ± 12 bits/min, with a maximum of ing is usually preserved. Thus, a BCI based on
123 bits/min for one of the subjects studied. auditory evoked potentials (AEP-BCI) becomes
As shown in Fig. 4.17d [40], in a phase- an alternative paradigm.
modulated VEP (p-VEP) BCI, several targets AEPs are the brain’s response to external au-
flicker at the same frequency but with different ditory stimuli. Two types of AEP-based BCIs
phases so that more targets can be presented have been explored. One uses auditory stimuli
in less time. Jia et al. [143] proposed a coding as feedback in order to help subjects learn to
method using a combination of frequency and regulate their sensorimotor rhythms [147] or to
phase information. With this method, they regulate the slow cortical potential [148]. The
developed a BCI system with 15 targets and second type of system uses an auditory “oddball”
only three stimulus frequencies. Through the paradigm [149, 150]. Most current AEP-based
optimization of lead position, reference phase, BCIs use an “oddball” paradigm [149, 150]. As
data segment length, and harmonic components, in the case of the visual P300 described earlier in
the average ITR exceeded 60 bits/min in a this chapter, the auditory stimuli in auditory odd-
simulated online test with ten subjects. ball BCIs are divided into two types: frequently
Wang et al. [40] and Bin et al. [144] summa- presented non-targets and rarely presented tar-
rized the pros and cons of VEP BCIs. The ad- gets. For example, spoken digits could comprise
vantages of VEP BCIs are their simplicity, lower a stimulus sequence. The digits would be pre-
training time, and high information transfer rate. sented in random order and used to represent the
The disadvantages of the system are the need possible selections. In the sequence, all the digits
for good gaze control (which people with severe would be standard non-target stimuli except for
164 B. He et al.
Fig. 4.18 Closed-loop system design of an SSVEP-based with an interval of 0.2 Hz. The phase interval between two
BCI speller with high information transfer rate. (a) System neighboring frequencies is 0.35π. (c) Examples of spelling
diagram of the BCI speller, which consists of four main characters “H” (15.0 Hz, 0.25π) and “I” (8.2 Hz, 0.35π)
procedures: visual stimulation, EEG recording, real-time with the BCI speller. An intertrial interval of 0.5 s is used
data processing, and feedback presentation. The 5 × 8 for directing gaze to a target before the stimulation matrix
stimulation matrix includes the 26 letters of the English starts to flash for 0.5 s. The 0.5-s-long EEG epoch with
alphabet, 10 numbers, and 4 symbols (i.e., space, comma, a delay of τ (∼140 ms) to the stimulation is extracted
period, and backspace). Real-time data analysis recog- for target identification. The target character can be deter-
nizes the attended target character through preprocessing, mined by the decoding algorithm based on the correlations
feature extraction, and classification. (b) Frequency and between the single-trial SSVEP and individual SSVEP
phase values used for encoding each character in the stim- templates. (From Chen et al. [145] with permission)
ulation matrix. The frequencies range from 8.0 to 15.8 Hz
one target stimulus, that is, the subject’s desired in a typical visual P300 speller were replaced
choice. The subject is instructed to pay attention by spoken digits. As in a visual P300 speller,
to the target digit and perform a mental task when the subjects using the auditory system were in-
the target digit is spoken (e.g., count each time structed to first select the row number and then
it is heard). The auditory ERPs in response to the column number containing the target letter.
the target stimulus are similar to those in visual The auditory system was first tested with healthy
P300-based BCIs. An auditory spelling system subjects. Nine of 13 subjects achieved accuracies
was proposed by Furdea et al. [149] and tested above 70% [149]. In the study by Kubler et al.
with four ALS patients [151]. To compare a user’s [151], four ALS patients used the system and
performance with the auditory and visual modali- performed above chance level.
ties, a 5x5 visual support matrix was displayed to Compared to the visual spelling system, users’
the participants. Rows were coded with numbers performance with the auditory speller was lower
1–5, and columns with numbers 6–10. The flashes and the peak latencies of the auditory ERPs were
longer. However, for severely disabled people motor imagery or P300-based tasks using EEG-
with compromised vision or loss of eye move- based signal detection [159].
ment control, AEP-based BCIs might provide a The main objectives of hybrid BCI develop-
preferred way to communicate with the external ment are (i) to increase the number of brain com-
world and thus are worthy of further study. Re- mands for control applications; (ii) to enhance the
cently, the research has shown that the proper BCI classification accuracy; and (iii) to shorten
training can improve the performance of the audi- the brain command detection time. In fact, non-
tory ERP-based BCI, specifically the information brain signals in hybrid BCIs such as EMG and
transfer rate [152]. EOG are useful either to increase the number of
commands or to remove motion artifacts in EEG
recordings to improve the classification accuracy
4.6.6 Hybrid BCI of the BCI system.
Hybrid BCI allows the potential patient
The concept of hybrid BCIs was proposed to candidates to fully utilize their reserved body
further improve the performance of BCIs beyond movement such as EOG to enhance the imperfect
that of BCIs with a single approach [153]. The BCI performance by decoding their brain waves
hybrid BCIs fulfill the following criteria: the ac- [160]. Soekadar et al. demonstrated a group of
tivity should be directly acquired from the brain; six naïve individuals performed independent
at least one of the multiple brain signal acquisition and self-initiated reaching and grasping activity
modalities should be employed in acquiring such outside of the laboratory [161].
activity; the signals must be processed in real- Hybrid BCIs are suited to both disable persons
time/online to establish communication between and healthy people. For healthy individuals, hy-
the brain and the computer; feedback describing brid BCIs can be useful in the environment with
the outcomes of the brain activity for communi- multiple tasks utilizing several devices [162] or
cation and control must be provided. entertainment [163]. Also, hybrid BCIs may give
Although BCI shows great promising applica- better information about the mental workload and
tions in the healthy population, stroke patients, fatigue, cognitive functions, and vigilance of a
ALS patients, etc., it still faces the challenge of person to avoid some accidents.
performance variation, relatively low information
transfer rate compared to the normal body func-
tion, to name a few. It is reasonable to combine the 4.6.7 Attention-Based BCI
users’ preserved body movements as one of the
control sources with the traditional BCI output to Attention-based BCIs could be implemented by
fully benefit the daily use or daily rehabilitation a covert attention or overt attention paradigm.
of the end users. In a covert attention paradigm, the subject is
Hybrid BCIs can be configured in two ways: instructed to look at a centrally located fixation
(i) a combination of two different brain signal ac- point. The subject’s task is to follow another point
quisition modalities (e.g., EEG and fNIRS) [154, without overt eye movement. In contrast, in an
155]; (ii) a combination of a brain signal acqui- overt attention, the subject’s task is to use overt
sition modality with one or more nonbrain sig- eye movements while they attend to a moving
nal acquisition modalities (e.g., EEG and EMG, object.
EOG, ECG) [156, 157]. Hong et al. presented a In a conventional SSVEP BCI system, the
comprehensive review of the recent development subject overtly directs attention to one of the
in hybrid BCIs [158]. stimuli by changing his or her gaze direction.
In addition to combining different signal ac- The attended stimulus elicits enhanced SSVEP
quisition modalities, some hybrid BCIs are de- responses at the corresponding frequency over
signed by decoding multiple tasks using a single occipital brain areas. This kind of system is
modality. For example, SSVEP is combined with considered a “dependent” BCI since muscle
166 B. He et al.
activity such as that producing gaze shifting dimensional BCI performance compared to the
may be necessary. Therefore, it might not be conventional motor imagery–based BCI [105].
usable by people who have lost control of gaze Furthermore, it was shown that overt spatial at-
direction. tention and motor imagery could function inde-
A large number of psychophysical and neuro- pendently and simultaneously. Thus, a 3-D BCI
physiological studies have shown that people can control is realized through the solely endoge-
covertly shift attention to different spatial loca- nous modulation of attentions by simultaneously
tions without redirecting gaze. In addition, shift- performing both the overt spatial attentional and
ing attention to one out of several superimposed sensorimotor rhythm modulations. Figure 4.19
objects can improve behavioral performance (re- illustrates high-dimensional cursor control BCI
action time and accuracy) and increase neuronal via the combination of overt spatial attention and
responses compared to paradigms in which the motor imagery modulation. The use of hybrid
object is unattended. This covert attention could control signals allowed achieving as high as 12
be decoded and applied to build a BCI system targets, leading to a group average information
[164]. Kelly et al. [165, 166] reported a BCI based transfer rate of 29.7 ± 1.6 bits/min in nine human
on spatial visual selective attention. Two bilateral subjects [105].
flickers with superimposed letter sequences were Visual selective attention-based BCIs have
presented to the subjects. The subjects covertly thus far provided only binary control. However,
attended to one of the two bilateral flickers for tar- their performance with gaze independence
get selection. Greater than 70% average accuracy encourages further study, including the devel-
was achieved with this system. Zhang et al. [167] opment of a multiple-selection system. These
explored a nonspatial visual selective attention- systems may be a good option for paralyzed
based BCI. Two sets of dots with different colors people who cannot control well gaze direction.
and flicker frequencies, rotating in opposite direc- It might enable them to achieve control of a BCI
tions, were used to induce the perception of two by employing covert attention shifts instead of
superimposed, transparent surfaces. Because the changes of gaze direction [170].
surfaces flickered at different frequencies, they
elicited distinguishable SSVEPs. By selectively
attending to one of the two surfaces, the SSVEP 4.6.8 BCIs for Brain-to-Brain
amplitude at the corresponding frequency was en- Communications
hanced so that the subjects could select between and Interactions
two different BCI outputs. This system was tested
in healthy subjects in a 3-day online training BCI has been explored beyond the setting of
program. An average online classification accu- a single brain to computer/device. Babiloni
racy of 72.6 ± 16.1% was achieved on the last and colleagues have shown multiple brain
training day. Tonin and colleagues used a covert communications by simultaneous recordings of
attention paradigm for a two-class classification EEG as revealed in functional connectivity that
problem [168, 169]. The BCI system operated existed among the multiple brains in a social
based on covert visuospatial attention without setting [171, 172]. Their work demonstrated
relying on any evoked responses. The mean on- brain-to-brain communications and suggested
line accuracy across eight healthy subjects was the possibility of multiple brain interactions.
70.6 ± 1.5% and 88.8 ± 5.8% for the best sub- An interesting approach integrating EEG BCI
ject. Previously, the covert attention was success- with transcranial magnetic stimulation (TMS)
fully used to build a one-dimensional online BCI to realize brain-to-brain interface where EEG
system. BCI was used to decode the intent and TMS was
A recent study demonstrated that decoding of used to transmit the information into a brain
overt spatial attention might be more efficient was reported [173, 174]. Recently, Rao and
and show comparable one-dimensional and two- colleagues showed brain-to-brain interactions in
Fig. 4.19 Realization of 3-D BCI for cursor control via control task. The highlighted bar indicated the target to hit.
the combination of overt spatial attention and motor im- (b) A scene of the 12 target 3-D cursor control task where
agery modulation. (a) A scene of the 8 target 3-D cursor the highlighted bar indicated the target to hit. (From Meng
et al. [105] with Permission, © 2018 IEEE)
Fig. 4.20 Direct brain-to-brain communication and in- brain via a computer–brain interface (CBI) based on TMS.
teraction using BCI. Two participants (“Sender 1” and After consciously processing the two inputs from the
“Sender 2”) each use an SSVEP BCI to convey infor- Senders, the Receiver uses a BCI based on EEG to execute
mation about a collaborative task directly to the brain of an action in the task. (From Jiang et al. [175], licensed
the third participant (“Receiver”). Information from each under CC BY 4.0)
Sender is transmitted over the internet to the Receiver’s
a social setting involving SSVEP BCI and TMS

for online transmitting and receiving information 4.7 BCI Performance Assessment
and interacting [175]. In a computer-based game and Training
setting, two senders each used an SSVEP BCI
to convey information to a third individual— A BCI user controls brain signal features that the
receiver—as coded by transcranial magnetic BCI can recognize and translate into control com-
stimulation (see Fig. 4.20). Such brain-to-brain mands. The performance of BCIs can be affected
communications and interactions may represent by the differences among users, by the varying
further applications, especially in the general signal-processing abilities of the BCI systems, or
population. by the signal acquisition protocols used in the BCI
systems. In order to better understand the impact
168 B. He et al.
of these factors, researchers usually assess BCI relationship between X and Y, whereas values
performance with respect to one factor at a time. close to 0 indicate that there is very little linear
For example, for communication systems, the correlation.
traditional unit of measure is the amount of in- In BCI systems, user performance can be de-
formation transferred in a unit of time. There- fined as the level of correlation between the user’s
fore, the performance measure can be indicated intent and the brain signal feature(s) that the BCI
by bits per trial and bits per minute. This pro- translates into its output commands.
vides a tangible measure for making intra-system
and inter-system performance comparisons. For
other systems aimed at replacing motor function, 4.7.2 System Performance
it is not only the attainment of the goal (i.e., Assessment
reaching a target location) that matters, but also
how well the continuous trajectories are recon- Many different BCI systems have been studied.
structed. Therefore, the performance measure can They differ in inputs, outputs, translation algo-
be indicated by statistical measures for goodness rithms, and other characteristics. To compare and
of fit, such as the coefficient of determination evaluate the performance of different BCI sys-
(r2 ). tems, an objective measure is required. BCIs pro-
vide the capability of communication between
brain signals and external devices. Therefore, the
4.7.1 User Performance Assessment information transfer rate (ITR) has been used as
one of the primary metrics to evaluate BCI system
The square of the Pearson product-moment corre- performance.
lation coefficient (PPMCC) is denoted as r2 and Most current BCI systems translate the user’s
has been widely used in the assessment of BCI brain signal features into output commands by a
user performance. regression method or by a classification method.
The PPMCC between two variables X and Y is The former has the advantage of requiring only
defined as the covariance of the two variables di- one translation function for each dimension of the
vided by the product of their standard deviations: matrix of possible output commands, while the
latter requires additional functions as additional
cov (X, Y ) E [(X − μX ) (Y − μY )] output commands are added.
ρX,Y = =
σX σY σX σY Currently, the most popular method for ITR
(4.1) calculation was defined by Wolpaw et al. in 1998
[176] and discussed further in McFarland et al.
where μx , μy , σ x , and σ y are the mean and stan- [177]. The definition is a simplified computa-
dard deviation of X and Y, respectively. tional model based on the Shannon channel the-
Substituting estimates of the covariances and ory under several assumptions. The measure of
variances based on samples gives the sample cor- ITR is the bit rate B (bits/symbol) as shown in Eq.
relation coefficient, commonly denoted by r: (4.3).
n

Xi − X Yi − Y B = log2 N + P log2 P
i=1
r= (4.2) + (1 − P ) log2 [(1 − P ) / (N − 1)]
n
2 n 2
Xi − X Yi − Y (4.3)
i=1 i=1
where N is the number of possible selections,
where r ranges between +1 and − 1. Its square P is the accuracy (probability that the desired
(r2 ) then has a value between 0 and 1. A selection will be selected), and B is the bits per
value of r2 close to 1 indicates a strong linear trial. If the execution time per symbol selection is
T, then the bits per minute Bt can be calculated as 4. The translation error is equally distributed am-
follows. ong all the remaining selections p (yj /xi ) =
j =i
1−p(yi /xi )
Bt = B ∗ (60/T ) (4.4) N −1
.
5. The translation accuracy is above the chance
It is worth noting that the use of Eq. (4.3) and level.
Eq. (4.4) is conditional, because the following
assumptions were used in the derivation of Eq. The resulting ITR by Eqs. (4.3) and (4.4) de-
(4.3). pends on both speed and accuracy. Figure 4.21
illustrates the relationship between accuracy and
1. BCI systems are memoryless and stable trans- bit rate for different numbers of selections.
mission channels. In reality, r2 and ITR are just two factors that
2. All the output commands (i.e., selections) have can be used for BCI performance assessment.
the same probability of selection (p(wi ) = 1/N) Other factors important for BCI evaluation in-
3. The translation accuracy is the same for all the clude invasiveness, training time, ease and com-
selections (p(yi /xi ) = p(yj /xj )). fort of use, cost, and others. The significance of
Fig. 4.21 Information transfer rate in bits/trial (i.e., bit- selected is always the same, and if each of the other (i.e.
s/selection) and in bits/min (for 12 trials/min) when the undesired) choices has the same probability of selection
number of possible choices (i.e., N) is 2, 4, 8, 16, or 32. (i.e., (1 – P)/(N – 1)), then bit rate, or bits/trial (B), is
As derived from Pierce [195] (and originally from [196]), B = log2 N + P log2 P + (1 – P)log2 [(1 – P)/(N – 1)]. For
if a trial has N possible choices, if each choice has the each N, bit rate is shown only for accuracy ≥100 = N (i.e.,
same probability of being the one that the user desires, if ≥chance). (From Wolpaw et al. [10], with permission)
the probability (P) that the desired choice will actually be
170 B. He et al.
these various factors may vary across different therefore be a major research goal. The biggest
BCI applications. challenge for an EEG-based BCI maybe the
further development of signal processing and
machine learning techniques that can reliably
4.8 Future Perspectives and accurately decode and delineate the intention
signals from relatively noisy EEG signals. This
4.8.1 Expectations would require innovations in machine learning,
signal processing, and classification algorithms,
BCI research and development evokes a great as well as advancement in systems neuroscience
deal of excitement in scientists, engineers, clin- research.
icians, and the public in general. This excite- BCIs that employ implanted electrodes (i.e.,
ment is largely in response to the considerable invasive BCIs) face a number of complex issues,
promise of BCIs. With continued development, some of which are not yet fully understood. These
they may replace or restore useful function to systems require hardware that: is safe and com-
people severely disabled by neuromuscular dis- pletely implantable; stays intact, functional, and
orders. In addition, BCIs might augment natural reliable for many years; records stable signals for
motor outputs for pilots, surgeons, other profes- many years; transmits the recorded signals using
sionals, or ordinary citizens for daily activities. telemetry; is able to be recharged in situ (or has
They might also give new opportunities and chal- batteries that last for many years); has external
lenges to artists, athletes, and video-gaming en- components that are durable, comfortable, con-
thusiasts. Furthermore, BCIs might also conceiv- venient, and unobtrusive; and interfaces readily
ably improve rehabilitation methods for people with a range of high-performance applications.
with strokes, head trauma, and other devastating While considerable progress has been made in
disorders. At the same time, it is clear that this the past several years, it is not yet clear which
exciting future can become reality only if BCI possible solutions will be most successful, or how
researchers and developers address and resolve successful they can be. Fundamental innovations
problems in crucial areas including signal ac- in sensor technology may be needed for invasive
quisition, BCI validation and dissemination, and BCIs to achieve their full promise.
reliability.
4.8.3 Clinical and Practical

4.8.2 Signal Acquisition Validation
and Processing
Various noninvasive and invasive BCIs are being
BCI systems depend on the sensors and the developed. As this work proceeds and BCIs start
related hardware that record the crucial brain to actually be used clinically, two key questions
signals. Improvements in this hardware are must be addressed: how capable and reliable a
needed. EEG-based (noninvasive) BCIs should: particular BCI can get; and which BCIs are the
have electrodes that do not need skin abrasion or best choices for a particular clinical or practical
conductive gel (i.e., so-called dry electrodes); be purpose. To address the first question, each can-
small and portable; use comfortable, convenient, didate BCI should be optimized and the limits
and attractive mountings; be easy to set up; work on users’ capacities with it should be determined.
for many hours without needing maintenance; Engaging the second question will require some
work reliably in any environment; use telemetry consensus among researchers concerning which
rather than connecting wires; and interface applications to use for comparing BCIs and con-
easily with many different applications. Reliable cerning how their performance should be mea-
performance in all relevant environments sured. One obvious example is the question of
may be especially hard to ensure and should whether BCIs that use intracortical signals can
perform better than BCIs that use ECoG signals, Many BCIs ask the user to perform specific
or even EEG signals, and if their performance cognitive tasks that generate recognizable EEG
justifies the necessary electrode implantation by components (i.e., components that the BCI can
surgery. For many people, invasive BCIs will need decode into intent). Motor imagery tasks have
to perform much better to be considered prefer- been the most widely used cognitive task. For
able to noninvasive BCIs. Although the degree of each selection, the user imagines or plans one
freedom for a neuroprosthetic control increased of the several motor movements (i.e., left- or
from seven to ten [178, 179] and the informa- right-hand movement) based on visual or aural
tion transfer rate has increased dramatically for cues. Research has shown that this generates brain
invasive BCIs [180, 181] in the past few years, signals (e.g., from sensorimotor cortex) that can
significant improvement was also achieved for be detected by EEG or fMRI [43, 63]. After sev-
noninvasive BCIs as well [13, 105, 145]. It is eral training sessions, the user is usually able to
as yet unclear whether they can do so. Contrary produce a specific pattern of signal features (e.g.,
to widespread expectations, the available data amplitudes in specific frequency bands at specific
seem not to provide a clear answer to this critical locations) by performing a specific cognitive task.
question. Other cognitive tasks can be used, such as
Furthermore, the widespread clinical usage of arithmetic (addition of a series of numbers), vi-
BCIs by people with disabilities requires definite sual counting (sequential visualization of num-
validation of their real-life value in efficacy, prac- bers), geometric figure rotation (visualization of
ticality, and effect on the quality of life. Such rotation of a 3-D object around an axis), letter
validation depends on multidisciplinary groups composition (nonvocal letter composition), and
able and willing to perform chronic studies of baseline (relaxation). Studies have shown that
real-life use in complex and frequently difficult these tasks produce components detectable in the
environments. These studies, which are just be- EEG [56, 183, 184].
ginning, are a critical step if BCIs are to achieve The EEG components produced by cognitive
their promise. The results of these studies could tasks are vulnerable to the amount of direction
also shape the development of BCIs for the gen- provided to the user. Motor imagery, for exam-
eral population. The clear validation of BCIs for ple, is subject to issues such as first-/third-person
functional rehabilitation after strokes or in other perspective, visualization of the action versus re-
disorders will be similarly demanding and will trieving a memory of the action performed earlier,
necessarily entail direct comparisons with the imagination of the task as opposed to a verbal
outcomes of conventional methods alone. narration, etc. Research has yet to prove whether
users can effectively control such fine details to
produce significant change in the components
4.8.4 BCI Training they produce.
The major focus of BCI development thus far
The effectiveness of a BCI depends on the capac- has been to provide communication for severely
ity of the user to produce brain signals that reflect disabled people. It is possible that some poten-
intent and that the BCI can decode accurately and tial users have disorders that are also cognitively
reliably into output commands that achieve that debilitating in ways that preclude their control
intent [10, 32, 182]. Control of brain activity is of signals from areas of the brain that may be
harder to achieve than control of motor activity important for BCI control. The left hemisphere
partly because the user can neither identify nor of the brain, for example, is the center of activity
discern the activity. The user can only compre- for tasks involving language, numbers, and logic,
hend EEG activity through the feedback received while the right hemisphere is more active during
from the BCI system. Different BCI systems use spatial relations and movement imagery. Users
different strategies to help users learn to control need to be paired with the cognitive tasks that best
the crucial brain signals. suit their capabilities.
172 B. He et al.
As indicated earlier, it is possible to discern of subjects who do not respond to certain BCI
different cognitive tasks based on the EEG com- modalities. The proportion of nonresponders for
ponents generated when the task is performed. the P300-based BCI [186] and SSVEP-based BCI
When using a set of cognitive tasks during train- [187] is generally small, that is, less than 10%.
ing, the overlap of EEG signals can occur if the However, there is ample evidence to show that
tasks require similar skills or cortical areas. It is there is a non-negligible number of subjects (esti-
important to choose tasks with contrasting EEG mated around 20%–30%) who could not generate
components for easy discrimination. reliable brain rhythms to be classified in sensori-
Another factor to consider during training is motor rhythm BCIs [188, 189]. They were named
the particular EEG component to use. P300 re- as “BCI illiterate” previously. In recent years,
sponses, for example, require less training time a lot of work has been done to find novel ap-
than that needed by a user learning to control sen- proaches improving the BCI performance in order
sorimotor rhythms. As mentioned earlier, choos- to reduce the number of BCI illiterates [185, 190]
ing contrasting cognitive tasks accelerates train- or to investigate the factors that might predict the
ing. It is also important to maintain consistent performance of BCI users [188, 191, 192]. The
training regiments to ensure that subjects retain recognition of BCI efficiency and inefficiency is
their ability to control their EEG components. an important issue. Because there might not be
The tasks used in training carry forward into a universal BCI paradigm that would be suit-
general BCI usage. The method of training, there- able for everybody, it is meaningful to find out
fore, is associated with the method of signal ac- what kind of population is suitable for a certain
quisition. Neuronal activity generated by specific type of BCI technology. Thus, the BCI nonre-
cognitive tasks is focused in specific areas of the sponders could be screened out for a particular
brain. This allows signal acquisition to occur over paradigm before more intensive experiments are
a few electrodes that encompass these areas. conducted. It would save both subjects’ and re-
Studies have suggested the use of mindful searchers’ time and cost for an inappropriate BCI
meditation helps subjects to perform better in mo- technology [102]. On the other hand, exploring
tor imagery paradigm BCI and learn faster [185]. the underlying factors or mechanism that might
Such mindful meditation may be considered as affect the BCI performance would be vital to ad-
preprocessing training as they prepare subjects vance the development of BCI technology itself.
better for the motor imagery tasks, thus leading Blankertz et al. suggested that the idling sensori-
to enhanced performance in the subsequent BCI motor rhythm during resting state might be an im-
experiments. portant predictor of BCI system based on endoge-
nous motor imagination [188]. Grosse-Wentrup
and Scholkopf suggested that performance varia-
4.8.5 Recognition of BCI Efficiency tion within subjects might be closely related to the
and Inefficiency attentional networks in the gamma band (>40 Hz)
[191]. Further, understanding these factors will
Until now, the total number of human patients help improve the recognition of the BCI ineffi-
recruited in the invasive BCI studies, especially ciency. Additionally, some other studies seek to
counting studies with implanted neural chips, is reduce the numbers of BCI illiterate by designing
still a small double-digit number. It is hard to say various new paradigms. For example, Cassady
whether every subject might be able to achieve et al. recruited participants with/without mindful
high performance yet. Most of the human BCI meditation experience and found that the medi-
studies are still using noninvasive recording tech- tation practitioners achieve similar good perfor-
nology due to its applicability to both the healthy mance in shorter training sessions, which sug-
population and the general patient population (ex- gested that practicing meditation might facilitate
cept for those with clinical needs of implanting BCI skill acquisition [185]. Yao et al. applied
electrodes). However, there is a certain proportion vibrotactile stimulations on subjects’ both wrists
and asked the participants to either sense the vi- output commands correspond to the intent of the
bration or performing conventional motor imag- user. Concurrently, the BCI needs to encourage
ination. They found a significant improvement and facilitate CNS plasticity that improves the
in BCI performance when using the combina- reliability and precision with which the brain
tion of sensation and motor imagery compared signals encode the intent of the user. In summary,
to either using motor imagery or sensation alone the BCI and CNS need to work together to master
[190]. More recently, Meng and He investigated and maintain a partnership that is reliable in all
the effect of training on BCI performance based circumstances. The work required to realize this
on motor imagery paradigm. Their results sug- essential partnership has just started. It engages
gested that training could improve subjects’ per- basic neuroscientific questions and may produce
formance quickly in three sessions of practice and valuable new insights into CNS function. Thus,
the improvement is particularly significant in the BCI research has importance for neuroscience
group of low BCI performers, that is, participants in general, independent of the practical uses that
who might be recognized as BCI illiterate using are the primary focus of most BCI research and
the conventional standard of 70% accuracy [102]. development.
Therefore, the BCI inefficiency might be depen- The fundamental importance of CNS adapta-
dent on a specific BCI paradigm and the subject tion implies that the key problems in BCI research
population. Future studies should carefully select are neurobiological. The principles that deter-
their population of subjects and specify their BCI mine how the CNS masters, improves, and pre-
experimental design when determining the BCI serves its natural muscle-based skills are likely to
inefficient subjects. be the best guide for designing BCI systems. CNS
control of actions is typically distributed among
multiple areas. While cortical areas may define
4.8.6 Reciprocal Learning Between the goal and the broad outlines of an action, the
the Machine and the Brain details (especially high-speed sensorimotor inter-
actions) are often managed subcortically. Further-
BCIs provide the CNS with the chance to mas- more, control is distributed in the CNS in accord
ter novel skills in which brain signals substitute with the demands of the task. Piano playing can
for the spinal motoneurons that produce natu- require cortical control of every finger individu-
ral muscle-based skills. Muscle-based skills rely ally, while merely grasping an object may not do
for their initial mastery and long-term preserva- so.
tion on continual activity-dependent plasticity in The performance of BCIs is also likely to
many CNS areas, from the cortex to the spinal benefit from comparable distribution of control.
cord. This plasticity, which can require practice In this case, the distribution would be between the
over many months or even years, allows infants BCI’s output commands (i.e., the user’s intent)
to learn to walk and talk, children to master and the application that receives the commands
reading, writing, and arithmetic, and adults to and then converts them into action. The most
acquire many different athletic and intellectual effective distribution will probably vary with the
skills. BCI and with the application.
The acquisition and maintenance of BCI- The natural muscle-based CNS outputs are
based skills, such as robust multidimensional products of the combined contributions of nu-
movement control, depend on comparable merous areas from the cortex to the spinal cord.
plasticity [13, 14, 18, 29, 44, 81, 193]. BCI This reality suggests that BCI performance might
operation requires the successful interaction be improved and stabilized by employing signals
of two adaptive controllers, the CNS and the from more than one brain area and by employing
BCI—continuous learning in machine learning brain signal features that represent relationships
algorithms used in BCIs and in the CNS through among different areas (e.g., coherences). By per-
neuroplasticity. The BCI needs to adapt so that its mitting the CNS to operate more in the way it does
174 B. He et al.
Fig. 4.22 Peer-reviewed BCI articles in the scientific been published in the last 6 years. (The statistics is from
literature. Over the past 30 years, BCI research, which Web of Science Core Collection by using keywords brain
was previously limited to a very few research groups, computer interface or brain machine interface, Language
has become an extremely active and rapidly growing English, Document Types: article. From 1980 to January
scientific field. The majority of research articles have 21, 2020)
in producing muscle-based actions, this approach illustrates the publication years of peer-reviewed
could substantially increase BCI reliability. BCI articles that have appeared to date according
Lastly, the feedback that present-day BCIs to the Web of Science database by inputting the
give their users is primarily visual and thus keywords “brain–computer interface” or “brain–
relatively slow and often imprecise. Natural machine interface” and shows that a majority of
muscle-based skills rely on multiple types of all the articles ever published have appeared just
sensory input (e.g., proprioceptive, cutaneous, in the past several years. These BCIs use a variety
visual, auditory). BCIs that control applications of different brain signals, recording techniques,
that produce complex high-speed movements and signal-processing methods. They can operate
(e.g., limb movements) would benefit from a wide variety of different applications, including
sensory feedback that is faster, more precise, communication programs, cursors on computer
and more comprehensive than vision alone. screens, drones, wheelchairs, and robotic arms. A
Work seeking to provide such feedback using small number of people with severe disabilities
stimulators in cortex or elsewhere has begun are already employing BCIs for simple commu-
[194]. The best techniques will almost certainly nication and control functions in their everyday
vary with the BCI, the application, and the lives. With improved signal acquisition hardware
user’s disability (e.g., peripheral inputs may and sensors, machine learning software, defini-
not be useful in many people with spinal cord tive clinical and practical validation, and, better
injuries). integration of neuroscience with machine learn-
ing, BCIs could become a major new technology
for people with disabilities, and for the general
4.9 Conclusion population as well.
Numerous researchers throughout the world are Acknowledgments This work was supported in part
realizing BCI systems that some years ago might by NIH AT009263, EB021027, EB006433, NS096761,
MH114233, NSF CBET-0933067 (B.H.), as well as by
have been considered science fiction. Figure 4.22
NSF of China-90820304 (S.G.).
Homework (4.1) What is the limitation to use a computer

monitor as the display of the flicker in
1. Answer the following questions about the a steady-state visual evoked potential
general aspects of BCI. (SSVEP)–based BCI?
(1.1) Define brain–computer interface (BCI) (4.2) Download one of the examples (shared
in your own words. data, e.g., S1.mat, http://thubci.org/
(1.2) Describe at least 3 examples of BCI en/index.php?s=/home/index/nr/id/
according to different signal resources 100/page/1.html) from the shared
and explain their pros and cons. data in the ‘Wang et al (2016). A
(1.3) Describe what the unique challenges of benchmark dataset for SSVEP-based
BCI research are. brain–computer interfaces. IEEE
(1.4) If you want to decode people’s imagery Transactions on Neural Systems and
movement, which brain areas do you Rehabilitation Engineering, 25(10),
want to choose in order to build an 1746-1752.’ Plot the power spectrum
EEG-based BCI? of electrode Oz from any one of the
2. Answer the following questions about the 40 targets in a single block and the
BCI signal acquisition. average from all of the six blocks.
(2.1) What is the spatial resolution of nonin- 5. Answer the following questions about the
vasive techniques such as EEG, MEG, motor imagery–based BCI.
and fMRI? (5.1) Download one of the examples (shared
(2.2) What is the spatial resolution of inva- data, e.g., S1_LR_20150130.mat)
sive techniques such as ECoG, multi- from the shared data in [14] and
unit recording? Readme file to learn the structure of
(2.3) What is the temporal resolution of the shared data.
noninvasive techniques such as EEG, (5.2) Extract the multichannel signals
MEG, and fMRI? of each trial; calculate the average
(2.4) For EEG-based BCI, does increasing feedback duration for the example
the electrode number help to improve session.
the decoding accuracy of motor imagi- (5.3) Calculate the average band power (8–
nation? Why? 13 Hz) of channel C3 and C4 over all
(2.5) Does the combination of different non- of the left trials, respectively.
invasive modalities help to improve the (5.4) Calculate the average band power (8–
decoding accuracy such as the simulta- 13 Hz) of channel C3 and C4 over all
neous acquisition of EEG and fMRI? of the right trials, respectively.
Please explain why? (5.5) Compare the above average band
3. Answer the following questions about the power for left trials and right trials.
BCI feature extraction. Describe the difference.
(3.1) What kind of features could be ex- 6. What kinds of classification algorithms are
tracted to decode the event-related po- commonly used in the EEG–based BCI?
tentials (ERP)? 7. Answer the following questions about
(3.2) Is it possible to decode the ERP in robotic arm control using BCI.
single trials? Please explain. (7.1) Please explain what are the pros and
(3.3) What kind of features could be used cons to control a prosthetic or robotic
to decode the motor imagery–induced arm by using different types of nonin-
sensorimotor rhythms? vasive BCI, such as SSVEP based and
4. Answer the following questions about the sensorimotor rhythm based.
SSVEP BCI. (7.2) What is the challenge for control of a
high degree of freedom (DoF) robotic
arm by noninvasive BCIs? Please de-
176 B. He et al.
scribe your solution of controlling a 6. L.A. Farwell, E. Donchin, Talking off the top of your
high DoF robotic arm. head: Toward a mental prosthesis utilizing event-
related brain potentials. Electroencephalogr. Clin.
8. Answer the following questions about BCI
Neurophysiol. 70(6), 510–523 (1988)
applications. 7. J.R. Wolpaw, D.J. McFarland, G.W. Neat, C.A.
(8.1) What BCI could be used as a tool? Forneris, An EEG-based brain-computer interface
Please describe at least three examples. for cursor control. Electroencephalogr. Clin. Neu-
rophysiol. 78(3), 252–259 (1991). https://doi.org/
(8.2) Please describe how BCIs could
10.1016/0013-4694(91)90040-B
be used to induce tactile sensation 8. B. He, S. Gao, H. Yuan, J.R. Wolpaw, Brain–
neurofeedback. Computer Interfaces, in Neural Engineering, 2nd
9. Answer the following questions about the edn. (Springer, Boston, MA, 2013), pp. 87–151
9. A. Vallabhaneni, T. Wang, B. He, Brain—Computer
hybrid BCI.
Interface, in Neural Engineering (Springer, Boston,
(9.1) Please describe an example of the hy- MA, 2005), pp. 85–121
brid BCI. 10. J.R. Wolpaw, N. Birbaumer, D.J. McFarland, G.
(9.2) Please describe your solution of driv- Pfurtscheller, T.M. Vaughan, Brain–computer inter-
faces for communication and control. Clin. Neuro-
ing a wheelchair mounting with an as-
physiol. 113(6), 767–791 (2002)
sistive robotic arm to help drinking and 11. G.K. Anumanchipalli, J. Chartier, E.F. Chang,
eating via a hybrid BCI. Speech synthesis from neural decoding of spoken
10. Answer the following questions about infor- sentences. Nature 568(7753), 493 (2019)
12. T. Carlson, J. Del R. Millan, Brain-controlled
mation transfer rate of BCI.
wheelchairs: A robotic architecture. IEEE Robot.
(10.1) What is the state-of-the-art informa- Autom. Mag. 20(1), 65–73 (2013)
tion transfer rate (ITR) of different 13. B. Edelman, J. Meng, D. Suma, C. Zurn, E. Nagara-
types of noninvasive–based BCIs? jan, B. Baxter, et al., Noninvasive neuroimaging en-
hances continuous neural tracking for robotic device
(10.2) Please describe a possible solution of
control. Sci. Robot. 4(31), eaaw6844 (2019)
increasing the ITR of a noninvasive 14. J. Meng, S. Zhang, A. Bekyo, J. Olsoe, B. Baxter, B.
sensorimotor rhythm–based BCI and He, Noninvasive electroencephalogram based con-
explain why it might work. trol of a robotic arm for reach and grasp tasks. Sci.
Rep. 6, 38565 (2016)
11. Answer the following questions about BCI
15. A.B. Ajiboye, F.R. Willett, D.R. Young, W.D. Mem-
development. berg, B.A. Murphy, J.P. Miller, et al., Restora-
(11.1) Please list three most important question of reaching and grasping movements through
tions to be addressed in order to sig- brain-controlled muscle stimulation in a person
with tetraplegia: A proof-of-concept demonstration.
nificantly improve the field of BCI.
Lancet 389(10081), 1821–1830 (2017)
(11.2) Please discuss the potential of BCI 16. C.E. Bouton, A. Shaikhouni, N.V. Annetta, M.A.
application in the clinical field. Bockbrader, D.A. Friedenberg, D.M. Nielson, et al.,
Restoring cortical control of functional movement in
a human with quadriplegia. Nature 533(7602), 247
(2016)
References 17. M.T. DeBettencourt, J.D. Cohen, R.F. Lee, K.A.
Norman, N.B. Turk-Browne, Closed-loop training
1. B. Graimann, B. Allison, G. Pfurtscheller, brain– of attention with real-time brain imaging. Nat.
computer interfaces: A gentle introduction, in Brain- Neurosci. 18(3), 470–478 (2015). https://doi.org/
computer Interfaces, (Springer, Berlin, Heidelberg, 10.1038/nn.3940
2009), pp. 1–27 18. J.R. Wolpaw, D.J. McFarland, Control of a two-
2. E.E. Fetz, Operant conditioning of cortical unit ac- dimensional movement signal by a noninvasive
tivity. Science 163(3870), 955–958 (1969) brain-computer interface in humans. Proc. Natl.
3. J.J. Vidal, Toward direct brain-computer communi- Acad. Sci. 101(51), 17849–17854 (2004)
cation. Annu. Rev. Biophys. Bioeng. 2(1), 157–180 19. C.I. Penaloza, S. Nishio, BMI control of a third
(1973) arm for multitasking. Sci. Robot. 3(20), eaat1228
4. J.J. Vidal, Real-time detection of brain events in (2018)
EEG. Proc. IEEE 65(5), 633–641 (1977) 20. A. Biasiucci, R. Leeb, I. Iturrate, S. Perdikis, A.
5. T. Elbert, B. Rockstroh, W. Lutzenberger, N. Bir- Al-Khodairy, T. Corbet, et al., Brain-actuated func-
baumer, Biofeedback of slow cortical potentials. tional electrical stimulation elicits lasting arm motor
I. Electroencephalogr. Clin. Neurophysiol. 48(3), recovery after stroke. Nat. Commun. 9(1), 2421
293–301 (1980) (2018)
21. B. Graimann, B. Z. Allison, G. Pfurtscheller (eds.), ligible speech from the human auditory cortex. Sci.
Brain-Computer Interfaces: Revolutionizing Hu- Rep. 9(1), 874 (2019)
mancomputer Interaction (Springer Science & Busi- 37. B.N. Pasley, S.V. David, N. Mesgarani, A.
ness Media, Springer, Berlin, Heidelberg, 2010) Flinker, S.A. Shamma, N.E. Crone, et al.,
22. A.J. Ijspeert, Central pattern generators for locomo- Reconstructing speech from human auditory
tion control in animals and robots: A review. Neural cortex. PLoS Biol. 10(1) (2012). https://doi.org/
Netw. 21(4), 642–653 (2008) 10.1371/journal.pbio.1001251
23. T. Saneyoshi, D.A. Fortin, T.R. Soderling, Regu- 38. E. Donchin, K.M. Spencer, R. Wijesinghe, The men-
lation of spine and synapse formation by activity- tal prosthesis: Assessing the speed of a P300-based
dependent intracellular signaling pathways. Curr. brain-computer interface. IEEE Trans. Rehabil. Eng.
Opin. Neurobiol. 20(1), 108–115 (2010) 8(2), 174–179 (2000)
24. J.R. Wolpaw, What can the spinal cord teach us 39. G. Townsend, B.K. LaPallo, C.B. Boulay, D.J.
about learning and memory? Neuroscientist 16(5), Krusienski, G. Frye, C. Hauser, et al., A novel P300-
532–549 (2010) based brain–computer interface stimulus presenta-
25. P. Shenoy, M. Krauledat, B. Blankertz, R.P. Rao, K.- tion paradigm: Moving beyond rows and columns.
R. Müller, Towards adaptive classification for BCI. Clin. Neurophysiol. 121(7), 1109–1120 (2010)
J. Neural Eng. 3(1), R13 (2006) 40. Y. Wang, X. Gao, B. Hong, C. Jia, S. Gao, Brain-
26. A.L. Orsborn, H.G. Moorman, S.A. Overduin, M.M. computer interfaces based on visual evoked poten-
Shanechi, D.F. Dimitrov, J.M. Carmena, Closed- tials. IEEE Eng. Med. Biol. Mag. 27(5), 64–71
loop decoder adaptation shapes neural plasticity (2008)
for skillful neuroprosthetic control. Neuron 82(6), 41. M. Angrick, C. Herff, E. Mugler, M.C. Tate, M.W.
1380–1393 (2014) Slutzky, D.J. Krusienski, T. Schultz, Speech synthe-
27. S. Perdikis, L. Tonin, S. Saeedi, C. Schneider, J. sis from ECoG using densely connected 3D convolu-
Del R. Millán, The Cybathlon BCI race: Successful tional neural networks. J. Neural Eng. 16(3), 036019
longitudinal mutual learning with two tetraplegic (2019)
users. PLoS Biol. 16(5), e2003787 (2018) 42. R. Teasell, N. Hussein, Clinical consequences of
28. J.S. Müller, C. Vidaurre, M. Schreuder, F.C. Mei- stroke, in Evidence-Based Review of Stroke Reha-
necke, P. Von Bünau, K.-R. Müller, A mathematical bilitation, (March 2018), pp. 1–29. http://ebrsr.com/
model for the two-learners problem. J. Neural Eng. evidence-review/2-clinical-consequences-stroke
14(3), 036005 (2017) 43. H. Yuan, T. Liu, R. Szarkowski, C. Rios, J. Ashe,
29. A.J. Doud, J.P. Lucas, M.T. Pisansky, B. He, Con- B. He, Negative covariation between task-related
tinuous three-dimensional control of a virtual heli- responses in alpha/beta-band activity and BOLD
copter using a motor imagery based brain-computer in human sensorimotor cortex: An EEG and fMRI
interface. PLoS One 6(10), e26322 (2011) study of motor imagery and movements. NeuroIm-
30. K. LaFleur, K. Cassady, A. Doud, K. Shades, E. Ro- age 49(3), 2596–2606 (2010)
gin, B. He, Quadcopter control in three-dimensional 44. D.M. Taylor, S.I.H. Tillery, A.B. Schwartz, Direct
space using a noninvasive motor imagery-based cortical control of 3D neuroprosthetic devices. Sci-
brain–computer interface. J. Neural Eng. 10(4), ence 296(5574), 1829–1832 (2002). https://doi.org/
046003 (2013) 10.1126/science.1070291
31. A.S. Royer, A.J. Doud, M.L. Rose, B. He, EEG con- 45. M. Velliste, S. Perel, M.C. Spalding, A.S. Whitford,
trol of a virtual helicopter in 3-dimensional space us- A.B. Schwartz, Cortical control of a prosthetic arm
ing intelligent control strategies. IEEE Trans. Neural for self-feeding. Nature 453(7198), 1098 (2008)
Syst. Rehabil. Eng. 18(6), 581–589 (2010) 46. L.R. Hochberg, M.D. Serruya, G.M. Friehs, J.A.
32. B. He, B. Baxter, B.J. Edelman, C.C. Cline, W.Y. Mukand, M. Saleh, A.H. Caplan, et al., Neuronal
Wenjing, Noninvasive brain-computer interfaces ensemble control of prosthetic devices by a human
based on sensorimotor rhythms. Proc. IEEE 103(6), with tetraplegia. Nature 442(7099), 164–171 (2006).
907–925 (2015) https://doi.org/10.1038/nature04970
33. X. Chen, B. Zhao, Y. Wang, X. Gao, Combination 47. W. Truccolo, G.M. Friehs, J.P. Donoghue, L.R.
of highfrequency SSVEP-based BCI and computer Hochberg, Primary motor cortex tuning to intended
vision for controlling a robotic arm. J. Neural Eng. movement kinematics in humans with tetraplegia. J.
16(2), 026012 (2019) Neurosci. 28(5), 1163–1178 (2008)
34. C.G. Coogan, B. He, Brain-computer interface con- 48. B. He, A. Sohrabpour, E. Brown, Z. Liu, Electro-
trol in a virtual reality environment and applications physiological source imaging: A noninvasive win-
for the internet of things. IEEE Access 6, 10840– dow to brain dynamics. Annu. Rev. Biomed. Eng.
10849 (2018) 20, 171–196 (2018)
35. B.J. Edelman, N. Johnson, A. Sohrabpour, S. Tong, 49. J.R. Manning, J. Jacobs, I. Fried, M.J. Kahana,
N. Thakor, B. He, Systems neuroengineering: Un- Broadband shifts in local field potential power spec-
derstanding and interacting with the brain. Engineer- tra are correlated with single-neuron spiking in hu-
ing 1(3), 292–308 (2015) mans. J. Neurosci. 29(43), 13613–13620 (2009)
36. H. Akbari, B. Khalighinejad, J.L. Herrero, A.D. 50. G. Pfurtscheller, B. Graimann, J.E. Huggins, S.P.
Mehta, N. Mesgarani, Towards reconstructing intel- Levine, L.A. Schuh, Spatiotemporal patterns of beta
178 B. He et al.
desynchronization and gamma synchronization in 64. E. Boto, N. Holmes, J. Leggett, G. Roberts, V. Shah,
corticographic data during self-paced movement. S.S. Meyer, et al., Moving magnetoencephalography
Clin. Neurophysiol. 114(7), 1226–1236 (2003) towards real-world applications with a wearable sys-
51. G. Schalk, K.J. Miller, N.R. Anderson, J.A. Wilson, tem. Nature 555(7698), 657 (2018)
M.D. Smyth, J.G. Ojemann, et al., Two-dimensional 65. J. Mellinger, G. Schalk, C. Braun, H. Preissl, W.
movement control using electrocorticographic sig- Rosenstiel, N. Birbaumer, A. Kübler, An MEG-
nals in humans. J. Neural Eng. 5(1), 75 (2008) based brain-computer interface (BCI). NeuroIm-
52. E.C. Leuthardt, C. Gaona, M. Sharma, N. Szrama, age 36(3), 581–593 (2007). https://doi.org/10.1016/
J. Roland, Z. Freudenberg, et al., Using the electro- j.neuroimage.2007.03.019
corticographic speech network to control a brain– 66. J. Van Der Werf, O. Jensen, P. Fries, W.P. Meden-
computer interface in humans. J. Neural Eng. 8(3), dorp, Neuronal synchronization in human posterior
036004 (2011) parietal cortex during reach planning. J. Neurosci.
53. C. Michel, B. He, EEG mapping and source imag- 30(4), 1402–1412 (2010)
ing, in Niedermeyer’s Electroencephalography, ed. 67. F. Darvas, R. Scherer, J.G. Ojemann, R. Rao, K.J.
by D. Schomer, F. L. da Silva, 7th edn., (Wolters Miller, L.B. Sorensen, High gamma mapping using
Kluwer & Lippincott Williams & Wilkins, Philadel- EEG. NeuroImage 49(1), 930–938 (2010)
phia, 2017) 68. H. Battapady, P. Lin, T. Holroyd, M. Hallett, X.
54. T.J. Bradberry, R.J. Gentili, J.L. Contreras-Vidal, Chen, D.-Y. Fei, O. Bai, Spatial detection of multiple
Reconstructing three-dimensional hand movements movement intentions from SAM-filtered single-trial
from noninvasive electroencephalographic signals. MEG signals. Clin. Neurophysiol. 120(11), 1978–
J. Neurosci. 30(9), 3432–3437 (2010) 1987 (2009)
55. S. Waldert, H. Preissl, E. Demandt, C. Braun, N. Bir- 69. P.A. Bandettini, E.C. Wong, R.S. Hinks, R.S. Tikof-
baumer, A. Aertsen, C. Mehring, Hand movement sky, J.S. Hyde, Time course EPI of human brain
direction decoded from MEG and EEG. J. Neurosci. function during task activation. Magn. Reson. Med.
28(4), 1000–1008 (2008) 25(2), 390–397 (1992)
56. H. Yuan, C. Perdoni, B. He, Relationship between 70. K.K. Kwong, J.W. Belliveau, D.A. Chesler, I.E.
speed and EEG activity during imagined and exe- Goldberg, R.M. Weisskoff, B.P. Poncelet, et al., Dy-
cuted hand movements. J. Neural Eng. 7(2), 026001 namic magnetic resonance imaging of human brain
(2010) activity during primary sensory stimulation. Proc.
57. T.J. Bradberry, R.J. Gentili, J.L. Contreras-Vidal, Natl. Acad. Sci. 89(12), 5675–5679 (1992)
Fast attainment of computer cursor control with 71. S. Ogawa, D.W. Tank, R. Menon, J.M. Eller-
noninvasively acquired brain signals. J. Neural Eng. mann, S.G. Kim, H. Merkle, K. Ugurbil, Intrin-
8(3), 036010 (2011) sic signal changes accompanying sensory stimula-
58. A.P. Georgopoulos, A.B. Schwartz, R.E. Kettner, tion: Functional brain mapping with magnetic res-
Neuronal population coding of movement direc- onance imaging. Proc. Natl. Acad. Sci. U. S. A.
tion. Science 233(4771), 1416–1419 (1986). https:// 89(13), 5951–5955 (1992). https://doi.org/10.1073/
doi.org/10.1126/science.3749885 pnas.89.13.5951
59. L. Qin, L. Ding, B. He, Motor imagery classification 72. S. Ogawa, T.-M. Lee, A.R. Kay, D.W. Tank, Brain
by means of source analysis for brain–computer in- magnetic resonance imaging with contrast depen-
terface applications. J. Neural Eng. 1(3), 135 (2004) dent on blood oxygenation. Proc. Natl. Acad. Sci.
60. B.J. Edelman, B. Baxter, B. He, EEG source 87(24), 9868–9872 (1990)
imaging enhances the decoding of complex right- 73. H. Yuan, C. Perdoni, L. Yang, B. He, Differ-
hand motor imagery tasks. IEEE Trans. Biomed. ential electrophysiological coupling for positive
Eng. 63(1), 4–14 (2016). https://doi.org/10.1109/ and negative BOLD responses during unilateral
TBME.2015.2467312 hand movements. J. Neurosci. 31(26), 9585–9593
61. B. Kamousi, A.N. Amini, B. He, Classification of (2011)
motor imagery by means of cortical current density 74. R.W. Cox, A. Jesmanowicz, J.S. Hyde, Real-time
estimation and Von Neumann entropy. J. Neural functional magnetic resonance imaging. Magn. Re-
Eng. 4(2), 17 (2007) son. Med. 33(2), 230–236 (1995)
62. B. Kamousi, Z. Liu, B. He, Classification of motor 75. G. Pfurtscheller, F.L. Da Silva, Event-related
imagery tasks for brain-computer interface applica- EEG/MEG synchronization and desynchronization:
tions by means of two equivalent dipoles analysis. Basic principles. Clin. Neurophysiol. 110(11),
IEEE Trans. Neural Syst. Rehabil. Eng. 13(2), 166– 1842–1857 (1999)
171 (2005) 76. G. Pfurtscheller, C. Neuper, Motor imagery and
63. H. Yuan, A. Doud, A. Gururajan, B. He, Cortical direct brain-computer communication. Proc. IEEE
imaging of event-related (de) synchronization dur- 89(7), 1123–1134 (2001). https://doi.org/10.1109/
ing online control of brain-computer interface us- 5.939829
ing minimum-norm estimates in frequency domain. 77. K.J. Miller, G. Schalk, E.E. Fetz, M. den Nijs, J.G.
IEEE Trans. Neural Syst. Rehabil. Eng. 16(5), 425– Ojemann, R.P. Rao, Cortical activity during motor
431 (2008)
execution, motor imagery, and imagery-based online 93. B. Blankertz, R. Tomioka, S. Lemm, M. Kawan-
feedback. Proc. Natl. Acad. Sci. 107(9), 4430–4435 abe, K.R. Müller, Optimizing spatial filters for ro-
(2010) bust EEG single-trial analysis. IEEE Signal Process.
78. T. Wang, J. Deng, B. He, Classifying EEG-based Mag. 25(1), 41–56 (2008). https://doi.org/10.1109/
motor imagery tasks by means of time–frequency MSP.2008.4408441
synthesized spatial patterns. Clin. Neurophysiol. 94. A.J. Bell, T.J. Sejnowski, An information-
115(12), 2744–2753 (2004) maximization approach to blind separation and
79. T. Wang, B. He, An efficient rhythmic component blind deconvolution. Neural Comput. 7(6), 1129–
expression and weighting synthesis strategy for clas- 1159 (1995)
sifying motor imagery EEG in a brain–computer 95. D.J. McFarland, J.R. Wolpaw, Sensorimotor
interface. J. Neural Eng. 1(1), 1 (2004) rhythm-based brain–computer interface (BCI):
80. N. Yamawaki, C. Wilke, Z. Liu, B. He, An enhanced Model order selection for autoregressive spectral
time-frequency-spatial approach for motor imagery analysis. J. Neural Eng. 5(2), 155 (2008)
classification. IEEE Trans. Neural Syst. Rehabil. 96. J. Deng, B. He, Classification of imaginary tasks
Eng. 14(2), 250–254 (2006) from three channels of EEG by using an artificial
81. D.J. McFarland, W.A. Sarnacki, J.R. Wolpaw, neural network. Proceedings of the 25th annual in-
Electroencephalographic (EEG) control of three- ternational conference of the IEEE engineering in
dimensional movement. J. Neural Eng. 7(3) (2010). medicine and biology society CD-ROM, 2003
https://doi.org/10.1088/1741-2560/7/3/036007 97. N. Lu, T. Li, X. Ren, H. Miao, A deep learning
82. H. Yuan, B. He, Brain–computer interfaces using scheme for motor imagery classification based on
sensorimotor rhythms: Current state and future per- restricted Boltzmann machines. IEEE Trans. Neural
spectives. IEEE Trans. Biomed. Eng. 61(5), 1425– Syst. Rehabil. Eng. 25(6), 566–576 (2017). https://
1435 (2014) doi.org/10.1109/TNSRE.2016.2601240
83. A. Kübler, B. Kotchoubey, J. Kaiser, J.R. Wol- 98. Y.R. Tabar, U. Halici, A novel deep learning ap-
paw, N. Birbaumer, Brain–computer communica- proach for classification of EEG motor imagery
tion: Unlocking the locked in. Psychol. Bull. 127(3), signals. J. Neural Eng. 14(1), 016003 (2016)
358 (2001) 99. A.L. Blum, P. Langley, Selection of relevant fea-
84. N. Birbaumer, N. Ghanayim, T. Hinterberger, I. tures and examples in machine learning. Artif. Intell.
Iversen, B. Kotchoubey, A. Kübler, et al., A spelling 97(1–2), 245–271 (1997)
device for the paralysed. Nature 398(6725), 297 100. H. Ramoser, J. Muller-Gerking, G. Pfurtscheller,
(1999) Optimal spatial filtering of single trial EEG during
85. E. Donchin, M.G. Coles, Is the P300 component a imagined hand movement. IEEE Trans. Rehabil.
manifestation of context updating? Behav. Brain Sci. Eng. 8(4), 441–446 (2000)
11(3), 357–374 (1988) 101. F. Lotte, L. Bougrain, A. Cichocki, M. Clerc, M.
86. R. Ortner, B.Z. Allison, G. Korisek, H. Gaggl, G. Congedo, A. Rakotomamonjy, F. Yger, A review
Pfurtscheller, An SSVEP BCI to control a hand of classification algorithms for EEG-based brain–
orthosis for persons with tetraplegia. IEEE Trans. computer interfaces: A 10 year update. J. Neural
Neural Syst. Rehabil. Eng. 19(1), 1–5 (2011). https:/ Eng. 15(3), 031005 (2018)
/doi.org/10.1109/TNSRE.2010.2076364 102. J. Meng, B. He, Exploring training effect in 42
87. J. Pan, X. Gao, F. Duan, Z. Yan, S. Gao, Enhanc- human subjects using a non-invasive sensorimotor
ing the classification accuracy of steady-state visual rhythm based online BCI. Front. Hum. Neurosci. 13,
evoked potential-based brain–computer interfaces 128 (2019)
using phase constrained canonical correlation anal- 103. M. Congedo, A. Barachant, R. Bhatia, Riemannian
ysis. J. Neural Eng. 8(3), 036027 (2011) geometry for EEG-based brain-computer interfaces;
88. D.J. McFarland, L.M. McCane, S.V. David, J.R. a primer and a review. Brain Comput Interfaces 4(3),
Wolpaw, Spatial filter selection for EEG-based com- 155–174 (2017)
munication. Electroencephalogr. Clin. Neurophys- 104. A. Barachant, S. Bonnet, M. Congedo, C. Jut-
iol. 103(3), 386–394 (1997) ten, Multiclass brain-computer interface classi-
89. B. He, R.J. Cohen, Body surface Laplacian ECG fication by Riemannian geometry. IEEE Trans.
mapping. IEEE Trans. Biomed. Eng. 39(11), 1179– Biomed. Eng. 59(4), 920–928 (2012). https://
1191 (1992) doi.org/10.1109/TBME.2011.2172210
90. B. Hjorth, An on-line transformation of EEG scalp 105. J. Meng, T. Streitz, N. Gulachek, D. Suma, B. He,
potentials into orthogonal source derivations. Elec- Three-dimensional brain–computer Interface con-
troencephalogr. Clin. Neurophysiol. 39(5), 526–530 trol through simultaneous overt spatial attentional
(1975) and motor imagery tasks. IEEE Trans. Biomed. Eng.
91. B. He, Brain electric source imaging: Scalp Lapla- 65(11), 2417–2427 (2018)
cian mapping and cortical imaging. Crit. Rev. 106. C.T. Moritz, S.I. Perlmutter, E.E. Fetz, Direct con-
Biomed. Eng. 27(3–5), 149–188 (1999) trol of paralysed muscles by cortical neurons. Nature
92. B. He, J. Lian, G. Li, High-resolution EEG: A new 456(7222), 639 (2008)
realistic geometry spline Laplacian estimation tech- 107. W.-K. Tam, K.-Y. Tong, F. Meng, S. Gao, A minimal
nique. Clin. Neurophysiol. 112(5), 845–852 (2001) set of electrodes for motor imagery BCI to control an
180 B. He et al.
assistive device in chronic stroke subjects: A multi- Clinical applications. Int. Rev. Neurobiol. 86, 107–
session study. IEEE Trans. Neural Syst. Rehabil. 117 (2009)
Eng. 19(6), 617–627 (2011) 121. R. Sitaram, T. Ros, L. Stoeckel, S. Haller, F.
108. F. Pichiorri, G. Morone, M. Petti, J. Toppi, I. Pisotta, Scharnowski, J. Lewis-Peacock, et al., Closed-loop
M. Molinari, et al., Brain–computer interface boosts brain training: The science of neurofeedback. Nat.
motor imagery practice during stroke recovery. Ann. Rev. Neurosci. 18(2), 86 (2017)
Neurol. 77(5), 851–865 (2015) 122. S. Ruiz, K. Buyukturkoglu, M. Rana, N. Birbaumer,
109. E. Buch, C. Weber, L.G. Cohen, C. Braun, M.A. R. Sitaram, Real-time fMRI brain computer inter-
Dimyan, T. Ard, et al., Think to move: A neuro- faces: Self-regulation of single brain regions to net-
magnetic brain-computer interface (BCI) system for works. Biol. Psychol. 95, 4–20 (2014)
chronic stroke. Stroke 39(3), 910–917 (2008) 123. J.A. Anguera, J. Boccanfuso, J.L. Rintoul, O. Al-
110. H. Ring, N. Rosenthal, Controlled study of neuro- Hashimi, F. Faraji, J. Janowich, et al., Video game
prosthetic functional electrical stimulation in sub- training enhances cognitive control in older adults.
acute post-stroke rehabilitation. J. Rehabil. Med. Nature 501(7465), 97 (2013)
37(1), 32–36 (2005) 124. R. Abiri, S. Borhani, E.W. Sellers, Y. Jiang, X.
111. A. Ramos-Murguialday, D. Broetz, M. Rea, L. Läer, Zhao, A comprehensive review of EEG-based brain-
O. Yilmaz, F.L. Brasil, et al., Brain-machine inter- computer interface paradigms. J. Neural Eng. 16(1)
face in chronic stroke rehabilitation: A controlled (2019). https://doi.org/10.1088/1741-2552/aaf12e
study. Ann. Neurol. 74(1), 100–108 (2013). https:/ 125. T.O. Zander, C. Kothe, S. Jatzev, M. Gaertner,
/doi.org/10.1002/ana.23879 Enhancing human-computer interaction with input
112. G. Pfurtscheller, R. Leeb, C. Keinrath, D. Fried- from active and passive brain-computer interfaces,
man, C. Neuper, C. Guger, M. Slater, Walking from in Brain-Computer Interfaces, (Springer, London,
thought. Brain Res. 1071(1), 145–152 (2006) 2010), pp. 181–199
113. F. Lotte, J. Faller, C. Guger, Y. Renard, G. 126. C. Brunner, G. Andreoni, L. Bianchi, B. Blankertz,
Pfurtscheller, A. Lécuyer, R. Leeb, Combining C. Breitwieser, S.I. Kanoh, et al., Bci software plat-
BCI with virtual reality: Towards new applications forms, in Towards Practical Brain-Computer Inter-
and improved BCI, in Towards Practical Brain- faces, (Springer, Berlin, Heidelberg, 2012), pp. 303–
Computer Interfaces, (Springer, Berlin, Heidelberg, 331
2012), pp. 197–220 127. G. Schalk, J. Mellinger, A Practical Guide to Brain–
114. J. Mercier-Ganady, F. Lotte, E. Loup-Escande, M. Computer Interfacing with BCI2000: General-
Marchal, A. Lécuyer, The Mind-Mirror: See your Purpose Software for Brain-Computer Interface Re-
brain in action in your head using EEG and aug- search, Data Acquisition, Stimulus Presentation,
mented reality. Paper presented at the 2014 IEEE and Brain Monitoring (Springer Science & Business
Virtual Reality (VR), 2014 Media, Springer, London, 2010)
115. N. Johnson, J. Carey, B. Edelman, A. Doud, A. 128. J. Lindgren, A. Lecuyer, OpenViBE and other BCI
Grande, K. Lakshminarayan, B. He, Combined software platforms, in Brain–Computer Interfaces
rTMS and virtual reality brain–computer interface 2: Technology and Applications, (2016), pp. 179–
training for motor recovery after stroke. J. Neural 198
Eng. 15(1), 016009 (2018) 129. Y. Renard, F. Lotte, G. Gibert, M. Congedo, E.
116. M. Tani, Y. Ono, M. Matsubara, S. Ohmatsu, Y. Maby, V. Delannoy, et al., Openvibe: An open-
Yukawa, M. Kohno, T. Tominaga, Action observa- source software platform to design, test, and use
tion facilitates motor cortical activity in patients with brain–computer interfaces in real and virtual en-
stroke and hemiplegia. Neurosci. Res. 133, 7–14 vironments. Presence Teleop. Virt. 19(1), 35–53
(2018) (2010)
117. M. Gonzalez-Franco, J. Lanier, Model of illusions 130. J.R. Wolpaw, Brain–computer interface research
and virtual reality. Front. Psychol. 8, 1125 (2017) comes of age: Traditional assumptions meet emerg-
118. I. Bermúdez, S. Badia, A. García Morgade, H. ing realities. J. Mot. Behav. 42(6), 351–353 (2010)
Samaha, P.F.M.J. Verschure, Using a hybrid brain 131. G. Pfurtscheller, D. Flotzinger, J. Kalcher, Brain-
computer interface and virtual reality system computer Interface-a new communication device
to monitor and promote cortical reorganization for handicapped persons. J. Microcomput. Appl.
through motor activity and motor imagery 16(3), 293–299 (1993). https://doi.org/10.1006/
training. IEEE Trans. Neural Syst. Rehabil. Eng. jmca.1993.1030
21(2), 174–181 (2013). https://doi.org/10.1109/ 132. E.W. Sellers, T.M. Vaughan, J.R. Wolpaw, A brain-
TNSRE.2012.2229295 computer interface for long-term independent home
119. J. Kamiya, The first communications about operant use. Amyotroph. Lateral Scler. 11(5), 449–455
conditioning of the EEG. J. Neurother. 15(1), 65–73 (2010)
(2011) 133. G. Cattan, C. Mendoza, A. Andreev, M. Con-
120. N. Birbaumer, A.R. Murguialday, C. Weber, P. Mon- gedo, Recommendations for integrating a P300-
toya, Neurofeedback and brain–computer interface: based brain computer interface in virtual reality en-
vironments for gaming. Computers 7(2), 34 (2018)
134. Z. Gu, Z. Chen, J. Zhang, X. Zhang, Z.L. Yu, An on- computer interface based on the self-regulation of
line interactive paradigm for P300 brain–computer slow cortical potentials. Neurorehabil. Neural Re-
Interface speller. IEEE Trans. Neural Syst. Rehabil. pair 19(3), 206–218 (2005)
Eng. 27(2), 152–161 (2019) 149. A. Furdea, S. Halder, D. Krusienski, D. Bross, F.
135. I. Käthner, A. Kübler, S. Halder, Rapid P300 brain- Nijboer, N. Birbaumer, A. Kübler, An auditory odd-
computer interface communication with a head- ball (P300) spelling system for brain-computer in-
mounted display. Front. Neurosci. 9, 207 (2015) terfaces. Psychophysiology 46(3), 617–625 (2009)
136. G. Townsend, V. Platsko, Pushing the P300-based 150. J. Guo, S. Gao, B. Hong, An auditory brain–
brain-computer interface beyond 100 bpm: Extend- computer interface using active mental response.
ing performance guided constraints into the tem- IEEE Trans. Neural Syst. Rehabil. Eng. 18(3), 230–
poral domain. J. Neural Eng. 13(2) (2016). https:// 235 (2010)
doi.org/10.1088/1741-2560/13/2/026024 151. A. Kübler, A. Furdea, S. Halder, E.M. Hammer, F.
137. B. Hong, F. Guo, T. Liu, X. Gao, S. Gao, N200- Nijboer, B. Kotchoubey, A brain–computer interface
speller using motion-onset visual response. Clin. controlled auditory event-related potential (P300)
Neurophysiol. 120(9), 1658–1666 (2009) spelling system for locked-in patients. Ann. N. Y.
138. G. Bin, X. Gao, Z. Yan, B. Hong, S. Gao, An online Acad. Sci. 1157(1), 90–100 (2009)
multi-channel SSVEP-based brain–computer inter- 152. S. Halder, T. Leinfelder, S.M. Schulz, A. Kübler,
face using a canonical correlation analysis method. Neural mechanisms of training an auditory event-
J. Neural Eng. 6(4), 046002 (2009) related potential task in a brain–computer inter-
139. M. Cheng, X. Gao, S. Gao, D. Xu, Design and imple- face context. Hum. Brain Mapp. 40(8), 2399–2412
mentation of a brain-computer interface with high (2019). https://doi.org/10.1002/hbm.24531
transfer rates. IEEE Trans. Biomed. Eng. 49(10), 153. G. Pfurtscheller, B.Z. Allison, G. Bauernfeind, C.
1181–1186 (2002) Brunner, T. Solis Escalante, R. Scherer, et al., The
140. F. Guo, B. Hong, X. Gao, S. Gao, A brain–computer hybrid BCI. Front. Neurosci. 4, 3 (2010)
interface using motion-onset visual evoked poten- 154. V. Kaiser, G. Bauernfeind, A. Kreilinger, T. Kauf-
tial. J. Neural Eng. 5(4), 477 (2008) mann, A. Kübler, C. Neuper, G.R. Müller-Putz, Cor-
141. P.-L. Lee, J.-C. Hsieh, C.-H. Wu, K.-K. Shyu, Y.-T. tical effects of user training in a motor imagery based
Wu, Brain computer interface using flash onset and brain–computer interface measured by fNIRS and
offset visual evoked potentials. Clin. Neurophysiol. EEG. NeuroImage 85, 432–444 (2014)
119(3), 605–616 (2008) 155. F. Putze, S. Hesslinger, C.-Y. Tse, Y. Huang, C.
142. G. Bin, X. Gao, Y. Wang, Y. Li, B. Hong, S. Gao, A Herff, C. Guan, T. Schultz, Hybrid fNIRS-EEG
high-speed BCI based on code modulation VEP. J. based classification of auditory and visual percep-
Neural Eng. 8(2), 025015 (2011) tion processes. Front. Neurosci. 8, 373 (2014)
143. C. Jia, X. Gao, B. Hong, S. Gao, Frequency and 156. M. Li, S. Guo, G. Zuo, Y. Sun, J. Yang, Removing
phase mixed coding in SSVEP-based brain - ocular artifacts from mixed EEG signals with Fast-
computer interface. IEEE Trans. Biomed. Eng. KICA and DWT. J Intell Fuzzy Syst 28(6), 2851–
58(1), 200–206 (2011). https://doi.org/10.1109/ 2861 (2015)
TBME.2010.2068571 157. B.-H. Yang, L.-F. He, L. Lin, Q. Wang, Fast re-
144. G. Bin, X. Gao, Y. Wang, B. Hong, S. Gao, VEP- moval of ocular artifacts from electroencephalogram
based brain-computer interfaces: Time, frequency, signals using spatial constraint independent compo-
and code modulations [Research Frontier]. IEEE nent analysis based recursive least squares in brain-
Comput. Intell. Mag. 4(4), 22–26 (2009) computer interface. Front. Inf. Technol. Electron.
145. X. Chen, Y. Wang, M. Nakanishi, X. Gao, T.P. Jung, Eng. 16(6), 486–496 (2015)
S. Gao, High-speed spelling with a noninvasive 158. K.-S. Hong, M.J. Khan, Hybrid brain–computer in-
brain-computer interface. Proc. Natl. Acad. Sci. U. terface techniques for improved classification accu-
S. A. 112(44), E6058–E6067 (2015). https://doi.org/ racy and increased number of commands: A review.
10.1073/pnas.1508080112 Front. Neurorobot. 11, 35 (2017)
146. M. Nakanishi, Y. Wang, X. Chen, Y.T. Wang, X. 159. J. Zhao, W. Li, M. Li, Comparative study of SSVEP-
Gao, T.P. Jung, Enhancing detection of SSVEPs and P300-based models for the telepresence control
for a high-speed brain speller using task-related of humanoid robots. PLoS One 10(11), e0142168
component analysis. IEEE Trans. Biomed. Eng. (2015)
65(1), 104–112 (2018). https://doi.org/10.1109/ 160. S. Crea, M. Nann, E. Trigili, F. Cordella, A. Baldoni,
TBME.2017.2694818 F.J. Badesa, et al., Feasibility and safety of shared
147. F. Nijboer, A. Furdea, I. Gunst, J. Mellinger, D.J. EEG/EOG and vision-guided autonomous whole-
McFarland, N. Birbaumer, A. Kübler, An auditory arm exoskeleton control to perform activities of
brain–computer interface (BCI). J. Neurosci. Meth- daily living. Sci. Rep. 8(1), 10823 (2018)
ods 167(1), 43–50 (2008) 161. S. Soekadar, M. Witkowski, C. Gómez, E. Opisso, J.
148. M. Pham, T. Hinterberger, N. Neumann, A. Kübler, Medina, M. Cortese, et al., Hybrid EEG/EOG-based
N. Hofmayer, A. Grether, et al., An auditory brain- brain/neural hand exoskeleton restores fully inde-
182 B. He et al.
pendent daily living activities after quadriplegia. Sci. 175. L. Jiang, A. Stocco, D.M. Losey, J.A. Abernethy,
Robot. 1(1) (2016). eaag3296: 3291–3298 C.S. Prat, R.P.N. Rao, BrainNet: A multi-person
162. H.-J. Hwang, J.-H. Lim, Y.-J. Jung, H. Choi, S.W. brain-to-brain Interface for direct collaboration be-
Lee, C.-H. Im, Development of an SSVEP-based tween brains. Sci. Rep. 9(1) (2019). https://doi.org/
BCI spelling system adopting a QWERTY-style 10.1038/s41598-019-41895-7
LED keyboard. J. Neurosci. Methods 208(1), 59–65 176. J.R. Wolpaw, H. Ramoser, D.J. McFarland,
(2012) G. Pfurtscheller, EEG-based communication:
163. Y. Li, G. Zhou, D. Graham, A. Holtzhauer, Towards Improved accuracy by response verification. IEEE
an EEG-based brain-computer interface for online Trans. Rehabil. Eng. 6(3), 326–333 (1998)
robot control. Multimed. Tools Appl. 75(13), 7999– 177. D. McFarland, D. Krusienski, J. Wolpaw, E. Wol-
8017 (2016) paw, BCI signal processing: Feature translation, in
164. A. Gunduz, P. Brunner, A. Daitch, E.C. Leuthardt, Brain–Computer Interfaces: Principles and Prac-
A.L. Ritaccio, B. Pesaran, G. Schalk, Decoding tice, (Oxford University Press, New York, 2012), pp.
covert spatial attention using electrocorticographic 147–164
(ECoG) signals in humans. NeuroImage 60(4), 178. J.L. Collinger, B. Wodlinger, J.E. Downey, W.
2285–2293 (2012) Wang, E.C. Tyler-Kabara, D.J. Weber, et al., High-
165. S.P. Kelly, E.C. Lalor, C. Finucane, G. McDarby, performance neuroprosthetic control by an individ-
R.B. Reilly, Visual spatial attention control in an ual with tetraplegia. Lancet 381(9866), 557–564
independent brain-computer interface. IEEE Trans. (2013)
Biomed. Eng. 52(9), 1588–1596 (2005) 179. B. Wodlinger, J. Downey, E. Tyler-Kabara,
166. S.P. Kelly, E.C. Lalor, R.B. Reilly, J.J. Foxe, Visual A. Schwartz, M. Boninger, J. Collinger, Ten-
spatial attention tracking using high-density SSVEP dimensional anthropomorphic arm control in a
data for independent brain-computer communica- human brain− machine interface: Difficulties,
tion. IEEE Trans. Neural Syst. Rehabil. Eng. 13(2), solutions, and limitations. J. Neural Eng. 12(1),
172–178 (2005) 016011 (2014)
167. D. Zhang, A. Maye, X. Gao, B. Hong, A.K. Engel, 180. V. Gilja, C. Pandarinath, C.H. Blabe, P. Nuyujukian,
S. Gao, An independent brain–computer interface J.D. Simeral, A.A. Sarma, et al., Clinical translation
using covert non-spatial visual selective attention. J. of a high-performance neural prosthesis. Nat. Med.
Neural Eng. 7(1), 016010 (2010) 21(10), 1142 (2015)
168. L. Tonin, R. Leeb, J. Del R. Millán, Time-dependent 181. C. Pandarinath, P. Nuyujukian, C.H. Blabe, B.L.
approach for single trial classification of covert visu- Sorice, J. Saab, F.R. Willett, et al., High performance
ospatial attention. J. Neural Eng. 9(4) (2012). https:/ communication by people with paralysis using an in-
/doi.org/10.1088/1741-2560/9/4/045011 tracortical brain-computer interface. elife 6, e18554
169. L. Tonin, R. Leeb, A. Sobolewski, J. del R Millán, (2017)
An online EEG BCI based on covert visuospatial 182. E.A. Curran, M.J. Stokes, Learning to control brain
attention in absence of exogenous stimulation. J. activity: A review of the production and control of
Neural Eng. 10(5), 056007 (2013) EEG components for driving brain–computer inter-
170. J. Ekanayake, C. Hutton, G. Ridgway, F. face (BCI) systems. Brain Cogn. 51(3), 326–336
Scharnowski, N. Weiskopf, G. Rees, Real-time (2003)
decoding of covert attention in higher-order visual 183. N. Birbaumer, A. Kübler, N. Ghanayim, T. Hinter-
areas. NeuroImage 169, 462–472 (2018) berger, J. Perelmouter, J. Kaiser, et al., The thought
171. L. Astolfi, J. Toppi, F. De Vico Fallani, G. Vecchiato, translation device (TTD) for completely paralyzed
F. Cincotti, C.T. Wilke, et al., Imaging the social patients. IEEE Trans. Rehabil. Eng. 8(2), 190–193
brain by simultaneous hyperscanning during subject (2000). https://doi.org/10.1109/86.847812
interaction. IEEE Intell. Syst. 26(5), 38–45 (2011). 184. W.D. Penny, S.J. Roberts, EEG-based communica-
https://doi.org/10.1109/MIS.2011.61 tion via dynamic neural network models. Proceed-
172. F. De Vico Fallani, V. Nicosia, R. Sinatra, L. ings of the IJCNN 99 International Joint Conference
Astolfi, F. Cincotti, D. Mattia, et al., Defecting on Neural Networks (Cat. No. 99CH36339), 1999
or not defecting: How to “read” human behavior 185. K. Cassady, A. You, A. Doud, B. He, The impact of
during cooperative games by EEG measurements. mind-body awareness training on the early learning
PLoS One 5(12) (2010). https://doi.org/10.1371/ of a brain-computer interface. Tech. 2(03), 254–260
journal.pone.0014187 (2014)
173. M.E.M. Mashat, G. Li, D. Zhang, Human-to-human 186. C. Guger, S. Daban, E. Sellers, C. Holzner, G.
closed-loop control based on brain-to-brain inter- Krausz, R. Carabalona, et al., How many people
face and muscle-to-muscle interface. Sci. Rep. 7(1) are able to control a P300-based brain–computer
(2017). https://doi.org/10.1038/s41598-017-10957- interface (BCI)? Neurosci. Lett. 462(1), 94–98
z (2009)
174. R.P.N. Rao, A. Stocco, M. Bryan, D. Sarma, T.M. 187. B. Allison, T. Luth, D. Valbuena, A. Teymourian, I.
Youngquist, J. Wu, C.S. Prat, A direct brain-to-brain Volosyak, A. Graser, BCI demographics: How many
interface in humans. PLoS One 9(11) (2014). https:/
/doi.org/10.1371/journal.pone.0111332
(and what kinds of) people can use an SSVEP BCI? 192. C. Jeunet, B. N’Kaoua, F. Lotte, Advances in user-
IEEE Trans. Neural Syst. Rehabil. Eng. 18(2), 107– training for mental-imagery-based BCI control: Psy-
116 (2010) chological and cognitive factors and their neural
188. B. Blankertz, C. Sannelli, S. Halder, E.M. Hammer, correlates, in Vol. 228. Progress in Brain Research,
A. Kübler, K.-R. Müller, et al., Neurophysiological (2016), pp. 3–35
predictor of SMR-based BCI performance. Neu- 193. K. Ganguly, J.M. Carmena, Emergence of a stable
roImage 51(4), 1303–1309 (2010) cortical map for neuroprosthetic control. PLoS Biol.
189. C. Guger, G. Edlinger, W. Harkam, I. Nieder- 7(7), e1000153 (2009)
mayer, G. Pfurtscheller, How many people are 194. S.N. Flesher, J.L. Collinger, S.T. Foldes, J.M. Weiss,
able to operate an EEG-based brain-computer in- J.E. Downey, E.C. Tyler-Kabara, et al., Intracortical
terface (BCI)? IEEE Trans. Neural Syst. Reha- microstimulation of human somatosensory cortex.
bil. Eng. 11(2), 145–147 (2003). https://doi.org/ Sci. Trans. Med.8(361), 361ra141 (2016)
10.1109/TNSRE.2003.814481 195. J.R. Pierce, An Introduction to Information Theory
190. L. Yao, J. Meng, D. Zhang, X. Sheng, X. Zhu, (Dover Press, New York, 1980)
Combining motor imagery with selective sensa- 196. C.E. Shannon, W. Weaver, The Mathematical The-
tion toward a hybrid-modality BCI. IEEE Trans. ory of Communication (University of Illinois Press,
Biomed. Eng. 61(8), 2304–2312 (2014). https:// 1964)
doi.org/10.1109/TBME.2013.2287245 197. J. Malmivuo, R. Plonsey, Bioelectromagnetism:
191. M. Grosse-Wentrup, B. Schölkopf, A review of per- Principles and Applications of Bioelectric and Bio-
formance variations in SMR-based Brain− Com- magnetic Fields (Oxford University Press, New
puter interfaces (BCIs), in Brain-Computer Inter- York, USA, 1995)
face Research, (Springer, Berlin, Heidelberg, 2013), 198. M.A. Dimyan, L.G. Cohen, Neuroplasticity in the
pp. 39–51 context of motor rehabilitation after stroke. Nat. Rev.
Neurol. 7(2), 76–85 (2011)
Intracortical Brain–Machine
Interfaces 5
Emily R. Oby, Jay A. Hennig, Aaron P. Batista, Byron M. Yu,
and Steven M. Chase
Abstract where in the brain to record them. The salient

features of the neural signal useful for control
A brain–machine interface, or BMI, directly
are extracted with a decoding algorithm. This
connects the brain to the external world, by-
algorithm translates the neural signal into an
passing damaged biological pathways. It re-
intended action which is executed by an output
places the impaired parts of the nervous system
device, such as a robotic limb, the person’s
with hardware and software that translate a
own muscles, or a computer interface. In Sect.
user’s internal motor commands into action.
5.3 we will discuss classification decoders and
In this chapter, we will discuss the four basic
components of an intracortical BMI: an in-
tracortical neural recording, a decoding algo- A. P. Batista
rithm, an output device, and sensory feedback. Department of Bioengineering, University of Pittsburgh,
In Sect. 5.2 we will discuss intracortical sig- Pittsburgh, PA, USA
nals, the electrodes used to record them, and Center for the Neural Basis of Cognition, University of
Pittsburgh and Carnegie Mellon University, Pittsburgh,
PA, USA
Electronic Supplementary Material: The online version University of Pittsburgh Brain Institute and Systems
of this chapter (https://doi.org/10.1007/978-3-030-43395- Neuroscience Center, Pittsburgh, PA, USA
to authorized users. B. M. Yu
Center for the Neural Basis of Cognition, University of
E. R. Oby Pittsburgh and Carnegie Mellon University, Pittsburgh,
Center for the Neural Basis of Cognition, University of PA, USA
Neuroscience Institute, Carnegie Mellon University,
PA, USA
Pittsburgh, PA, USA
Department of Neurobiology, University of Pittsburgh,
Pittsburgh, PA, USA
University of Pittsburgh Brain Institute and Systems
Department of Electrical and Computer Engineering,
Neuroscience Center, Pittsburgh, PA, USA
Carnegie Mellon University, Pittsburgh, PA, USA
J. A. Hennig
S. M. Chase ()
PA, USA
PA, USA
Pittsburgh, PA, USA
Pittsburgh, PA, USA
Machine Learning Department, Carnegie Mellon

186 E. R. Oby et al.
how they can be implemented in a BMI for than the rest of the brain combined, is involved
communication. In Sect. 5.4 we will discuss in coordinating movements. Motor control only
continuous decoders for moment-by-moment seems easy because we don’t tend to think about
control of a computer cursor or robotic arm. In it very much: we just do it. In fact, the only time
Sect. 5.5, we will discuss a BMI that controls we really think about motor control is when it is
electrical stimulation to directly activate a pa- impaired.
tient’s own paralyzed muscles and reanimate Movements can become impaired for a num-
their arm. Finally, in Sect. 5.6, we will dis- ber of reasons, including neurological injury or
cuss ongoing work toward expanding sensory disorders at the level of the brain, spinal cord, pe-
feedback with the goal of making intracortical ripheral nerves, and muscles. The most common
BMIs a clinically viable option for treating causes of paralysis are stroke and spinal cord in-
paralysis, as well as other research trends. jury. There are approximately 291,000 Americans
currently living with spinal cord injuries, with
Keywords more than 17,500 new cases each year (National
Spinal Cord Injury Statistical Center). About 40%
Neural population activity · Motor cortex ·
of these individuals are paraplegic, i.e., their legs
Motor control · Neural decoding ·
are paralyzed, and 60% are quadriplegic, i.e.,
Probabilistic models · Classifier · Kalman
their arms and legs are paralyzed. Fewer than 1%
filter · Functional electrical stimulation
of patients fully recover from spinal cord injuries.
Spinal cord injuries disrupt the natural pathway
between the brain and the muscles but leave the
cortical neurons involved in generating the move-
5.1 What Is a Brain–Machine ment signals intact. If we could leverage these
Interface? intact control signals, and decode motor intent,
we could create assistive technologies that bypass
Humans are capable of a nearly endless repertoire the damaged pathway to restore motor control to
of movements: we can walk, run, skip, reach, those who have lost it. This is the clinical goal of
grab, kick, throw, dance, and more. The ease brain–machine interfaces.
with which most of us perform these movements A brain–machine interface, or BMI, directly
conceals the fact that motor control is one of connects the brain to the external world, bypass-
the most complex tasks the brain performs. More ing damaged biological pathways. It replaces the
brain resources are devoted to the problem of impaired parts of the nervous system with hard-
controlling our movements than are devoted to ware and software that translate a user’s internal
any other task we might perform. The primary motor commands into action. BMI technologies
motor cortex, as its name indicates, is the area of serve as a neural engineering solution to replace
the brain chiefly responsible for sending axons to or restore motor or sensory function to patients
the spinal cord to exert control over the muscles. with neurological injury or disease.
In addition to primary motor cortex, there are at An intracortical BMI (iBMI) is driven by the
least six other cortical areas that also send axons action potentials recorded from individual neu-
down the spinal cord to help control muscles: rons. Action potentials, or “spikes,” are the elec-
dorsal premotor cortex, ventral premotor cortex, trical signals by which neurons transmit informa-
supplementary motor area, and three separate re- tion. Intracortical BMIs involve implanting elec-
gions of the cingulate motor area. In addition to trodes directly into the cortex. This neural record-
cortex, several subcortical regions are engaged ing method provides greater spatial and temporal
during motor control, including major parts of the specificity than noninvasive recording techniques
thalamus, basal ganglia, and the spinal cord. The because the electrodes are only microns from
cerebellum, which is composed of more neurons the neurons. The greater specificity increases the
5 Intracortical Brain–Machine Interfaces 187
decoding accuracy and allows for higher degree- correlated with extrinsic motor control variables
of-freedom control. (e.g., the direction of the arm in space) as well as
intrinsic motor control variables (e.g., the force
exerted by the arm). These results have been the
5.1.1 History of Intracortical BMIs basis of BMI development since.
Over the following decades, technology devel-
In 1969, Eberhard Fetz created what is consid- oped that enabled researchers to record from pop-
ered to be the first modern-day intracortical BMI. ulations of tens to hundreds of neurons. In 1999,
In this first example of an iBMI, he showed neural signals recorded simultaneously from rat
that monkeys could learn to volitionally modu- motor cortex were used to control a robotic arm
late the activity of a single neuron in primary [1]. Soon after, the hand trajectories of primates
motor cortex in order to receive a food reward. were predicted from the activity of a population
To do so, Fetz recorded the spikes from a neu- of neurons [2], and monkeys were using neural
ron while providing the monkey with visual or activity to control computer cursors [3, 4] and
auditory feedback about the number of spikes robotic arms for reaching and grasping [5]. This
that neuron generated per unit time (i.e., firing was the beginning of a now-flourishing field of
rate). When the monkey increased the firing rate iBMI development and research.
above a certain threshold, he was rewarded. Fol- In 2006, a group of researchers at BrainGate
lowing this operant conditioning, monkeys would performed the first clinical trial to establish an
quickly and consistently increase the firing rate iBMI in a human [42]. They recorded neural
of the recorded neuron to earn rewards. Monkeys population activity from a paralyzed person as he
were also asked to separately control two neurons, imagined limb movements and used that activity
increasing the rate of one and decreasing the rate to drive the movement of a cursor on a computer
of the other. The independent control of two neu- screen. Then, in 2012, the same group demon-
rons demonstrated that the control was not simply strated that a person who had been paralyzed by
achieved by a general increase of all neurons’ brainstem stroke could directly control a robotic
firing rates. This early study provided the first arm [43]. Specifically, she was able to control
proof of concept that a person might someday the velocity of the robot’s hand to make reaches,
be able to modulate neural activity to control a and she simultaneously controlled a decoder that
computer cursor or robotic arm. could execute one of four hand actions to grasp
The Fetz study was near the beginning of objects. With this level of control, she was able to
several decades of intensive work to define the use the robotic arm to grasp a bottle and bring it to
nature of the signals encoded in motor cortex. In her mouth for a drink. Compared to natural reach-
1970, Humphrey, Schmidt, and Thompson con- ing and grasping movements, the brain-controlled
ducted a set of experiments that addressed the robot’s reaching and grasping were slower and
possibility of using neural signals to make quan- less accurate. However, this result showed that
titative predictions of simple motor behaviors. it is possible to use tens of neurons to control a
Using recordings from a small set of neurons robotic device and interact with objects. Shortly
during a wrist flexion and extension task, they afterward, a team at the University of Pittsburgh
predicted arm position, velocity, and net force demonstrated that a person could control ten de-
exerted about the joint. They showed that force grees of freedom of a robotic arm [6, 7]. The ten
was quite accurately predicted and that arm posi- degrees of freedom consisted of three dimensions
tion and velocity could also be predicted, though of translation, three dimensions of orientation of
typically not as well. In the 1980s, Apostolos the robot’s hand, and four dimensions of hand
Georgopoulos showed that neural activity dur- shape. By including the hand shape dimensions,
ing whole-arm reaches predicted the direction the researchers could increase the repertoire of
of the reach quite well. Together these findings possible movements to include dexterous manip-
suggested that motor cortical neural activity was ulation of objects. Ultimately, the person could
Fig. 5.1 An iBMI user at the University of Pittsburgh controls a robotic arm to eat a chocolate bar as members of the
research team look on. (Photo credit: UPMC)
perform skillful and coordinated reach and grasp as six wrist and hand movements, allowing him
movements like those that are essential for daily to perform some activities of daily living. In
activities, such as shaking hands or feeding one- 2017, a group of researchers at Case Western Re-
self (Fig. 5.1). serve University demonstrated that an individual
The ultimate goal of iBMI systems for people with a high cervical spinal cord injury could use
with paralysis is to restore the function of their his own cortical activity to control a chronically
own arms, hands, and legs. Currently, the best implanted FES system to perform coordinated
prospect for this is to use neural commands to reaching and grasping movements with his own
activate the muscles with electrical stimulation. paralyzed arm and hand [9]. He could volitionally
Two groups have recently shown progress in iB- perform reaches to drink coffee and feed himself
MIs to control functional electrical stimulation (Fig. 5.2). These studies are a major step toward
(FES) of a user’s own muscles. The electrical a clinically viable BMI for reaching and grasping
stimulation activates the muscles, causing them after paralysis.
to contract and thus to generate movement. In We have been focusing on iBMIs for move-
2016, a group of researchers at Battelle was the ment, but the principles of decoding motor intent
first to demonstrate successful control of mus- can also be applied to solve the problem commu-
cle activation using intracortically recorded sig- nication, allowing users to “type” by moving a
nals in a human [8]. Neural activity was de- computer cursor to different letters on a screen.
coded to control the stimulation of muscles in the There is a group of patients for whom restoring
forearm via electrodes in a custom-built sleeve. communication is crucial. These patients are re-
With the iBMI-controlled FES, the person was ferred to as “locked in” because, although they
able to independently control his fingers as well are awake and aware, they have lost control of
Fig. 5.2 An iBMI user controls functional electrical stimulation of muscles in his arm and hand to feed himself as part
of the BrainGate2 clinical trials. (Photo credit: Russell Lee; Case Western Reserve University/Cleveland FES Center)
all voluntary muscles, except (in some cases) describe the components of an iBMI, the state-of-
those that control vertical eye movements and the-art control, and future directions of research
blinking, due to brainstem stroke or amyotrophic and development.
lateral sclerosis (ALS). These patients have no
way of speaking or producing facial expressions,
so restoring some form of communication would 5.1.2 Components
dramatically enhance their quality of life. For of an Intracortical BMI
people with paraplegia or quadriplegia who can
still speak, a communication BMI could provide A BMI consists of four basic components: a neu-
an important means to interact with others via ral recording, a decoding algorithm, an output
email or texting. A number of studies have estab- device, and sensory feedback (Fig. 5.4). Intra-
lished the feasibility of iBMIs for communication cortical BMIs begin by recording neural signals
[10, 11]. In 2017, a group at Stanford University from electrodes implanted in the cortex. Next,
developed a high-performance iBMI for commu- the salient features of the neural signal useful for
nication that allows users to control a computer control are extracted with a decoding algorithm.
tablet to perform activities like browsing the web This algorithm translates the neural signal into an
and texting (Fig. 5.3; [12]). Users were able to intended action which is executed by an output
perform typing tasks that simulated real-world device, such as a robotic limb, the person’s own
applications such as typing messages at a conver- muscles, or a computer interface. Finally, the
sational pace, with a typing rate of 24 characters user receives sensory feedback about the action,
per minute. allowing them to make corrections if they move
In this chapter, we will discuss motor iBMIs off course and also allowing them to improve over
for both movement and communication. We will time with learning.
Fig. 5.3 Two iBMI users who are part of BrainGate text each other. (Credit: BrainGate Collaboration)
Neural
Recording
neurons
Decoder
time
Output
Device
Q K C G V J
S
DEL I N D
W T H E A M
Sensory
Feedback
U O R L DEL
Z B F Y P X
Fig. 5.4 The components of an iBMI consist of intracortical neural recordings, a decoder to translate the neural input
into a control signal, an output device, and sensory feedback
Building an effective iBMI depends on choos- turn, these choices influence which motor cortical
ing a brain area for the neural recordings, a de- area is most appropriate to record from and what
coding algorithm, an output device, and feedback type of signals to record. We will discuss the
for the desired use. These choices are interrelated. considerations related to the neural recordings in
The most appropriate command signal to control Sect. 5.2.
the output device will depend on the goal of the Once the neural signals have been recorded, a
task and the particular device being controlled. In decoding algorithm (or “decoder”) translates the
user’s movement intentions into a control signal feedback into BMI systems. Emerging bidirec-
suitable for guiding the output device. There are tional technology aims to “close the loop” by
two classes of BMI decoders: discrete and contin- inputting sensory signals conveying naturalistic
uous. A discrete decoder estimates one of several proprioceptive or somatosensory information di-
possible movement goals by solving a classifi- rectly to the nervous system via electrical stim-
cation problem. The most common use for this ulation. We will discuss recent progress toward
is a communication device, where patients use closing the loop with somatosensory feedback in
their iBMI to type letters, much as one would if Sect. 5.6.
one were composing a text or an email. Commu-
nication BMIs focus on the speed and accuracy
with which a desired key on a keyboard can be 5.2 Choosing the Input for iBMIs
selected. We will discuss how these devices work
in Sect. 5.3. The most appropriate control signal for an iBMI
A continuous decoder estimates the moment- will depend on the goal of the task and the device
by-moment details of a movement trajectory. This being controlled. In turn, these aspects of the
is needed for guiding a computer cursor or robotic iBMI influence the choice of brain area from
limb along a desired path. For example, a person which to record. In this section, we will discuss
may wish to guide a robotic limb to pick up a the intracortical input signals, the electrodes that
glass of milk without knocking over the milk can be used to record these signals, and the motor
carton. We will talk about continuous decoders in neurophysiology that underlies this choice.
Sect. 5.4.
The decoder produces a control signal that is
then fed into an output device. There are a variety 5.2.1 Neural Signal Recordings
of output devices for BMIs depending on the
particular needs of the user. One common device Intracortical recording techniques provide access
is a computer cursor, where a person controls the to signals that consist of neural activity, which
cursor by thinking about making a movement, can come from individual neurons or groups of
much as they would control a computer mouse. neurons near the electrode. There are three types
Other common output devices for BMI users in- of signals that can be recorded with intracortical
clude robotic arms and motorized wheelchairs. electrodes: single-unit activity, multiunit activity,
Another type of output device is perhaps the most and local field potentials (LFP). Single-unit ac-
natural one: the person’s own limb. In Sect. 5.5, tivity consists of action potentials which emanate
we will talk about recent efforts using electrical from a single neuron. Multiunit activity consists
stimulation to directly activate a patient’s muscles of action potentials from a small group of neu-
to reanimate their own arm. rons near the electrode tip that are not clearly
The final element of the BMI control loop discriminable from one another. The LFP signal
is sensory feedback. The most common sensory is thought to reflect the summation of local neural
feedback is visual: a user can look at the device activity, mostly changes in membrane potentials,
they are controlling and see how it is responding, and is comprised of the activity of perhaps hun-
which allows them to make corrective movements dreds to thousands of neurons. Single-unit activ-
and learn to better control the device. Feedback ity has the most specific information about the
has been shown to dramatically improve BMI fine details of intended movements, with each
performance. Some tasks, however, require more neuron responding uniquely to different aspects
than just visual feedback to be performed dexter- of movement. Multiunit activity and LFPs arise
ously. Consider putting on a necklace. To fasten from averaging over many neurons. Thus, the re-
the necklace behind our head, we must rely on sulting activity consists of a signal that is common
touch to manipulate the necklace clasp. Such to the contributing neurons. While multiunit and
tasks have motivated the inclusion of nonvisual LFP signals are correlated with movement, the
information is not as specific as that obtained

from individual neurons.
Of the three signals, single-unit activity pro-
vides the most specific information about the
fine details of intended movements and has been
Band Pass Filter
shown to lead to good iBMI performance. Single-
600 - 6,000 Hz
unit activity is identified through a process known
as “spike sorting” (Fig. 5.5). The action potentials
recorded with a single electrode can come from
potentially multiple neurons, and in spike sort-
ing, we attempt to classify which action potential
came from which neuron using the neurons’ char-
acteristic waveform shapes. Waveform shapes are
determined by the particular combination of ion
channels expressed by a neuron and the proximity
of that neuron to the recording site and so pro-
vide a “fingerprint” that can be used to uniquely
identify action potentials specific to that neuron.
To identify which action potentials belong to a
given neuron, the recorded voltage trace is typi-
cally first band-pass filtered (e.g., 600 Hz–6 kHz).
After that, the waveform snippets are aligned to SUA1
the time at which the voltage crosses a prede- SUA2
termined threshold. The snippets are then sorted Noise
(i.e., clustered) based on the specific shapes of the
waveforms.
Multiunit activity also leads to good iBMI
performance and does not require the intensive
processing involved in identifying single-unit ac-
tivity. Instead, a voltage threshold is set, and
Fig. 5.5 Neural signal processing cascade for threshold
all waveform snippets that exceed that threshold
crossings (TC) and single-unit activity (SUA). The raw
(i.e., “threshold crossings,” Fig. 5.5) are counted voltage trace recorded from a single electrode is band-pass
with no further assignment to particular neurons. filtered. Then a voltage-based threshold is used to identify
The type and quality of information that can be TCs. TCs can be further sorted into activity attributed to a
single neuron or noise based on waveform shape. (Adapted
extracted from multiunit activity depend on the
from Perel et al. [13])
threshold setting, because the threshold impacts
the effective sampling radius of the electrode. A
selective threshold (i.e., a threshold farther from farther from the electrode than those captured
zero) results in threshold crossings that are likely with a selective threshold.
due to spikes from individual neurons within a LFP signals can also be used for iBMIs, but
small sampling radius, close to the electrode, they do not offer as much specific information
akin to single-unit activity. A permissive thresh- as single- and multiunit activity about the move-
old (i.e., a threshold closer to zero) results in ment. However, LFP signals are not as susceptible
more threshold crossings, because it enlarges the to degradation over time, so there is a trade-
effective sampling radius of the electrode. The off between resolution and duration in choosing
larger effective sampling radius captures thresh- a neural signal for BMI control. Often, LFPs
old crossings from smaller neurons and neurons can provide an alternative signal if single- and
multiunit activity is no longer available. LFPs can has an impedance of roughly 80–150 k , and is
be obtained from different band-pass filtering of separated from its neighbors by 400 μm.
the same electrode signal, typically between 0.3
and 300 Hz. Ultimately, the viability of iBMIs
in a clinical setting will depend on the longevity 5.2.3 Motor Neurophysiology
of the implanted electrodes and their ability to
reliably record signals that are informative about Decisions about where to implant multielectrode
movement. Using LFP as a secondary input signal arrays and how to design decoding algorithms
could extend the lifetime of the iBMI. to extract movement information are guided by
our understanding of how movement is controlled
naturally. Let’s consider the multiple processes
involved in picking up a cup for a sip of coffee.
5.2.2 Multielectrode Arrays The sight of the coffee cup might inspire a desire
for a sip of coffee, and the desire for coffee is
The goal of controlling an output device with translated into a plan to reach for the cup. The
multiple degrees of freedom drove the develop- hand is then shaped to grasp the cup, and the arm
ment of multielectrode recording arrays. Record- extends to bring the hand toward the cup. The cup
ing arrays come in many shapes and sizes and is then grasped with an appropriate level of force,
enable recordings from a large number of neurons and the cup is brought to the mouth. Throughout
at the same time. Multielectrode arrays are the this process, visual, tactile, and proprioceptive
primary recording technology used for iBMIs. feedback are used to adjust the movement to
Each electrode within the array records from a ensure our actions are successful. For an iBMI
small population of neurons close to the elec- to work as seamlessly as natural movement, we
trode tip. There are three main types of multielec- will likely want to make use of the natural control
trode arrays: microwires, flexible polymer-based signals. For decades, neuroscientists have worked
microelectrode arrays, and silicon-based arrays. to understand how the motor cortex produces arm
Microwires are typically made of stainless steel movements in healthy individuals. Understanding
or platinum–iridium. They can be customized to how movement occurs naturally can inform the
include the desired number of electrodes in the design of technologies like iBMI to improve the
desired configuration and of the desired length. quality of life for individuals with injury or dis-
Flexible arrays are made of polymers that are not ease.
as stiff as microwires and, as such, are a closer Primary motor cortex (M1) has long been
mechanical match to the soft brain tissue into thought to be an ideal location for recording
which they are implanted. This design can lead BMI control signals because it is involved in
to less damage to the tissue, a lower inflammatory generating voluntary movements (Fig. 5.7).
response, and consequently better quality signals. However, other brain areas also have signals
While microwires and flexible arrays have many related to aspects of movement. For example,
attractive features, they can be fragile. One of premotor cortex (PMd) has movement planning
the most popular electrode arrays for iBMIs is a signals, and posterior parietal cortex (PPC) has
silicon-based array, the Utah Array (Fig. 5.6). It is been shown to be involved in the transformation
the only array currently approved for clinical tri- from visual representations of reach goals to the
als with human patients by the US Food and Drug movement itself. The brain’s “sensory areas”
Administration (FDA). It is a silicon-machined also often reflect internal representations of
device which permits the simultaneous implanta- stimuli and movements. Each of these brain
tion of 100 platinum–iridium electrodes in a small areas is comprised of neurons that modulate their
region of cortex (16 mm2 ). Each electrode array activity in association with different aspects of
consists of a silicon base with a 10 × 10 grid movement. Thus, there are neural signals from
of electrode shanks etched into it. Each electrode many brain areas that could be used as the input
Fig. 5.6 The Utah array is a 100-electrode microelectrode array that is commonly used to record neural signals for
iBMIs. (Credit: Utah Array- ©2019 Blackrock Microsystems)
SMA M1
central
sulcus
SMA Cingulate Cortex
M1
PMd
S1
Posterior
Parietal
Cortex
PMv
Cerebellum
Cerebellum
Fig. 5.7 Lateral (left) and medial (right) views of a human brain with some of the brain areas involved in motor control
highlighted
to an iBMI depending on the type of information area (SMA), posterior parietal cortex (PPC),
to be extracted and the decoding algorithm. and primary somatosensory cortex (S1) can all
The desired use of the BMI helps determine contribute information to real-time predictions of
the choice of brain area from which to record hand position, velocity, grip force, and muscle
the input signal. Recordings from primary activity [5]. However, the different cortical
motor cortex (M1), dorsal and ventral premotor areas vary in the quality of their predictions
cortex (PMd and PMv), supplementary motor of different aspects of movement. Thus, the
source of the input signal should be a brain area where y is the firing rate of the neuron, b is its
which is informative about the desired aspects of baseline firing rate (i.e., the mean firing rate), m is
movement. the modulation depth (i.e., the difference in firing
One way to design a BMI that controls the rate between the baseline firing rate and the max-
movement of a device in a natural way would be imum firing rate), θ is the direction of movement,
to mimic how the brain controls arm movements. and θ− →
p is the direction of movement that elicited
To do this requires an understanding of how the the highest firing rate (i.e., the neuron’s preferred
brain produces arm movements. However, for the direction). This study showed that, in addition to
purposes of a BMI, we can do quite well by reflecting forces and muscle activity, M1 activity
just considering how neural signals are corre- also reflects the direction of arm movement.
lated with different aspects of movement. There Since these experiments, a number of groups
is a debate in the field of motor control research have reported a correlation between M1 firing
about whether M1 generates arm movements by rates and various kinematic variables, including
directly controlling muscle activity or by signal- direction and distance of targets, as well as di-
ing aspects of movement like position, velocity, rection, speed, and spatial path of hand displace-
and acceleration that are then transformed into ment. Other groups have found M1 firing rates
muscle activity by downstream neural networks to be related to forces and even to muscle activ-
such as the spinal cord. There is evidence for ity. It appears that M1 includes a heterogeneous
both representations, which can be summarized representation of both the kinematics and kinetics
by the results of two seminal studies. The first of limb movements. The good news, from the
was an experiment performed by Ed Evarts in perspective of designing an iBMI, is that either
awake behaving monkeys [14]. He trained mon- representation can be exploited as a BMI control
keys to make wrist flexion and extension move- signal, depending on the intended function of the
ments against opposing or assistive loads while device. If we can accurately extract information
recording single-neuron activity in M1 and mus- about the position, velocity, or acceleration of
cle activity (EMG). The experimental design dis- the desired movement from the neural activity,
sociated the movement itself from the force re- that type of control signal can be used to move
quired to produce it. For example, a given flexion the robotic arm. If we can accurately extract in-
displacement under an opposing load required formation about muscle activity from the neural
greater activity of the wrist flexor muscles than activity, that control signal can be used to drive
under an assistive load. The experimental results muscle stimulators.
showed that force was reflected in the firing rate The control signal for a BMI output device
of the M1 neurons he recorded. Thus, activity could conceivably be any of the aspects of move-
in the motor cortex reflects kinetic aspects of ment with which neural signals are correlated.
movement, i.e., force or muscle activity. Most current BMIs utilize kinematic signals to
The second relevant study was an experiment control external actuators such as computer cur-
by Georgopoulos and his colleagues in which sors or robotic limbs. We can drive robotic limbs
they found that most (75%) of the neurons they because kinematics (i.e., position and velocity)
recorded had a firing rate which varied with the can be directly decoded from neural activity and a
direction of hand movement [15]. Neural sig- kinematic signal could drive the endpoint position
nals in M1 were recorded while a monkey made of the limb (see Sects. 5.3 and 5.4). BMIs could
reaches from the center of a workspace out to also take advantage of the kinetic (i.e., force-
eight peripheral targets. The relationship between related) signals in M1. A demonstration of a
firing rate and direction looked like a sinusoid and kinetic BMI is cortically controlled stimulation
could be described by the equation of paralyzed muscles. We can reanimate the arm
because muscle activity can be directly decoded
y = b + m cos θ − θ−
→
p (5.1) from neural activity and a kinetic signal could
control electrical stimulation of paralyzed muscle to type. Rather than solving the neural encoding
tissue (see Sect. 5.5). of speech, we only need to solve a classification
The decision about which brain area one problem: of all the letters on the screen, which
should record from should take into account character is the person trying to select?
which aspects of movement are best suited for the In this section, we will describe classification
intended type of control. A kinetic BMI, like one decoders, including an example of how to imple-
to control stimulation of muscles, would benefit ment a classifier. We will discuss how to estimate
from a muscle-like input signal. Such a BMI is the model parameters of the classifier and how to
likely to be implanted in M1, which has shown use the resulting classifier in a BMI context.
strong correlations to muscle activity. On the
other hand, a user who will be engaging primarily
in computer cursor control might benefit more 5.3.1 Classification Decoders
from an implant in an area of the brain that
strongly encodes kinematic signals, such as the The goal of classification is to take an input and
location of movement goals. Activity in premotor assign it to one of K discrete classes (Fig. 5.8).
cortex (PMd and PMv; Fig. 5.7) reflects target For an iBMI, the input is the spike counts across
positions [16, 17] and could function as the signal a population of neurons (in Fig. 5.8, two neu-
source for a BMI to be used for a communication rons are illustrated). We ask which of K discrete
interface involving target selection similar to movements most likely corresponds to the user’s
typing [18]. For endpoint control of a robotic intended movement. To begin, we need labeled
limb, it would be advantageous to decode a training data. For example, we could record a
kinematic signal such as hand position or velocity. user’s neural activity while he or she imagines
Although this has been most notably done with reaching to various letters displayed like keys on
signals from M1 [19], areas PMd, SMA, and S1 a keyboard, as instructed by the experimenter. As
also contain information about hand position and shown in Fig. 5.8a, the training data consists of
velocity [5]. the class label (i.e., the imagined letter, depicted
as different colors) and the value of each data
point (i.e., the activity of two neurons, y1 and y2 ).
5.3 Intracortical Spelling In the training phase, we fit a probability model
Devices to the training data, a process we will describe
in detail below. This training phase defines a set
The goal of a communication BMI is to provide a of decision boundaries between the classes (Fig.
means of communication for the user. This might 5.8b). Once the classifier has been trained, we can
hold particular value for locked-in patients, who then use the classifier to predict the label of a new
are no longer able to speak. Ideally, we would data point (Fig. 5.8c). We do this by comparing
record neural signals from the parts of the brain where the data point falls relative to the classi-
responsible for speech, decode the intended mes- fier’s decision boundaries. In the example shown
sage, and use that to drive a speech synthesizer in Fig. 5.8c, the new data point would be assigned
or a speech transcription program. However, the to class 3.
neural encoding of speech is only now beginning Before we describe how a classifier works in
to be understood. Instead, current devices lever- detail, let’s start with a simplified example to
age our understanding of the neural representa- build intuition. In this example, the iBMI user is
tion of intended movements to design spelling typing one of three different letters, either “E,”
devices through control of an onscreen keyboard. “Q,” or “A,” while we record activity from a
Users imagine reaching toward the letter they single neuron (y). First, we ask the user to re-
would like to type. By decoding the intended peatedly imagine reaching to the letter “E” while
movement from motor and premotor cortex, it is we record the neuron’s spiking activity. Because
possible to infer which letter the person is trying neurons are noisy, the neuron’s spike counts will
A Training Data B Define decision boundaries C Test Data

class 1
class 2 class 1 class 1
class 3
y2
y2
y2
class 2 class 3 class 2 class 3
y1 y1 y1
Fig. 5.8 (a) To train a classifier, we collect a set of labeled the different classes of data points into different regions.
training data, consisting of a set of data points where each (c) Given a new data point (open circle) for which we do
data point consists of values, y1 and y2 , along with their not know the true class label, the classifier will predict the
corresponding class labels, depicted here as the color of class of that new data point using its decision boundaries.
each data point. (b) The classifier uses the training data to In this case, the new data point would be assigned to class 3
create a set of decision boundaries (black lines) that divide
not be exactly the same every time (Fig. 5.9a). ambiguous. In general, the range of ambiguous
Instead, we obtain a distribution that reflects the spike counts of the two neurons will not overlap
conditional probability of measuring a particular because neurons have different preferred stimuli.
spike count given that the person is intending to Thus, adding even just one more neuron will
reach to the letter “E” (Fig. 5.9b), here idealized likely enable our classifier to make a more ac-
as a Gaussian distribution. We can repeat this curate guess about the intended letter. Similar
process for the letter “Q” and again for the third to the single-neuron example, we can estimate
letter, “A.” the distribution of spike counts of two (or more)
How can we use this spiking activity to build a neurons conditioned on the user’s intentions to
classifier that will classify the letter a user intends select each of the three letters (Fig. 5.9c). Now,
from only the neural activity? First, let’s suppose the distribution of spike counts corresponding to
the neuron spiked 30 times. We would probably each letter is a region in a plane (for two neurons)
guess that the user was intending to reach to the or in an N-dimensional space for N neurons. This
“A” because that is the letter that is the most prob- looks very much like the scenario depicted in Fig.
able for that spike count. Similarly, if the neuron 5.8 for which classifiers are designed.
spiked five times, by comparing the probability Now that we have established some intuition,
distributions, we would guess that the user was in- let’s talk about how to implement the classifier
tending to reach to the “E.” In general, we would using a probability model. Here we’ll suppose
like our classifier to predict the most likely letter that the user is typing one of K different letters,
given the recorded spike counts by comparing which we’ll refer to as c1 , c2 , . . . , cK . (In the
the measurement to the conditional distributions above example, we had K = 3, with c1 = “E,”
of the neural activity. If we recorded 20 spikes, c2 = “Q,” and c3 = “A”). While the user is
what letter would we guess the user intended? intending to type these different letters, we record
This time it is not obvious because intending the spike counts, y ∈ Rd , from d different neurons.
to reach to either the “Q” or the “A” would be To build the classifier, we would like to be able to
equally likely to generate that measurement. Now predict which letter, ck , the user was most likely
suppose we repeated this exercise for a second intending just from observing the neural activity,
neuron. Neuron 2 will also have spike counts y. In other words, we want to know P(ck | y).
for which the classification will be unambiguous What our training data provides, however, is the
and spike counts for which the classification is reverse: the distribution of spike counts given the
A E Q A
trials
time
B C
(spikes/second)
Neuron 2
p(y|Ck)
0 10 20 30
Neuron 1
(spikes/second)
Neuron 1
(spikes/second)
Fig. 5.9 (a) A user imagines typing the letter “E” many a different distribution of spike counts, here idealized as
times while neural activity is recorded from one neuron. Gaussians. (c) If we record from a second neuron’s spike
We can plot the neural activity across time as a raster plot, counts simultaneously, we can plot the joint distribution
in which each row is a trial and each mark represents the of spike counts in both neurons for each letter. For each
time that a spike occurred. We count the spikes occurring set of spike counts, we can fit a multivariate Gaussian
within a given window (black box) and repeat this process to the data by estimating the mean (unfilled circle) and
for other letters (e.g., “Q” and “A”). (b) We can plot the covariance (ellipse). Adding additional neurons will likely
conditional distribution of spike counts we recorded given improve our ability to discriminate which letter the user
that the user imagined typing the letter “E” (red), “Q” was imagining
(black), or “A” (blue). For each letter, we will likely see
intended letter, P(y | ck ). Additionally, because we cally, given y, the classifier will predict the class
know the true letters the user was intending in the as follows:
training data, we also know P(ck ), the proportion
P (y|ck ) P (ck )
of data points with a given class label. We can k̂ = argmaxP (ck |y) = argmax
relate all of these terms using Bayes’ rule: k k P (y)
(5.3)
P (y|ck )P ( ck )
P (ck |y) = (5.2) We can ignore the denominator in Eq. 5.3 because
P (y)
P(y) is the same for every class k, so it does not
To predict the letter the subject was intending affect which k yields the maximum.
given the neural activity y, we will simply choose Equation 5.3 tells us how we can predict the
the class (k) that has the largest value of P(ck | letter the user was most likely intending, given
y). This is similar to how in our single-neuron only the spiking activity, y. The right-hand side of
example, we chose the letter that had the highest the equation includes two terms: the conditional
probability in Fig. 5.9b. To write this mathemati- probability of spiking given the intended letter,

P(y| ck ), and the prior probability of each letter, L μk , Σk ; y1 , . . . , yN, ck = P (y1 , . . . , yN |ck )
P(ck ). We now discuss how we can use our train- (5.7)
ing data to estimate these two quantities.
Let’s begin by discussing the first term. What The approach of maximum likelihood estimation
we would like is to use the training data to deis to choose the parameters most consistent with
scribe the distribution of neural activity observed the observed data. In other words, we will choose
for each letter. In Fig. 5.9b, c, we note that a Gaus- the parameters that maximize the probability of
sian captures the first and second moments (i.e., the neural activity that we observed:
mean and covariance) of the distribution of neural
activity. This means when the user is intending a μk , Σk = argmax L (μ, Σ; y1 , . . . , yN , ck )
particular letter (e.g., ck = “E”), the distribution μ,Σ
of observed spike counts (y) can be described by (5.8)

a Gaussian with a mean and covariance:
By maximizing the log likelihood function for
P (y|ck ) = N (μk , Σk ) (5.4) each class k, we can find μk , k for each of the
distributions P(y| ck ). We would find (with a few
where y ∈ Rd is a vector of spike counts from lines of math, omitted here) that the parameters
a population of d neurons, μk describes their μk , k are the sample mean and sample covari-
mean spike counts, and the covariance matrix k ance of the spike counts recorded with class k:
describes any correlations that might exist among
1 N
neurons. To estimate these mean and covariance μk = yi (5.9)
N i=1
parameters, we seek to find the parameters μk
and k that maximize the probability of hav-
ing observed the activity that we observed. This
1 N
widely used procedure is known as maximum Σk = (yi − μk ) (yi − μk ) (5.10)
likelihood estimation (MLE). Let’s suppose we N i=1
have N examples (or trials) of recorded neural ac- If we had assumed that the covariance of neu-
tivity, {y1 , . . . , yN } when the user was imagining ral activity was the same across classes (i.e.,
typing the same letter ck . Then the probability of 1 = 2 = . . . = k ), it can be shown that
recording a particular yi (i.e., on a single trial) is the resulting decision boundaries between classes
−1
are linear, as shown in Fig. 5.8c. As a variant of
P (yi |ck ) = (2π)−d/2 |Σk |−1/2 e−(yi −μk ) Σk (yi −μk )/2 this Gaussian classifier, we could instead describe
(5.5) the distributions of neural activity P(y| ck ) using a
Poisson distribution and perform the same MLE
Assuming the neural activity recorded across
procedure to estimate its parameters.
trials is conditionally independent, the probability
The other term we need to know in order
of observing {y1 , . . . , yN } is the product of ob-
to implement the classifier is P(ck ), the prior
serving each individual trial:
probability of each class. This is the probability
N that a user is likely to want to type each letter
P (y1 , . . . , yN |ck ) = (2π)−d/2 |Σk |−1/2 · without having observed any neural activity. For
i=1

Σk −1 (yi −μk )/2 example, in the English language, “E” is a much
e−(yi −μk ) more common letter than “Q,” which means we
(5.6) should expect to observe “E” more often than
“Q.” Or, imagine that instead of typing letters,
This joint probability indicates the “likeli-
the goal was to select among different icons on
hood” of observing the spike counts given that
a computer screen. It might be that each icon is
the true parameters were μk and k . For this, we
expected to get the same amount of use. In this
write
case, P(ck ) = 1/K for each k = 1, . . . , K. Using 4 targets 8 targets

Bayes’ rule, the classifier accounts for the prior
probability when making its predictions (see Eqs.
5.2 and 5.3).
As an example of using a classification de- 100 7
Decode accuracy (% correct)

coder in a BMI setting, researchers recorded neu- 90 6
ral activity from premotor cortex in monkeys 80 5
to predict the intended reach target from neural
ITR (bps)
70 4
activity while a monkey planned a reach [20]. The 60 3
researchers assumed that neurons were Poisson 50 2
and conditionally independent of one another.
40 1
They then decoded the most likely target given the
30 0
observed neural activity using a method similar
200 500 200 500
to the approach described above. In this case, the Trial length (ms) Trial length (ms)
monkey made arm reaches to each of eight targets
Fig. 5.10 The relationship between single-trial decod-
that were equally likely by design. The classifier
ing accuracy and information transfer rate (ITR). Per-
successfully decoded the correct target from the formance was measured during iBMI experiments for a
neural activity on 90% of the trials. This work four-target configuration and an eight-target configuration
was the first demonstration that neural activity across varying trial lengths. Each data point represents
performance calculated from one experiment (hundreds of
recorded during the movement planning period
trials). (Adapted from Santhanam et al. [18])
could be decoded as a useful control signal for
a classification iBMI.
Performance of classification decoders is as- classification decoder can convey 6.5 bits per
sessed based on the speed and accuracy of target second or 2–3 targets per second with greater
selection. In general, fast and accurate classifica- than 90% accuracy. This would allow users to
tion is difficult because neural activity is variable type at a speed of 15 words per minute. While
(cf. Fig 5.9a). The approach is to average the neu- this is an improvement over noninvasive BMIs,
ral activity over a longer window of time. Because this is not yet up to the average typing speed
neural variability is Poisson-like, averaging over a of 40 words per minute. State-of-the-art commu-
longer window reduces the “noise” and results in nication BMIs, such as the one shown in Fig.
a more accurate prediction of the target. However, 5.3, combine discrete and continuous decoders.
with longer time windows, fewer predictions are We will discuss continuous decoders in the next
made each second, resulting in slower decoding. section.
There is a speed accuracy trade-off that makes the
choice of the particular duration and placement of
the time window an important design choice. 5.4 Intracortical Control
Rather than evaluating BMI performance on of Continuous Effectors
accuracy alone, one should include some measure
of speed as well. One metric that is often used In the spelling device, and in classification de-
for this purpose is the information transfer rate coders in general, we decode the intended target
(ITR; [18]). ITR measures how much information (or letter) directly from the neural activity. How-
is conveyed per unit time. ITR increases with ever, if we want to control a robotic arm, we need
window duration but then decreases (Fig. 5.10). to specify the path that the arm will take so that,
This is because ITR takes into account both how for example, the user can prevent the arm from
accurately and how quickly each target is se- bumping into objects in the workspace. To specify
lected. Accuracy fails to increase rapidly enough the reach trajectory, we need to decode the evolu-
to overcome the slowdown in target selection rate tion of the desired movement at progressive time
with longer window durations. An intracortical steps. In other words, we need a continuous de-
coder. Accurate decoding of a continuous control associated with a given firing rate. In Fig. 5.11,
signal is necessary for controlling not only robotic suppose that the neuron is firing at 30 spikes
arms but also computer cursors or a patient’s per second. This could correspond to the subject
own paralyzed limb. In this section, we discuss reaching at 0◦ (yellow) or 135◦ (cyan). As we saw
three continuous decoders for BMI control: the in the classification example above, the activity
population vector algorithm, the optimal linear of just one neuron can be ambiguous, but we can
estimator, and the Kalman filter. Variations on the solve this problem by recording from a population
Kalman filter are the current state of the art for of neurons in order to reduce uncertainty in our
continuous decoders. estimate of the reach direction.
As its name suggests, the population vector
algorithm (PVA) utilizes the activity of a popula-
5.4.1 Population Vector Algorithm tion of neurons to estimate the desired movement.
Each neuron contributes a “push” in the direction
One of the first continuous decoders was the of its preferred direction. This push is weighted
population vector algorithm (PVA). The PVA by the neuron’s normalized firing rate, given by
was proposed by Apostolos Georgopoulos in wi = yim−bi i , where yi is the measured firing rate
the 1980s as a way of decoding movement of neuron i, bi is the neuron’s baseline firing rate,
direction from a population of neurons. As and mi is its modulation depth. The algorithm then
mentioned in Sect. 5.2, the firing rates of averages all of the neurons’ contributions together
neurons in motor cortex reflect the direction of a to yield the resulting command. Mathematically,
reach, as the relationship between a neuron’s the PVA decoder is a weighted vector sum of
firing rate and the arm’s reach direction is each of the recorded neurons. Taken together, the
approximately cosine tuned (Fig. 5.11). This prediction of the intended movement direction is
means that there is a reach direction for which the the resulting population vector.
neuron fires maximally. We refer to this direction Figure 5.12 shows a simple example of the
as the neuron’s “preferred direction,” θ− →
p . The PVA using two neurons at one time point. Each
neuron’s firing rate decreases gradually as the of the neurons has a different preferred direc-
reach direction moves away from this preferred tion (Fig. 5.12a). The preferred direction of the
direction. Different neurons have different neuron determines the direction of its push. The
preferred directions, so together the activity of red neuron will push “up,” while the blue neuron
a population of neurons can uniquely specify the will push “left” (Fig. 5.12b). The measured firing
arm’s direction of movement. Specifically, when rates determine the magnitude of the pushes (Fig.
the arm is moving in a particular direction, θ , we 5.12c). Let’s consider the firing rates specified by
can describe the firing rate, y, of one neuron as the gray shaded box. The red neuron has a firing
shownin Eq. 5.1.
The firing rate is linearly related rate of yred = 25 spikes per second. Given that this
to cos θ − θ p , and this relationship is the basis
−
→ neuron’s tuning parameters are bred = 35 spikes
of PVA. In motor cortex, studies have used cosine per second and mred = 15 spikes per second,
tuning to describe movement direction, velocity, the magnitude of the red neuron’s push is then
speed, position, force, and torque. Cosine tuning wred = − 0.66. This contribution is negative, so
has also been used to describe neural activity in its push is now “down.” Let’s repeat this process
other nonmotor brain areas. for the blue neuron. The blue neuron has a firing
The fact that a neuron’s firing rate has a sys- rate yblue = 45 spikes per second. Given that this
tematic relationship with reach direction suggests neuron’s tuning parameters are bblue = 35 spikes
that we can accurately decode a subject’s in- per second and mblue = 15 spikes per second,
tended reach direction from the activity of a single the magnitude of the blue neuron’s push is then
neuron. However, it is not easy to estimate di- wblue = 0.66. This contribution is positive, so its
rection of movement from one cosine-tuned neu- push stays “left.” Taking the weighted sum of the
ron because there are multiple reach directions two pushes, the population vector points down
A B 60
Preferred
Direction
(spikes/sec)
Firing Rate
trials
time 40
m
b0
20
0 100 200 300

o
Target Angle ( )
Fig. 5.11 (a) Spike trains for repeated reaches to each of for which the neuron shows the maximal firing rate. The
eight reach directions. (b) A cosine tuning curve describes baseline firing rate, b0 , is the mean firing rate. The modu-
the relationship between the neural firing rate and the lation depth, m, is the difference in firing rate between the
reach directions. The preferred direction is the direction baseline firing rate and the maximum firing rate
A 60 B C D
direction of push magnitude of push population vector
(spikes/sec)
Firing Rate
40
0.66
-0.66
20
0 100 200 300

Target Angle (o)
Fig. 5.12 A two-neuron example of the population vector each neuron’s push is determined by its firing rate. (d)
algorithm. (a) Tuning curves of the two neurons (red The population vector algorithm outputs a resultant vector
and blue). (b) Each neuron contributes a push in the represented by the black arrow that is the weighted sum of
direction of its preferred direction. (c) The magnitude of each neuron’s push
and to the left, corresponding to a movement of tor cortex cause slight contractions of muscles
around 225◦ (Fig. 5.12d). This procedure repeats that push the arm in their preferred directions.
at each time point, as the activity of the neurons However, PVA suffers from statistical biases: if
varies over time. there is a nonuniform distribution of preferred
In a BMI use scenario, the goal is to record directions in the recorded neural population, the
neural activity from a population of neurons and PVA will systematically misestimate the intended
convert it into the position of a cursor on a screen reach direction. In practice, it is rare to record a
over time. In contrast to the classification de- population of neurons with a uniform distribution
coders of the previous section, here we decode of preferred directions. While this bias can be
the cursor position at each point in time. This mitigated by recording from a large number of
provides the user with continuous control over the neurons or by sub-selecting neurons that have
trajectory that the cursor takes. a uniform distribution of preferred directions, a
The PVA is a biologically inspired decoder, better approach is to use an unbiased decoding
where the idea is that perhaps neurons in mo- algorithm. The optimal linear estimator and the
Kalman filter, both of which we will discuss In the OLE, our estimate of velocity is the velocity
shortly, are examples of unbiased decoders. that maximizes the above probability with respect
to the observed neural activity. This velocity is
−1
5.4.2 Optimal Linear Estimator vˆt = B T Σ −1 B B T Σ −1 (yt − b0 ) (5.13)
As discussed above, if we do not have a uniform Note that this estimate of velocity is a linear
distribution of preferred directions, we do not function of the recorded firing rates. Further, the
want to use a PVA decoder. Instead, we should OLE decoder corrects for any nonuniformity in
specify a statistical model that describes the re- the distribution of preferred directions, resulting
lationship between the intended movement and in an unbiased estimate of intended velocity. This
the activity of each neuron. This encoding model, is why this decoder is called the optimal linear
a probabilistic description of how neural activity estimator.
(y) varies based on the intended movement (x), Which decoder should we implement, a PVA
is written as P(y|x). The encoding model has decoder or an OLE decoder? The PVA decoder is
parameters which are estimated during a decoder simpler than statistical approaches like OLE, but
calibration phase. Applying Bayes’ rule, we can the OLE decoder is optimal given the specified
then use this encoding model to create a decod- encoding model. Empirically, if we were to use
ing model, P(x|y), which is our estimate of the each decoder to reconstruct arm trajectories, we
intended movement given the observed neural would see that OLE performs significantly better
activity. than PVA and with fewer neurons [21]. However,
An example of an unbiased continuous de- both decoders do comparably well in a BMI use
coder is the optimal linear estimator (OLE). The scenario because the users can incorporate feed-
OLE makes two assumptions: that firing rates back and correct errors quickly enough to com-
are linearly related to intended movement and pensate for any theoretical differences in system
that neural variability is described by a Gaussian performance.
distribution. We can rewrite the cosine tuning
model in matrix form:
5.4.3 Kalman Filter
yt = b0 + Bvt + εt , εt ∼ N (0, Σ) , (5.11)
Is it possible to do even better? Both PVA and
where yt is the n × 1 vector of firing rates from n
OLE estimate movement velocity given only the
neurons, b0 is the n × 1 vector of baseline firing
neural activity. But there is other information we
rates, B is the n × 2 matrix of tuning coefficients,
could also incorporate into our estimates. For
vt is the 2 × 1 intended velocity, and εt is the n × 1
instance, we know that the cursor or arm should
noise vector. move smoothly. During arm reaches, the arm can-
We can now ask a question that is very similar
not teleport from one location to another instanta-
to the classification problem we solved in Sect.
neously. Rather, there are finite constraints to the
5.3: What is the most likely velocity given a accelerations and decelerations that muscles can
measurement of firing rates from our population?
produce. We can use this information about the
From our encoding model (Eq. 5.11), we know kinematics of the arm during natural reaching to
the probability of the firing rates given the in-
influence how we allow our estimate of the de-
tended movement direction: sired trajectory to change with time. As a simple
example, if we know where the arm is currently
P (yt | vt ) = (2π )−n/2 |Σ|−1/2 (current state), and how fast the arm is moving

1 (state dynamics), we can predict where the arm
exp − (yt − Bvt − b0 ) Σ −1 (yt − Bvt − b0 ) will go next (future state). To use this information
2
(5.12) to improve our ability to decode arm velocity, we
need to combine it with the information we have yt = Bxt + εt , εt ∼ N (0, Σ) (5.14)

from the neural activity.
Consider trying to track a satellite. What where yt ∈ Rn × 1 is the vector of spike counts
sources of information might we use to do so? measured from all n neurons at time step t,
We could simply measure its position. But what B ∈ Rn × d is the matrix of tuning coefficients,
if it goes behind a cloud or over a region of xt ∈ Rd × 1 is the vector of the intended movement
Earth with no sensors? We could potentially use kinematics (e.g., cursor velocity) at time step t, d
Newton’s laws to predict the satellite’s trajectory. is the number of kinematic variables (e.g., d = 2
But what if something collides with our satellite for a two-dimensional velocity), and εt ∈ Rn × 1 is
and changes its course? Intuitively, the best way the vector of additive Gaussian noise, drawn from
to track a satellite would be to combine our a distribution with mean 0 and covariance matrix
measurements and our predictions, weighting ∈ Rn × n . Note that this is the same observation
each source of information according to how model that the OLE uses (Eq. 5.11), with two
reliable it tends to be. exceptions. First, we have assumed that the
These are the intuitions captured by the baseline, b0 from Eq. 5.11, is already subtracted
Kalman filter: it predicts the current state of a from the spike counts. This just simplifies the
system based on our estimate at the previous state derivations below. Second, we denote the state
combined with new observations of data. The as x instead of v, as it is common to incorporate
two key components of a Kalman filter are a state other kinematic variables in addition to velocity,
model that describes how the movement evolves such as the position and acceleration, in a Kalman
over time and an observation model that describes filter. Though for simplicity, we will assume that
how the observations relate to the movement. The x contains only velocity in what follows.
way the state and observation models combine The state model is defined as
can be visualized graphically as in Fig. 5.13, with
the red arrows indicating the state model and the xt = Axt−1 + ωt , ωt ∼ N (0, Q) (5.15)
black arrows indicating the observation model.
We outline below how these two models are where A ∈ Rd × d describes how the velocity
combined to update our predictions at each time evolves from one time step to the next, ωt ∈ Rd × 1
step t. The Kalman filter is based on linear- is additive Gaussian noise to the velocity, and
Gaussian relationships. First, we define the Q ∈ Rd × d is the covariance matrix of the velocity
observation model: noise. Notice how in the observation model (Eq.
5.14), the current state is linearly related to the
observed neural activity and in the state model
(Eq. 5.15), the state at time t is linearly related to
xt-2 xt-1 xt
the state at time t − 1.
To calibrate the decoder, we estimate the pa-
rameters B, , A, and Q in the observation and
state models. Typically, in a decoder calibration
session, the neural activity is recorded, while the
states of the arm or cursor are known. This can
yt-2 yt-1 yt be done in a number of ways. Some decoders
are calibrated based on arm movements. Other
decoders are calibrated by moving a cursor on the
Fig. 5.13 Graphical model of a Kalman filter. Each verti- screen and having the user intend or imagine that
cal slice represents a time step. The nodes in red represent
the state (e.g., movement velocity), while the nodes in blue
they are moving the cursor. In this way, the state
represent the observations (e.g., spike counts). Each arrow of the cursor is known or assumed during decoder
represents a probabilistic relationship between the nodes calibration. Because Eqs. 5.14 and 5.15 are linear-
Gaussian, when both the states and neural activity For the particular state and observation models
are known, we can find the parameters using defined in our example (Eqs. 5.14 and 5.15), we
multivariate linear regression on the calibration can simplify Eqs. 5.16 and 5.17. Because the
data. We leave the derivation of the equations relationships in our state and observation models
for the parameters from the observation and state (Eqs. 5.14 and 5.15) are linear-Gaussian, this
models to homework problem #7. means that all of the relevant marginal, condi-
How do we use these models to decode move- tional, and joint distributions are also Gaussian.
ment trajectories from neural activity for a BMI? Thus, all we need to do is compute the mean and
What we would like to know is P(xt | y1 , . . . , yt ), covariance of each distribution.
which describes the probability of the intended We start with the state estimate at the
movement velocity at a particular time step given previous time step t − 1, P(xt − 1 | y1 , . . . , yt − 1 ).
all of the recorded neural activity up to that time Let its mean and covariance be μt − 1 and
step. The decoded movement is the movement Φ t − 1 , respectively. The mean and covari-
that maximizes this probability. ance of the one-step prediction distribution
To decode the movement velocity at all time are μ− t = E [xt |y1 , . . . , yt−1 ] and − t =
steps, we will compute P(xt | y1 , . . . , yt ) sequen- Var [xt |y1 , . . . , yt−1 ]. We can solve for these by
tially starting from t = 1. In order to do this, we plugging the state model into Eq. 5.16:
will first need to find P(xt | y1 , . . . , yt − 1 ). This is
called a “one-step prediction” and can be found μ−
t =AE [xt−1 |y1 , . . . , yt−1 ]
from the previous time step and the state model (5.18)
+ E [ωt |y1 , . . . , yt−1 ] = Aμt−1
as follows:
Similarly, for the covariance:
P (xt |y1 , . . . , yt−1 )

−
t = Var [Axt−1 + ωt |y1 , . . . , yt−1 ]
= P (xt |xt−1 ) P (xt−1 |y1 , . . . , yt−1 ) dxt−1 (5.19)
(5.16) = At−1 AT + Q
This equation describes our current estimate In the measurement update, we use the new
of the movement velocity at time t given all of observation yt to update the one-step prediction to
our observations up to time t − 1, along with compute the state estimate at the current time step
our knowledge of how the movement kinematics t, P(xt | y1 , . . . , yt ). Let its mean and covariance be
evolve over time. That is, it’s a prediction of μt and Φ t , respectively. To compute μt and Φ t ,
where the state may have gone since our last we first obtain the joint distribution of xt and yt
measurement. We then augment this prediction given y1 , . . . , yt − 1 . Using the one-step prediction
with a “measurement update” that describes how and the observation model, we find:
this prediction changes when we observe yt :

yt | y1 , . . . , yt−1
P (yt |xt ) P (xt |y1 , . . . , yt−1 )
P (xt |y1 , . . . , yt ) = xt | y1 , . . . , yt−1
P (yt |y1 , . . . , yt−1 )
−
(5.17) Bμ− B−
t B + Σ Bt
T
∼N t
,
μ− t −t B
T −t
The one-step prediction and measurement (5.20)
update (Eqs. 5.16 and 5.17) are general. They can
be used for any state and observation model Then, using the theorem of conditioning for
as long as the graphical model is as shown jointly Gaussian random variables, we can solve
in Fig. 5.13. for

μt = μ− −
t + Kt yt − Bμt (5.21) cursor or robotic limb at each time step t. Φ t is
the uncertainty around that estimate. These steps
are illustrated in Fig. 5.14.
t = (I − Kt B) −
t (5.22)
The Kalman filter and OLE have advantages
over methods such as PVA because their
where the Kalman gain Kt is the d × n matrix:
assumptions are made explicitly and they provide
−1 an uncertainty around the state estimate. Having
Kt = − T − T
t B Bt B + Σ (5.23) explicit assumptions means that we can easily
change our assumptions and derive a different
The Kalman gain indicates how much the mea- continuous decoder. In practice, the leading iBMI
surement influences the update. When the uncer- decoders in the field today are variants of the
tainty in the measurement is large compared to Kalman filter.
uncertainty in the state estimate, the Kalman gain An example of a high-performance closed-
is small. On the other hand, when uncertainty loop iBMI [22] using a Kalman filter is illustrated
in the measurement is small compared to uncer- in Fig. 5.15. This approach involved two key
tainty in the state estimate, the Kalman gain is modifications to the basic Kalman filter. First, the
large. experimenters assumed that the user intended to
To summarize, we implement the Kalman fil- produce velocities straight to the instructed target
ter by iterating between the one-step prediction at every time step, rather than produce the cursor
(Eqs. 5.18 and 5.19) and the measurement update velocities that were actually decoded (red vectors
(Eqs. 5.21 and 5.22) for time steps t = 1, . . . , in Fig. 5.15) following the original calibration.
T. Using this procedure, we obtain the estimated They used this information to improve the de-
kinematics μt that is used to move the computer
A BMI user’s workspace B previous state C one-step prediction

time t
N(μt, Φt)
target
N(μt-1, Φt-1)
xt | y1, ..., yt~ N(μt, Φt)

time t-1
D new observation E measurement

update
xt-1 | y1, ..., yt-1~ N(μt-1, Φt-1) N(μt, Φt) N(μt, Φt)
N(μt, Φt)
BMI cursor
(xt = velocity)
P(yt | xt) P(yt | xt)
Fig. 5.14 Implementing a Kalman filter. (a) At each time tion. (d) When we observe a new measurement of neural
step t, we update the cursor’s position (black circles) by activity, yt , the Kalman observation model provides us
adding to it the cursor velocity, xt , which we estimate with additional information about the likelihood of the
using a Kalman filter (panels b–e). (b) At time step t − 1, intended cursor velocity. (e) In the final step, we use
we have an estimate of the previous state of the cursor the measurement update to combine the two sources of
velocity, a Gaussian with mean μt − 1 (pink dot), and a information about cursor velocity to arrive at our final
covariance t − 1 (pink ellipse). (c) At time step t, we estimate of the cursor velocity μt , which is then used to
first update the estimated velocity distribution using our update the position of the cursor in panel a
Kalman state model, according to the one-step predic-
BMI user’s workspace intention-trained Kalman filter or ReFIT-KF [22].

Compared to a standard-velocity Kalman filter,
BMI
the ReFIT-KF increased performance by reducing
cursor target the time required to move a computer cursor to hit
t=1 a target. The ReFIT-KF is currently being used in
clinical trials (Fig. 5.3).
t=3
t=2
t=4 5.5 Reanimating Paralyzed
Limbs
Observed cursor kinematics
BMI Our vision is that one day paralyzed people will

cursor walk, shake hands, interact with objects, and
target
overall behave in a manner that is virtually
indistinguishable from healthy individuals.
Although this goal is far from realized, ongoing
research is promising. Consider the development
of the pacemaker over the past 50 years. The
original pacemaker recipient was confined to
ReFIT estimated cursor kinematics a wheelchair due to the extensive externalized
devices and required daily maintenance from
Fig. 5.15 Top: A series of cursor positions (black circles) trained care givers. With the assistance of
and cursor velocities (red vectors) from an example trial on
which the user was trying to navigate a cursor toward the
the pacemaker, he lived another 43 years and
target (yellow circle). Axes are the horizontal and vertical passed away at 86 from causes unrelated to his
position in the BMI user’s workspace (e.g., as seen on a heart. These days one would be hard-pressed to
computer screen). Bottom: Decoding with a Kalman filter determine who has a pacemaker and who does
can be improved by assuming that the user was trying
to produce a velocity straight to the instructed target at
not without an X-ray machine. We anticipate a
every time step. Rather than calibrating the decoder on similar development for iBMI systems.
the observed cursor kinematics (red vectors), the assumed Most current iBMIs decode kinematic control
kinematics are obtained by rotating the observed velocities signals, such as desired velocity, in order to con-
toward the instructed target (purple vectors). The estimate
of intended kinematics is regressed against neural activity
trol a computer cursor or a robotic arm. As an
to obtain the parameters of the ReFIT-KF output device, robotic arms are most appropriate
for amputees, but many potential iBMI users have
intact limbs. If we could decode desired muscle
coder by rotating the decoded cursor velocities activity directly from the brain, we could use
to point toward the target (purple arrows in Fig. functional electrical stimulation (FES) to directly
5.15) and using the resulting velocities in a sec- activate a patient’s muscles to reanimate their
ond round of calibration. This requires knowledge own limbs. This would allow a person with paral-
of the intended target during the decoder calibra- ysis to regain the ability to interact with the world
tion phase. The second change was a causal inter- with their own limbs. Although FES applications
vention in which the feedback the user received have been developed for upper and lower ex-
about the cursor position was taken to be known tremity function, bowel and bladder control, and
with no uncertainty. Doing so meant that the respiratory function, here we will focus on FES
user’s estimate and the algorithm’s estimate of the for grasping. In this section, we will describe
cursor position were the same, effectively remov- how to decode desired muscle activity and how
ing the uncertainty in the cursor position. Taking to deliver electrical stimulation to the muscles in
these changes together, they called their extension order to match that desired activity.
of the Kalman filter the recalibrated feedback
5.5.1 Functional Electrical municate directly with muscles and are ultimately
Stimulation responsible for generating movement. They cause
muscle contractions by releasing the neurotrans-
Functional electrical stimulation (FES) is mitter acetylcholine at the synapse of the alpha
neuromuscular stimulation used to restore motor motor neuron onto skeletal muscle. This synapse
function to paralyzed limbs. This is possible is termed the neuromuscular junction. Acetyl-
because neurons are electrically excitable. There choline binds to receptors on the muscle fiber
is an electric potential maintained across the cell and generates a muscular action potential that
membrane. Physiologically, synaptic inputs to a causes the muscle to contract. Although the mus-
neuron cause a change in the membrane potential, cle tissue itself is electrically excitable, most FES
and action potentials are generated when the systems target the alpha motor neurons because
membrane is depolarized past a certain threshold. they require less current to generate action po-
Electrical stimulation can artificially depolarize tentials than activating the muscle fibers directly
the membrane in a similar way to generate action (Fig. 5.16). Thus, FES requires the alpha motor
potentials. neuron to be intact and the neuromuscular junc-
FES electrically stimulates the neurons that tion and muscle to be healthy. These requirements
are responsible for generating movement, called exclude patients with polio, amyotrophic lateral
alpha motor neurons. Alpha motor neurons com- sclerosis, peripheral nerve injuries, and muscular
M1
Corticospinal Damage in
patients with SCI
Tract
X
Spinal Cord
Alpha Motor
Neuron
Muscle FES device
Fig. 5.16 Many neurons in M1 extend down the corti- as the neuromuscular junction. Spinal cord injury (SCI)
cospinal tract in the spinal cord and synapse either directly interrupts the connection between M1 and the muscles.
onto alpha motor neurons or onto interneurons that in turn Functional electrical stimulation (FES) artificially gener-
synapse onto alpha motor neurons. Alpha motor neurons ates action potentials at the alpha motor neuron to generate
synapse onto muscle fibers at a specialized contact known movement in people who are otherwise paralyzed
dystrophies. Patients who can benefit from FES the amplitude, the stronger the depolarizing
include those with spinal cord injury, stroke, head effect. This recruits more neurons and results
injuries, cerebral palsy, or multiple sclerosis. in a stronger muscle contraction. Adjusting these
A single alpha motor neuron makes synapses parameters changes the strength of the evoked
onto several (10–100) muscle fibers. These mus- muscle contraction. Regardless of the particular
cle fibers are driven only by that single mo- parameters, FES stimulation typically consists of
tor neuron. Small motor neurons innervate slow- biphasic, charge-balanced pulses (i.e., the amount
twitch, fatigue-resistant muscle fibers that proof charge injected into the tissue is balanced by
duce low forces. Large motor neurons innervate the amount of charge drawn out of the tissue) to
fast-twitch, fatigable muscle fibers that produce minimize adverse effects on the tissue and the
large forces. Together, a motor neuron and the electrodes.
muscle fibers it innervates are known as a motor FES electrodes are broadly of two classes:
unit. surface electrodes and intramuscular electrodes.
Surface electrodes are positioned on the skin over
the targeted muscles. Intramuscular electrodes
5.5.2 FES Systems are implanted near the neurons that innervate the
targeted muscles. Implanted electrodes have the
An FES system consists of a controller, benefit of being able to recruit muscle fibers more
electrodes, and a stimulator. The controller selectively, because they are positioned closer
regulates the timing and intensity of the delivered to the neuromuscular junctions. However, sur-
stimulation. Stimulation is delivered in the face electrodes are less invasive and easier to
form of pulses of current with waveform replace.
patterns such as a square wave or a sine The majority of FES systems in use today
wave. These waveform patterns are described do not rely on cortical control signals but rather
by their frequency, duration, and amplitude rely on signals from intact, residual movements.
(Fig. 5.17). Frequency refers to the number For example, quadriplegics can use a sip/puff
of pulses per second. For FES applications, tube to control the initiation of a preprogrammed
typically low frequencies are used to produce stimulation pattern. Patients who have spinal cord
a smooth contraction at low force levels while injuries at the level of the fifth to sixth vertebrae
minimizing muscle fatigue. The time span of of the cervical spinal column retain voluntary
a single pulse is the pulse duration or width. control of the muscles above the injury and can
Increasing pulse duration tends to recruit more shrug their shoulders. Some systems detect the
motor units. Stimulation amplitude describes electrical activity when these muscles contract
the strength of the current applied. The higher and use it as a control signal for the FES. This is
known as myoelectric control. One-dimensional
control signals such as these allow the user to
pulse
duration
control only one degree of freedom. For example,
shrugging the shoulder might control the stim-
ulation to open or close the hand and control
pulse the degree to which the hand opens and closes.
amplitude This ultimately limits the number of movements
to a few preprogrammed grasps. The first FES
system for grasp was developed in the 1960s. It
current
(amps)
frequency = # of pulses per second

consisted of surface stimulation to open and close
the hand [23, 24]. Since then, advances in the
time
(seconds) electrodes and stimulation paradigms have led to
implantable systems.
Fig. 5.17 Square pulse waveform
To access a wider repertoire of movements and from the activity of the recorded M1 neurons and
ideally greater dexterity, a higher-dimensional then stimulate the muscles to artificially generate
control signal is necessary. Using a cortical that EMG.
control signal for FES would enable higher As a proof of concept, researchers at North-
degree-of-freedom control and thus more western University simultaneously recorded
complex movements. A cortical control signal is EMG activity and neural activity in M1 from an
also more natural because it taps into the neural able-bodied monkey while the monkey performed
activity that controls muscle activity in normal a reaching task [26]. They then used a linear
reaching. The goal is that, when the user thinks filter (Fig. 5.18a) with multiple inputs (i.e., the
about reaching, the brain-controlled FES would recorded neural activity) to predict a single output
generate a movement as seamlessly as a normal (i.e., EMG activity from one muscle). The filter
reach. can be fit by minimizing the squared error of
the predicted EMG. The predicted EMG is a
weighted linear combination of the recent history
5.5.3 Brain-Controlled FES of neural responses from many neurons:
For a brain-controlled FES system, neural ac-

N
L
tivity is mapped to the stimulation of paralyzed EMGlinear (t) = wk,l yk (t − l) (5.24)

k=1 l=0
muscles. A simple way to do this is to have
the firing rate of a neuron directly control the
where l is a time lag, yk is the firing rate of neuron
intensity of the simulation. A group at the Univer-
k, N is the number of neurons in the population,
sity of Washington showed that monkeys could
and wk, l is the weight that characterizes the effect
modulate the firing rate of one or two neurons
of neuron k’s firing rate at time (t − l) on the EMG
to control the stimulation of temporarily para-
signal at time t. Typically, the time history is a few
lyzed wrist muscles to flex and extend the wrist.
hundred milliseconds in length.
In this demonstration, when the firing rate of
A linear filter of this type does quite well at
the neuron crossed a certain threshold, current
predicting force signals and muscle activity but
was delivered through the FES in proportion to
often fails to capture specific features of EMG
the neuron’s firing rate, allowing the monkey to
signals. In particular, linear filters often fail to
produce graded muscle contraction force [25].
capture the peaks of activity and adequately
Brain control of more complex behaviors requires
characterize the quiescent periods between
more muscles and more neurons. However, it is
movements. A nonlinear decoder can address
not as straightforward as controlling each muscle
these issues, improving predictions by up to 10%.
with a different neuron because the activity of
One such option is a Wiener cascade, which is a
neurons in primary motor cortex is correlated.
linear combination of the neural activity passed
Instead populations of neurons are used to drive
through a static nonlinearity:
the coordinated activity of the muscles.
We can measure the electrical activity in a

N
L
muscle while it is contracting. This technique is EMG(t) = P w0 + wk,l yk (t − l)
called electromyography or EMG. The amplitude k=1 l=0
of the EMG signal is a measure of motor unit (5.25)
activity during muscle activation and is propor-
tional to the magnitude of muscle force. The more Here, P is a nonlinear function (e.g., often a poly-
active motor units, the higher the measured EMG nomial) and w0 is a bias term. The polynomial is
amplitude and the greater the resulting force. For fit between the output of the linear filter and the
brain-controlled FES, the goal is to decode the EMG activity (Fig. 5.18b). The static nonlinearity
EMG signal that would have naturally resulted acts to increase the gain of the peaks and decrease
Measured y(t) EMGlinear(t) Predicted

M1 activity EMG activity
B Wiener cascade
Measured y(t) EMG(t) Predicted

M1 activity polynomial
EMG activity
C
EMG
Wiener Cascade
Fig. 5.18 Decoding muscle activity from neural activity. (a) Block diagram for a linear filter. (b) Block diagram of a
Wiener cascade. (c) Using a Wiener cascade to predict EMG activity
low-level noise. This is particularly important in was possible with the preprogrammed grasps
an FES application because it reduces unneces- available through existing FES systems.
sary stimulation during the quiescent periods. Because people who are paralyzed cannot gen-
The group at the Northwestern University erate EMG activity, the aforementioned methods
further showed that the approach of predicting (which require knowing the intended EMG activ-
EMG activity from neural population activity ity) cannot be directly applied to train a brain-
could be used in a closed-loop iBMI-FES system controlled FES for these people. One way to
in monkeys. They simultaneously recorded neural overcome this issue is to take advantage of the
activity in M1 and EMG activity while the fact that the patterns of muscle activity produced
monkeys performed wrist movements or grasping in a given task (e.g., grasping) are stereotyped
movements. They then temporarily paralyzed between different individuals. To train a decoder
the monkeys by injecting lidocaine around that predicts EMG, it is possible to use the EMG
the nerves innervating the forearm and hand activity measured from a healthy individual as a
muscles. Temporarily paralyzed monkeys could template and record appropriate neural activity by
use the iBMI-FES system to control stimulation cueing the user to attempt to generate forces that
of muscles in the forearm to flex and extend correspond to the EMG activity. This approach
the wrist and to grasp and release a ball (Fig. was successful in monkeys [28], suggesting it
5.19; [27]). This demonstration showed that a would be possible to use a similar approach to
brain-controlled FES system could allow for train a brain-controlled FES for paralyzed people.
more flexible and dexterous movements than Indeed, a group from Case Western University
showed that a decoder for FES in a person
Fig. 5.19 Grasp performance during four consecutive rewarded (green dashed). When the iBMI-controlled FES
trials in a iBMI-controlled FES ball grasp-and-release system is working well, the monkey modulates his neural
task. (a) Neural activity plotted as a raster colored by activity to drive the stimulation of his muscles, success-
firing rate. (b) Predicted muscle activity (red) for two fully completing the grasp to earn a reward (green dashed).
muscles involved in flexing the fingers (flexor digitorum In addition, when the FES system is turned off during
superficialis, FDS; flexor digitorum profundus, FDP). The “catch” trials, the monkey is unable to complete the trial in
predicted muscle activity was translated into stimulus the allotted time (red dashed). Note that during this trial,
commands (black) executed by the stimulator. The vertical the neurons are firing (a) and there is a prediction of EMG
dashed lines indicate the progression of successful trials: activity (red), but no commands (black) are sent to the FES
a go cue (black dashed), the ball was picked up (blue stimulator and the monkey fails to complete the task within
dashed), and the ball was released and the monkey was 5 seconds (red dashed). (Adapted from Ethier et al. [27])
with spinal cord injury can be trained from could perform point-to-point movements with
the neural activity evoked during attempted 80–100% accuracy and, in one session, was
movements (Fig. 5.20; [9]). An initial decoder successful in 11 of 12 attempts at reaching to
was trained from the neural activity recorded grab a mug of coffee.
while the participant watched a virtual arm make
goal-directed movements and simultaneously
attempted to make the same movements. This 5.5.4 Challenges for FES
initial decoder was refined during a virtual reality
condition in which his neural activity controlled An important challenge in the development of
the movements of a virtual arm. Once the decoder FES systems relates to the way motor units are
parameters were fixed, the participant performed recruited by electrical stimulation. Henneman’s
volitional multi-joint movements of his own FES- size principle states that the natural physiological
actuated arm under brain control (Fig. 5.2). He order of motor unit recruitment is from small
Fig. 5.20 A iBMI-controlled FES system for reaching trodes implanted in the biceps, triceps, forearm, and hand
and grasping. Neural activity was recorded from two ar- muscles. The neural signals also actuated the mobile arm
rays implanted into motor cortex. These neural signals support. (Reprinted from Ajiboye et al. [9] with permis-
were used to control stimulation of intramuscular elec- sion)
to large units (Fig. 5.21). That is, motor units channels and a correspondingly larger input re-
that generate small amounts of force are recruited sistance. Thus, they require less synaptic current
first (allowing for precise control of small move- to change the membrane potential enough to fire
ments), and as the force requirements grow larger, action potentials. Similarly, larger neurons have
the units that generate larger forces (but are con- more membrane surface area, more ion channels,
sequently not as finely controlled) are recruited. and a lower input resistance. So, they require
This natural recruitment order arises from Ohm’s more synaptic current to change the membrane
law, V = Isynaptic Rinput , where V is voltage, I is potential enough to fire action potentials. Given a
the synaptic current, and R is the input resis- common synaptic drive, the smaller motor units
tance. Smaller neurons have smaller membrane will be recruited before the larger motor units.
surface area, which means they have fewer ion This natural recruitment order minimizes fatigue
Fig. 5.21 Motor units are HIGH

naturally recruited from
smallest to largest
according to Henneman’s
fast-twitch
Activation Threshold
size principle. This means
fatigable
that slow-twitch,
fatigue-resistant motor
units (red) are recruited at
lower activation thresholds
than fast-twitch, fatigable
motor units (purple)
slow-twitch
fatigue-resistant
LOW
LOW HIGH
Force Production
by recruiting fatigue-resistant muscle fibers first to be activated by the subsequent stimulation

and only recruiting fatigable fibers when high pulse. This type of pre-pulse paradigm has the
forces are necessary. This is why it is possible potential to reduce fatigue under FES conditions
to walk for hours but only sprint for minutes at by recruiting fatigue-resistant fibers earlier.
a time. Another attempt to mitigate the fatigue prob-
By contrast, in an FES system, electrical stim- lems encountered with FES is to pretreat the
ulation recruits motor units in the reverse order of muscles with low-level stimulation. This has two
Henneman’s size principle. With electrical stim- benefits. First, the low level of stimulation acts
ulation, the injected current creates an electric as exercise for the muscles, counteracting the
potential across the membrane. The larger neu- observed increase in fatigability of chronically
rons have more ions and are easier to depolarize. paralyzed muscles due to disuse [29]. The second
Thus, in FES the larger motor units are recruited benefit is that motor units can actually be changed
before the smaller motor units. This means that from fatigable to fatigue-resistant through ex-
the fatigable motor units are recruited before the ercise. Indeed, fatigability profiles can also be
fatigue-resistant motor units, which is the reverse changed with low levels of electrical current [30].
of the natural physiological order. The reverse re- Patients implanted with FES systems are often
cruitment order limits dexterity and it also causes pretreated with low-level stimulation to change
fatigue. A muscle that is fatigued will produce muscle fibers toward fatigue-resistant fibers.
less force for the same stimulation than a muscle
that is not fatigued.
There are some ways to counteract the re- 5.6 The Future of iBMIs
verse recruitment order. One such proposal is
the utilization of a pre-pulse. A pre-pulse is a Brain–machine interfaces have shown promise
pulse several hundred microseconds in duration for restoring motor function to patients with neu-
which precedes the stimulation and hyperpolar- rological injury or disease. However, there are
izes the neurons, making them less easily ex- still many improvements before iBMIs become
citable. As with other stimulation, a pre-pulse a widespread treatment for paralysis. In this sec-
preferentially affects the larger-diameter motor tion, we will discuss ongoing work toward mak-
units. With the appropriate parameters, the pre- ing iBMIs a clinical reality, such as including
pulse can be selectively applied so that only the somatosensory feedback and building better elec-
large-diameter axons are hyperpolarized, leaving trodes. Finally, we will end by discussing an
the small-diameter, fatigue-resistant motor units emerging new field in which iBMIs are used
to answer basic science questions about motor One type of movement that is particularly
learning and motor control that are currently too aided by somatosensory feedback is grasping.
difficult to tackle any other way. Grasping an object requires information about
contact forces that is difficult to get from visual
feedback alone. For example, consider the
5.6.1 Restoring Somatosensory difference in the forces on the hand when lifting
Feedback an egg versus a suitcase. Our hands have a variety
of touch sensors providing information about
Thus far, we have discussed motor control from shape, weight, size, and texture – information
the perspective of controlling the movements of critical for effective grasping. Incorporating
our body. A critical component of motor control similar information into a BMI could improve
is sensory feedback or sensory information about the degree to which a user could reach and grab
ongoing movements [31]. All of the iBMIs we a wide variety of objects. A bidirectional BMI
have discussed to this point have relied solely (i.e., one that incorporates both motor output and
on visual feedback. For example, when using an sensory input) could potentially both improve
iBMI to control a robotic limb, the user can see motor function and restore the sense of touch.
where the limb moves. Another equally (if not
more) important source of sensory information
which we have not yet discussed is somatosensa- 5.6.2 Building Better Electrodes
tion, which is the sensation of touch, temperature,
and proprioception (e.g., body position). A major obstacle to building clinically viable iB-
Without somatosensation, the everyday move- MIs is that the signals recorded from chronically
ments that many of us take for granted would implanted electrode arrays degrade over time.
be much more difficult. For example, when we This happens because the electrodes trigger an
carry a heavy box, the texture receptors in our inflammatory response in the brain that eventu-
fingertips let us know when the box starts to slip, ally encapsulates the electrodes with a protective
allowing us to adjust our grip. When we reach to layer of glia, forming a “glial scar” [34]. This
grab a cup of coffee, the temperature receptors in encapsulation reduces the quality of the recorded
our fingertips tell us that the coffee is too hot to signals because the neurons are pushed farther
drink. And when we get ready to go outside, our away from the electrode tips. Typically, it is pos-
sense of proprioception lets us slide our arms into sible to record neural signals from chronically
the sleeves of our jacket without having to turn implanted electrodes for months to a few years
around. Given the importance of somatosensation before the signal degrades. However, it would
during movements such as these, it makes sense be unreasonable to expect patients to replace an
that a BMI would benefit from incorporating so- iBMI (a process involving brain surgery and the
matosensory feedback. associated risks) every few years for the rest of
In principle, sensory percepts can be restored their lives.
by electrically stimulating the neural structures There have been a number of attempts to
responsible for sensation and perception [32]. minimize this problem of signal degradation.
For amputees, somatosensory feedback could be One such approach is to coat the electrodes with
provided by stimulating the peripheral sensory a chemical to minimize scar formation. L1, a
nerves. However, for patients with quadriplegia neuronal specific cell adhesion molecule, has
due to spinal cord injury, stimulating the nerves been shown to minimize glial scar formation
would not work because the pathway between the [35]. Other neurotrophic chemicals are also being
brain and the limb has been disrupted. In this case, tested. Another approach is to make electrodes
the somatosensory cortex could be stimulated that have mechanical characteristics that are more
directly with intracortical electrodes [33]. similar to brain tissue. Most electrode arrays
today are made with rigid materials, such as makes motor control easier to study: all the output
tungsten or silicon. The mismatch in the stiffness neurons are recorded, the mapping from these
of the electrodes and the soft brain tissue can neurons to the movement is specified by the ex-
induce damage and exacerbate the inflammatory perimenter, and the dynamics of the cursor or
response. To minimize this problem, electrodes robotic limb are known and can be made to be
can be made from soft materials that are a closer simple (e.g., linear). As a result, it is possible to
match to the mechanical properties of the brain. make scientifically causal statements about the
Yet another approach is to reduce the diameter relationship between neural activity and behavior
of the electrodes. For example, Neuralink is (in this case, cursor or robotic limb movements)
developing electrodes the size of neurons that that are not currently possible when studying arm
can be sewn into the cortex with a robot that acts movements. Together these features make iBMIs
like a sewing machine [36]. Although this work a powerful tool for studying sensorimotor control
is at its infancy, the hope is that with enough (Table 5.1).
investment, these approaches will lead to longer A key feature of sensorimotor control is the
lasting, more information-rich neural recordings. ability to learn, adapt, and refine motor skills over
time. Our understanding of learning is grounded
in concepts of synaptic plasticity and cortical map
5.6.3 iBMIs for Basic Science plasticity. However, we lack an explanation for
how such changes give rise to new behavioral
As discussed in Sect. 5.2, the more we under- capacities. We can leverage an iBMI to establish
stand about natural motor control, the better BMI a causal link between learning-related changes
systems will be. In turn, the inverse is true: by in the brain and new behavioral capacities, be-
studying how the brain functions during con- cause in an iBMI, we record from all of the
trol of a BMI, we can gain new insights into neurons that drive the behavior, and we as the
the natural processes of motor planning, control, experimenter define the relationship between the
and learning. A BMI is a simplified motor con- activity of those neurons and the behavior. As
trol system compared to arm movement control. discussed in the previous sections, we can begin
When we move our arms, there are hundreds of with a BMI decoder that relates neural activ-
thousands of output neurons; the mapping from ity patterns to cursor velocities in a way that
these neurons to movement is unknown; and the provides proficient control without requiring the
arm has nonlinear dynamics that are difficult to user to learn. We can then induce learning by
measure. All of these characteristics make it dif- presenting a novel decoder from neural activity
ficult to study sensorimotor control during arm to behavior (i.e., cursor velocity). This is akin
reaching. By comparison, an iBMI simplifies all to giving somebody a flipped computer mouse
of the features of natural arm reaching, and this and asking them to learn to control the cursor.
Table 5.1 Comparison of BMI control to arm reaching

Arm reaching iBMI
Effector Arm Cursor or robotic limb
Number of non-output neurons Millions Millions
Number of output neurons Thousands Tens to hundreds
(only a subset are recorded) (all are recorded)
Neuron-to-movement mapping Unknown Known
Effector dynamics Difficult to measure, nonlinear Known, can be linear
Sensory feedback Tied to the arm Flexibly manipulable
From Golub et al. [37]
Entries in bold indicate components of an iBMI that make it a simplified, well-defined, and easily manipulated system
for studying sensorimotor control
Through trial and error, the person can learn to (b) Suppose we now assume that the
use a flipped mouse. Similarly, through trial and BMI user moves “left” twice as often
error, the user can learn to use a novel BMI de- as “right” (i.e., P(left) = 2/3 and
coder [38]. Because we know exactly how neural P(right) = 1/3. For each of the cases
activity relates to cursor movement in the iBMI, below, would we classify “left” or
any observed improvement in behavior (e.g., ac- “right”?
curacy of cursor movements) can be attributed
to an observed change in the neural activity. We y (Spikes/second) 2 5 8 11 14 17
have found that the way in which neurons are Classification
interconnected can shape learning that occurs on a
timescale of hours [39]. In particular, it is easier to
learn tasks requiring population activity patterns 2. In Sect. 5.3 we showed how to implement
that are consistent with the underlying network a classifier with Gaussian firing statistics,
constraints than tasks requiring novel population where the neural activity for class k is mod-
activity patterns. On a time scale of days to weeks, eled as y∼N(μk , k ), where μk ∈ Rd and
populations of neurons can produce new patterns k ∈ Rd × d are the mean and covariance of
of activity to enable new behavioral capacities the activity of a population of d neurons. Here
[40]. These findings can inform the design of we will assume that the covariance matrix is
future iBMIs in which we can leverage the user’s the same for each class k = 1, . . . , K (i.e.,
ability to learn to create even higher-performance 1 = 2 = . . . = K ).
iBMI systems [41]. (a) First, suppose we have a new record-
ing of neural activity, y. Also, suppose
that P(ck ) = π k . Using Bayes’rule, find
Homework logP(ck | y), up to the normalizing con-
stant.
1. Consider designing a BMI to classify move- (b) Find the decision boundary used for de-
ment to the right or left, and we want to test termining whether the point y came from
how well it works with one neuron. If the class j or class k, and simplify the expres-
BMI user intends to move right, the neuron’s sion.
firing rate is drawn from a Gaussian distribu- (c) Is the decision boundary linear?
tion with mean μright = 8 spikes/second and 3. In this problem we will derive the equations
standard deviation σ right = 5 spikes/second. to implement a classifier based on Poisson
If the BMI user intends to move left, the neu- spike counts. The spike count of neuron i
ron’s firing rate is drawn from a Gaussian dis- given class k is Poisson-distributed with pa-
tribution with mean μleft = 12 spikes/second rameter λik . We will assume that the D neu-
and standard deviation σ left = 6 spikes/sec- rons, y1 , . . . , yD are conditionally indepen-
ond. dent given the class j. In other words, given
(a) Suppose we make one measurement of neural activity y ∈ RD , the probability that y
the firing rate, y, and we assume the came from class k is as follows:
prior probability of “left” and “right” are D
equal. For each of the cases below, would P (y|ck ) = P (yi |ck ) , where
i=1
we classify “left” or “right”?
y
y (Spikes/second) 2 5 8 11 14 17 P (yi |ck ) = exp (−λik ) λiki /yi !
Classification
(a) Let P(ck ) = π k . Find P(ck | y) using Bayes 7. Derive the expressions for the training phase
rule. of the Kalman filter in Sect. 5.4:
Now simplify the expression above by −1
T T
taking the log: log P(ck | y). B= t=1 yt xt
T T
t=1 xt xt

(b) Given a new point y, we want to deter- Σ = T1 Tt=1 (yt − Bxt ) (yt − Bxt )T (Note
mine to which class this point belongs. that here we use the B found above.)
−1
Derive the decision boundary that deter- A= T
x x T T
x x T
t=2 t t−1 t=2 t−1 t−1
1 T
mines whether we classify a new point
y as belonging to either class j or class Q = T −1 t=2 (x t − Ax t−1 ) (x t −Ax t−1 )
T
k. Use the expression that you derived in (Note that here we use the A found above.)
part a. 8. Consider using a BMI to play Pong with one
(c) Is the decision boundary linear? neuron. That is, we will use a Kalman filter to
4. In Sect. 5.3 we provided the following decode position along a one-dimensional axis
expressions for the training phase of a
from the firing rate of a single neuron. Let the
classifier: μk = N1 N i=1 y i (Eq. 5.9) and
1 N
state model be xt = xt − 1 + ωt , ωt ∼N(0, q)
Σk = N i=1 y i − μk y i − μk (Eq. and the observation model be yt = bxt + εt ,
5.10), where yi ∈ Rd for all i = 1, . . . , N εt ∼N(0, σ ).
is the neural activity recorded with class k,
(a) Show that the estimate of the position on
μk ∈ Rd , and k ∈ Rd × d . Show that these
values of μk and k maximize the following time step t, μt , can be written in the form
equation for the likelihood:

y
t
L μk , Σk |y 1 , . . . , y N, ck = P y 1 , . . . , y N |ck μt = (1 − α) μt−1 + α
b
N −d/2
= i=1 (2π) |Σk |−1/2 (b) Prove that 0 ≤ α ≤ 1.
(c) When does α approach 0? Under this
−1
exp − 2 y i − μk
1
Σk y i − μk case, why does it make sense for
μt = μt − 1 ?
5. In Sect. 5.4, we considered a two-neuron ex- (d) When does α approach 1? Under this
ample of the PVA decoder where the neurons case, why does it make sense for
had orthogonal preferred directions (e.g., one μt = yt /b?
neuron preferred 90◦ , while the other neuron 9. You decide to speed up the implementation of
preferred 180◦ ). Show that if the two neu- your Kalman filter by skipping the one-step
rons do not have orthogonal tuning direc- prediction. Whereas normally you would
tions, the directions decoded by PVA will be solve the measurement update (Eq. 5.17) and
biased. one-step predictions iteratively on each time
6. Show that a neuron that exhibits cosine tun- step (Eq. 5.16).
ing also shows linear tuning to velocity. That You instead decide to just iterate the mea-
is, suppose that given a reach in the θ direc- surement update step, by directly plugging in
tion with speed s, a neuron’sfiring rate
can be the velocity estimate
written as y = b0 + ms cos θ − θ− →
from the previous time
p , where step, P x t−1 |{y}1t−1 , without making a one-
b0 is the neuron’s baseline firing rate, m is step prediction:
its modulation depth, and θ−→p is the neuron’s

preferred direction. Show that this means we P y t |x t P x t−1 |{y}1t−1
can also write y = b0 + bT v, where b and v P x t |{y}t1 =
P y t |{y}1t−1
are both 2D vectors.
Describe qualitatively what will happen to Sect. 5.4 whereby we would first find the
the velocity estimate over time. joint distribution P(xt , yt | y1 , . . . , yt − 1 ),
(Hint: when in doubt, try simulating it or and then use the theorem of conditioning
solving the 1D case.) for jointly Gaussian random variables to
10. The goal of the measurement update of the find P(xt | y1 , . . . , yt ). Here we will derive
Kalman filter is to find P(xt | y1 , . . . , yt ). the means and covariances of the joint
To do so, we adopted the strategy in distribution
−
y t | y 1 , . . . , y t−1 Bμ−t B−
t B + Σ Bt
T
∼N ,
x t | y 1 , . . . , y t−1 μ−
t −t B
T −t
(a) Find the mean of y t | y 1 , . . . , y t−1 .

(b) Find the variance of y t | y 1 , . . . , y t−1 . 11. Use PVA decoder to estimate the movement
(c) Find the covariance of xt , yt when both velocity during a center out task.
are conditioned on yt , . . . , yt − 1 . (a) Fit the parameters of the decoder using
11–12. We have provided a dataset (https:// the 180 trials of training data.
github.com/emilyoby/bmi-data-set) consist- (b) Test the decoder on the eight test trials.
ing of center-out arm reaches and neural ac- Plot the decoded trajectories and the ac-
tivity recorded from a Utah electrode array tual movement trajectories on the same
implanted in M1. The following describes the plot.
data format. The .mat file has two data struc- (c) Try improving the decoding by smooth-
tures: ‘trainTrials’ contains 180 trials to be ing the firing rates by using a running
used as training data, and testTrials contains average of the firing rates during the pre-
8 trials to be used as test data. Each data vious 250 ms.
structure contains ‘spikes’, ‘handPos’, and 12. Use a Kalman filter decoder to estimate the
‘handVel’ variables, representing the spiking movement trajectory for each trial.
activity, hand position, and hand velocity, (a) Fit the parameters of the decoder using
respectively, on each trial in which a mon- the 180 trials of training data.
key reached to one of eight different targets. (b) Test the decoder on the eight test trials.
The ‘spikes’ variable contains, for each trial, Plot the decoded trajectories and the ac-
the number of threshold crossings in 50 ms tual movement trajectories on the same
bins recorded simultaneously from the 91 plot.
electrodes and has dimensions (n time steps)
× (91 electrodes), where n is the number
of time steps within a particular trial. For References
example, ‘trainTrials.spikes{i}(n,k)’ contains
1. J.K. Chapin, K.A. Moxon, R.S. Markowitz, M.A.
the number of threshold crossings recorded Nicolelis, Real-time control of a robot arm using
on the kth electrode in the nth time step of simultaneously recorded neurons in the motor cortex.
the ith trial. The ‘handPos’ and ‘handVel’ Nat. Neurosci. 2(7), 664–670 (1999). https://doi.org/
10.1038/10223
variables are structured similarly and contain
2. J. Wessberg, C.R. Stambaugh, J.D. Kralik, P.D. Beck,
the 2D hand position (in mm) and velocity M. Laubach, J.K. Chapin, J. Kim, S.J. Biggs, M.A.
(in mm/sec), respectively, for the same time Srinivasan, M.A. Nicolelis, Real-time prediction of
steps as in the ‘spikes’ variable. hand trajectory by ensembles of cortical neurons in
primates. Nature 408(6810), 361–365 (2000). https://
For the problems below, use the provided
doi.org/10.1038/35042582
neural and kinematic data to implement the 3. M.D. Serruya, N.G. Hatsopoulos, L. Paninski, M.R.
continuous decoders discussed in Sect. 5.4. Fellows, J.P. Donoghue, Instant neural control of
a movement signal. Nature 416(6877), 141–142
(2002). https://doi.org/10.1038/416141a
4. D.M. Taylor, S.I. Helms Tillery, A.B. Schwartz, Di- 14. E.V. Evarts, Relation of pyramidal tract activity to
rect Cortical Control of 3D Neuroprosthetic Devices. force exerted during voluntary movement. J. Neuro-
Science (New York, N.Y.) 296(5574), 1829–1832 physiol. 31(1), 14–27 (1968). https://doi.org/10.1152/
(2002). https://doi.org/10.1126/science.1070291 jn.1968.31.1.14
5. J.M. Carmena, M.A. Lebedev, R.E. Crist, J.E. 15. A.P. Georgopoulos, J.F. Kalaska, R. Caminiti, J.T.
O’Doherty, D.M. Santucci, D.F. Dimitrov, P.G. Patil, Massey, On the relations between the direction of
C.S. Henriquez, M.A.L. Nicolelis, Learning to con- two-dimensional arm movements and cell discharge
trol a brain-machine interface for reaching and grasp- in primate motor cortex. J. Neurosci. Off. J. Soc.
ing by primates. PLoS Biol. 1(2), E42 (2003). https:/ Neurosci. 2(11), 1527–1537 (1982)
/doi.org/10.1371/journal.pbio.0000042 16. N. Hatsopoulos, J. Joshi, J.G. O’Leary, Decoding
6. J.L. Collinger, B. Wodlinger, J.E. Downey, W. Wang, continuous and discrete motor behaviors using motor
E.C. Tyler-Kabara, D.J. Weber, A.J.C. McMorland, and premotor cortical ensembles. J. Neurophysiol.
M. Velliste, M.L. Boninger, A.B. Schwartz, High- 92(2), 1165–1174 (2004). https://doi.org/10.1152/
performance neuroprosthetic control by an indi- jn.01245.2003
vidual with tetraplegia. Lancet (London, England) 17. S. Kakei, D.S. Hoffman, P.L. Strick, Direction of ac-
381(9866), 557–564 (2013). https://doi.org/10.1016/ tion is represented in the ventral premotor cortex. Nat.
S0140-6736(12)61816-9 Neurosci. 4(10), 1020–1025 (2001). https://doi.org/
7. B. Wodlinger, J.E. Downey, E.C. Tyler-Kabara, 10.1038/nn726
A.B. Schwartz, M.L. Boninger, J.L. Collinger, Ten- 18. G. Santhanam, S.I. Ryu, B.M. Yu, A. Afshar, K.V.
dimensional anthropomorphic arm control in a human Shenoy, A high-performance brain-computer inter-
brain-machine interface: Difficulties, solutions, and face. Nature 442(7099), 195–198 (2006). https://
limitations. J. Neural Eng. 12(1), 016011 (2015). doi.org/10.1038/nature04968
https://doi.org/10.1088/1741-2560/12/1/016011 19. M. Velliste, P. Sagi, M. Chance Spalding, A.S. Whit-
8. C.E. Bouton, A. Shaikhouni, N.V. Annetta, M.A. ford, A.B. Schwartz, Cortical control of a prosthetic
Bockbrader, D.A. Friedenberg, D.M. Nielson, G. arm for self-feeding. Nature 453(7198), 1098–1101
Sharma, P.B. Sederber, B.C. Glenn, W.J. Mysiw, (2008). https://doi.org/10.1038/nature06996
A.G. Morgan, M. Deogaonkar, A.R. Rezai, Restor- 20. K.V. Shenoy, D. Meeker, S. Cao, S.A. Kureshi,
ing cortical control of functional movement in a hu- B. Pesaran, C.A. Buneo, A.P. Batista, P.P. Mitra,
man with quadriplegia. Nature533(7602), 247–250 J.W. Burdick, R.A. Andersen, Neural prosthetic con-
(2016). https://doi.org/10.1038/nature17435 trol signals from plan activity. Neuroreport 14(4),
9. A.B. Ajiboye, F.R. Willett, D.R. Young, W.D. Mem- 591–596 (2003). https://doi.org/10.1097/00001756-
berg, B.A. Murphy, J.P. Miller, B.L. Walter, et al., 200303240-00013
Restoration of reaching and grasping movements 21. S.M. Chase, A.B. Schwartz, R.E. Kass, Bias, op-
through brain-controlled muscle stimulation in a per- timal linear estimation, and the differences be-
son with tetraplegia: A proof-of-concept demon- tween open-loop simulation and closed-loop perfor-
stration. Lancet (London, England) 389(10081), mance of spiking-based brain-computer Interface al-
1821–1830 (2017). https://doi.org/10.1016/S0140- gorithms. Neural Netw. 22(9), 1203–1213 (2009).
6736(17)30601-3 https://doi.org/10.1016/j.neunet.2009.05.005
10. V. Gilja, C. Pandarinath, C.H. Blabe, P. Nuyujukian, 22. V. Gilja, P. Nuyujukian, C.A. Chestek, J.P. Cunning-
J.D. Simeral, A.A. Sarma, B.L. Sorice, et al., Clinical ham, B.M. Yu, J.M. Fan, M.M. Churchland, et al., A
translation of a high-performance neural prosthesis. high-performance neural prosthesis enabled by con-
Nat. Med. 21(10), 1142–1145 (2015). https://doi.org/ trol algorithm design. Nat. Neurosci. 15(12), 1752–
10.1038/nm.3953 1757 (2012). https://doi.org/10.1038/nn.3265
11. B. Jarosiewicz, A.A. Sarma, D. Bacher, N.Y. Masse, 23. C. Long, An electrophysiologic splint for the hand.
J.D. Simeral, B. Sorice, E.M. Oakley, et al., Vir- Arch. Phys. Med. Rehabil. 44(September), 499–503
tual typing by people with tetraplegia using a self- (1963)
calibrating intracortical brain-computer interface. 24. L. Vodovnik, C. Long, J.B. Reswick, A. Lippay, D.
Sci. Transl. Med. 7(313), 313ra179 (2015). https:// Starbuck, Myo-electric control of paralyzed muscles.
doi.org/10.1126/scitranslmed.aac7328 I.E.E.E. Trans. Biomed. Eng. 12(3), 169–172 (1965).
12. C. Pandarinath, P. Nuyujukian, C.H. Blabe, B.L. https://doi.org/10.1109/tbme.1965.4502374
Sorice, J. Saab, F.R. Willett, L.R. Hochberg, K.V. 25. C.T. Moritz, E.E. Fetz, Volitional control of single
Shenoy, J.M. Henderson, High performance commu- cortical neurons in a brain-machine interface. J. Neu-
nication by people with paralysis using an intracor- ral Eng. 8(2), 025017 (2011). https://doi.org/10.1088/
tical brain-computer interface. elife 6 (2017). https:// 1741-2560/8/2/025017
doi.org/10.7554/eLife.18554 26. E.A. Pohlmeyer, S.A. Solla, E.J. Perreault, L.E.
13. S. Perel, P.T. Sadtler, E.R. Oby, S.I. Ryu, E.C. Tyler- Miller, Prediction of upper limb muscle activity from
Kabara, A.P. Batista, S.M. Chase, Single-unit activity, motor cortical discharge during reaching. J. Neural
threshold crossings, and local field potentials in motor Eng. 4(4), 369–379 (2007). https://doi.org/10.1088/
cortex differentially encode reach kinematics. J. Neu- 1741-2560/4/4/003
rophysiol. 114(3), 1500–1512 (2015). https://doi.org/
10.1152/jn.00293.2014
27. C. Ethier, E.R. Oby, M.J. Bauman, L.E. Miller, 36. T.L. Hanson, C.A. Diaz-Botia, V. Kharazia, M.M.
Restoration of grasp following paralysis through Maharbiz, P.N. Sabes, The ‘sewing machine’ for
brain-controlled stimulation of muscles. Nature minimally invasive neural recording. BioRxiv, March.
485(7398), 368–371 (2012). https://doi.org/10.1038/ https://doi.org/10.1101/578542 (2019)
nature10987 37. M.D. Golub, S.M. Chase, A.P. Batista, B.M. Yu,
28. C. Ethier, D. Acuna, S.A. Solla, L.E. Miller, Adaptive Brain-computer interfaces for dissecting cognitive
neuron-to-EMG decoder training for FES neuropros- processes underlying sensorimotor control. Curr.
theses. J. Neural Eng. 13(4), 046009 (2016). https:// Opin. Neurobiol. 37(April), 53–58 (2016). https://
doi.org/10.1088/1741-2560/13/4/046009 doi.org/10.1016/j.conb.2015.12.005
29. K. Singh, F.J. Richmond, G.E. Loeb, Recruitment 38. K. Ganguly, J.M. Carmena, Emergence of a sta-
properties of intramuscular and nerve-trunk stimulat- ble cortical map for neuroprosthetic control. PLoS
ing electrodes. IEEE Trans. Rehabil. Eng. 8(3), 276– Biol. 7(7), e1000153 (2009). https://doi.org/10.1371/
285 (2000) journal.pbio.1000153
30. J.T. Mortimer, Motor prostheses, in Handbook of 39. P.T. Sadtler, K.M. Quick, M.D. Golub, S.M.
physiology: The nervous system II, (American Physi- Chase, S.I. Ryu, E.C. Tyler-Kabara, B.M. Yu, A.P.
ological Society, Bethesda, MD, 1981) Batista, Neural constraints on learning. Nature
31. S.J. Bensmaia, L.E. Miller, Restoring sensorimotor 512(7515), 423–426 (2014). https://doi.org/10.1038/
function through intracortical interfaces: Progress and nature13665
looming challenges. Nat. Rev. Neurosci. 15(5), 313– 40. E.R. Oby, M.D. Golub, J.A. Hennig, A.D. De-
325 (2014). https://doi.org/10.1038/nrn3724 genhart, E.C. Tyler-Kabara, B.M. Yu, S.M. Chase,
32. M.C. Dadarlat, J.E. O’Doherty, P.N. Sabes, A A.P. Batista, New neural activity patterns emerge
learning-based approach to artificial sensory with long-term learning. Proc. Natl. Acad. Sci. U.
feedback leads to optimal integration. Nat. Neurosci. S. A. 116(30), 15210–15215 (2019). https://doi.org/
18(1), 138–144 (2015). https://doi.org/10.1038/ 10.1073/pnas.1820296116
nn.3883 41. K.V. Shenoy, J.M. Carmena, Combining decoder de-
33. S.N. Flesher, J.L. Collinger, S.T. Foldes, J.M. Weiss, sign and neural adaptation in brain-machine inter-
J.E. Downey, E.C. Tyler-Kabara, S.J. Bensmaia, A.B. faces. Neuron 84(4), 665–680 (2014). https://doi.org/
Schwartz, M.L. Boninger, R.A. Gaunt, Intracortical 10.1016/j.neuron.2014.08.038
microstimulation of human somatosensory cortex. 42. L.R. Hochberg, M.D. Serruya, G.M. Friehs, J.A.
Sci. Transl. Med. 8(361), 361ra141 (2016). https:// Mukand, M. Saleh, A.H. Caplan, A. Branner, D.
doi.org/10.1126/scitranslmed.aaf8083 Chen, R.D. Penn, J.P. Donoghue. Neuronal ensem-
34. V.S. Polikov, P.A. Tresco, W.M. Reichert, Response ble control of prosthetic devices by a human with
of brain tissue to chronically implanted neural elec- tetraplegia. Nature. 442(7099), 164–71 (2006).
trodes. J. Neurosci. Methods 148(1), 1–18 (2005). 43. L.R. Hochberg, D. Bacher, B. Jarosiewicz, N.Y.
https://doi.org/10.1016/j.jneumeth.2005.08.015 Masse, J.D. Simeral, J. Vogel, S. Haddadin, J. Liu,
35. J.R. Eles, A.L. Vazquez, N.R. Snyder, C. Lagenaur, S.S. Cash, P. Van Der Smagt, J.P. Donoghue. Reach
M.C. Murphy, T.D.Y. Kozai, X.T. Cui, Neuroadhesive and grasp by people with tetraplegia using a neurally
L1 coating attenuates acute microglial attachment to controlled robotic arm. Nature. 485(7398), 372–5
neural electrodes as revealed by live two-photon mi- (2012).
croscopy. Biomaterials 113, 279–292 (2017). https://
doi.org/10.1016/j.biomaterials.2016.10.054
Deep Brain Stimulation: Emerging
Technologies and Applications 6
Aysegul Gunduz
Abstract innovative electrode designs, computational

models to guide stimulation, closed-loop
Deep brain stimulation (DBS) is a neurosur-
DBS, emerging clinical indications, and future
gical technique that consists of continuous
noninvasive strategies.
delivery of electrical pulses through chron-
ically implanted electrodes connected to a
Keywords
neurostimulator, programmable in amplitude,
pulse width, frequency, and stimulation Deep brain stimulation · Implantable pulse
channel. DBS is a promising treatment option generator · Ablation · Stereotactic surgery ·
for addressing severe and drug-resistant Basal ganglia · Thalamus · Parkinson’s
movement disorders. The success of DBS disorder · Tremor
therapy stems from a combination of surgical
implantation techniques, device technologies,
and clinical programming strategies. Changes
in device settings require highly trained and 6.1 Introduction
experienced clinicians to achieve maximal
therapeutic benefit for each targeted symptom, Deep brain stimulation (DBS) is a neurosurgical
and optimization of stimulation parameters treatment option for addressing severe and drug-
can take many clinic visits. Thus, the resistant movement disorders, such as Parkin-
development of innovative DBS technologies son’s disease and essential tremor, and is show-
that can optimize the clinical implementation ing promise for the treatment of neuropsychiatric
of DBS will lead to wider-scale utilization. disorders, such as major depression. Historically,
This chapter aims to discuss engineering DBS is a product of the interplay between neu-
approaches that have the potential to improve roscientific and engineering advances translated
clinical outcomes of DBS, focusing on successfully to functional neurosurgery.
the development novel temporal patterns, In the 1950s, development of stereotaxic at-
lases and frames led the way to ablative surgery
of “malfunctioning” brain areas [1, 2]. Surgical
Electronic Supplementary Material: The online version ablations of the globus pallidus interna (GPi) [3,
of this chapter (https://doi.org/10.1007/978-3-030-43395- 4] in Parkinsonian patients and the ventral inter-
mediate nucleus (Vim) of the thalamus [5, 6] in
essential tremor and Parkinson’s disease patients
A. Gunduz () were used as a therapy for suppressing tremor;
J. Crayton Pruitt Family Department of Biomedical
however, the ablations provided little benefit for
Engineering, University of Florida, Gainesville, FL, USA
e-mail: [email protected] bradykinesia in the latter cohort. Moreover, bi-

224 A. Gunduz
lateral ablations led to irreversible side effects way for new targets for DBS therapy [15]. At the
such as speech and swallowing deficits and gait same time, implantable medical device technol-
and balance issues. Electrical stimulation during ogy had advanced, and chronically implanted de-
stereotactic surgery was being used for the local- vices such as cardiac pacemakers and spinal cord
ization of the targeted areas [7], and various stud- stimulators for pain treatment were commonly
ies demonstrated that high-frequency stimulation implanted in patients. These scientific and tech-
(50–100 Hz) resulted in a reduction of tremor nical advances brought the inception of chronic
symptoms, while low-frequency stimulation (5– DBS treatment as it is applied today, which is
10 Hz) exacerbated tremor [8]. Furthermore, both widely credited to Benabid and colleagues [16,
outcomes were reversible with the termination of 17]. Contemporary DBS therapy targets the Vim
stimulation. Although many studies in this early nucleus of the thalamus for the treatment of es-
era explored the therapeutic effects of DBS [9– sential tremor, the subthalamic nucleus (STN) or
12], the introduction of levodopa as a “miracle the GPi of the basal ganglia for Parkinson’s dis-
drug” for the treatment of Parkinson’s disease ease, and the GPi for dystonia. The application of
in the 1960s curtailed surgical interventions for DBS within these dysfunctional circuits not only
movement disorders. allows alleviation of symptoms but also presents
The excitement surrounding levodopa lasted a unique opportunity for probing the function
for two decades, but by the 1980s, it became ap- of these circuits. DBS offers a portal into the
parent that Parkinsonian patients developed resis- dynamics of brain circuits in relation to behavior,
tance to levodopa after long-term treatment and not previously possible with lesion surgery, and
experienced dyskinesias as a side effect. In the thus has grown into a vibrant area of scientific
meantime, the neurophysiological underpinnings inquiry.
of movement within the basal ganglia and their It has been reported that DBS can bring about
role in Parkinson’s disease were introduced by >50% improvement in clinical ratings of mo-
Albin [13] and DeLong [14] with the segregated tor symptoms in patients with movement dis-
functional circuit model (Fig. 6.1), paving the orders appropriately screened and selected [18].
Fig. 6.1 (a) The segregated functional basal ganglia jections to the cortex. This model led to the STN and GPi
(show in blue) circuit model proposed by Albin [13] and to be utilized as targets for DBS therapy. SNc substantia
DeLong [14]. (b) Parkinsonian model of the basal ganglia nigra pars compacta, GPe globus pallidus externus, GPi
shows reduced and increased projections along the direct globus pallidus internus, STN subthalamic nucleus
and indirect pathways that cause decreased excitatory pro-
6 Deep Brain Stimulation: Emerging Technologies and Applications 225
These promising clinical outcomes of modern

DBS therapy arise from a combination of neuro-
surgical techniques that rely on imaging, bioin-
strumentation design, and clinical decision mak-
ing regarding the stimulation contact and stimu-
lation pattern. The first stage is functional neu-
rosurgery [19], in which electrodes are implanted
into deep brain nuclei via stereotactic radiological
imaging. Many institutions also perform electro-
physiological recordings to validate the place-
ment of the electrode array in the target structure.
The electrodes are subsequently connected to a
neurostimulator, also known as an implantable
pulse generator (IPG), that is fully implanted in
the patient’s body. The neurostimulator generates
an electrical pulse train that has programmable
variables, namely, the frequency, amplitude, and
pulse width of the pulse train [20]. The implanted
electrode array is a single rod with four to eight
cylindrical ring electrodes (or contacts) at its dis-
tal end. Extension leads or connectors from the
electrode array are tunneled through the neck
and connected to the IPG, which is typically
implanted in the chest cavity (similar to a cardiac
pacemaker). Figure 6.2 depicts all the implantable
components of a unilateral DBS system.
Clinical programming, which is critical for the
success of DBS, requires highly trained and ex- Fig. 6.2 Implanted components of deep brain stimula-
tion (DBS) therapy. A four- or eight-contact electrode
perienced clinicians to achieve maximal patient- is stereotactically implanted into a target brain structure
specific therapeutic benefit (18–36 h per patient through a burr hole. An implantable pulse generator (IPG)
in total [22]). Given that the stimulation param- is surgically placed under the collarbone. The electrode
eter space is extremely high dimensional, opti- array is connected to the pulse generator via a connector
lead that is surgically tunneled through the neck. (Adapted
mization within this space can take many visits from [21])
over several months [23]. Newer neurostimula-
tor designs therefore need to be accompanied
by guided and/or automated programming strate-
gies to improve DBS outcomes. In this chapter,
6.2 State of the Art in DBS
we first introduce the current state of the art
Instrumentation
in DBS instrumentation technology. This is fol-
As introduced earlier, DBS technology was
lowed by the current understanding of the mecha-
built on advances in implantable biomedical
nisms of DBS therapy. Next, we present the recent
instrumentation such as cardiac pacemakers
engineering advancements in bioinstrumentation
and spinal cord stimulators for the treatment
design and neuromodulation delivery strategies
of chronic pain. Similar to a pacemaker, neu-
aimed at improving DBS outcomes. Finally, we
rostimulators are implanted under the collarbone,
discuss other disorders which can potentially ben-
and they are connected to the electrode array via
efit from DBS therapy and close with emerging
a connector that is tunneled through the neck
noninvasive technologies that may complement
(see Fig. 6.1). From a hardware perspective,
or eventually challenge DBS therapy.
226 A. Gunduz
stimulators can be designed to be current- or quent subsections. But first, we present the cur-
voltage-controlled (i.e., regulated current versus rent understanding in the literature of how DBS
regulated voltage stimulation). Although with therapy mechanistically provides symptom relief.
regulated voltage stimulation, nonlinear or
changing impedances of the electrode–tissue
interface lead to varying current flow, commercial 6.3 Current Understanding
neurostimultors historically have been voltage of the Therapeutic
controlled. Newer neurostimulators on the market Mechanisms of DBS
are designed to be current-controlled to deliver
constant therapeutic levels [24]; nevertheless, Today, DBS is an established therapy for the treat-
they still have limits to their output voltage and ment of Parkinson’s disease, essential tremor.
other safety measures to prevent tissue irritation In the United States, the use of DBS for these
or damage (e.g., in instances of electrode defects). indications, as well as for dystonia, is approved
The standard stimulation waveform is an by the US Food and Drug Administration (FDA).
asymmetric biphasic pulse train with passive Nevertheless, the mechanisms by which DBS im-
recharge, i.e., a standard rectangle pulse in proves pathological symptoms remain to elude
the first phase, followed by a low-amplitude both scientists and clinicians [30]. This gap in
but long-decaying tail in the second phase.1 knowledge can hinder our efforts to improve the
This waveform allows for a charge-balanced benefits of DBS therapy and mitigate its side
stimulation for tissue safety, and passive recharge effects. Moreover, it can hinder its translation into
provides energy efficiency. Currently, most IPGs more complex disorders of the brain. Given that
are not rechargeable, and depleted batteries have clinical DBS leads are macroscale electrodes, it is
to be replaced with a new IPG via surgery. likely that the electrical fields generated by DBS
Though there are some commercially available nonselectively affects local neurons, afferent in-
rechargeable IPGs on the market today, they also puts, and fibers of passage. This macroscale effect
have a limited shelf life and have a limitation on limits our ability to study the individual roles
the number times they can be recharged. of functionally different cells or cells projecting
Stimulation is typically delivered in a cathodic onto different nuclei in the overall mechanism
monopolar fashion, in which the electrode is set of DBS. In addition, studying the electrophys-
as the cathode (negative pole) and the case of iological effects of DBS modulation is highly
the IPG is set as the anode (positive pole). Ca- challenging due to large stimulus artifacts [31].
thodic stimulation requires a lower stimulation Given that DBS provides outcomes similar
amplitude and/or pulse width than anodic stim- to those of ablative surgery, historically it was
ulation to achieve equivalent therapeutic benefit widely accepted that DBS brings about a tem-
[27–29]. Monopolar stimulation leads to a wide porary reversible lesion that reduces its output
electric field that is relatively symmetric in all [16] (Fig. 6.2). Several animal and clinical studies
directions. For a more spatially constrained elec- have in fact reported inhibited activity within
tric field, stimulation can be delivered between the target when DBS was delivered to STN or
two electrode contacts. Different active contact GPi [32–37]. However, other electrophysiologi-
combinations can be used to direct current flow cal studies put forth contradicting results suggest-
through desired target areas or to prevent stim- ing excitation at the target as a result of stim-
ulating undesired areas that may lead to side ulation with increased activity toward projected
effects. Novel waveform patterns and electrode nuclei [38, 39]. Functional imaging studies dur-
designs, as well as computational modeling of the ing DBS delivery reported increased blood flow
stimulation fields will be discussed I the subse- toward and increased blood oxygenation levels
at downstream nuclei consistent with increased
1 More discussion on waveform shapes will follow. See output at the stimulated structure [40–42].
Fig. 6.4b for a drawing of this waveform.
Overall, the mechanisms by which DBS masks

pathological symptoms most likely rely on a
combination of several phenomena [43–45]. The
pathological neural activity in the basal ganglia-
thalamo-cortical motor network is likely related
to increased neuronal synchronization, which is
reflected in low-frequency rhythmic oscillations
[46–48]. It is possible that DBS masks these low-
frequency pathological oscillations and over-
writes them with tonic high-frequency output.
This “informational lesioning” hypothesis [49]
suggests that tonic high-frequency stimulation,
though devoid of any informational content,
overrides the pathophysiological activity, akin
to a communication jammer. The message
sent downstream is thought to be replaced by
a stimulation-induced regular pattern, which
can be more easily mitigated by the remaining
elements of the network [31, 43, 44]. In fact,
random patterns of DBS in the STN, even
when delivered at a high average frequency
that would be considered therapeutic, were not
Fig. 6.3 Effects of (a) surgical lesioning versus (b) deep
effective in relieving bradykinesia in patients brain stimulation (DBS) on neuronal firing rates and local
with Parkinson’s disease [50]. These findings field potentials (LFPs) in the Parkinsonian subthalamic
reinforce the importance of regularization of nucleus (STN). Historically, DBS was thought to cause a
pathological activity in the network for the temporary lesion effect, but electrophysiological studies
have demonstrated the target response is rather complex.
effectiveness of DBS [51]. Local field potential (Adapted from [53])
(LFP) recordings from DBS electrodes in
Parkinsonian STN and GPi nuclei have shown
pathologically elevated amplitudes in the beta temporal patterns of DBS. Figure 6.4a, third row,
rhythm that reduce with levodopa intake [52] demonstrates a regular pulse train, which were
(Fig. 6.3). DBS has also been shown to reduce biphasic pulses with passive recharge (see Fig.
these pathologically high beta rhythms [54], 6.4b). The dark lines in Fig. 6.4a represent the in-
regularizing the network (Fig. 6.2). stance of the first phase of the pulses. Other rows
in Fig. 6.4a are examples of temporal patterns that
deviate from evenly spaced regular pulse trains,
6.4 Novel Temporal Patterns which have been investigated in DBS treatment
of Stimulation as of Parkinson’s disease and essential tremor. In
a Therapeutic Innovation some of these novel patterns, investigators studied
the absence of pulses (Fig. 6.4a, top row) or in-
The impact of the temporal pattern of stimulation troduced higher-frequency short bursts of pulses
(regular pulses versus random pulses) on thera- in the regular pattern (Fig. 6.4a, second row).
peutic outcomes discussed in the previous subsec- In other cases, the distributions of instantaneous
tion inspired researchers to test many other novel pulse frequencies were fitted to a log-uniform dis-
228 A. Gunduz
Fig. 6.4 (a) Novel temporal patterns compared to regular with an overall 20% coefficient of variance (CV). (b) A
pattern (middle row). Top row: Patterns with short periods monophasic active pulse with passive charge. (c) A bal-
of absence. Second row: Patterns with the presence of anced biphasic pulse with active recharge. (d) Parameter
short bursts of pulses. Fourth row: Highly non-regular space of active recharge biphasic pulse design (a1 , a2 ,
pulses with log-normal distribution of instantaneous pulse amplitudes of the two phases; w1 , w2 , pulse widths of the
frequencies. Bottom row: Pulses consisting of the same two phases; d, inter-pulse interval). (Adapted from [25,
average stimulation frequency as the regular pattern but 26])
tribution (Fig. 6.4a, third and fourth rows). These life would result in fewer battery replacement
novel patterns applied to DBS STN stimulation in surgeries. A primate study of novel DBS patterns,
patients with Parkinson’s diseases yielded equiv- on the other, did not yield improved bradykinesia
alent or improved treatment of symptoms com- scores compared to traditional stimulation pat-
pared to traditional patterns [25]. These clini- terns when the GPi was selected as the stimulation
cal outcomes were accompanied by a substantial target [55].
reduction in battery consumption, which is an Another study investigated biphasic active
important consideration for prolonging the life recharge patterns in Parkinson’s disease and
of a non-rechargeable device. Prolonged battery essential tremor and reported improved clinical
scores [26] (see Fig. 6.4d). These biphasic stimulation patterns. Computational modeling
pulse patterns expedite the recovery of charge will be discussed further in this chapter.
in tissue at the expense of increased power
consumption, shortening the battery life. Biphasic
active recharge patterns need not be programmed 6.5 Innovations in DBS Electrode
symmetrically and can be delivered with Design
asymmetric phase amplitudes and durations,
as well as an inter-phase delay (see Fig. 6.4d). The most widely used DBS electrode arrays con-
Other studies are investigating the therapeutic sist of four cylindrical electrode contacts at their
effects of biphasic stimulation patterns with distal ends (see Fig. 6.5a). The neural tissue that
active recharge in dystonia [56]. Rechargeable is affected by an electrical field generated by a
neurostimulators can compensate for the extra set of stimulation parameters is called the volume
current drain required by the secondary phase of tissue activation (VTA). If an electrode array
and may thus facilitate the translation of these is not positioned optimally, a wider VTA may be
patterns into clinical use. However, the success necessary to activate the target. This may come
of all rechargeable neurostimulators will rely on at the cost of activating other areas that may lead
patient compliance to maintain sufficient battery to undesired side effects [62]. To mitigate this
charge levels. problem, novel electrode designs have been pro-
In order to overwrite the pathological syn- posed that aim to directionally steer stimulation
chronization in the Parkinsonian basal ganglia currents toward the target, while avoiding other
[57], Tass et al. proposed a spatially distributed areas. This method of controlling the shape of the
stimulation pattern called coordinated reset [58]. VTA can personalize and optimize therapies. Fig-
In this stimulation scheme, aimed at desynchro- ure 6.5b shows four novel lead designs [63–65].
nizing an overly synchronous system, brief high- Three of these designs (Abbott Infinity™, Aleva
frequency pulse trains are delivered in the STN directSTNAcute™, and Boston Scientific Ver-
across different electrode contacts with varying cise™ Systems) have segmented ring electrodes.
order. The authors report significant improvement Two of the cylindrical contacts have equidistant
of in clinical motor outcome scores [59]. 90◦ angular segments. An intraoperative study
Although the non-regular patterns discussed involving 11 Parkinsonian and 2 essential tremor
above show promise in Parkinson’s disease, they patients with the Aleva lead revealed directional
patterns did not provide therapeutic benefit in stimulation required 43% lower stimulation mag-
essential tremor [51]. This result may have been nitudes to achieve equivalent therapeutic benefits
caused by sufficiently long gaps in the stimulation [63]. A fourth electrode that is in preclinical trials
patterns that allowed undesired propagation of (from Medtronic-Sapiens) has 8–10 rows of 4
pathological tremor activity within the thalamo- electrodes per row, and with an offset of 45◦ per
cortical network [60]. row. This creates a total of eight radial directions.
Altogether these findings substantiate the A desired VTA can be shaped with various com-
value of novel temporal patterns to increase binations of independently active electrodes [64].
the efficacy of the therapy. However, large and However, having a high number of electrodes and
well-powered studies are necessary for them their combination would significantly complicate
to be translated into clinical practice, along clinical programming and prolong the clinical
with devices that can support them. In the visit [23]. Fast computational modeling for vi-
meantime, computational models can guide sualization of the VTA based on patient-specific
our understanding of the effect of these novel imaging may reduce this burden [66] but would
patterns. In fact, computational models will be still increase the amount of required training on
essential in the design and optimization of novel the clinician’s part.
230 A. Gunduz
Fig. 6.5 (a) Current clinical DBS leads (Adapted from entific Neuromodulation; SJM, Abbott-St. Jude Medical;
Abbott, Aleva, and Medtronic DBS brochures). (b) MDT, Medtronic). (Adapted from [61])
Emerging DBS electrode lead designs (BSN, Boston Sci-
the spatial spread of stimulation, or the VTA,

6.6 Imaging and Computational relative to the anatomical structures surrounding
Tools for Personalized DBS the electrode implant. If the electrode placement
is suboptimal, or if the VTA spreads across
Clinical DBS programming is typically per- nontarget areas without immediate overt side
formed without visualization tools to observe effects, clinical programming and interpreting the
Fig. 6.6 (a) Implanted electrode overlaid on patient MRI. distribution in the tissue medium determined from a finite
(b) Extracted 3D thalamic and subthalamic nuclei (in element model with the total current amplitude divided
yellow and green, respectively). (c) The computational between adjacent electrode contacts (Adapted from [68])
model of the volume of tissue activation. (d) The voltage
outcomes can be challenging. Software platforms Finally, these models can help us understand
that would enable the dynamic visualization the mechanisms of DBS. For instance, a recent
of the VTA as a function of the stimulation computational study [73] shed light on the
parameters and electrode location within the mechanisms of cathodic and anodic stimulation,
brain as yielded by imaging could provide demonstrating that cathodic stimulation prefer-
an engineering solution to this problem [67, entially activates axon segments passing adjacent
68]. Figure 6.6a depicts three-dimensional to the electrode, whereas anodic stimulation
subcortical structures constructed from a patient’s preferentially activates orthogonal axon segments
preoperative MRI using a brain atlas. Figure approaching or leaving the electrode. Hence,
6.6b highlights the implanted electrode array’s anodic stimulation, which has been traditionally
location with reference to the subthalamic excluded from DBS practice without a complete
nucleus and the thalamus (the green and yellow understanding of its mechanism, can now be
structures, respectively). The electric field that adopted to target fiber orientation selectively.
can be generated by this extracted location of
the electrode and specific shape of electrode is
modeled using finite element models. Moreover, 6.7 Development
tissue conductivity properties can be derived of Closed-Loop DBS Systems
from diffusion tensor imaging. Altogether these
imaging and modeling techniques yield the Although DBS is a widely used treatment option
VTA as a function of stimulation contact(s), for medication refractory movement disorders,
amplitude, pulse width, frequency, as well as the stimulation is delivered in an “open-loop”
electrode shape, electrode, and tissue impedance fashion, meaning no feedback regarding the pa-
(see Fig. 6.4d). Computational models will tient’s current symptom(s), medication status, or
be essential in guiding the optimization of the side effects they are experiencing are pro-
stimulation parameters of high-density electrode vided back to the DBS system. A “closed-loop
arrays (as discussed in the previous subsection). DBS” system, on the other hand, would be able
Recent studies utilizing computational models in to identify a biomarker reflective of a patient’s
guiding clinical parameters report significantly current disease state and adapt the delivery and
reduced programming time [69, 70], reduced parameters of stimulation. Such a smart system
power consumption [69], and reduced side effects may yield improved patient outcomes due to its
(e.g., stimulation induced cognitive decline) [71]. tailored approach and increased battery life as
In addition, computational modeling can also unnecessary stimulation would be avoided when
assist in the design of novel temporal patterns, symptoms are manageable without DBS. Figure
prior to conducting long and expensive clinical 6.7 depicts a block diagram of a closed-loop DBS
trials [72]. system.
232 A. Gunduz
Fig. 6.7 A closed-loop DBS system that is utilizing responsively activates and terminates stimulation or adap-
biomarkers from the DBS leads, from additional subdural tively changes the stimulation parameters based on the
ECoG strips, and/or from wearable sensors. A controller extracted biomarkers. (Adapted from [74])
The first step toward achieving closed-loop (LFP) recordings in the STN and GPi. Increased
DBS is the characterization of biomarkers asso- power in beta activity in the basal ganglia
ciated with disease symptoms. Such biomarkers has been correlated with Parkinsonian motor
can be studied during DBS implantation surgery symptoms, and this exaggerated beta power is
with consent, as patients remain awake. Other reduced with medication (levodopa) and with
short-term studies can be performed with DBS in responders [52]. More recent studies
externalized leads, while patients are closely have also reported on the shape and duration of
being monitored in hospital surgical care units. beta oscillations. Cole et al. [79] demonstrated
Next-generation neurostimulators enable chronic that beta oscillations in the human primary motor
recording of brain signals directly from the cortex have sharp, asymmetric, non-sinusoidal
implanted electrodes (the NeuroPace RNS [75], features, specifically asymmetries in the ratio
the Medtronic Activa PC + S [76], the Medtronic between the sharpness of the beta peaks compared
Summit RC + S [77], and the CorTec Brain with the troughs. These beta oscillations become
Interchange [78]). With regulatory approvals more symmetric with optimal DBS therapy.
(such as investigational device exemptions), Other studies showed that Parkinsonian beta
these systems can allow the translation of disease activity is not continuously elevated but fluctuates
biomarkers into closed-loop therapies and may to give beta bursts in the STN [80] and GPi
provide groundbreaking discoveries in human [81]. Incidence of prolonged beta bursts is
neuroscientific research. shown to be positively correlated with clinical
Figure 6.8 presents a summary of biomarkers impairment in Parkinson’s disease patients.
that have been shown to correlate with disease Similar short-term studies in dystonia patients
symptoms. The most extensively investigated lead to the discovery of theta oscillations as a
DBS biomarker intraoperatively and through correlate of mobile dystonic symptoms, which
externalized leads is the Parkinsonian beta band could be suppressed with DBS therapy [82].
(11–30 Hz) amplitude in the local field potential GPi peak theta activity has also been shown
Fig. 6.8 Pathological oscillations in disorders repre- GPi during dystonic movements (c). Similarly, patients
sented in power spectral densities are potential biomark- with Tourette syndrome have increased activity in lower
ers for closed-loop DBS development. (a) In Parkinson’s frequencies (1–10 Hz) within the GPi and Cm-Pf complex
disease, pathologically high power in the beta (11–30 Hz) of the thalamus. (d) In essential tremor patients, oscilla-
frequency range has been recorded in several areas in the tions within the Vim and cortex have been coherent with
basal ganglia and even cortex. (b) In dystonia patients, peripheral tremor frequency as recorded by electromyog-
increases in theta rhythms (4–7 Hz) were reported in the raphy (EMG). (Adapted from [53])
to significantly correlate with preoperative Pf thalamus reported similar improvements in

symptom severity in cervical dystonia subjects tic severity scores, with 36% improvements in
[83]. Next, intraoperative and chronic studies expected battery lifetimes, again in comparison
have reported that elevated low-frequency power to open-loop DBS [90].
(<10–13 Hz) in the centromedian-parafascicular Recording and stimulating from the same elec-
(Cm-Pf) complex of the thalamus coincides trode array, however, inject stimulation artifacts
with involuntary tics in patients with Tourette into the recorded signals that confound the detec-
syndrome [84, 85]. Finally, patients with essential tion of biomarkers, even when the stimulation and
tremor experience tremor within 4–8 Hz as recordings are performed on different electrodes.
measured by inertial sensors or electromyography This is mainly because stimulation amplitudes
(EMG). Electrophysiological studies have are significantly larger than the neural biomarker
reported coherence between thalamic (Vim) and amplitudes. Another confounding factor in the
primary motor cortical field potentials and EMG neural signal is the subharmonics of the stimula-
activity at the tremor frequency [86, 87]. tion frequency caused by aliasing during analog-
Many researchers are translating these to-digital conversion. Some mitigation methods
biomarkers to closed-loop DBS systems. Little for these artifacts include front-end filtering of
et al. (2013) [54] have shown intraoperatively stimulation frequencies, heterodyning the spec-
that closed-loop DBS is possible by increasing tral range of the biomarker, and common-mode
stimulation amplitude in response to increased rejection via recording differentially from the two
beta power. They report 28% improvement in adjacent electrodes surrounding the stimulation
motor scores and 56% reduction in stimulation electrode [76]. Additional electrodes implanted in
time compared to standard DBS. Rosa et al. other nodes of the pathological network can also
[88] presented a case report of a freely moving provide a solution. If these electrodes are placed
patient on closed-loop DBS via externalized far from the stimulation area, they will not be
leads. They observed that closed-loop DBS significantly contaminated. For movement disor-
significantly improved the subject’s main ders, electrocorticography (ECoG) strips may be
symptom, bradykinesia, in comparison to open- placed subdurally on the surface of the primary
loop DBS. Other groups are working toward motor cortex to extract disease biomarkers. This
chronic implementation of closed-loop DBS [88, additional electrode strip can be placed through
89]. A case study using a NeuroPace device the same surgical burr hole used for the DBS elec-
with bilateral responsive stimulation of the Cm- trode. Cortical signals yield with much greater
234 A. Gunduz
amplitudes compared to deep brain structures. A a clinical manifestation. Another challenge of

chronic study with five PD patients reported a wearable sensor-based DBS is establishing the
fourfold root-mean-square (RMS) difference be- communication channel between external sensors
tween cortical ECoG recordings and subthalamic and the implanted neurostimulator without
nucleus recordings [91]. A cortical biomarker for draining the implanted battery. Still, there is
dyskinetic events has been reported in Parkinso- notable utility for wearable sensor technology
nian patients [92]. Dyskinesias are hyperkinetic for movement disorders. For instance, wearable
activity in multiple body regions that manifest sensors could yield objective clinical measures
when dopamine levels of patients who are on both of motor symptom severity. Continuous wear
medication and stimulation cannot be properly of these sensors can provide a highly sampled
regulated. This dyskinesia marker could termi- record of patient symptoms [99]. Current clinical
nate or decrease stimulation to prevent dyskine- scales are assessed by scoring a video recording
sias in a closed-loop fashion. Two research groups (examination by a specialist), who then assigns a
are studying ECoG biomarkers to detect intention categorical rating to each item. This approach is
of movement from motor cortices to deliver DBS highly subjective and can only be implemented
responsively in essential tremor patients with in- completely during patient visits. The use of
tention tremor [93]. This patient population does wearable technologies could yield continuous
not suffer from rest tremor and experience tremor scoring of hypokinetic or hyperkinetic symptoms
only when they make a goal-directed movement. through inertia measurements, which can capture
Opri et al. [94] demonstrated the first fully em- tremor, rigidity, and bradykinesia. It will also be
bedded closed-loop system for intention tremor. important to create telemedicine solutions that
In addition, cortical electrodes enable the net- will make it easier for patients to obtain DBS
work study of basal ganglia- or thalamocorti- devices and to be programmed and managed
cal interactions. In intraop and chronic studies, from remote locations.
Opri et al. [95] showed that low-frequency phase Overall, Hoang and Turner [100] summarize
from the Vim couples to the broadband high- that an optimal biomarker for closed-loop DBS
frequency amplitude of the primary motor cortex should have the following general characteristics:
when subjects are at rest. This coupling dissolves (i) directly correlated to clinical symptoms, (ii)
with volitional movement, suggesting a gating able to dynamically reflect symptom improve-
mechanism from the thalamus to the cortex that ment, (iii) stable in real-world settings and daily
needs to be released for movement execution. The activities, (iv) differentiable from background
coupling is not present between the Vim phase noise, and (v) consistent across patients, yet
and the somatosensory cortex amplitude, suggest- tunable to account for patient variability.
ing a functional role for the coupling (i.e., gating Although this is a comprehensive list, it is also
motor function). Malekmohammadi et al. [96] important to consider that long-term plasticity is
also reported the same phenomena with patients necessary for symptom relief in disorders such
at rest but additionally showed that the coupling as dystonia and depression. It is therefore highly
dissolved under general anesthesia. possible that a biomarker capable of reflecting
For movement disorders, motor symptoms or predicting clinical improvement may be
could be detected with wearable sensors. Case present in a different part of the network than
studies in Parkinson’s disease [97] and essential the stimulation target. Such a biomarker could
tremor [98] provide initial evidence for the guide clinical programming in these disorders,
feasibility of using wearable sensors for closed- which is very challenging and can take many
loop DBS. Wearable sensors, however, will yield months, as symptom relief is not immediate. In
symptom biomarkers after they manifest. Hence, paroxysmal disorders such as Tourette syndrome
any adjustments to DBS will come after symptom or epilepsy, since tics or seizures are not
manifestation. Neural signals, on the other hand, always present, the absence of a tic or seizure
likely modulate and yield biomarkers prior to biomarker may not be reflective of whether
stimulation parameters are optimized. In such cases to levodopa), to minimally conscious states
cases, a biomarker that directly correlates with due to traumatic brain injury (TBI). Freezing of
observable symptoms may not reflect whether gait is a debilitating Parkinsonian symptom that
stimulation settings are optimally selected. leads to falls and significantly limits the inde-
Nevertheless, the field of neuromodulation is pendence of patients. Traditional high-frequency
rapidly uncovering biomarkers of disorders and STN or GPi DBS does not resolve this symp-
biomarkers of treatment efficacy with the goal of tom. However, new clinical trials are targeting
developing smarter implantable neurostimulators low-frequency STN [103] and pedunculopontine
for improved symptom relief. nucleus (PPN) of the brain stem [104]. Tremors
In the previous four sections, we discussed the in multiple sclerosis (MS) are targeted with Vim
emerging technologies in DBS. As DBS tech- or dual Vim and ventral oralis (VO) nucleus of
nology advances, it will be important to develop the thalamus [105]. Another emerging indication,
easy-to-use tools for clinicians and end users. One Alzheimer’s disease, is the most common form
danger in the field is to develop solutions that of adult-onset dementia. Electrical stimulation of
are too complex to implement on a large scale. the fornix, a node in the Papez memory circuit
As engineers, we need to remind ourselves that [106, 107], was targeted in a Phase I clinical
input from stakeholders (patients, clinicians, and trial for improving working memory and cogni-
caregivers) is the most important component of tive function in patients with Alzheimer’s-type
any design and optimization process. dementia with mixed results. DBS is also being
investigated as a potential treatment option for
patients in minimally conscious state following
6.8 Emerging Indications traumatic brain injury [108]. The surgical target is
the central thalamus, which is interposed between
Other advances in the field of DBS therapy brain stem and basal forebrain arousal systems
arise not from technological advancements and frontal cortical supervisory attentional sys-
but our understanding of the mechanisms of tems during wakefulness. Another layer of ethi-
disorders. This leads the way for new brain cal challenges surrounds this therapy, as subjects
areas to be investigated as potential targets for cannot provide informed consent. Moreover, pa-
DBS therapy in emerging indications. Table 6.1 tients may gain an “awareness of a situation to
summarizes current and potential indications which [they] had previously been unaware, strip-
that are being treated by DBS. As discussed ping away a protective veneer that spared [them]
previously, essential tremor and Parkinson’s knowledge of the severity of [their] injury and its
disease are indications that are approved for DBS associated burdens” [109]. Hence, it is important
in the United States by the FDA. For emerging for engineers and clinicians to work closely with
indications, DBS is a last line of surgical neuroethicists as these new treatments are being
treatment, when all other options fail. Dystonia studied.
and obsessive compulsive disorder (OCD) have Furthermore, many neuropsychiatric disorders
humanitarian device exemption (HDE) approval are emerging as potential targets for DBS.
for DBS. In OCD, the most common targets are Chronic pain, which is more traditionally treated
the nucleus accumbens (NAcc) [101], which is with spinal cord stimulation, may be treated with
known to be part of the brain’s reward circuit, and DBS of the anterior cingulate cortex (ACC). The
the intersection of the ventral capsule and ventral dorsal ACC has been implicated in the affective
striatum (VC/VS) [102]. These areas are thought aspect of pain, emotional reward processing, and
to have an effect in the impulsive behaviors of addiction [110]. For the treatment of refractory
these subjects. depression, cingulate white matter tracts near
Emerging neurological indications range from Brodmann’s area 25 have been targeted [111], as
freezing of gait in Parkinson’s disease, which positron emission tomography (PET) in patients
does not respond to DBS therapy (and in some with major depression showed increased blood
236 A. Gunduz
Table 6.1 Current and emerging indications for DBS therapy

Indication Surgical target(s) Regulatory approvalsa
Essential tremor Vim Approved
Parkinson’s disease STN, GPi Approved
Dystonia GPi Humanitarian device exemption
Obsessive compulsive disorder NAcc, VC/VS, anterior limb of Humanitarian device exemption
internal capsule, STN
Freezing of gait in Parkinson’s disease STN, PPN Investigational
Multiple sclerosis Vim, Vim + VO Investigational
Tourette syndrome Cm-Pf, GPi, NAcc Investigational
Dementia in Alzheimer’s disease Fornix Investigational
Minimally conscious state Central thalamus Investigational
Chronic pain ACC Investigational
Major depression BA 25 white matter tracts
Post-traumatic stress disorder BLA Investigational
Morbid obesity LH, VMH, NAcc Investigational
Addiction NAcc Investigational
ACC Anterior cingulate cortex, BA 25 Brodmann’s area 25 (subgenual cingulate cortex), BLA Basolateral amygdala,
Cm-Pf Centromedian-parafascicular complex (thalamus), GPi Globus pallidus internus (basal ganglia), LH Lateral hy-
pothalamus, NAcc Nucleus accumbens (basal ganglia), PPN Pedunculopontine nucleus (brain stem), STN Subthalamic
nucleus (basal ganglia), VC/ VS Ventral capsule/ventral striatum (white matter/basal ganglia), Vim Ventral intermediate
nucleus (thalamus), VMH Ventromedial hypothalamus, VO Ventral oralis nucleus (thalamus)
a Regulatory approvals in the United States as granted by the Food and Drug Administration (FDA)
flow into that region compared to matched them, will be critical for the success of these
controls [112]. Post-traumatic stress disorder trials.
(PTSD) is another indication that is under
investigation for DBS. The amygdala is involved
in the neurocircuitry of fear conditioning, and 6.9 Nonsurgical Approaches for
thus the basolateral amygdala is the most Deep Brain Stimulation
commonly used target for investigational studies
of DBS for PTSD therapy [113]. Halpern et al. 6.9.1 Focused Ultrasound
[114] identified three potential neural targets
that are believed to be associated with excessive While effective, stereotactic lesioning of the brain
food consumption: the lateral hypothalamus for the treatment of movement disorders has been
(LH), the ventromedial hypothalamus (VMH), largely abandoned with the development of DBS
and the NAcc through animal lesioning studies. [117]. However, progress in transcranial MR-
Respectively, these three areas have been guided high-intensity focused ultrasound (HIFU)
implicated in feeding behavior and energy technology has renewed an interest in stereotactic
expenditure, appetite regulation, and the value of lesioning due to the potential for continuous
food regardless of appetite [115]. The NAcc, as a MRI guidance of a sharp focus “incisionless”
key structure in the mesolimbic reward pathway, thalamotomy [118]. As a propagating wave, ultra-
is a potential DBS target for the treatment of sound can penetrate biological tissues including
drug addiction [116]. Overall, large-scale clinical the skull, and its energy can be concentrated
trials need to be conducted to establish DBS as in a small target. Early feasibility trials have
a viable treatment for any new indication. Our demonstrated significant improvements in hand
neuroscientific understanding of these disorders, tremor in patients with severe essential tremor
as well as the mechanism of DBS in treating following HIFU thalamotomy [119–121], and
further clinical trials are ongoing. These studies
have suggested functional improvements in DBS by exploiting the effects of high- and low-
activities, disabilities, and quality of life with frequency oscillating electric fields on neuronal
minimal morbidity. Proponents of HIFU further activity and temporal interference. The technique
highlight that an incision and burr hole are not employs two sinusoidal delivered from noninva-
required to perform the procedure, offering a sive electrodes placed on the skull at frequencies
“lower-cost, less invasive” alternative to DBS higher than 1 kHz. These fields are low-pass
that eliminates both the risks of penetrating the filtered by the neural membrane and therefore
brain and the inconvenience and costs imposed can pass through the superficial cerebral cortex
by implanted hardware [122]. Furthermore, without affecting the neuronal firing within their
transcranial ultrasound at low intensities can be field. In contrast, low-frequency sinusoidal fields,
used to manipulate deep brain circuitry through which are in the dynamic range of neural firing
noninvasive brain mapping prior to lesioning. activity, promote neuronal discharges. The ab-
On the other hand, patients must remain awake sence of neuronal firing was verified in mouse
during the procedure and must lie flat within the hippocampal cells, whereas a 10 Hz neuronal
MRI scanner for a few hours while the target is bursting was reported in response to 10 Hz stimu-
localized. The MRI environment, while offering lation. The latter, however, also activates overly-
the potential for procedural monitoring, can be ing cortical cells.
difficult to work in, and some patients cannot Selective activation of hippocampal neurons
tolerate these image-guided procedures. More without activating cortical neurons was achieved
importantly, HIFU is an ablative and irreversible by exploiting the temporal interference of two
lesion and can result in adverse effects especially high-frequency sinusoidal electric fields. To de-
when used bilaterally. scribe temporal interference, we will borrow the
When delivered at low intensities and short concept of interference patterns of beats from
durations, focused ultrasound can be used to ma- simple acoustics. A high-frequency sine wave at
nipulate deep brain circuitry, facilitating nonin- 2 kHz produces an audible tone (red trace in Fig.
vasive brain mapping with high spatial precision. 6.7). If we add to this tone another sine wave
Focused ultrasound can excite or inhibit cellular that differs slightly in frequency (yellow trace
activity, depending on specific stimulation pain Fig. 6.7), for example, by 10 Hz, the waves
rameters [123], and stimulate a volume of sev- follow each other closely, amplifying each other,
eral millimeters when applied through the human but over time they drift out of phase and begin
skull [124]. Furthermore, low-intensity focused to attenuate each other (blue trace in Fig. 6.7).
ultrasound can induce long-term changes in neu- Thus, their sum oscillates between high and low
ral activity of the stimulated circuits when applied volume. If we listen to this signal, we perceive this
for periods longer than 10 seconds [125, 126]. periodic variation of volume as a beat. Moreover,
The mechanisms underlying these short-duration the beat frequency happens to be the difference in
or lasting effects are not well understood. Still, frequency between the two original waves (pur-
focused ultrasound has the potential to become ple trace in Fig. 6.7). Grossmann and colleagues
a new tool for causal mapping of brain func- show that they can temporally interfere 2 kHz and
tion, and further studies will show whether it 2.01 kHz waves within the mouse hippocampus,
has potential to become a noninvasive treatment creating a beat frequency at 10 Hz, the difference
for neurological and psychiatric disorders without between the two. The result is hippocampal neu-
the need for ablating the target focus. ronal firing at 10 Hz (Fig. 6.9).
Using this technique, high frequencies are
low-pass filtered so as not to affect the overlying
6.9.2 Temporal Interference cerebral cortex. This can be seen by looking at
expression of the gene c-fos – a marker of neural
Grosmann and colleagues [127] recently intro- activity in mice cortex and hippocampus [129] –
duced a practical nonsurgical method to apply which verified that ipsilateral hippocampal
238 A. Gunduz
Fig. 6.9 Model of temporal interference and its effects electrodes), no neural recruitment is observed. The in-
on neuronal activity as delivered from two noninvasive terference field (blue trace) and a 10 Hz field (purple
electrodes on the skull. The interference of two oscillating trace) both lead to 10 Hz neural firing but with selective
electric fields with slightly varying frequencies (f1 and recruitment at the interference locus and with broad re-
f1 + 10 Hz) creates a beat at 10 Hz (where f1 > > 1 kHz; not cruitment on the path of the low-frequency electric field,
drawn to temporal scale). When the same high-frequency respectively. (Adapted from [128])
field is applied from both electrodes (e.g., f1 from both
neurons were activated. The contralateral are commonly observed in DBS studies with hu-
hippocampal and ipsilateral cortical neurons, man participants. Furthermore, the fixation of the
however, did not express c-fos, substantiating this noninvasive electrodes and the calculation of the
technique’s selective activation. Moreover, the interference loci must be reliable. These factors
locus of interference can be changed by merely could be limiting in clinical translation, especially
modulating the ratio of the two electric field for patients requiring continuous DBS. Finally,
amplitudes, while keeping the amplitude sum longer durations of safety testing should be con-
constant. Furthermore, by moving the locus of ducted prior to human studies. Overall, the tech-
the interference to the sensorimotor cortex, the nique did not induce seizures or increase tissue
authors were able to evoke movements in the temperature, and histology showed preservation
contralateral forepaw. of neuronal density without DNA damage. How-
Although the proposed methodology of this ever, the interfering fields were only applied in
technique seems practical, a necessary next step short bouts, and the histological effects were only
would be to test the clinical effectiveness in an- investigated at a single time point, 24 h post stim-
imal models of movement and neuropsychiatric ulation. The technical issues, practicality, safety,
disorders. The selection of a beat frequency in and effectiveness will need to be addressed in
the range of theta bursts was a suitable choice for order to translate this finding into a treatment for
the hippocampus; however, DBS is typically de- human beings.
livered in the thalamus and basal ganglia around
100–180 Hz. Thus, the functional spectral range
of the methodology should be investigated. More- 6.10 Discussion
over, the volume of tissue of activation will need
to be delineated to identify the specificity of the Though DBS has become a widely used ther-
interference effect. If the noninvasive methodol- apy over the last two decades, no significant ad-
ogy is not specific in its neural activation, the vancement in DBS device technology has been
unintended spread of DBS current could lead to a demonstrated. Clinical programming of DBS still
range of motor and non-motor side effects, which requires the involvement of highly experienced
clinicians to yield maximal therapeutic benefit Redo Problem 1 to attain equal TEED for
in each patient. Progress in imaging will enable the two different stimulation frequencies.
better specificity in brain circuit targeting. Novel 5. Redo Problem 2 to attain equal TEED for the
stimulation patterns, novel electrodes, computa- two different stimulation frequencies.
tional modeling for current steering, and closed- 6. Redo Problem 3 to attain equal TEED for the
loop DBS have the potential to transform DBS two different stimulation frequencies.
therapy by improving the therapeutic benefit-to- 7. What would be the amount of current drain
side effect ratio. We will likely see many more off of a voltage-controlled pulse generator if
feasibility studies on closed-loop DBS enabled by the tissue impedance is reduced by an aver-
next-generation DBS devices capable of chronic age of 10% across time?
recordings and supported by public–private part- 8. What would be the amount of current drain
nerships. Chronic brain recordings from humans off of a voltage-controlled pulse generator
will bring about improved scientific understand- if the tissue impedance is increased by an
ing of the neurophysiology of movement disor- average of 10% across time?
ders and other indications. The collective knowl- 9. What would be the amount of current drain
edge will inform the DBS mechanisms of action off of a current-controlled pulse generator if
and guide the path for new design goals in future the tissue impedance is reduced by an aver-
devices. Advances in these areas will better serve age of 10% across time?
patients, as well as clinicians. 10. What would be the amount of current drain
off of a current-controlled pulse generator
if the tissue impedance is increased by an
Homework average of 10% across time?
1. Assume you want to do a controlled experi-

ment between low-frequency DBS at 50 Hz References
and high-frequency DBS at 100 Hz by de-
livering the same amount of charge into the 1. E.A. Spiegel et al., Stereotaxic apparatus for opera-
tions on the human brain. Science 106(2754), 349–
tissue. If you keep your pulse width the same,
350 (1947)
what would be the ratio of amplitudes in 2. J. Talairach et al., Atlas d’anatomie stereotaxique
these two stimulation schemes with a current- (Masson, Paris, 1957)
controlled pulse generator? 3. H. Narabayashi, T. Okuma, Procaine-oil blocking of
the globus pallidus for the treatment of rigidity and
2. In the scenario above, how would you adjust
tremor of parkinsonism. Proc Japan Acad 29, 134–
your pulse width if that was the only param- 137 (1953)
eter you could change? 4. E. Svennilson et al., Treatment of parkinsonism
3. If in the above scenario (question 1), safety by stereotatic thermolesions in the pallidal region.
A clinical evaluation of 81 cases. Acta Psychiatr.
limits only allow you to increase your ampli-
Scand. 35, 358–377 (1960)
tude by 25%, what kind of adjustment would 5. K. Matsumoto, F. Shichijo, T. Fukami, Long-term
you have to make to your pulse width? follow-up review of cases of Parkinson’s disease af-
4. A recent study [86] determined the total ter unilateral or bilateral thalamotomy. J. Neurosurg.
60(5), 1033–1044 (1984)
electrical energy delivered (TEED) by
6. R. Hassler, T. Riechert, Indications and localization
nonrechargeable implantable voltage- of stereotactic brain operations. Nervenarzt 25(11),
controlled pulse generators based on the 441–447 (1954)
stimulation parameters as given below: 7. G. Guiot, J. Hardy, D. Albe-Fessard, Delimitation
precise des structures sous-corticales et indentifi-
TEED = cation de noyuax thalamiques chez l’homme par
l’electrophysiologie stereotaxique. Neurochirurgia
amplitude (V )2 ×frequency (H z) ×pulse width 5, 1–18 (1961)
impedance 8. D. Albe Fessard et al., Characteristic electric ac-
tivites of some cerebral structures in man. Ann. Chir.
× 1 sec 17, 1185–1214 (1963)
240 A. Gunduz
9. J. Delgado, H. Hamlin, W. Chapman, Technique 26. U. Akbar et al., Randomized, blinded pilot testing of
of intracranial electrode implacement for recording nonconventional stimulation patterns and shapes in
and stimulation and its possible therapeutic value Parkinson’s disease and essential tremor: Evidence
in psychotic patients. Confin. Neurol. 12, 315–319 for further evaluating narrow and biphasic pulses.
(1952) Neuromodulation 19(4), 343–356 (2016)
10. N. Bekthereva et al., Utilisation of multiple elec- 27. J. Ranck, Which elements are excited in electrical
trodes implanted in the subcortical structure of the stimulation of mammalian central nervous system:
human brain for the treatment of hyperkinesis. Zh. A review. Brain Res. 98, 417–440 (1975)
Nevropatol. Psikhiatr. Im. S S Korsakova 63, 3–8 28. P. Basser, B. Roth, New currents in electrical stimu-
(1963) lation of excitable tissues. Annu. Rev. Biomed. Eng.
11. C. Sem-Jacobsen, Depth electrographic stimulation 2, 377–397 (2000)
and treatment of patient with Parkinson’s disease 29. T. Zhang, W. Grill, Modeling deep brain stimulation:
including neurosurgical technique. Acta Neurol. Point source approximation versus realistic repre-
Scand. Suppl. 13, 365–377 (1965) sentation of the electrode. J. Neural Eng. 7, 066009
12. I. Cooper, Cerebellar Stimulation in Man (Raven (2010)
Press, New York, 1978) 30. C.C. McIntyre et al., How does deep brain stimula-
13. R. Albin, A. Young, J. Penney, The functional tion work? Present understanding and future ques-
anatomy of basal ganglia disorders. Trends Neu- tions. J. Clin. Neurophysiol. 21(1), 40–50 (2004)
rosci. 12(10), 366–375 (1989) 31. T. Hashimoto, C.M. Elder, J.L. Vitek, A template
14. G. Alexander, M. DeLong, P. Strick, Parallel orga- subtraction method for stimulus artifact removal in
nization of functionally segregated circuits linking high-frequency deep brain stimulation. J. Neurosci.
basal ganglia and cortex. Annu. Rev. Neurosci. 9, Methods 113(2), 181–186 (2002)
357–381 (1986) 32. A. Benazzouz et al., Effect of high-frequency stim-
15. H. Bergman, T. Wichmann, M. DeLong, Reversal of ulation of the subthalamic nucleus on the neuronal
experimental parkinsonism by lesions of the subtha- activities of the substantia nigra pars reticulata and
lamic nucleus. Science 249, 1436–1438 (1990) ventrolateral nucleus of the thalamus in the rat.
16. A. Benabid et al., Combined (thalamotomy and Neuroscience 99(2), 289–295 (2000)
stimulation) stereotactic surgery of the VIM thala- 33. J.O. Dostrovsky et al., Microstimulation-induced in-
mic nucleus for bilateral Parkinson disease. Appl. hibition of neuronal firing in human globus pallidus.
Neurophysiol. 50, 344–346 (1987) J. Neurophysiol. 84(1), 570–574 (2000)
17. A. Benabid et al., Long-term suppression of tremor 34. M.L. Welter et al., Effects of high-frequency
by chronic stimulation of the ventral intermediate stimulation on subthalamic neuronal activity in
thalamic nucleus. Lancet 337, 403–406 (1991) parkinsonian patients. Arch. Neurol. 61(1), 89–96
18. S. Miocinovic et al., History, applications, and (2004)
mechanisms of deep brain stimulation. JAMA Neu- 35. W. Meissner et al., Subthalamic high frequency
rol. 70(2), 163–171 (2013) stimulation resets subthalamic firing and reduces
19. A. Machado et al., Deep brain stimulation for abnormal oscillations. Brain 128(Pt 10), 2372–2382
Parkinson’s disease: Surgical technique and peri- (2005)
operative management. Mov. Disord. 21(Suppl 14), 36. E.B. Montgomery Jr., Effects of GPi stimulation on
S247–S258 (2006) human thalamic neuronal activity. Clin. Neurophys-
20. J. Volkmann, E. Moro, R. Pahwa, Basic algorithms iol. 117(12), 2691–2702 (2006)
for the programming of deep brain stimulation in 37. J.L. Vitek et al., External pallidal stimulation im-
Parkinson’s disease. Mov. Disord. 21(Suppl 14), proves parkinsonian motor signs and modulates neu-
S284–S289 (2006) ronal activity throughout the basal ganglia thalamic
21. M.S. Okun, Deep-brain stimulation for Parkinson’s network. Exp. Neurol. 233(1), 581–586 (2012)
disease. N. Engl. J. Med. 367(16), 1529–1538 38. M. Anderson, N. Postupna, M. Ruffo, Effects of
(2012) high-frequency stimulation in the internal globus
22. K. Hunka et al., Nursing time to program and as- pallidus on the activity of thalamic neurons in the
sess deep brain stimulators in movement disorder awake monkey. J. Neurophysiol. 89, 1150–1160
patients. J. Neurosci. Nurs. 37(4), 204–210 (2005) (2003)
23. E. Moro et al., Subthalamic nucleus stimulation: Im- 39. T. Hashimoto et al., Stimulation of the subthalamic
provements in outcome with reprogramming. Arch. nucleus changes the firing pattern of pallidal neu-
Neurol. 63(9), 1266–1272 (2006) rons. J. Neurosci. 23, 1916–1923 (2003)
24. Group, S.D.S, Subthalamic deep brain stimulation 40. R. Jech et al., Functional magnetic resonance imag-
with a constant-current device in Parkinson’s dis- ing during deep brain stimulation: A pilot study in
ease: An open-label randomised controlled trial. four patients with Parkinson’s disease. Mov. Disord.
Lancet Neurol. 11(2), 140–149 (2012) 16(6), 1126–1132 (2001)
25. D.T. Brocker et al., Improved efficacy of temporally 41. J.S. Perlmutter et al., Blood flow responses to deep
non-regular deep brain stimulation in Parkinson’s brain stimulation of thalamus. Neurology 58(9),
disease. Exp. Neurol. 239, 60–67 (2013) 1388–1394 (2002)
42. T. Hershey et al., Cortical and subcortical blood flow 60. B.D. Swan et al., Short pauses in thalamic deep brain
effects of subthalamic nucleus stimulation in PD. stimulation promote tremor and neuronal bursting.
Neurology 61(6), 816–821 (2003) Clin. Neurophysiol. 127(2), 1551–1559 (2016)
43. J.L. Vitek, Mechanisms of deep brain stimulation: 61. P.J. Rossi et al., Proceedings of the third annual deep
Excitation or inhibition. Mov. Disord. 17(Suppl 3), brain stimulation think tank: A review of emerging
S69–S72 (2002) issues and technologies. Front. Neurosci. 10, 119
44. E.B. Montgomery Jr., K.B. Baker, Mechanisms of (2016)
deep brain stimulation and future technical develop- 62. M.S. Okun et al., Management of referred deep brain
ments. Neurol. Res. 22(3), 259–266 (2000) stimulation failures: A retrospective analysis from
45. A.L. Benabid, A. Benazzous, P. Pollak, Mechanisms 2 movement disorders centers. Arch. Neurol. 62(8),
of deep brain stimulation. Mov. Disord. 17(Suppl 3), 1250–1255 (2005)
S73–S74 (2002) 63. C. Pollo et al., Directional deep brain stimulation:
46. H. Bergman et al., Physiological aspects of informa- An intraoperative double-blind pilot study. Brain
tion processing in the basal ganglia of normal and 137(Pt 7), 2015–2026 (2014)
parkinsonian primates. Trends Neurosci. 21(1), 32– 64. M.F. Contarino et al., Directional steering: A novel
38 (1998) approach to deep brain stimulation. Neurology
47. G. Deuschl et al., The pathophysiology of tremor. 83(13), 1163–1169 (2014)
Muscle Nerve 24(6), 716–735 (2001) 65. J.L. Vitek, P.A. Starr, Deep Brain Stimulation
48. J.L. Vitek, M. Giroux, Physiology of hypokinetic (DBS) for the treatment of Parkinson’s disease
and hyperkinetic movement disorders: Model for (INTREPID) (2013). Available from: Available at:
dyskinesia. Ann. Neurol. 47(4 Suppl 1), S131–S140 https://clinicaltrials.gov/ct2/show/NCT01839396
(2000) 66. G. Aad et al., Search for supersymmetry using final
49. W.M. Grill, A.N. Snyder, S. Miocinovic, Deep brain states with one lepton, jets, and missing transverse
stimulation creates an informational lesion of the momentum with the ATLAS detector in radicals=7
stimulated nucleus. Neuroreport 15(7), 1137–1140 TeV pp collisions. Phys. Rev. Lett. 106(13), 131802
(2004) (2011)
50. A.D. Dorval et al., Deep brain stimulation alleviates 67. C.R. Butson, C.C. McIntyre, Current steering to
parkinsonian bradykinesia by regularizing pallidal control the volume of tissue activated during deep
activity. J. Neurophysiol. 104(2), 911–921 (2010) brain stimulation. Brain Stimul. 1(1), 7–15 (2008)
51. M.J. Birdno et al., Stimulus features underlying re- 68. C.R. Butson et al., Patient-specific analysis of the
duced tremor suppression with temporally patterned volume of tissue activated during deep brain stimu-
deep brain stimulation. J. Neurophysiol. 107(1), lation. NeuroImage 34(2), 661–670 (2007)
364–383 (2012) 69. M.H. Pourfar et al., Model-based deep brain stim-
52. S. Little, P. Brown, What brain signals are suitable ulation programming for Parkinson’s disease: The
for feedback control of deep brain stimulation in GUIDE pilot study. Stereotact. Funct. Neurosurg.
Parkinson’s disease? Ann. N. Y. Acad. Sci. 1265, 9– 93(4), 231–239 (2015)
24 (2012) 70. C.R. Butson et al., Evaluation of interactive visual-
53. R.S. Eisinger et al., A review of basal ganglia circuits ization on mobile computing platforms for selection
and physiology: Application to deep brain stimula- of deep brain stimulation parameters. IEEE Trans.
tion. Parkinsonism Relat. Disord. 59, 9–20 (2019) Vis. Comput. Graph. 19(1), 108–117 (2013)
54. S. Little et al., Adaptive deep brain stimulation in 71. A.M. Frankemolle et al., Reversing cognitive-motor
advanced Parkinson disease. Ann. Neurol. 74(3), impairments in Parkinson’s disease patients using
449–457 (2013) a computational modelling approach to deep brain
55. K.B. Baker, J. Zhang, J.L. Vitek, Pallidal stimula- stimulation programming. Brain 133(Pt 3), 746–761
tion: Effect of pattern and rate on bradykinesia in the (2010)
non-human primate model of Parkinson’s disease. 72. D.T. Brocker et al., Optimized temporal pattern of
Exp. Neurol. 231(2), 309–313 (2011) brain stimulation designed by computational evolu-
56. L. Almeida et al., A pilot trial of square biphasic tion. Sci. Transl. Med. 9(371) (2017)
pulse deep brain stimulation for dystonia: The BIP 73. D. Anderson et al., Anodic stimulation misunder-
dystonia study. Mov. Disord. 32(4), 615–618 (2017) stood: Preferential activation of fiber orientations
57. C. Hammond, H. Bergman, P. Brown, Pathological with anodic waveforms in deep brain stimulation. J.
synchronization in Parkinson’s disease: Networks, Neural Eng. 16, 016026 (2019)
models and treatments. Trends Neurosci. 30, 357– 74. A. Gunduz, K.D. Foote, M.S. Okun, Reengineer-
364 (2007) ing deep brain stimulation for movement disorders:
58. P.A. Tass et al., Coordinated reset has sustained Emerging technologies. Curr Opin Biomed Eng 4,
aftereffects in parkinsonian monkeys. Ann. Neurol. 97–105 (2017)
72(5), 816–820 (2012) 75. M.J. Morrell, R.N.S.S.i.E.S. Group, Responsive cor-
59. I. Adamchic et al., Coordinated reset neuromodula- tical stimulation for the treatment of medically in-
tion for Parkinson’s disease: Proof-of-concept study. tractable partial epilepsy. Neurology 77(13), 1295–
Mov. Disord. 29(13), 1679–1684 (2014) 1304 (2011)
242 A. Gunduz
76. S. Stanslaski et al., Design and validation of a fully 93. J.A. Herron et al., Chronic electrocorticography for
implantable, chronic, closed-loop neuromodulation sensing movement intention and closed-loop deep
device with concurrent sensing and stimulation. brain stimulation with wearable sensors in an essen-
IEEE Trans. Neural Syst. Rehabil. Eng. 20(4), 410– tial tremor patient. J. Neurosurg. 127(3), 580–587
421 (2012) (2017)
77. S. Stanslaski et al., Creating Neural “Co- 94. E. Opri, et al., Responsive cortico-thalamic closed-
processors” to Explore Treatments for Neurological loop deep brain stimulation for the improved treat-
Disorders (ISSCC, San Francisco, 2018) ment of essential tremor. in review, 2019
78. J. Rickert. Available from: https:// 95. E. Opri et al., The functional role of thalamocortical
commonfund.nih.gov/sites/default/files/ coupling in the human motor network. J. Neurosci.
CoreTec%20Info.pdf (2019). in review
79. S.R. Cole et al., Nonsinusoidal beta oscillations 96. M. Malekmohammadi et al., Propofol-induced loss
reflect cortical pathophysiology in Parkinson’s dis- of consciousness is associated with a decrease in tha-
ease. J. Neurosci. 37(18), 4830–4840 (2017) lamocortical connectivity in humans. Brain 142(8),
80. G. Tinkhauser et al., Beta burst dynamics in Parkin- 2288–2302 (2019)
son’s disease OFF and ON dopaminergic medica- 97. M. Malekmohammadi et al., Kinematic adaptive
tion. Brain 140(11), 2968–2981 (2017) deep brain stimulation for resting tremor in Parkin-
81. R. Lofredi et al., Pallidal beta bursts in Parkinson’s son’s disease. Mov. Disord. 31(3), 426–428 (2016)
disease and dystonia. Mov. Disord. 34(3), 420–424 98. J.A. Herron et al., Chronic electrocorticography for
(2019) sensing movement intention and closed-loop deep
82. E. Barow et al., Deep brain stimulation suppresses brain stimulation with wearable sensors in an essen-
pallidal low frequency activity in patients with pha- tial tremor patient. J. Neurosurg., 1–8 (2016)
sic dystonic movements. Brain 137(11), 3012–3024 99. Q.W. Oung et al., Technologies for assessment of
(2014) motor disorders in Parkinson’s disease: A review.
83. W.J. Neumann et al., A localized pallidal phys- Sensors (Basel) 15(9), 21710–21745 (2015)
iomarker in cervical dystonia. Ann. Neurol. 82(6), 100. K.B. Hoang, D.A. Turner, The emerging role of
912–924 (2017) biomarkers in adaptive modulation of clinical brain
84. S. Marceglia et al., Thalamic single-unit and local stimulation. J. Neurosurg. (2019). in press
field potential activity in Tourette syndrome. Mov. 101. D. Denys et al., Deep brain stimulation of the nu-
Disord. 25(3), 300–308 (2010) cleus accumbens for treatment-refractory obsessive-
85. J.B. Shute et al., Thalamocortical network activ- compulsive disorder. Arch. Gen. Psychiatry 67(10),
ity enables chronic tic detection in humans with 1061–1068 (2010)
Tourette syndrome. Neuroimage Clinical 12, 165– 102. B.D. Greenberg et al., Deep brain stimulation of
172 (2016) the ventral internal capsule/ventral striatum for
86. D.J. Pedrosa et al., Thalamomuscular coherence in obsessive-compulsive disorder: Worldwide experi-
essential tremor: Hen or egg in the emergence of ence. Mol. Psychiatry 15(1), 64–79 (2010)
tremor? J. Neurosci. 34(43), 14475–14483 (2014) 103. R.A. Ramdhani et al., Early use of 60 Hz frequency
87. J.F. Marsden et al., Coherence between cerebellar subthalamic stimulation in Parkinson’s disease: A
thalamus, cortex and muscle in man: Cerebellar case series and review. Neuromodulation 18, 664–
thalamus interactions. Brain 123(Pt 7), 1459–1470 669 (2015)
(2000) 104. T.A. Mestre et al., Long-term double-blinded uni-
88. M. Rosa et al., Adaptive deep brain stimulation in lateral pedunculopontine area stimulation in Parkin-
a freely moving Parkinsonian patient. Mov. Disord. son’s disease. Mov. Disord. 31(10), 1570–1574
30(7), 1003–1005 (2015) (2016)
89. H. Bronte-Stewart, Adaptive Closed Loop Neuro- 105. H.A. Wishart et al., Chronic deep brain stimulation
modulation and Neural Signatures of Parkinson’s for the treatment of tremor in multiple sclerosis:
Disease (2015) Review and case reports. J. Neurol. Neurosurg. Psy-
90. R. Molina et al., Chronic responsive deep brain chiatry 74(10), 1392–1397 (2003)
stimulation for Tourette syndrome: Proof of concept. 106. S.D. Vann et al., Impaired recollection but spared
J. Neurosurg. 129(2), 308–314 (2017) familiarity in patients with extended hippocampal
91. N.C. Swann et al., Chronic multisite brain record- system damage revealed by 3 convergent methods.
ings from a totally implantable bidirectional neural Proc. Natl. Acad. Sci. U. S. A. 106(13), 5442–5447
interface: Experience in 5 patients with Parkinson’s (2009)
disease. J. Neurosurg. 128(2), 605–612 (2018) 107. C. Hamani et al., Memory enhancement induced
92. N.C. Swann et al., Gamma oscillations in the by hypothalamic/fornix deep brain stimulation. Ann.
hyperkinetic state detected with chronic human Neurol. 63(1), 119–123 (2008)
brain recordings in Parkinson’s disease. J. Neurosci. 108. N.D. Schiff, Central thalamic deep brain stimula-
36(24), 6445–6458 (2016) tion for support of forebrain arousal regulation in
the minimally conscious state. Handb. Clin. Neurol.
116, 295–306 (2013)
109. N.D. Schiff, J.T. Giacino, J.J. Fins, Deep brain stim- 120. M. Wintermark et al., Imaging findings in MR
ulation, neuroethics, and the minimally conscious imaging-guided focused ultrasound treatment for
state: Moving beyond proof of principle. Arch. Neu- patients with essential tremor. AJNR Am. J. Neu-
rol. 66(6), 697–702 (2009) roradiol. 35(5), 891–896 (2014)
110. J.F. Russo, S.A. Sheth, Deep brain stimulation of the 121. W.J. Elias et al., A pilot study of focused ultrasound
dorsal anterior cingulate cortex for the treatment of thalamotomy for essential tremor. N. Engl. J. Med.
chronic neuropathic pain. Neurosurg. Focus. 38(6), 369(7), 640–648 (2013)
E11 (2015) 122. N. Lipsman et al., MR-guided focused ultra-
111. H.S. Mayberg et al., Deep brain stimulation for sound thalamotomy for essential tremor: A proof-
treatment-resistant depression. Neuron 45(5), 651– of-concept study. Lancet Neurol. 12(5), 462–468
660 (2005) (2013)
112. H.S. Mayberg et al., Reciprocal limbic-cortical func- 123. M. Plaksin, E. Kimmel, S. Shoham, Cell-type-
tion and negative mood: Converging PET findings in selective effects of intramembrane cavitation as a
depression and normal sadness. Am. J. Psychiatry unifying theoretical framework for ultrasonic neu-
156(5), 675–682 (1999) romodulation. eNeuro 3(3) (2016)
113. A. Lavano et al., DBS in treatment of post-traumatic 124. P. Ghanouni et al., Transcranial MRI-guided focused
stress disorder. Brain Sci. 8(1) (2018) ultrasound: A review of the technologic and neuro-
114. C.H. Halpern et al., Deep brain stimulation in the logic applications. AJR Am. J. Roentgenol. 205(1),
treatment of obesity. J. Neurosurg. 109(4), 625–634 150–159 (2015)
(2008) 125. V.A. Velling, S.P. Shklyaruk, Modulation of the
115. R. Kumar et al., Obesity and deep brain stimulation: functional state of the brain with the aid of focused
An overview. Ann. Neurosci. 22(3), 181–188 (2015) ultrasonic action. Neurosci. Behav. Physiol. 18(5),
116. U.J. Muller et al., Deep brain stimulation of the 369–375 (1988)
nucleus accumbens for the treatment of addiction. 126. R.F. Dallapiazza et al., Noninvasive neuromodula-
Ann. N. Y. Acad. Sci. 1282, 119–128 (2013) tion and thalamic mapping with low-intensity fo-
117. R. Dallapiazza et al., Feasibility and safety of MR- cused ultrasound. J. Neurosurg. 128(3), 875–884
guided focused ultrasound lesioning in the setting (2018)
of deep brain stimulation. Stereotact. Funct. Neuro- 127. N. Grossman et al., Noninvasive deep brain stimu-
surg. 93(2), 140–146 (2015) lation via temporally interfering electric fields. Cell
118. T. Wang, R. Dallapiazza, W. Elias, Neurological 169(6), 1029–1041 (2017). e16
applications of transcranial high intensity focused 128. A. Gunduz, M.S. Okun, A new non-surgical ap-
ultrasound. Int. J. Hyperth., 285–291 (2015) proach for deep-brain stimulation. Lancet Neurol.
119. M. Wintermark et al., Thalamic connectivity in 16(8), e1 (2017)
patients with essential tremor treated with MR 129. L.G. Reijmers et al., Localization of a stable neural
imaging-guided focused ultrasound: In vivo fiber correlate of associative memory. Science 317(5842),
tracking by using diffusion-tensor MR imaging. Ra- 1230–1233 (2007)
diology 272(1), 202–209 (2014)
Transcranial Magnetic Stimulation:
Principles and Applications 7
Lari M. Koponen and Angel V. Peterchev
Abstract efficiency of the pulse generator as well as the

design of the stimulation coil. Ancillary effects
Transcranial magnetic stimulation (TMS) is a
such as heating, electromagnetic forces, and
noninvasive method for focal brain stimula-
interactions with other devices are considered.
tion, with applications in research, diagnostics,
Then, the underlying physics and its mod-
and treatment. In basic research, TMS can help
eling are presented, including the magnetic
establish a causal link between a brain cir-
field of the coil and the impact of the sub-
cuit and a behavior. Clinically, repetitive TMS
ject’s head on the induced electric field. This
can alter the long-term excitability of specific
is followed by a description of the biophysics
brain regions to treat psychiatric and neuro-
of neuronal activation due to TMS, includ-
logical disorders. This chapter aims to support
ing the cable equation, leaky integrate-and-fire
engineers and researchers to understand and
neural membrane dynamics, and morphologi-
innovate TMS technology. It introduces the
cally realistic neuron models. Various methods
basics of TMS spanning engineering, physics,
to measure the responses to TMS are sum-
biophysics, paradigms, and applications. First,
marized, spanning observations of behavior,
the principles of TMS devices are explained
electromyography, epidural recordings, elec-
including the electrical circuit topologies and
troencephalography, functional near-infrared
spectroscopy, functional magnetic resonance
Electronic Supplementary Material: The online version imaging, and positron emission tomography.
of this chapter (https://doi.org/10.1007/978-3-030-43395- The chapter concludes with an overview of
6_7) contains supplementary material, which is available stimulation paradigms encompassing single-
pulse, paired-pulse, and repetitive TMS, along
L. M. Koponen with their applications in basic research and
Department of Psychiatry & Behavioral Sciences, Duke the clinic. The chapter includes ten problems
that cover the presented material.
A. V. Peterchev ()
Keywords
Department of Psychiatry & Behavioral Sciences, Duke
University, Durham, NC, USA Brain stimulation · Transcranial magnetic
Department of Biomedical Engineering, Duke University, stimulation · TMS · TMS pulse generator ·
Durham, NC, USA TMS coil · Electric field · Physics of TMS ·
Department of Electrical & Computer Engineering, Duke Biophysics of TMS · Neuron model · TMS
stimulation paradigm · TMS applications
Department of Neurosurgery, Duke University, Durham,
NC, USA

246 L. M. Koponen and A. V. Peterchev
energy, in order to produce suprathreshold stimu-

7.1 Introduction
lation of the cortex. In contrast, noninvasive elec-
trical stimulation through the skull—transcranial
Transcranial magnetic stimulation (TMS) is a
electrical stimulation (TES)—requires currents
noninvasive brain stimulation method. In TMS, a
less than an ampere (<1 A), corresponding to
rapidly changing magnetic field (B-field) is used
energy on the order of 0.01 J, to evoke direct
to induce an electric field (E-field) inside the
motor responses. From an engineering point of
brain, mostly limited to the superficial parts of the
view, the energy and power levels required in
cortex. The E-field drives ionic currents causing
TMS remain somewhat challenging, limiting the
local hyperpolarization or depolarization of the
range of possible stimuli.
excitable membrane of the neurons. Sufficiently
TMS, however, has the critical benefit that it
large membrane depolarization results in firing
is tolerable. The skull has low electrical conduc-
of action potentials. Thus, despite its name, TMS
tivity and presents a barrier for the current flow
is essentially electrical stimulation, where the de-
in TES, resulting is very strong E-field in the
livery of an E-field to the brain is mediated by a
scalp relative to the brain. This makes TES at
magnetic field.
near-threshold current levels very painful, due to
The basic understanding of the physics
massive activation of sensory nerves and muscles
underlying magnetic stimulation dates back
in the scalp. In contrast, the skull is essentially
to late nineteenth century. In 1896, in the
transparent to the magnetic field, and therefore
first documented TMS experiment, D’Arsonval
TMS results in markedly lower scalp stimulation.
observed that “phosphenes [brief flashes of light
Consequently, TMS is usually painless—a typical
in the visual field] and vertigo, and in some
TMS pulse feels like a gentle tap on the head.
persons, syncope [fainting]” can be induced by
The tolerability advantage of TMS is the main
sending current through a large stimulation coil
reason for its widespread adoption, despite the
surrounding the whole head [1]. It is possible,
relatively high energy and power requirements of
however, that these early observed effects were
the equipment.
not of cortical origin, as phosphenes are more
This chapter begins with a discussion of the
readily observed due to retinal stimulation and
principles and types of TMS devices. We then
vertigo may be due to stimulation of the inner ear.
build a basic understanding of the TMS physics
After the early experiments, magnetic stimulation
and biophysics. Subsequently, we introduce
had a long hiatus, mostly due to lack of suitable
methods suitable for simultaneous measurements
power electronics to reach the required intensities
of the effects of TMS. Finally, we present the
with brief, controlled pulses. The first modern
typical uses of TMS in research and clinical
experiments on pulsed magnetic stimulation of
applications—to probe the brain without lasting
peripheral nerves were carried out in the 1970s.
changes in its function and to modulate its
Finally in 1985, Barker and his collaborators
function with repetitive stimulation.
demonstrated the first device capable of evoking
motor responses by transcranially stimulating
the human cortex [2]. The other key component
7.2 Devices
of modern TMS devices followed soon after
when Ueno and colleagues proposed the figure-
TMS devices are comprised of two main compo-
of-eight (figure-8) coil capable of focal brain
nents illustrated in Fig. 7.1: a pulse generator and
stimulation [3].
an electromagnetic coil placed over the subject’s
In terms of required energy, TMS and mag-
head. The pulse generator controls the tempo-
netic nerve stimulation in general are very ineffi-
ral waveform and amplitude of the TMS pulse,
cient methods to activate neurons. The current in
whereas the coil shape and placement determine
a multi-turn TMS coil must reach several kiloam-
the spatial distribution of the E-field induced in
peres (>1000 A), corresponding to about 100 J of
the brain.
7 Transcranial Magnetic Stimulation: Principles and Applications 247
Fig. 7.1 A TMS device comprises two main components: arm. A full TMS setup usually includes other devices, such
a pulse generator (a) and an electromagnetic coil (b). The as a stereotactic neuronavigation system (c) with optical
coil is held on the scalp of the TMS subject (sitting) by trackers both on the coil (d) and on the subject’s head—
an operator or, alternatively, a static or robotic mechanical here attached to a pair of goggles (e)
7.2.1 Pulse Generators with the coil L. Therefore, the coil current has a
dampened sine shape:
7.2.1.1 Circuit Topology
The principle of TMS devices is to generate a VC (0)
IL (t) = sin (ωt) exp (−αt) , t ≥ 0 (7.1)
magnetic pulse by rapidly transferring energy ωL
from a capacitor in the pulse generator to the coil.
where t is time, VC (0) is the initial capacitor
Three representative TMS device circuit topolo-
voltage, α = r/2L characterizes the damping
from
gies are shown in Fig. 7.2. The conventional
the circuit series resistance r, ω = 1/LC − α 2
monophasic stimulator (in the left column) uses a
is the damped resonance frequency of the pulse,
high-voltage power supply to charge capacitor C
and L and C are, respectively, the coil inductance
to a high voltage (typically <3000 V). To generate
and energy storage capacitance. The maximum
a TMS pulse, a high-power semiconductor switch
peak coil current of a conventional TMS device
Q is triggered, and the energy of capacitor C
is several kiloamperes (typically <8000 A). In
is transferred to the TMS coil L. Switch Q is
the circuit diagrams, the resistor r lumps the
typically implemented with a silicon-controlled
resistance of the coil windings, cable, connector,
rectifier, a type of thyristor. The silicon-controlled
switch Q (when turned on), and capacitor. The
rectifier is turned on by a trigger pulse applied
coil voltage is proportional to the derivative of the
to its gate terminal, but it cannot be turned off
coil current:
in a controlled way—it stops conducting only
when the coil current IL drops to zero. During
VL (t) = L dIdt
L (t)
. (7.2)
its discharge, capacitor C forms a resonant circuit
Controller Gate drive

Q Q1
Q + D1
IL IL VC1
R + -
C1 L IL
+ + D - r
VC VL L VC VL L -
C - D C - +
+ VL +
Power -
VC2
supply r r -
C2 D2
Q2
Fig. 7.2 Circuit topologies of representative TMS de- energy storage capacitor C, the power switch gate drives,
vices producing coil a current (IL ) and a proportional and the system controller are common to all topologies
magnetic field pulse that is monophasic sinusoidal (left), but shown only in the left column for concision. The
biphasic sinusoidal (middle), and nearly triangular (right). circuit component values for the shown waveforms are
The induced E-field is proportional to the coil voltage, VL , C = 185 μF, C1 = 370 μF, C2 = 1,500 μF, L = 16 μH,
and results in estimated neuronal membrane polarization r = 50 m , R = 100 m , VC1 = VC , and VC1 /VC2 = 4. The
change Vm . (The y-axes units are arbitrary.) To make all pulse duration of the initial rising current for the monopha-
three stimulators deliver strongest neural activation in the sic nearly triangular pulse is 51 μs, which produces iden-
same direction, the polarity of the coil has been reversed tical membrane depolarization, Vm , as the 81-μs-long
for the middle column. The power supply charging the initial rising current for the monophasic sinusoidal pulse
Therefore, for the underdamped sinusoidal switched between the two capacitors. Moreover,
current of Eq. 7.1: C1 and C2 have significantly larger capacitance
than those in the conventional stimulators dis-
α
VL (t) = VC (0) cos (ωt) − sin (ωt) cussed above, enabling slower discharge of the
ω (7.3) capacitors, which results in more linear slopes of
× exp (−αt) , t ≥ 0. the coil current and therefore more rectangular
shape of the induced E-field. Finally, instead of
For the monophasic stimulator, Eqs. 7.1 and thyristors, the capacitors are connected to the
7.3 hold until the capacitor discharges to zero, coil with transistors (specifically, insulated-gate
i.e., for VC (t) ≥ 0. When VC reaches zero, the coil bipolar transistors) which can be not only turned
current is at its (positive) peak and continues to on but also turned off by driving their gate—this
discharge VC into negative values. This results in allows control over the duration (width) of the
diode D turning on, which adds a damping resis- pulse phases. Like the silicon-controlled rectifiers
tor R in parallel with the capacitor. The damping used in conventional devices, insulated-gate bipo-
effect of R now dominates that of r, and the lar transistors can conduct current in only one
circuit configuration becomes mostly a parallel direction, whereas diodes D1 and D2 allow cur-
LCR resonant circuit, with R selected to produce rent to flow in the opposite direction. This device
an overdamped response. Representative shapes can generate monophasic, biphasic, or polyphasic
of the coil current and voltage waveforms for the magnetic pulses by appropriate switching of the
monophasic sinusoidal pulse are illustrated in the coil between the two capacitors. A monophasic
left column of Fig. 7.2. Note that the coil B-field pulse and a biphasic pulse are illustrated in Fig.
is proportional to IL (t) and the E-field induced by 7.2. The monophasic pulse is generated by con-
the coil is proportional to VL (t) (see Eq. 7.8). necting the coil first to C1 and then to C2 . In the
The circuit topology of the conventional biphasic pulse, the coil is first connected to C2 ,
biphasic stimulator is represented in the middle then to C1 , and finally back to C2 .
column of Fig. 7.2. Unlike the monophasic
stimulator discussed above, there is no deliberate 7.2.1.2 Energy Efficiency
damping of the pulse energy, and the capacitor and Repetitive TMS
and coil are allowed to resonate for a full period The total electric energy stored in the TMS device
of T = 2π /ω seconds. This is achieved by capacitor(s) is
connecting diode D in antiparallel to Q to allow

the coil energy, minus circuit losses, to return WC (t) = 1
2 i Ci VC2i (t) (7.4)
back to the capacitor. The resulting underdamped
sinusoidal coil current is biphasic, and the coil where the summation is over all capacitors in the
and capacitor voltages have an underdamped circuit—one for conventional monophasic and
cosine triphasic waveform. biphasic devices and two for the advanced topol-
The right column of Fig. 7.2 illustrates an ogy shown in Fig. 7.2, right column. The B-field
advanced circuit topology that allows more flex- energy stored in the coil is
ible shaping of the TMS pulse waveform [4]. It
deploys two energy storage capacitors, C1 and WL (t) = 12 LIL2 (t). (7.5)
C2 . By turning on switch Q1 or Q2 , the coil is
connected across capacitor C1 or C2 , respectively. To produce neural depolarization in the brain,
This allows independent control of the rate of TMS pulse energies in the 50–200 J range are
rise and fall of the coil current and therefore the typically required. For rTMS pulse trains, the
amplitude of the positive and negative phases of average power circulated through the coil is the
the induced E-field. The coil current and voltage product of the pulse energy and the repetition
still follow Eqs. 7.1 and 7.2 with appropriate frequency. This power is substantial—for exam-
initial conditions that change each time the coil is ple, 0.5–2 kW for a 10 Hz pulse train. Presently,
many rTMS devices can deliver pulse trains with with briefer and more rectangular E-field pulses,
frequencies up to 50 Hz, and some can exceed ηrecov can reach 90%.
100 Hz. Energy efficiency is therefore important,
especially for rTMS devices. The energy recovery
efficiency of an rTMS device can be quantified by 7.2.2 Coils
the ratio of the capacitor energy at the end of a
pulse (t = T) to that at the beginning of the pulse The simplest TMS coil type is the round coil
(t = 0): which has a single circular winding (Fig. 7.3a).
The round coil induces a circular E-field pattern
ηrecov = WC (T )
WC (0) (7.6) in the brain. Therefore, the resultant stimulation
is nonfocal. In the figure-8 configuration (Fig.
where T is the pulse duration. For example, 7.3b), two round coils are placed next to each
T = 2π /ω is the period of the sine wave for a other with current flowing in opposite directions
conventional sinusoidal biphasic stimulator. If in the two windings, creating a focal E-field peak
resistance r is the only loss mechanism in the where the two loops meet. Because of its fo-
circuit, then the energy loss can be calculated cality, the figure-8 coil is the most commonly
from the coil current (Eq. 7.1) by integrating the used TMS coil. A variation of the figure-8 coil
power loss is the double-cone coil (Fig. 7.3c), in which the
two loops are larger in diameter, increasing the

T depth of stimulation in the brain and angled with
Wloss = WC (0) − WC (T ) = rIL2 (t)dt. (7.7)
0
respect to each other, improving the magnetic
field coupling to the brain. Hence a double-cone
The circuit topologies in Fig. 7.2 are ordered coil requires 53% less energy than a figure-8 coil
by increasing energy efficiency. The conventional to evoke comparable responses [6]. The increased
monophasic stimulator deliberately dissipates all depth and energy efficiency, however, cost re-
energy returning from the coil into resistor R. duced focality [7]. Another variation involves
Therefore, ηrecov = 0 and capacitor C has to be adding a ferromagnetic core into the figure-8 coil
fully recharged by the power supply before each (Fig. 7.3d). The ferromagnetic material has high
pulse. For this reason, conventional monophasic magnetic permeability and consequently focuses
stimulators are not well suited for high-frequency the magnetic field energy toward the subject’s
pulse trains (>1 Hz). In contrast, the conven- head. Consequently, a ferromagnetic core coil
tional biphasic stimulator in the middle column has been shown to require 73% less energy to
recharges capacitor C with the coil energy re- stimulate neurons and heats up less compared to
maining at the end of the pulse. The energy losses air core coils [8]. Another approach to increase
in this topology are from nonidealities of the the coil efficiency is to optimize computationally
circuit components, especially the effective series the winding pattern, with a reported gain of 41%
resistance r. Consequently, the power supply has (Fig. 7.3e) [9]. Finally, coils with steerable locus
to only top off the capacitor charge, reducing and direction of the E-field can be implemented
significantly the power requirements of the de- by layering windings with orthogonal field char-
vice. The topology in the right column of Fig. acteristics and controlling them with independent
7.2 recycles pulse energy by transferring it be- pulse generators (Fig. 7.3f) [10]. A general trade-
tween the two capacitors. Moreover, it requires off in TMS coil design is that larger coils generate
less energy for stimulation, since rectangular E- deeper but less focal E-field than smaller coils
field pulses are more efficient at depolarizing [11, 12]; this is discussed further in Sec. 7.3.3.2.
neurons than conventional cosine pulses [4, 5]. In The coil induces a magnetic field B propor-
conventional sinusoidal biphasic rTMS devices, tional to the total current flowing through the
typically ηrecov = 50 – 70%, whereas in devices coil cross section, NIL , where N is the number of
winding turns. The E-field induced by the TMS
Fig. 7.3 TMS coil types: (a) round, (b) figure-8, (c) coil. The induced E-field distributions are shown for a
double cone, (d) figure-8 with ferromagnetic core, (e) spherical head model
computationally optimized winding, and (f) multi-locus
pulse is proportional to the rate of change of the determined by N. Typical TMS coil inductances
magnetic field, therefore, using Eq. 7.2: range from 10 to 25 μH.
E(t) ∝ dB
dt
∝ N dIdtL (t) = N
V (t).
L L
(7.8) 7.2.2.1 Coil Heating
In the absence of coil cooling, the coil winding
Thus, the induced E-field waveform is directly temperature rises at a rate proportional to the
proportional to the voltage of the coil inductor. product of the energy loss in the coil, Wloss , and
The inductance of the coil is given by the pulse repetition frequency. This can result
in substantial increases of the coil temperature,
L = μr μ0 N 2 SlLL (7.9) especially for rapid pulse trains. Therefore, the
majority of rTMS coils have forced air or liq-
where μr is the effective relative permeability of uid cooling. Strategies discussed above such as
the magnetic flux path (μr = 1 for air-core coils) ferromagnetic cores, computationally optimized
and μ0 = 4π × 10−7 H/m is the permeability of windings, and rectangular E-field pulse shape can
free space. Parameters SL and lL are, respectively, be used to increase the energy efficiency and
the average cross-sectional area and length of the hence mitigate coil heating and the need for active
magnetic flux path and are related to the spatial cooling.
profile of the coil B-field. Section 7.3 discusses
how to calculate the spatial distribution of the 7.2.2.2 Coil Forces
TMS coil magnetic field, which determines SL The high currents in TMS coils result in substan-
and lL . The ratio SL /lL scales proportionally to tial internal coil forces. The strongest of these
the coil radius. For a given size and shape of the is the net outward radial force acting on circular
coil, the SL /lL ratio is fixed, and the inductance is windings:
F (t) = dWL (t)

dρ
= 1 dL 2
I
2 dρ L (t) ∝ N 2 IL2 (t), (7.10) induced currents includes highly conductive EEG
electrodes and metal plates on the skull, which
where ρ is the coil radius [13]. The second can form low-resistance conductive paths for the
strongest force is the compression of neighboring current, resulting in heating.
turns within the coil: despite the net radial force
pointing outward, the force on the outermost turns
points inward. These forces produce mechanical 7.3 Physics
deformation and oscillation of the coil surface
that generate an audible clicking sound and Electrically excitable cells, such as neurons and
tapping on the subject’s scalp. The peak sound muscle cells, can be activated by imposing an E-
pressure of this sound can exceed 120 dB, field across them. In TMS, this E-field is applied
necessitating TMS subjects to wear hearing across neurons in the brain by means of a time-
protection. varying electromagnetic field. The electromag-
netic field is governed by Maxwell’s equations,
namely, Faraday’s law of induction
7.2.3 Device Safety
∇ × E = − ∂B
∂t
(7.12)
TMS devices must be designed to operate
safely. Specific safety considerations include and Ampere’s circuital law
appropriate insulation from high voltages in
the device, mechanical integrity of the coil for ∇ × B = μ0 J + ε0 ∂E
∂t
, (7.13)
repeated high electromagnetic forces in the
windings, safe operating temperature of the where E is the electric field, B is the magnetic
coil, mitigation of the coil clicking sound, and field, J is the current density, μ0 is the permeabil-
protections from erroneous delivery of high ity of free space, and ε0 is the permittivity of free
amplitude and/or frequency pulse trains that space. Equation 7.13 is strongly coupled to Eq.
could induce a seizure. For example, as medical 7.12 as, in addition to the so-called displacement
electrical equipment, commercial TMS devices current (ε0 ∂E/∂t), the E-field determines the in-
have to meet safety standards such as the IEC duced current density through Ohm’s law
60601.
J = σ E, (7.14)
7.2.3.1 Interaction with Other Devices
Implants that have wires placed in the scalp where σ is the electrical conductivity, for exam-
and/or the brain can be adversely affected ple, of tissue.
by voltages and currents induced by TMS. For modeling TMS, Eqs. 7.12 and 7.13 can
These include cochlear implants which have an fortunately be approximately decoupled. First, as
inductive coil implanted under the scalp as well seen in Sec. 7.2, a typical biphasic TMS pulse
as deep brain stimulation (DBS) implants which lasts about 300 μs, corresponding to a charac-
can have looping of lead wire under the scalp. The teristic frequency of 3.3 kHz. This is a very low
induced voltage in a wire loop can be estimated frequency for an electromagnetic field, having a
using Faraday’s law: wavelength of about 90 km. Therefore, we can
safely perform a few approximations: the TMS
Vind = − dΦ (7.11) coil current is in phase in all parts of the coil, and
dt
the B-field in the volume of interest is in phase
where = Sind • B is the magnetic flux associated with the coil current, the displacement currents
with magnetic field B through a loop of area Sind . are negligible, and no tissue exhibits apprecia-
Other hardware that can be adversely affected by ble magnetization. This electromagnetic regime
is referred to as “quasi-static” [14]. Further, the
magnetic field generated by the currents induced

in the subject’s body by TMS is insignificant.
Even for the most conductive biological tissues,
such as the cerebrospinal fluid, the conductivity
is only on the order of 2 S/m. Thus, the induced
currents in Eq. 7.14 are tiny: At its maximum
output, a typical TMS device induces peak E-
fields less than 250 V/m [15], resulting in current
densities of at most 0.5 mA/mm2 . In contrast, the
current density in the windings of a typical TMS
coil is about 200,000 mA/mm2 . Therefore, we
only have to consider the magnetic field of the Fig. 7.4 Magnetic field of circular coil. The direction of
TMS coil. With these approximations in place, the magnetic field is related to the direction of the current
by the right hand rule
we can first compute the B-field of the coil from
Eq. 7.13 using magnetostatics and use that B-field
to solve the E-field induced in the subject’s head 7.3.2 Induced Electric Field
with Eq. 7.12.
The E-field induced by TMS in the subject’s head
can be obtained from Faraday’s law of induction
7.3.1 Magnetic Field (Eq. 7.12). We can solve the E-field by splitting
it into a primary E-field, which is a direct result
According to Ampère’s law (Eq. 7.13), a current of the change in the B-field and which would
density gives rise to a B-field. Under the quasi- be present in an infinite homogeneous medium,
static approximation, the B-field can be obtained and a secondary E-field originating from electric
from the Biot–Savart law charge accumulation at tissue interfaces in the

J (r ,t ) dV ×(r−r )
head due to the primary E-field. To obtain this
B (r, t) = μ0
4π V |r−r |3
, (7.15) split, we must express B and E in terms of an
electric scalar potential ϕ and a magnetic vector
where B(r, t) is the magnetic field at point r and potential A

J(r , t)dV is a differential volume current element

at point r . Given the magnetostatic assumption of B =∇ ×A (7.17)
no induced volume currents, for an air core TMS
coil, Eq. 7.15 further simplifies to and
dr ×(r−r )
B (r, t) = μ0
4π
I (t) p |r−r |3 , (7.16) E = −∇ϕ − ∂A
∂t
. (7.18)
where I(t) is the coil current, p is the closed These two potentials are not uniquely defined,

path formed by the coil windings, and r is the as the divergence of the magnetic vector potential
integration variable following this path. For fer- (∇ · A) can have any value [16]. By fixing the
romagnetic core coils, we also need to include divergence with the Coulomb gauge (∇ · A = 0),
the magnetization currents in the core for the we isolate the primary and the secondary fields to
integration, which requires a numerical solver in correspond to the two terms of Eq. 7.18
most cases. Figure 7.4 illustrates the magnetic
field of a circular coil computed using Eq. 7.16. E primary = − ∂A
∂t
(7.19)
The peak magnetic field strength near a TMS coil
is typically on the order of 1 tesla (see Problem 1). and
E secondary = −∇ϕ. (7.20) at most a few microseconds [17]. As we will

learn in Sec. 7.4, the stimulation is proportional
With the Coulomb gauge, Eq. 7.16 can be to the sustained component of the E-field. Con-
rewritten as sequently, we are not interested in this transient
behavior, and the easiest way to obtain the sec-
dr
A (r, t) = μ0
4π
I (t) p |r−r | . (7.21) ondary currents is from Ohm’s law (Eq. 7.14) and
the conservation of charge
Thus, we obtain a closed-form solution for the
primary E-field ∇· J = − ∂ρ
∂t
, (7.23)
dr
E primary (r, t) = − 4π
μ0 dI (t)
dt p |r−r | . (7.22) where ρ is the charge density. The quasi-static
steady state does not allow for a time-dependent
Equation 7.22 can be interpreted as the pri- charge accumulation, which further simplifies
mary E-field being a smoothed mirror image of Eq. 7.23 to
the coil current. For example, a circular coil in-
duces a circular E-field pattern with largest mag- ∇· J = 0. (7.24)
nitude under the coil winding annulus. The minus
sign in Eq. 7.22 represents Lenz’s law, which Consequently, the scalar potential for the sec-
states that direction of the induced current, and ondary E-field in Eq. 7.18 satisfies Laplace’s
hence Eprimary , is such that the magnetic field cre- equation
ated by the induced current opposes the changing
field B. 0 = ∇· E = 0 + ∇· E secondary = ∇ 2 ϕ, (7.25)
The secondary E-field partially cancels out the
primary E-field, as illustrated in Fig. 7.5. The within each region of uniform conductivity, since
secondary field arises from charge accumulation the divergence of the primary E-field is zero.
at conductivity boundaries, which, from the TMS Eq. 7.24 also gives us the boundary conditions
point of view, happens near instantaneously in for the total E-field: using Gauss’s law, e.g., the
Fig. 7.5 Left: The primary E-field of a typical figure-8 symmetric volume conductor (approximating a human
coil (Magstim 70 mm Double Coil) when its current is brain) centered 85 mm below the coil surface. Right: The
increasing at a rate of 100 A/μs (corresponding to 1600 V total E-field is the vector sum of the primary and secondary
applied across the coil with inductance of 16 μH). The fields. The E-field distribution is more superficial than the
combination of two circular loops with current rotating primary field alone, and the peak magnitude is reduced to
in opposite directions produces a focal E-field peak of about 150 V/m. As a special property of the spherically
about 240 V/m at 15 mm from the intersection of the symmetric geometry, the secondary field cancels the nor-
two loops. Middle: The secondary E-field in a spherically mal component of the primary field everywhere inside the
volume
divergence theorem [16], we can convert Eq. 7.24 7.3.3 Electric Field Models
into an integral form
In the two previous sections, we derived the
0= 0dV = (∇· J ) dV physics controlling the total E-field induced by
Ω Ω TMS, namely, Eq. 7.22 to obtain the primary E-
(7.26)
field, Eq. 7.28 to obtain the boundary conditions
= J · n̂ dS , for the secondary E-field at tissue interfaces, and
∂Ω
Eq. 7.25 to obtain the secondary E-field within the
where is any arbitrary volume, dV is a differen- tissues. These three equations form a basis for the
tial volume element in it, ∂ is the closed surface direct approach for computing the E-field inside
of the volume , and n̂ and dS are, respectively, a volume conductor model of the subject’s head,
its surface normal vector and differential surface combining the head geometry and conductivity
element. Following the classical textbook deriva- information at each point inside the head. Once
tion of the boundary condition for E- or B-fields the head model has been formed, the E-field can
with a “very shallow pillbox” [16], we obtain the be solved by discretizing the head model and
boundary condition at tissue interfaces computing the E-field distribution with finite
element, finite difference, or boundary element
J 1 · n̂ = J 2 · n̂, (7.27) methods, all of which can be made sufficiently
accurate by appropriately dense discretization of
where J1 and J2 are the current densities in adja- the head and coil models [20].
cent tissues 1 and 2, respectively. Substituting Eq. The head model is usually made by segment-
7.14 into Eq. 7.27, we obtain ing a magnetic resonance imaging (MRI) dataset,
containing at least a T1-weighted scan, into the
σ1 E 1 · n̂ = σ2 E 2 · n̂, (7.28) major tissue types present in the head, and giv-
ing these tissues conductivity values from the
where σ 1 and σ 2 are the conductivities of tissues 1 literature. A simple head model comprises a sin-
and 2 and E1 and E2 are the total E-fields in these gle volume defined by the head surface and as-
tissues, respectively. At the scalp–air interface, signed uniform and isotropic conductivity. A typ-
this simplifies to E scalp · n̂ = 0. Consequently, ical anatomically detailed model contains sev-
near the head surface, the secondary field will eral different tissues, for example, scalp, compact
have to cancel out any normal component of bone, spongy bone, cerebrospinal fluid, gray mat-
the primary field so that the total E-field has no ter, and white matter. For a list of conductivity
normal component near the outside surface of the values of these tissues, see Table 7.1. The lim-
head (see Fig. 7.5). In the typical case of a focal iting factor in the accuracy of the head model
figure-8 coil, the secondary E-field also reduces is the accuracy of the segmentation and the un-
the magnitude of the tangential component of the certainty associated with the tissue conductivity
total E-field to about two-thirds of that of the values [21]. For example, the common estimates
primary field.
With the exception of a few special cases,
such as spherically symmetric geometry [18] or Table 7.1 A typical range of conductivity values for the
tissues present in the head [21]
cylindrical or semi-infinite geometry [19], there is
Tissue Conductivity (S/m)
no closed-form solution for the secondary E-field
White matter 0.1–0.4
and hence the total E-field. Rather, the secondary
Gray matter 0.1–0.6
field is usually computed with numerical methods
Cerebrospinal fluid 1.2–1.8
discussed in the next section.
Compact bone 0.003–0.012
Spongy bone 0.015–0.04
Scalp 0.2–0.5
of the conductivity of the gray matter have a metric head geometries. The latter is used to
sixfold range. Further, as the cerebrospinal fluid approximate the E-field in real-time neuronavi-
is about 100 times more conductive than the sur- gation systems [26]. The reciprocal approach can
rounding compact bone, any geometrical error in also be used to speed up the computation in a
the shape of the inner skull segmentation may realistic head geometry to suit real-time E-field
have large effects on the predicted secondary E- computation [27].
field inside the head. Accurate skull segmentation
may require the combination of several types of 7.3.3.2 Fundamental Limitations
scans, for example, T1- and T2-weighted images of Induced Electric Field
with and without fat suppression [22]. The maximum magnitude of the TMS-induced
In the notation for Ohm’s law in Eq. 7.14 and E-field is always superficial, since the peak E-
consequently in the boundary conditions in Eq. field in each tissue must be on its surface [24].
7.28, we implicitly assumed a linear homoge- This fundamental limitation of the E-field spatial
neous tissue. For the finite difference and finite distribution has two important implications to
element methods, however, we can simply replace TMS.
the scalar conductivity with a conductivity ten- First, the TMS-induced E-field diminishes
sor to model linear anisotropic tissue [23]. The very rapidly as the distance from the coil is
anisotropy of tissue conductivity can be estimated increased. The primary E-field of a figure-8
with MRI diffusion tensor imaging. coil attenuates as 1/r3 with increasing distance
from the coil surface, and the total E-field in a
7.3.3.1 Reciprocity volume conductor attenuates even faster with
to Magnetoencephalography increasing depth (see Fig. 7.5). Indeed, in a
Magnetoencephalography (MEG) is a functional spherical conductor, the total E-field in the center
brain imaging technique that is essentially the of the sphere is exactly zero for all coils. Further,
inverse of TMS: a coil placed on the subject’s the E-field from focal coils attenuates faster than
scalp is used to detect the tiny magnetic fields that of nonfocal coils, which results in a strict
generated by the electrical activity of cerebral limit in penetration depth at a given focality, as
neurons. Deploying again the quasi-static approx- demonstrated in Fig. 7.6.
imation introduced in the beginning of Sec. 7.3, Second, there is an optimal coil size that de-
we can derive a reciprocal relationship between pends on the size of the stimulated head. An
the induced E-field in TMS due to the changing overly large coil has limited coupling to a small
current in the TMS coil and the magnetic flux head, resulting in the lack of focality and reduced
through the pickup coil in MEG due to the source peak E-field compared to a larger head despite the
currents in the brain [24] short distance between the coil and the brain (see

Fig. 7.7). Unfortunately, this limitation cannot be

Q· E r Q = − dIdt(t) S B· n̂ dS, (7.29) circumvented simply by miniaturization of the
TMS coil: the required coil current does not scale
where B is the magnetic field due to a current down linearly with coil size, and thus smaller
dipole Q at a location rQ in the brain, I(t) is coil coils have larger power losses and overheat con-
current at time t, S is a surface limited by the siderably faster than larger coils. Consequently,
coil windings, and E is the resulting induced E- even the smallest practical figure-8 coils cannot
field in the brain at location rQ . As with the direct achieve focality in small animals similar to that
approach to TMS E-field computation, for most of regular figure-8 coils for humans.
head models computing B due to Q requires a Finally, a fundamental limitation of the tem-
numerical approach, for example, the boundary poral waveform of TMS induced E-fields is that
element method [25]. The reciprocal approach, it must always have both positive and negative
however, allows obtaining closed-form solutions phases, with the total area under the curve for all
of semi-infinite, cylindrical, and spherically sym- positive phases being equal to that for the negative
Fig. 7.6 Trade-off between depth and spread of the E- coil designs [11], and the red curve is the computationally
field. More focal (smaller spread) coils have lower pen- optimized limit [12]. (Reproduced with permission from
etration depth. The numbered dots correspond to specific Gomez et al. [12])
Fig. 7.7 Induced E-field distribution in mouse, monkey, when the size of the head is decreased and is very inef-
and human from figure-8 TMS coils with 25-mm or 70- ficient for the stimulation of a small animal. Both coils
mm-loop diameter for matched stimulator output (capac- produce nonfocal stimulation in the mouse. (Reproduced
itor voltage). The large coil loses its efficiency advantage with permission from Alekseichuk et al. [28])
phases. Another way to state this is that the time end of the pulse. Consequently, there can be no
integral over the E-field pulse waveform must monophasic E-field pulses in TMS: monophasic
equal zero. The reason is that the electric field is magnetic pulses produce biphasic E-field pulses
proportional to the derivative of the coil current (see Fig. 7.2).
and the coil current must drop back to zero at the
2
7.4 Biophysics λ2 ∂ (ΔV )
2 − (ΔV ) −τ
∂(ΔV )
∂t = λ2 ∂(E·x)
∂x =f (x, t) ,
∂x
(7.30)
As already mentioned, TMS is not really
magnetic stimulation but electric stimulation where V is the membrane depolarization mea-
mediated by the magnetic field. Consequently, sured as a difference from the resting membrane
√
much of the biophysics of TMS is common potential, λ = rm / (ri + re ) is the length con-
with that of electrical stimulation. The two stant of the axon, and τ = rm cm is its time con-
stimulation modalities have, however, a few stant [30]. Parameter cm stands for the membrane
notable differences in the E-field characteristics capacitance per surface area, and the three lower
relevant for neural activation. First, in TMS the case rs represent the membrane resistivity (re-
maximum E-field magnitude and the maximum sistance pre unit surface area) of the membrane,
E-field gradient are in different spatial locations, intracellular space, and extracellular space, re-
whereas in electric stimulation, both occur next spectively.
to the stimulating electrode. Second, TMS pulses The cable equation, Eq. 7.30, suggests that the
are always relatively brief, typically between stimulation occurs at the maximum gradient of
150 and 1000 μs, whereas electrical stimulation the E-field, rather than its maximum magnitude.
pulses can be from a few μs to dc stimulation. For peripheral neurons, which are generally long
Third, TMS pulses are always “charge balanced” and straight, this does indeed hold true [31]. In
since the circulated charge is proportional to the brain, however, the experimentally observed
the temporal integral of the E-field which site of activation is near the strongest E-field
is always zero, as discussed above, whereas underneath the center of a figure-8 coil [32–34].
electrical stimulation can inject a nonzero net This apparent discrepancy results from violation
charge, for example, with monophasic electrical of one of the basic assumptions in Eq. 7.30, as
pulses. Finally, the E-field induced by TMS is cortical axons are not long and straight (see Fig.
mostly tangential to the scalp surface, whereas 7.8). Rather, in the cortex (gray matter), axons
transcranial electrical stimulation can generate begin from the soma (cell body), have bends
E-field with a strong component normal to the and branches, and—with the exception of pyra-
scalp surface. midal axons which pass into the white matter—
terminate at synapses with neighboring neurons.
All of these conditions result in effective gradi-
7.4.1 Neuronal Membrane ents of the E-field along the axon, leading to sites
Depolarization in Response of local depolarization of the membrane due to the
to Electric Field external field (observed experimentally by Amas-
sian et al. [35]). The strongest depolarization
A TMS pulse induces an E-field, which drives occurs at axonal terminals (synapses), resulting
electric currents into the cell membranes of in lowest threshold activation there [36]. If we
neurons in the brain. At the small spatial scale assume that such site of activation is small com-
of an excitable neuronal membrane, we can no pared to the length constant, the second spatial
longer assume the quasi-static approximation and derivative of the membrane depolarization along
omit capacitive effects. Rather, at the level of the the cable will be tiny, and we can approximate the
neuronal membrane, the capacitive effect of the cable equation as
membrane is dominant, and will cause a time-
dependent charge accumulation requiring us to ΔV + τ ∂(ΔV
∂t
)
≈ −f (x = 0, t) ∝ E(t)· n,
consider the coupled dynamics in the temporal (7.31)
and spatial domains. For a thin, long straight
axon, these result in the famous cable equation where n is the unit vector aligned with the
[29], which for TMS is fiber direction at the terminal. This model fits
scalp Essentially, the time course of this model’s

gyral crown
skull
l fluid
membrane voltage follows the E-field waveform
gyral lip cerebrospina filtered by a first-order low-pass filter with a time
gre y ma tte r
sulcal wall
white matte constant τ . For a rectangular E-field pulse, where
sulcus
the induced E-field is approximately constant for
axon the duration of the initial rising coil current (Fig.
branches
7.2), Eq. 7.32 further simplifies to
bend

t
t−t
soma
dendrites axon hillock
pyramidal axon ΔV (t) ∝ 0 exp − τ
dt ∝ 1 − exp − τt .
(7.33)
Fig. 7.8 A simplified drawing of the brain geometry rel-
evant to cortical activation by TMS. Illustrated are two
pyramidal neurons in the cortex (not to scale). They com- Thus, in response to a step in the E-field, the
prise a dendritic tree to which axons from other neurons neural membrane voltage changes exponentially
are terminating via synapses, a pyramidal soma that nar- to a new level.
rows down towards the axon hillock, and an axon with two
short lateral branches terminating in grey matter and a long
main branch passing into the white matter. The somato-
dendritic axis of the pyramidal neurons is perpendicular 7.4.2 Neural Activation Models
to the pial surface. The sites of lowest threshold activation
by TMS appear to be axonal terminals (synapses) in the In the previous section, we derived the basic
lip and crown of gyri
physics behind the membrane depolarization due
to TMS. The passive membrane (with capaci-
experimental data qualitatively. In most cases, tance and leakage resistance) gave rise to first-
the stimulation outcome has a strong dependency order linear dynamics. Besides linear resistance,
on the E-field direction: for example, in primary the neuronal membrane has active ion channels
motor cortex, we observe largest responses when that have a strongly nonlinear resistance and are
the E-field is pointing toward the frontal wall of responsible for the generation and propagation of
the central sulcus [37]. In some cases, however, action potentials. Here, we discuss a few different
no directional dependency is observed: with approaches to this and the pros and cons of each
suitable paired-pulse protocol (such protocols are of these approaches.
discussed in more detail in Sec. 7.6), inhibition First, we discuss the simplest approach that
shows no orientation dependency [38]. Thus, is capable of providing a decent amount of ex-
more than one kind of neurons may be activated planatory power. By assigning a sharp “firing”
by TMS: Pyramidal neurons have asymmetric threshold for the membrane depolarization aris-
axonal arbors and are therefore sensitive to the ing from the model of Eq. 7.32, we get a sim-
direction of the E-field. In contrast, interneurons ple integrate-and-fire neural model which can be
have more symmetric axonal arbors and are used to predict the E-field amplitude required for
largely insensitive to the E-field direction [36, neural activation as a function of the E-field pulse
39]. waveform. The rheobase is defined as the lowest
For a given E-field direction, Eq. 7.31 results E-field amplitude to activate a neuron. For the
in the famous leaky integrator model of the neural integrate-and-fire neuron model, Erheobase is the
membrane response [40]. Solving this first-order amplitude of an infinitely long rectangular E-field
differential equation, the membrane depolariza- pulse at the neuronal activation threshold. Thus,
tion is proportional to the “leaky” temporal inte- based on Eq. 7.33, a rectangular pulse of duration
gral of the E-field T requires an E-field amplitude of

t 1−exp(− ∞
t−t τ )
ΔV (t) ∝ E(t) exp − dt . (7.32) Ethreshold = E
1−exp(− Tτ ) rheobase
= Erheobase
1−exp(− Tτ
0 τ )
(7.34)
to activate a neuron with a rheobase Erheobase and In order to study the site of activation and
membrane time constant τ . For non-rectangular to compare different kinds of neurons, we need
pulses, the principle is the same, but the integral to model the effects of the cell geometry. Mam-
in Eq. 7.33 has to be evaluated for the specific E- malian axons are typically myelinated—the axon
field waveform [41]. This model is indeed what is wrapped in a myelin sheath with relatively high
we have implicitly assumed several times in this resistance and low capacitance. There are only
chapter so far. For example, in Fig. 7.2, we used narrow openings called nodes of Ranvier between
this approach to explain why biphasic stimulation myelinated segments of the axon. By discretiz-
has higher energy efficiency than monophasic ing the cable equation (Eq. 7.30) at these nodes,
stimulation and why rectangular E-field pulses the second-order partial differential equation be-
were more efficient than conventional sinusoidal comes an ordinary finite difference equation [43]
pulses. Specifically, the increased efficiency is
contributed by a larger membrane depolarization ∂ λ2
τ [ΔVn ] +In,ion = 2 [ΔVn+1 −2ΔVn +ΔVn−1 ]
for a given amount of energy in the TMS pulse. ∂t l
In Fig. 7.9, we use the integrate-and-fire model to λ2
derive the relationship between the length of the − ΔVn − [En+1 − En ] ,
l
TMS pulse and the required peak stimulation cur- (7.35)
rent and voltage. These predictions match quan-
titative experimental results quite well for such a where In, ion models any additional ion channels at
simple model [42]. Because of its simplicity and node n, E is the E-field component tangential to
relatively good accuracy, the integrate-and-fire the axon, and the subscripts index the node. This
model is often used when simulating large net- formulation makes it relatively straightforward
works of neurons. This simple model, however, to construct a set of equations to model the full
provides no insight into the site of activation in the geometry of the axon, including the hillock, initial
brain, as it contains no mechanism for spatially segment, bends, branches, and terminations [44].
coupling the neuron to the E-field. The current In, ion is typically a summation of
the contributions of various ion channels. These
include channels with complex nonlinear dynam-
ics described by Hodgkin–Huxley-type equations
that support the generation and propagation of
action potentials [45–47]. For complex neuronal
morphologies and membrane dynamics, Eq. 7.35
can be solved numerically using simulation pack-
ages such as NEURON [36, 45, 48].
7.5 Measuring Responses

to Stimulation
Fig. 7.9 Peak E-field, B-field, and energy to stimulate In most applications, TMS is paired with a quan-
a neuron as a function of the duration of a TMS pulse tification of a physiological, behavioral, cogni-
with a rectangular E-field waveform. The E-field and B- tive, or emotional response. Depending on the
field are proportional to the TMS coil voltage and current,
respectively. The results are from a leaky integrate-and- research question or clinical protocol at hand,
fire neuron model with a 200 μs time constant. The dashed these approaches span visual observation of mus-
vertical line shows the duration of a typical monophasic cle twitches or other behavioral changes; tests
TMS pulse. The E-field is normalized to its rheobase— of memory, task performance, or mood; elec-
the stimulation threshold for an infinitely long pulse. The
B-field and energy (proportional to the square of the B- trophysiological recording of neural or muscle
field) are normalized to those of an arbitrarily brief pulse responses; and imaging of changes in related sig-
nals such as brain blood oxygenation.
7.5.1 Characterization of Behavior,

Cognition, or Emotional State
TMS can cause changes in behavior, cognition,

or emotional state that can be detected by obser-
vation, task performance quantification, or sub-
jective reports. For example, the TMS intensity is
usually normalized to the subject’s (or patient’s)
individual motor threshold, defined as the low-
est pulse amplitude that will consistently evoke
a motor response from a peripheral muscle. A
Fig. 7.10 Typical motor-evoked potentials (MEPs) re-
typical choice for such muscle is either the ab- sulting from TMS of the primary motor cortex and
ductor pollicis brevis (one of the muscles mov- recorded from the abductor pollicis brevis finger muscle
ing the thumb) or the first dorsal interosseous with electromyography (EMG). The TMS pulse was de-
(one of the muscles moving the index finger). livered at time zero (most EMG systems would record
a stimulation artifact when the TMS pulse is delivered;
While maximizing the sensitivity and precision of in this case the amplifier had a sample-and-hold cir-
the motor threshold measurement requires elec- cuit to disable the measurement during the TMS pulse).
tromyography (EMG; see next section), finding EMG has very high signal-to-noise ratio: a typical maxi-
the corresponding cortical representation area of mum MEP amplitude is several millivolts (peak to peak)
compared to a noise level of less than 10 μV. TMS
the desired muscle usually benefits from visual MEPs originate from the cortex and have large ran-
observation of twitching in the target and ad- dom variability, illustrated by the four responses to TMS
jacent muscles when the coil is moved around. pulses of identical strength (120% of the resting motor
Moreover, visual observation of the evoked mus- threshold). In contrast, magnetic stimulation of the pe-
ripheral nerves produces responses with little variation
cle contractions is commonly deployed to deter- in amplitude. Besides peak-to-peak amplitude (ranging
mine the TMS motor threshold in clinical settings from barely measurable to about 10 mV), the MEP on-
[49]. In a more complex example of behavior set latency (usually between 20 and 25 ms for finger
observation, a subject is asked to perform an muscles) is also of interest as it provides information
about the specific activated neural elements in the cor-
object-naming task and the TMS coil positions tex and the neural conduction speed in the corticospinal
that disrupt this ability form a map of speech tract
areas [50]. A similar approach can demarcate
regions related to speech but not singing [51]
EMG is fairly simple to use, has a high signal-to-
or to recognition of objects [52]. The effect of
noise ratio (SNR), and a high temporal resolution
rTMS on working memory can be quantified,
(<1 ms). However, it is limited to studying the
for example, with a mental alphabetization and
cortical areas that impact motor output. EMG
recall task [53]. Finally, standard clinical rat-
is typically used to measure the amplitude
ings can be used to quantify the therapeutic ef-
and latency of the muscle response elicited by
fects of rTMS, for instance, on mood [54] or
TMS, the motor-evoked potential (MEP). Figure
compulsivity [55].
7.10 illustrates that the MEPs due to identical
consecutive TMS stimuli can vary drastically in
their amplitude, latency, and/or waveform. This
7.5.2 Electrophysiological
variability originates from the central nervous
and Imaging Methods
system, as magnetic peripheral nerve stimulation
produces markedly more stable responses.
Electromyography (EMG)
Therefore, to estimate the MEP response for a
The most common quantitative physiological
given TMS pulse amplitude, one must average
measure combined with TMS is EMG. TMS–
the observed amplitudes and latencies of several
MEPs.
Epidural Recordings help with EEG artifacts specific to TMS, such as

Whereas EMG quantifies noninvasively the mus- electromagnetic interference, induced voltages in
cle response to TMS, epidural recordings from the scalp, scalp muscle contractions, and auditory
the spinal cord provide an invasive “upstream” activation by the TMS clicking sound, since they
measurement of the descending volley of neural are synchronous and overlap in time with the
signals that ultimately trigger the muscle. Similar directly evoked brain response [56]. The scalp
to EMG, epidural recordings provide high tem- muscle artifact alone can be on the order of one
poral resolution. The amplitude of a descending millivolt. Fortunately, there are plenty of regions
volley, however, is just a few tens of microvolts, of the brain that can be targeted with TMS with
and consequently the SNR for epidural record- limited scalp-muscle activation [57].
ings is much smaller than that of EMG. Epidural
spinal recordings reveal that TMS behaves much Functional Near-Infrared Spectroscopy
like invasive cortical electrical stimulation. The (fNIRS)
result of primary motor cortex TMS is a volley Functional near-infrared spectroscopy (fNIRS)
of potentials that can contain both direct (D) as allows measuring the hemodynamic responses
well as indirect (I) waves. Whereas D-waves are to TMS with pulsed red and near-infrared
thought to originate from direct activation of the light, similar to pulse oximetry. fNIRS has
pyramidal axon forming the white matter (see limited spatial resolution of a few centimeters,
Fig. 7.8), I-waves are thought to originate from comparable to EEG. Like fMRI (which will
activation of neurons feeding into the output pyra- be discussed next), the temporal resolution of
midal neurons. For TMS, generating D-waves re- fNIRS is limited by the rate of the underlying
quires a considerably higher stimulation intensity hemodynamic signal. Unlike EEG and fMRI,
than I-waves [39]. The epidural recordings allow fNIRS is immune to the electromagnetic artifacts
distinguishing between effects due to central ver- of TMS, although the physiological artifacts of
sus peripheral processing. For example, voluntary scalp activation remain.
muscle contraction has much larger effect on
MEPs than on the descending volleys suggesting Functional Magnetic Resonance Imaging
that muscle activity causes larger changes “down- (fMRI) and Positron Emission Tomography
stream” in spinal excitability than “upstream” in (PET)
cortical excitability [39]. For high spatial resolution, TMS can be combined
with simultaneous functional magnetic resonance
Electroencephalography (EEG) imaging (fMRI) or positron emission tomography
Moving away from motor regions, we can no (PET). Both of these techniques provide informa-
longer observe responses from the periphery tion on changes of the brain metabolic activity,
but must instead measure them directly from which correlates with neural activity. For both
the brain. With currently existing functional modalities, the high spatial resolution renders
imaging methods, this increases the complexity them virtually immune to some of the artifacts
of the measurements considerably. One common in the TMS–EEG signal, like muscle activation.
method is to combine TMS with electroen- PET and fMRI, however, have their own sets
cephalography (EEG). EEG recordings have of limitations. The high spatial resolution comes
a very high temporal resolution (<1 ms) but at the cost of significantly lower temporal res-
somewhat limited spatial resolution (of a few olution, on the order of seconds for fMRI and
centimeters). Unlike EMG, the EEG signals are minutes for PET. In TMS–fMRI, there are a few
tiny, on the order of a few microvolts. In ordinary engineering challenges related to mitigating the
EEG, this issue can be mitigated by averaging effects of electromagnetic noise, stronger elec-
multiple trials. In contrast, averaging does not tromechanicalforces, and louder sound emission
of the TMS coil in the scanner. Finally, PET age, an MEP of a specific amplitude, commonly
requires the injection of radioactive tracers. 50 μV peak to peak. If the targeted muscle is at
rest during the measurement, the result is called
resting motor threshold, whereas if the muscle is
7.6 Stimulation Paradigms partially contracted, the result is referred to as ac-
and Applications tive motor threshold. The active motor threshold
is lower than the resting motor threshold since the
7.6.1 Single Pulses underlying motor circuits are pre-activated. The
most common TMS dosing strategy, not only for
A sufficiently strong single TMS pulse can evoke motor cortex but for other cortical targets as well,
action potentials in a population of neurons. Even is to set the TMS pulse amplitude relative to the
for very focal TMS, the relevant population of resting or active motor threshold of the individ-
neurons spans about 1 square centimeter of cortex ual subject. This adjustment of the stimulation
and contains on the order of ten million individual intensity can account, in part, for both individual
neurons. The neuronal response depends on the anatomical features such as scalp-to-cortex dis-
amplitude and waveform of the induced E-field tance as well as for individual differences in the
pulse, as well as on the endogenous excitability physiological excitability of the cortex. However,
state of the neuron and the circuits it is embedded since these factors can vary across the cortical
into. The most common TMS activation measure sites, motor-threshold-based dosing is likely sub-
is the amplitude of a hand-muscle MEP evoked optimal outside motor cortex.
by stimulation of the contralateral primary motor
cortex [58]. The MEP is often characterized as a 7.6.2 Paired Pulses
function of the stimulus location and/or intensity,
comprising input–output (or recruitment) curves. Paired-pulse paradigms involve the delivery
The MEP amplitude is also used to assess cortical of two TMS pulses that are closely spaced in
excitability changes, for example, as a result of time, with interstimulus time ranging from 1 to
a neuromodulation intervention, such as rTMS, 250 ms [58]. This method allows characterization
or another manipulation of the brain state. For of connectivity within and between cortical
instance, to quantify the change of excitability regions. For example, a subthreshold stimulus
resulting from an rTMS protocol, the MEP am- can decrease the amplitude of an MEP generated
plitude resulting from a series of single TMS by a test pulse delivered to motor cortex through
pulses is first measured, then the rTMS protocol is the same coil 1–5 ms later. This paradigm is
applied, followed by remeasurement of the MEP known as short-interval intracortical inhibition,
amplitude for single TMS pulses. Importantly, and it is likely mediated by local inhibitory
the MEP amplitude fluctuates significantly even interneurons. If the test pulse is delivered 7–20 ms
when the same TMS pulse amplitude is applied, after the first pulse, then the MEP amplitude
due to variations of the endogenous excitability is increased—intracortical facilitation. These
state of the motor cortex (see Fig. 7.10). For two short-interval effects are illustrated in Fig.
this reason, a sequence of several TMS pulses 7.11. If two suprathreshold pulses are delivered
has to be acquired with inter-pulse interval of 100–150 ms apart, then a strong inhibition of
about 10 s, and the resultant MEP amplitudes the second (test) MEP is observed. This effect
have to be averaged to obtain a stable estimate. is called long-interval intracortical inhibition,
The reason to wait for 10 s between pulses is and it results likely from activating additional
that each pulse modulates the cortical excitability cortical inhibitory mechanisms compared to
for several seconds, which would confound the short-interval intracortical inhibition. Applying
measurement from the subsequent pulse [59]. two suprathreshold pulses with interstimulus
Another common measure is the motor thresh- interval in the range of 1–5 ms reveals on
old. The motor threshold is typically defined as oscillatory pattern of intracortical facilitation.
the TMS pulse amplitude that elicits, on aver- Delivering the two pulses to two separate coils
rTMS pulse amplitudes are usually in the vicinity

of the motor threshold, typically ranging from
80% of the active motor threshold to 120% of the
resting motor threshold. Safety guidelines have
been developed to ensure that the pulse train
parameters used in rTMS protocols do not induce
a seizure [61].
7.6.4 Clinical Applications
The clinical applications of TMS are either di-

agnostic or therapeutic. One diagnostic use is to
assess noninvasively the conduction of signals
through the corticospinal tract. The simplest form
of this test is to apply single TMS pulses to the
Fig. 7.11 Short-interval intracortical inhibition and facil- primary motor cortex and use EMG to measure
itation. Left: TMS stimulation waveforms, right: the sizes the threshold, amplitude, duration, and latency
of typical resulting MEPs. Top two rows show, respec- of MEPs in various muscles. For example, the
tively, a subthreshold TMS pulse, which alone would elicit
no MEP, and a suprathreshold pulse. If these two pulses are
MEP threshold is higher, latency is larger, and
delivered consecutively, separated by 1–5 ms, the resulting duration is prolonged in individuals with multiple
MEP will be decreased in amplitude (third row). If the two sclerosis [62]. Remarkably, MEPs with normal
pulses are separated by 7–20 ms, the resulting MEP will latency have been detected in paralyzed forearm
be enhanced (bottom row)
muscles of individuals with spinal cord injury,
suggesting rehabilitation opportunities [63]. An-
positions over different brain regions, paired- other diagnostic application uses single pulses or
pulse approaches can be extended to investigate short-pulse trains to map cortical function. For ex-
the connectivity between the two brain regions. ample, by mapping the brain regions where TMS
activates muscles or the regions where speech
is disrupted, these motor and language areas of
7.6.3 Pulse Trains cortex can be spared during resection surgery for
intractable epilepsy or tumor removal [64].
To produce lasting neuromodulation effects, TMS Therapeutic applications are usually pred-
has to be applied in trains of pulses referred to icated on the lasting neuromodulatory effect
as repetitive TMS (rTMS). Low-frequency (1 Hz) of rTMS. rTMS is currently cleared by the
rTMS trains tend to produce lasting inhibitory US Food and Drug Administration (FDA)
effects, whereas higher frequencies (5–20 Hz) are for the treatment of depression and obsessive
excitatory. Patterning the pulse train with more compulsive disorder [55]. For depression, there
than one frequency can produce powerful in- are three FDA-cleared protocols based on 10 Hz,
hibitory or excitatory effects. For example, theta 18 Hz, or theta burst (50 Hz bursts repeating
burst stimulation is comprised of bursts of three at 5 Hz) pulse trains, all targeting the left
pulses at 50 Hz, repeated at 5 Hz. A short train prefrontal cortex. The pivotal clinical trials of
(40 s, 600 pulses) of continuous TBS produces these protocols demonstrated response rates
inhibition lasting for nearly an hour, whereas of 15%–49% (patients getting significantly
turning the TBS train on for 2 s and off for 8 s better) and remission rates of 14%–33% (patients
for the same total number of pulses produces considered recovered from depression) [54, 65–
excitatory effects lasting about 20 min [60]. The 67]. An example of a depression treatment rTMS
paradigm is shown in Fig. 7.12. There is research
tion of cortical processing resulting in “virtual

lesions.” By disabling specific nodes in cortical
circuits, this approach can provide causal infor-
mation about the brain function, as opposed to
the correlational information provided by imag-
ing. Curiously, if TMS is applied with the right
coil location, stimulus parameters, and timing, it
can also enhance the subject’s performance on
tasks involving perceptual, motor, and executive
processing [69].
TMS can be combined with imaging modal-
ities including EEG, fMRI, PET, and fNIRS, as
discussed in the previous section. Both sequential
(offline) and simultaneous (online) combinations
of TMS with imaging provide information about
the effects of TMS on the brain. Brain connectiv-
Fig. 7.12 rTMS pulse train parameters of a depression ity inferred from imaging can be used for effective
treatment protocol [49]. The pulse train is applied with a targeting of deeper brain structures that are not
focal figure-8 coil daily for 5 days per week, and a typical directly accessible to the TMS-induced E-field.
treatment lasts 4 to 6 weeks. In the top row, a single TMS
pulse at 120% of resting motor threshold is delivered to This approach leverages the fact that the brain is
left dorsolateral prefrontal cortex. The second row shows interconnected, so activating a particular superfi-
a train of 40 such pulses given at interstimulus intervals cial cortical region with TMS sends action poten-
of 0.1 s (i.e., at 10 Hz). On the third row, the 4 s train is tials through synapses to other brain regions.
repeated every 30 s (i.e., there is an inter-train interval of
26 s). The daily treatment consists of 75 such trains for a
total duration of 37.5 min and a total of 3000 pulses per day
(bottom row). The treatment is thought to increase neural 7.7 Conclusions
excitability of the targeted brain region
TMS uses electromagnetic induction to bypass
on the therapeutic application of rTMS in many the poorly conductive skull, allowing noninva-
other neurologic and psychiatric disorders, sive, focal, and tolerable activation of superficial
including pain, schizophrenia, substance use brain structures. TMS is a versatile brain stimula-
disorders, epilepsy, and tinnitus. Interestingly, tion method with applications ranging from basic
single-pulse TMS can also have therapeutic brain research to characterization and treatment
effects: single-pulse TMS targeted to the occipital of neurological and psychiatric disorders.
cortex is FDA-cleared for mitigating migraine This chapter introduced the basic mechanisms
episodes [68]. and applications of TMS, from both engineering
and biological points of view. An engineer has to
understand both to develop new uses for existing
7.6.5 Research Applications TMS devices or new devices with enhanced ca-
pabilities.
There is a wide and expanding range of research
applications of TMS in disciplines spanning psy-
chology, cognitive neuroscience, psychiatry, and Homework
neurology. Single pulses are commonly used to
test neural excitability and circuit connectivity. Some of the problems require the use of physics
Single pulses or short bursts with specific timing equations, parameters values, and computer pro-
relative to a task can produce temporary disrup- grams that are not covered in this chapter. There-
fore, like a real-world engineer, the student may
need to consult other resources. Problems 3, 5, coil voltage is 800 V. The coil has an induc-
and 7 involve integrals that can be evaluated nu- tance of 16 μH, the stimulator has an energy
merically. These problems can be solved analyti- storage capacitance of 185 μF, and the total
cally as well for an extra challenge and deeper un- series resistance of the pulse generator and
derstanding of the underlying scaling laws. Prob- coil is 50 m . You can assume that the high-
lems with increased difficulty are denoted by an voltage power supply used to recharge the
asterisk (∗ ). capacitor is 80% efficient and that it can be
operated at all times (even during the pulses).
1. The peak magnetic field of a TMS pulse is Hint: Evaluate the Wloss integral of Eq. 7.7.
about 1 tesla. Verify this claim by computing 4. A person has a deep brain stimulator (DBS)
the maximum magnetic field of a circular to control the motor symptoms of Parkin-
TMS coil. son’s Disease. The DBS electrode in the sub-
(a) The coil has 13 turns, with a mean di- thalamic nucleus and the implanted pulse
ameter of 90 mm, and is driven with a generator (IPG) in the chest are connected
peak coil current of 5000 A. You may with a lead that is coiled between the scalp
further assume that the peak B-field is in and the skull, forming three loops of 5 cm di-
the center of the coil. ameter. The impedance through the person’s
(b) (∗ ) Each turn in the coil windings is body between the DBS electrode contact and
7-mm-tall and 2-mm-wide (i.e., inner the IPG can be approximated as a 1 k resis-
winding diameter is 64, and outer tor. Other impedances in the DBS circuit are
winding diameter 116 mm). The current negligible, unless otherwise indicated. The
density in the wire is uniform due to person needs to receive rTMS treatment for
the use of litz wire. The windings are depression. For the coil placement used for
surrounded from all sides by 3 mm this treatment and at maximum device out-
of nonmagnetic plastic. Compute the put, each DBS lead loop encircles a uniform
maximum B-field at the surface of the magnetic flux density of 0.5 T. The mag-
coil. Were the approximations made in netic pulse is sine shaped with a period of
part (a) reasonable? 300 μs.
2. Electrical safety implications of TMS: (a) Assume that during the TMS procedure,
(a) A monophasic TMS device has a maxi- the IPG is turned off but can still conduct
mum capacitor voltage of 2800 V and a current. Calculate the current induced by
185 μF capacitor. Compute the amount TMS through the electrode contacts at
of energy stored in such a system. Com- maximum device output. How does this
pare this energy to (1) the battery in your compare to the typical DBS electrode
smartphone and (2) the energy stored in current of 1 mA.
a men’s Olympic javelin (mass of 800 g) (b) Repeat part (a) under the assumption that
thrown at 100 km/h. What implications the IPG does not conduct any current
does such energy storage system have for until the voltage across it reaches 5 V.
safety? (c) What can the neurosurgeon implanting
(b) Compute the dielectric breakdown dis- the DBS system do to reduce the current
tance for such a voltage (in air). Consider induced by the TMS pulse in the DBS
the high-voltage breakdown of typical electrode?
insulation materials (e.g., polyethylene 5. Reusable solid silver EEG cup electrodes are
film); does a 0.18-mm-thick polyethy- considered for a TMS–EEG study. Each elec-
lene electrical tape provide adequate in- trode is approximately a disk of 10 mm diam-
sulation at TMS voltages? eter and 0.5 mm thickness. In the center of the
3. Compute the required power level to sustain disk, there is a circular hole of 2 mm diame-
biphasic rTMS at 10 Hz when the required ter. (In reality, such cup electrodes are dome-
shaped with a height of about 3 mm, but for decides to reduce the capacitance to
this problem such details can be omitted.) 100 μF. How should the capacitor volt-
The TMS protocol delivers a peak magnetic age be changed to maintain the original
field of 0.5 T perpendicular to the electrode. range of stimulation strength relative to
The magnetic pulse is sine shaped with a the neural activation threshold? What
period of 300 μs. These pulses are delivered would be the relative energy savings
at 1 Hz for a total of 1000 pulses. resulting from this design change?
(a) Calculate the worst case peak current How does this change the resistive
density induced in the EEG electrodes. losses in the coil, ignoring eddy current
(b) (∗ ) Calculate the corresponding average effects? Compute also the numbers for
power dissipation in the EEG electrode an idealized coil with zero resistance
during the TMS pulse train. (similar to part (a)), and compare the
(c) (∗ ) Calculate the increase in temperature results. Where does the additional
of the EEG electrode by the end of the efficiency come from, and what other
TMS pulse train. changes would you suggest as a designer
(d) Would the electrode temperature exceed for the next coil?
41 ◦ C which is considered the safety 8. (∗ ) The cable equation (Eq. 7.30) indicates
limit? that, for straight nerve fibers, the site of max-
(e) Suggest ways to mitigate the electrode imum membrane depolarization is where the
heating. E-field gradient, rather than the E-field mag-
6. The circular coil of Problem 1 is placed inside nitude, is maximum. This is relevant for mag-
a 3 T MRI device. Compute the worst-case netic stimulation of long straight nerves in
torque that the coil undergoes during a TMS the periphery but not for TMS. To see why
pulse. Hint: The TMS coil orientation in the this is the case, compare (1) the peak E-
MRI magnet affects the torque. field gradient in the cortex and (2) the peak
7. A TMS device designer aims to increase the effective E-field gradient along an axon due
device efficiency. The existing device uses a to a rounded bend of the axon. Hints: In Fig.
185 μF energy storage capacitor with a peak 7.5, the E-field drops to 70% in 1.5–2.5 cm
voltage of 1600 V and a 16 μH coil with 18 depending on direction. In Fig. 7.8, the bend
turns. The neural membrane time constant is in axon must fit inside a gyrus that is about
assumed to be 200 μs. 1 cm wide.
(a) The designer evaluates an alternative ap- 9. The action potential conduction velocity
proach in which the inductance of the of the myelinated nerve fibers in the
coil is reduced to about 10 μH, while corticospinal tract is approximately 7 cm/ms.
the shape and size of the coil are pre- Considering the latency between the time
served. What is the new number of turns a TMS pulse is applied to the primary
in the coil? Assuming the same capacitor, motor cortex and an MEP is detected in
how should its voltage be changed to a finger muscle, what difference do you
maintain the original range of stimula- expect between a subject who is 190 cm
tion strength relative to the neural acti- tall compared to one who is 155 cm tall?
vation threshold? What would be the rel- 10. A researcher wants to optimize the depres-
ative energy savings resulting from this sion treatment protocol illustrated in Fig.
design change? For this part you can ig- 7.12. Considering the TMS safety guidelines
nore the resistance of the pulse generator for a Class 2 study [61], how much should
and coil and assume a purely sinusoidal the following rTMS pulse train parameters
pulse waveform. be decreased from their default values (given
(b) (∗ ) Keeping the original 16 μH coil that in parentheses) so that the stimulation is still
has resistance of 50 m , the designer considered safe?
(a) Duration of each short train (4 s) when and implementation. Brain Stimul. 11, 849–855
increasing intensity from 120% of resting (2018). https://doi.org/10.1016/j.brs.2018.03.014
11. Z.-D. Deng, S.H. Lisanby, A.V. Peterchev, Electric
motor threshold (RMT) to 130% RMT.
field depth-focality tradeoff in transcranial magnetic
(b) Number of pulses per short trains (40 stimulation: Simulation comparison of 50 coil de-
pulses) when increasing pulse repetition signs. Brain Stimul. 6, 1–13 (2013). https://doi.org/
rate from 10 Hz to 20 Hz. 10.1016/j.brs.2012.02.005
12. L.J. Gomez, S.M. Goetz, A.V. Peterchev, Design of
(c) Intensity and total number of trains
transcranial magnetic stimulation coils with optimal
(120% RMT and 75 trains) when trade-off between depth, focality, and energy. J. Neu-
decreasing the interval between short ral Eng. 15, 046033 (2018). https://doi.org/10.1088/
trains from 26 s to 5 s. 1741-2552/aac967
13. J. Ruohonen, R.J. Ilmoniemi, Physical principles
for transcranial magnetic stimulation, in Handbook
of Transcranial Magnetic Stimulation, ed. by A.
References Pascual-Leone, N. J. Davey, J. C. Rothwell, et al.,
(Arnold, London, 2002), pp. 18–30
1. A. d’Arsonval, Dispositifs pour la mesure des 14. R. Plonsey, D.B. Heppner, Considerations of quasi-
courants alternatifs de toutes fréquences. Comput stationarity in electrophysiological systems. Bull.
Rend Soc Biol 3, 450–457 (1896) Math. Biophys. 29, 657–664 (1967). https://doi.org/
2. A.T. Barker, R. Jalinous, I.L. Freeston, Non-invasive 10.1007/bf02476917
magnetic stimulation of human motor cortex. Lancet 15. J.O. Nieminen, L.M. Koponen, R.J. Ilmoniemi,
1, 1106–1107 (1985). https://doi.org/10.1016/s0140- Experimental characterization of the electric
6736(85)92413-4 field distribution induced by TMS devices. Brain
3. S. Ueno, T. Tashiro, K. Harada, Localized stimulation Stimul. 8, 582–589 (2015). https://doi.org/10.1016/
of neural tissues in the brain by means of a paired con- j.brs.2015.01.004
figuration of time-varying magnetic fields. J. Appl. 16. J.D. Jackson, Classical Electrodynamics, 3rd edn.
Phys. 64, 5862–5864 (1988). https://doi.org/10.1063/ (Wiley, New York, 1999)
1.342181 17. H. Eaton, Electric field induced in a spherical volume
4. A.V. Peterchev, D.L. Murphy, S.H. Lisanby, conductor from arbitrary coils: Application to mag-
Repetitive transcranial magnetic stimulator with netic stimulation and MEG. Med. Biol. Eng. Com-
controllable pulse parameters. J. Neural Eng. 8, put. 30, 433–440 (1992). https://doi.org/10.1007/
036016 (2011). https://doi.org/10.1088/1741-2560/ bf02446182
8/3/036016 18. J. Sarvas, Basic mathematical and electromagnetic
5. S.M. Goetz, C.N. Truong, M.G. Gerhofer, et al., Anal- concepts of the biomagnetic inverse problem. Phys.
ysis and optimization of pulse dynamics for magnetic Med. Biol. 32, 11–22 (1987). https://doi.org/10.1088/
stimulation. PLoS One 8, e55771 (2013). https:// 0031-9155/32/1/004
doi.org/10.1371/journal.pone.0055771 19. P. Ravazzani, J. Ruohonen, F. Grandori, G. Tognola,
6. Y. Terao, Y. Ugawa, R. Hanajima, et al., Predom- Magnetic stimulation of the nervous system: Induced
inant activation of I1-waves from the leg motor electric field in unbounded, semi-infinite, spherical,
area by transcranial magnetic stimulation. Brain Res. and cylindrical media. Ann. Biomed. Eng. 24, 606–
859, 137–146 (2000). https://doi.org/10.1016/s0006- 616 (1996). https://doi.org/10.1007/bf02684229
8993(00)01975-2 20. L.J. Gomez, M. Dannhauer, L.M. Koponen, A.V.
7. Z.-D. Deng, S.H. Lisanby, A.V. Peterchev, Coil de- Peterchev, Conditions for numerically accurate TMS
sign considerations for deep transcranial magnetic electric field simulation. Brain Stimul. 13, 157–166
stimulation. Clin. Neurophysiol. 125, 1202–1212 (2019). https://doi.org/10.1016/j.brs.2019.09.015
(2014). https://doi.org/10.1016/j.clinph.2013.11.038 21. G.B. Saturnino, A. Thielscher, K.H. Madsen, et
8. C.M. Epstein, K.R. Davey, Iron-core coils for tran- al., A principled approach to conductivity uncer-
scranial magnetic stimulation. J. Clin. Neurophys- tainty analysis in electric field calculations. NeuroIm-
iol. 19, 376–381 (2002). https://doi.org/10.1097/ age 188, 821–834 (2019). https://doi.org/10.1016/
00004691-200208000-00010 j.neuroimage.2018.12.053
9. L.M. Koponen, J.O. Nieminen, T.P. Mutanen, et 22. M. Windhoff, A. Opitz, A. Thielscher, Electric field
al., Coil optimisation for transcranial magnetic calculations in brain stimulation based on finite ele-
stimulation in realistic head geometry. Brain ments: An optimized processing pipeline for the gen-
Stimul. 10, 795–805 (2017). https://doi.org/10.1016/ eration and usage of accurate individual head mod-
j.brs.2017.04.001 els. Hum. Brain Mapp. 34, 923–935 (2013). https://
10. L.M. Koponen, J.O. Nieminen, R.J. Ilmoniemi, doi.org/10.1002/hbm.21479
Multi-locus transcranial magnetic stimulation-theory 23. M. De Lucia, G.J.M. Parker, K. Embleton, et
al., Diffusion tensor MRI-based estimation of the
influence of brain tissue anisotropy on the ef- 36. A.S. Aberra, B. Wang, W.M. Grill, A.V. Peterchev,
fects of transcranial magnetic stimulation. NeuroIm- Simulation of transcranial magnetic stimulation in
age 36, 1159–1170 (2007). https://doi.org/10.1016/ head model with morphologically-realistic cortical
j.neuroimage.2007.03.062 neurons. Brain Stimul. 13, 175–189 (2019). https://
24. L. Heller, D.B. van Hulsteyn, Brain stimulation using doi.org/10.1016/j.brs.2019.10.002
electromagnetic sources: Theoretical aspects. Bio- 37. K.R. Mills, S.J. Boniface, M. Schubert, Magnetic
phys. J. 63, 129–138 (1992). https://doi.org/10.1016/ brain stimulation with a double coil: The importance
S0006-3495(92)81587-4 of coil orientation. Electroencephalogr. Clin. Neu-
25. A. Nummenmaa, M. Stenroos, R.J. Ilmoniemi, et rophysiol. 85, 17–21 (1992). https://doi.org/10.1016/
al., Comparison of spherical and realistically shaped 0168-5597(92)90096-t
boundary element head models for transcranial 38. U. Ziemann, J.C. Rothwell, M.C. Ridding,
magnetic stimulation navigation. Clin. Neurophys- Interaction between intracortical inhibition and
iol. 124, 1995–2007 (2013). https://doi.org/10.1016/ facilitation in human motor cortex. J. Physiol.
j.clinph.2013.04.019 Lond. 496, 873–881 (1996). https://doi.org/10.1113/
26. J. Ruohonen, J. Karhu, Navigated transcranial mag- jphysiol.1996.sp021734
netic stimulation. Neurophysiol. Clin. 40, 7–17 39. V. Di Lazzaro, D. Restuccia, A. Oliviero, et al., Ef-
(2010). https://doi.org/10.1016/j.neucli.2010.01.006 fects of voluntary contraction on descending volleys
27. M. Stenroos, L.M. Koponen, Real-time computa- evoked by transcranial stimulation in conscious hu-
tion of the TMS-induced electric field in a realistic mans. J. Physiol. Lond. 508(Pt 2), 625–633 (1998).
head model. NeuroImage 203, 116159 (2019). https:/ https://doi.org/10.1111/j.1469-7793.1998.625bq.x
/doi.org/10.1016/j.neuroimage.2019.116159 40. L. Lapicque, Recherches quantitatives sur l’excitation
28. I. Alekseichuk, K. Mantell, S. Shirinpour, A. Opitz, electrique des nerfs traitee comme une polarization.
Comparative modeling of transcranial magnetic and Journal de Physiologie et de Pathologie Generalej 9,
electric stimulation in mouse, monkey, and human. 620–635 (1907)
NeuroImage 194, 136–148 (2019). https://doi.org/ 41. A.V. Peterchev, S.M. Goetz, G.G. Westin, et al.,
10.1016/j.neuroimage.2019.03.044 Pulse width dependence of motor threshold and input-
29. B. Katz, Electric Excitation of Nerve (Oxford Univer- output curve characterized with controllable pulse pa-
sity Press, Oxford, 1939) rameter transcranial magnetic stimulation. Clin. Neu-
30. R.J. Ilmoniemi, J. Ruohonen, J. Karhu, Transcranial rophysiol. 124, 1364–1372 (2013). https://doi.org/
magnetic stimulation: A new tool for functional imag- 10.1016/j.clinph.2013.01.011
ing of the brain. Crit. Rev. Biomed. Eng. 27, 241–284 42. E. Corthout, A.T. Barker, A. Cowey, Transcra-
(1999) nial magnetic stimulation. Which part of the cur-
31. P.J. Maccabee, V.E. Amassian, L.P. Eberle, R.Q. rent waveform causes the stimulation? Exp. Brain
Cracco, Magnetic coil stimulation of straight and Res. 141, 128–132 (2001). https://doi.org/10.1007/
bent amphibian and mammalian peripheral nerve s002210100860
in vitro: Locus of excitation. J. Physiol. Lond. 43. S.S. Nagarajan, D.M. Durand, E.N. Warman, Effects
460, 201–219 (1993). https://doi.org/10.1113/ of induced electric fields on finite neuronal struc-
jphysiol.1993.sp019467 tures: A simulation study. I.E.E.E. Trans. Biomed.
32. A. Bungert, A. Antunes, S. Espenhahn, A. Thielscher, Eng. 40, 1175–1188 (1993). https://doi.org/10.1109/
Where does TMS stimulate the motor cortex? Com- 10.245636
bining electrophysiological measurements and real- 44. N. De Geeter, L. Dupré, G. Crevecoeur, Model-
istic field estimates to reveal the affected cortex po- ing transcranial magnetic stimulation from the in-
sition. Cereb. Cortex 27, 5083–5094 (2017). https:// duced electric fields to the membrane potentials along
doi.org/10.1093/cercor/bhw292 tractography-based white matter fiber tracts. J. Neu-
33. I. Laakso, T. Murakami, A. Hirata, Y. Ugawa, Where ral Eng. 13, 026028 (2016). https://doi.org/10.1088/
and what TMS activates: Experiments and modeling. 1741-2560/13/2/026028
Brain Stimul. 11, 166–174 (2018). https://doi.org/ 45. A.S. Aberra, A.V. Peterchev, W.M. Grill, Biophysi-
10.1016/j.brs.2017.09.011 cally realistic neuron models for simulation of cortical
34. K. Weise, O. Numssen, A. Thielscher, et al., A stimulation. J. Neural Eng. 15, 066023 (2018). https:/
novel approach to localize cortical TMS effects. Neu- /doi.org/10.1088/1741-2552/aadbb1
roImage 209, 116486 (2019). https://doi.org/10.1016/ 46. A.L. Hodgkin, A.F. Huxley, A quantitative de-
j.neuroimage.2019.116486 scription of membrane current and its application
35. V.E. Amassian, L. Eberle, P.J. Maccabee, R.Q. to conduction and excitation in nerve. J. Physiol.
Cracco, Modelling magnetic coil excitation of human Lond. 117, 500–544 (1952). https://doi.org/10.1113/
cerebral cortex with a peripheral nerve immersed in a jphysiol.1952.sp004764
brain-shaped volume conductor: The significance of 47. W.A. Wesselink, J. Holsheimer, H.B. Boom, A
fiber bending in excitation. Electroencephalogr. Clin. model of the electrical behaviour of myelinated sen-
Neurophysiol. 85, 291–301 (1992). https://doi.org/ sory nerve fibres based on human data. Med. Biol.
10.1016/0168-5597(92)90105-k Eng. Comput. 37, 228–235 (1999). https://doi.org/
10.1007/bf02513291
48. B. Wang, W.M. Grill, A.V. Peterchev, Coupling 59. C. Möller, N. Arai, J. Lücke, U. Ziemann, Hysteresis
magnetically induced electric fields to neurons: effects on the input-output curve of motor evoked po-
Longitudinal and transverse activation. Biophys. tentials. Clin. Neurophysiol. 120, 1003–1008 (2009).
J. 115, 95–107 (2018). https://doi.org/10.1016/ https://doi.org/10.1016/j.clinph.2009.03.001
j.bpj.2018.06.004 60. Y.-Z. Huang, M.J. Edwards, E. Rounis, et al.,
49. J.C. Horvath, J. Mathews, M.A. Demitrack, A. Theta burst stimulation of the human motor cortex.
Pascual-Leone, The NeuroStar TMS device: Con- Neuron 45, 201–206 (2005). https://doi.org/10.1016/
ducting the FDA approved protocol for treatment of j.neuron.2004.12.033
depression. J. Vis. Exp. 45, e2345 (2010). https:// 61. S. Rossi, M. Hallett, P.M. Rossini, et al., Safety,
doi.org/10.3791/2345 ethical considerations, and application guidelines
50. A. Pascual-Leone, J.R. Gates, A. Dhuna, Induction of for the use of transcranial magnetic stimulation
speech arrest and counting errors with rapid-rate tran- in clinical practice and research. Clin. Neurophys-
scranial magnetic stimulation. Neurology 41, 697– iol. 120, 2008–2039 (2009). https://doi.org/10.1016/
702 (1991). https://doi.org/10.1212/wnl.41.5.697 j.clinph.2009.08.016
51. L. Stewart, V. Walsh, U. Frith, J. Rothwell, Tran- 62. J.L. Neva, B. Lakhani, K.E. Brown, et al., Multi-
scranial magnetic stimulation produces speech ar- ple measures of corticospinal excitability are associ-
rest but not song arrest. Ann. N. Y. Acad. Sci. ated with clinical features of multiple sclerosis. Be-
930, 433–435 (2001). https://doi.org/10.1111/j.1749- hav. Brain Res. 297, 187–195 (2016). https://doi.org/
6632.2001.tb05762.x 10.1016/j.bbr.2015.10.015
52. P. Lioumis, A. Zhdanov, N. Mäkelä, et al., A novel 63. M. Cortes, G.W. Thickbroom, J. Elder, et al., The cor-
approach for documenting naming errors induced by ticomotor projection to liminally-contractable fore-
navigated transcranial magnetic stimulation. J. Neu- arm muscles in chronic spinal cord injury: A transcra-
rosci. Methods 204, 349–354 (2012). https://doi.org/ nial magnetic stimulation study. Spinal Cord 55, 362–
10.1016/j.jneumeth.2011.11.003 366 (2017). https://doi.org/10.1038/sc.2016.161
53. L. Beynel, S.W. Davis, C.A. Crowell, et al., On- 64. J.-P. Lefaucheur, T. Picht, The value of preoper-
line repetitive transcranial magnetic stimulation dur- ative functional cortical mapping using navigated
ing working memory in younger and older adults: TMS. Neurophysiol. Clin. 46, 125–133 (2016). https:/
A randomized within-subject comparison. PLoS /doi.org/10.1016/j.neucli.2016.05.001
One 14, e0213707 (2019). https://doi.org/10.1371/ 65. D.M. Blumberger, F. Vila-Rodriguez, K.E. Thorpe, et
journal.pone.0213707 al., Effectiveness of theta burst versus high-frequency
54. M.S. George, S.H. Lisanby, D. Avery, et al., repetitive transcranial magnetic stimulation in pa-
Daily left prefrontal transcranial magnetic stimu- tients with depression (THREE-D): A randomised
lation therapy for major depressive disorder: A non-inferiority trial. Lancet 391, 1683–1692 (2018).
sham-controlled randomized trial. Arch. Gen. Psy- https://doi.org/10.1016/S0140-6736(18)30295-2
chiatry 67, 507–516 (2010). https://doi.org/10.1001/ 66. Y. Levkovitz, M. Isserles, F. Padberg, et al., Efficacy
archgenpsychiatry.2010.46 and safety of deep transcranial magnetic stimulation
55. L. Carmi, A. Tendler, A. Bystritsky, et al., for major depression: A prospective multicenter ran-
Efficacy and safety of deep transcranial magnetic domized controlled trial. World Psychiatry 14, 64–73
stimulation for obsessive-compulsive disorder: (2015). https://doi.org/10.1002/wps.20199
A prospective multicenter randomized double- 67. J.P. O’Reardon, H.B. Solvason, P.G. Janicak, et al.,
blind placebo-controlled trial. Am. J. Psychiatry Efficacy and safety of transcranial magnetic stimu-
176, 931–938 (2019). https://doi.org/10.1176/ lation in the acute treatment of major depression: A
appi.ajp.2019.18101180 multisite randomized controlled trial. Biol. Psychi-
56. V. Nikouline, J. Ruohonen, R.J. Ilmoniemi, The role atry 62, 1208–1216 (2007). https://doi.org/10.1016/
of the coil click in TMS assessed with simultaneous j.biopsych.2007.01.018
EEG. Clin. Neurophysiol. 110, 1325–1328 (1999). 68. R.B. Lipton, D.W. Dodick, S.D. Silberstein, et al.,
https://doi.org/10.1016/s1388-2457(99)00070-x Single-pulse transcranial magnetic stimulation for
57. T. Mutanen, H. Mäki, R.J. Ilmoniemi, The effect of acute treatment of migraine with aura: A randomised,
stimulus parameters on TMS-EEG muscle artifacts. double-blind, parallel-group, sham-controlled trial.
Brain Stimul. 6, 371–376 (2013). https://doi.org/ Lancet Neurol. 9, 373–380 (2010). https://doi.org/
10.1016/j.brs.2012.07.005 10.1016/S1474-4422(10)70054-5
58. P.M. Rossini, D. Burke, R. Chen, et al., Non-invasive 69. B. Luber, S.H. Lisanby, Enhancement of human
electrical and magnetic stimulation of the brain, cognitive performance using transcranial magnetic
spinal cord, roots and peripheral nerves: Basic princi- stimulation (TMS). NeuroImage 85, 961–970 (2014).
ples and procedures for routine clinical and research https://doi.org/10.1016/j.neuroimage.2013.06.007
application. An updated report from an I.F.C.N. Com-
mittee. Clin. Neurophysiol. 126, 1071–1107 (2015).
https://doi.org/10.1016/j.clinph.2015.02.001
Transcranial Electrical Stimulation
8
Dennis Q. Truong, Niranjan Khadka, and Marom Bikson
Abstract dose can be applied. Stimulation impedance

measurements monitor contact quality, while
Transcranial electrical stimulation (tES)
current control is typically used to ensure
includes a range of devices where electric
consistent current delivery despite electrode
current is applied to electrodes on the
impedance unknowns. Computational current
head to modulate brain function. Various
flow models support device design and
tES devices are applied to indications
programming by informing dose selection
spanning neurological and psychiatric
for a given outcome. Consensus on the safety
disorders, neuro-rehabilitation after injury,
and tolerability of tES is protocol-specific, but
and altering cognition in healthy adults. All
medical-grade tES devices minimize risk.
tES devices share certain common features
including a waveform generator (typically
Keywords
current controlled), disposable electrodes or
electrolyte, and an adhesive or headgear to Transcranial · Electrical · Stimulation · tES ·
position the electrodes. tES “dose” is defined tDCS · tACS · tPCS · Neuromodulation ·
by the size and position of electrodes and Electrode design · Noninvasive · Medical
the waveform (current pattern, duration, and devices
intensity). Many subclasses of tES are named
based on dose. This chapter is largely focused
on low-intensity (few mA) tES. Low-intensity
tES includes transcranial direct-current 8.1 Basics of tES Devices
stimulation (tDCS), transcranial alternating- and Dose
current stimulation (tACS), and transcranial
pulsed-current stimulation (tPCS). Electrode tES dose is defined as the current waveform ap-
design is important for reproducibility, plied to the body and the number, shape, and
tolerability, and influences when and what location of electrodes placed on the scalp. The
electrodes guide the waveform into the head and
serve as the interface between the device and the
Electronic Supplementary Material: The online version body. A tES device should be designed to reliably
of this chapter (https://doi.org/10.1007/978-3-030-43395- deliver the target dose, including any operator
controls, safety features, and instructions for use.
The electrode number, shape, and location are
D. Q. Truong · N. Khadka · M. Bikson () collectively the montage. There are minimum of
Department of Biomedical Engineering, The City College
two electrodes. The waveform is produced by a
of New York of CUNY, New York, NY, USA
e-mail: [email protected] powered device that can be directly attached to

272 D. Q. Truong et al.
Fig. 8.1 Example of a tES device and material used for chemical electrodes) are placed inside the sponge layers.
electrical stimulation with sponge electrodes. Shown are Lead wires connect the device to the conductive rubber
conventional sponges (yellow) soaked with a controlled electrodes. Sponge electrodes are then secured on the
volume of saline using a syringe. Each electrode is made scalp using a headgear. For the case of tDCS, the rubber
of two layers of sponge. Conductive rubbers (electro- electrodes are energized using corresponding anode and
cathode wires connected to the stimulator
the electrodes using connector leads (Fig. 8.1). cases stimulation is applied for several minutes
A headgear is used to hold the electrodes in the (e.g., 10 min) using two electrodes (typically a
desired positions, or the electrodes are adhesive. few cm2 ) on the head. Often the distinguishing
If the device is small, it may be attached to the feature of different subclasses of tES is the wave-
headgear, but more typically it is a handheld or form – the peak intensity, options for electrode
benchtop device. Electrode design is key for tol- placements, and period of use are often compara-
erability (side effects) and what doses can be ap- ble across low-intensity tES approaches.
plied; as such electrodes are a key consideration When the waveform generated by the device
in device design and considered in this chapter in is sinusoidal alternating current (AC) stimulation,
detail. tES is classified as transcranial alternating current
Subclasses of tES are defined by a specific stimulation (tACS) (Fig. 8.2d). The frequency is
dose. For example, a form of tES that delivers varied typically in a range below 100 Hz, though
high intense stimulation (1000 mA) to intention- higher frequencies have been tested. When the
ally produce a seizure in a anesthetized patient is waveform generated by the device is a train of
called electroconvulsive therapy (ECT) [1]. This pulses, tES is called transcranial pulsed current
chapter is largely focused on low-intensity ap- stimulation (tPCS) (Fig. 8.2a). There are many
proaches that are well below the intensity needed further subclasses (variations) of tPCS waveform
to produce seizures, typically only a few mA [2]. including in duration of each pulse, pulse fre-
These low-intensity approaches are comfortable quency, and if pulses are monophasic or bipha-
when applied to alert individuals, who may be en- sic (Fig. 8.2a,b,c,e,f). Pulses are typically ap-
gaged in different activities during stimulation. In plied repetitively in a train, where the inverse
fact, low-intensity tES typically does not provide of the time between pulses equals the stimula-
an overt response related to brain stimulation – tion frequency. Individual pulses are typically
with any changes in brain function subtle – but rectangular with a pulse duration and amplitude.
can produce overt sensations such as tingling that A monophasic waveform has pulses of a single
are not related to direct brain modulation. In most polarity, while a biphasic waveform has pulses
8 Transcranial Electrical Stimulation 273
Fig. 8.2 Different types of waveforms used in tES and bursts, typically on the scale of hundreds of ms. (c) On/off
their parameters. (a) The pulse shape includes the pulse protocols indicate when stimulation is applied intermit-
duration and amplitude. In biphasic stimulation, pulses tently, typically on the scale of minutes. (d) Non-pulse
are applied in pairs of opposite polarities. The opposite waveforms that are applied include DC, AC, square wave,
polarity pulses may have the same or different duration and various forms of noise. (e) and (f) show examples of
and amplitude. The pulses are delivered in trains with a how all the waveform features in aggregated define dose
frequency. (b) Pulse trains may be continuous or applied in
that invert polarity, typically in paired opposite- such as tDCS and monophasic tPCS, each elec-
polarity pulses (i.e., positive, negative, positive, trode has a fixed assignment of either anode or
negative, etc.) [3]. cathode. For tES devices where the waveform is
When the waveform is a sustained direct biphasic, such as tACS and biphasic tPCS, each
current (DC), tES is classified as transcranial electrode alternates between functioning as an
direct-current stimulation (tDCS) (Fig. 8.2d). anode or cathode. When there are two electrodes,
Additional terminology refers to further the current at one electrode is always the opposite
variations in waveform such as transcranial of the other (1 mA at a single anode indicates
random noise stimulation (tRNS) and cranial −1 mA at a single cathode). When there are more
electrotherapy stimulation (CES). A single tES than two electrodes, the summed current across
device may be programmable to deliver different anode electrodes must equal the summed current
waveforms, e.g., a tDCS mode and a tRNS mode, across the cathode electrode [9] – that is because
or a device may be designed to provide a single of conservation of current where the total current
waveform. Devices made for research typically entering the body must equal the total current
provide more flexibility, while those made for exiting the body.
treatment, especially self-application by patients,
provide one or a limited number of waveforms.
Many tES devices will include an intensity 8.2 General Design Aspects
ramp up and ramp down. The ramp up and down of tES Electrodes
is considered to increase the tolerability of tES,
as skin sensation can accommodate over time, for Key technical contributors to the broad adap-
example, a 30-second linear increase in amplitude tion of tES are the portability and ease of use,
at the start of a session. Some tES devices include along with the tolerability profile of most tES
an interface for subjects or operators to adjust techniques. For limited-intensity tES techniques,
intensity in real time based on sensation, which adverse events are largely limited to effects that
then reduces the intensity if the subject reports occur at the skin such as transient skin sensations
high levels of discomfort [4]. (e.g., perception of warmth, itching, and tingling)
A tES device is essentially a (medical-grade) and redness [10]. Because adverse events are
powered current-controlled stimulator that gener- limited to the skin, the design and preparation
ates the stimulation waveform. tES devices that of tES electrodes are considered central to tol-
deliver low-intensity stimulation, such as tDCS, erability. Electrode design, in turn, can govern
tACS, and tPCS, are typically battery powered. which waveforms will be tolerated. When es-
tES devices used for ECT and devices that apply tablished electrode protocols are not followed or
brief high-intensity stimulation for neurophysio- poor electrode design used, tES produces unnec-
logical evaluation (e.g., a single 1000 mA pulse) essary significant skin irritation and burns. Elec-
are wall powered. In addition to waveform, electrode design also underpins reliable dose delivery.
trode number and shape determine dose and in In addition, electrode design should also address
some cases further inform the subclass of tES ease and robustness of use (e.g., potential for
classification. For example, the use of small elec- home use). For clinical trials, since sensations
trode arrays is classified as high definition (e.g., also determine effective blinding, tES electrodes
high-definition tDCS [5–7], high-definition tACS also impact blinding reliability. Finally, to the ex-
[8]). tent tES electrode design (separate from montage)
The anode electrode is defined as the elec- shapes current flow through the brain [11], and
trode where current enters the body, and at the electrode selection and preparation are critical for
cathode electrode, current exits the body [3]. At the reproducibility and efficacy.
any instant of stimulation, there must be at least The typical tES devices uses just two elec-
one active anode and one active cathode. For tES trodes, of comparable size, each positioned on the
devices where the waveform polarity is fixed, head [12]. However, strategies with asymmetric
electrode size, an electrode at or below the neck of conductive rubber or metal with skin during
[13], or increasing number of electrodes (using tES is avoided as this compromises tolerability
high-definition electrodes) have been investigated and introduces risk of significant skin irritation.
to alter tES spatial focality. This is the main reason why the more involved
Electrodes can be positioned based on head an electrode preparation technique is, and so the
anatomical landmark. These can be modestly more prone it is to set up error (e.g., insufficient
sophisticated requiring a trained operator, for electrolyte thickness in a free-paste electrode),
example, using the EEG 10/10 system (e.g., the less deployable it is, while electrodes intended
anode on C3), while more simplistic placement for wide or deployed use should require mini-
techniques are based on gross anatomical mum preparation (e.g., adhesive electrodes, pre-
landmarks (e.g., over the eyebrow). When a saturated sponge electrodes).
headgear is used, it is either designed to support There are two essential functions of the elec-
the determination of specific electrodes positions trolyte and by extension materials used to sup-
(e.g., a cap or marked straps [14]), or the port the electrolyte shape such as sponge, hy-
headgear is used for generic mechanical support drogel polymer, and/or other support materials
(e.g., rubber bands [15]), and so independent that contain a viscous electrolyte (such as the
measurement is used to position the electrodes. HD case). Both functions of the electrolyte relate
More sophisticated placement techniques such to preventing direct contact between metal/con-
as neuronavigated [16–18], functional [19], non- ductive rubber electrode and the skin. The first
neuronavigated [20], or image-based approaches function relates to electrochemical products, in-
(e.g., EEG reciprocity [21]) have been developed. cluding changes in pH, that occur only at the
tES electrodes include two essential compo- metal/rubber and electrolyte interface [26] such
nents: (1) a conductive rubber or metal separated that a “thick” electrolyte (e.g., realized by a thick
from the skin by (2) a saline-soaked sponge, sponge, gel, or holder) minimizes these reactions
gel, or paste – which are collectively called the from reaching the skin. The second function re-
electrolyte [12]. Additional components of the lates to normalizing current flow patterns through
electrode are often intended to provide mechan- the skin; related to this, the saline, conductive
ical support to the conductive rubber/metal or paste, or conductive gel is used to maintain good
electrolyte or otherwise facilitate use (e.g., facil- contact quality at the skin [5, 27, 28]. If as result
itate connection). In electrochemistry terms, the of poor electrode design (e.g., conductive met-
conductive rubber or plate would be the elec- al/rubber not fully protected from the skin) or
trode, while the saline, gel, or paste would be preparation (e.g., a metal/rubber electrode pushed
the electrolyte [3], but in tES literature, the en- through paste) the metal/rubber contacts the skin,
tire assembly is called the electrode. Here, we these electrochemical changes or poor current
refer to the electrochemical electrode as metal or density patterns can adversely impact the skin,
conductive rubber which includes the interface and aggravated skin irritation is likely.
between the metal/rubber and the electrolytes. The overall cardinal functions of electrodes
This interface is where electrochemical reactions used in tES is to (1) support reliable delivery of
(e.g., pH changes) occur. As noted, in tES when the desired dose and (2) protect the skin from
electrode size is described (e.g., 5 x 5 cm2 ), it is electrochemical reactions occurring at the surface
the interface (surface) between the skin and the of the metal/rubber including normalizing current
electrolyte. Nonetheless, the configuration of all density across the skin (e.g., minimize hot spots)
electrolyte and electrochemical-electrode dimen- and preventing any electrochemical reactions (oc-
sions and materials is important to control and curring at the electrochemical electrode) from
document as this affects tolerability [12, 22–25]. impacting the skin. Because electrochemical con-
The thickness of the sponge or paste essentially cerns are key concern, all electrodes designed
controls the minimum distance between the con- for tES include some mechanism to separate the
ductible rubber or metal and the skin. Contact metal/rubber from the skin. The electrolyte being
Table 8.1 Categories of tES electrodes and usability features

Electrode type On the hair? Preparation? Headgear required? Focal optimization? Electrode sizes
Sponge Yes Yesb Yes No Large
Self-adhesive No a Noa Noa No Variable
HD Yes Yes c Yes Yes Small
Handheld Yes Yesc No No Large
Free paste Yes Yesc No No Large
Dry Unknown No Yes No Variable
a Except if supplement with additional preparation adding liquid gel
b Except single-use pre-saturated snap design
c And gel or paste residue cleanup
the conductive element contacting the skin thus 6. Dry electrodes: Novel designs that that are
takes on importance in general performance. As not adhesive and leave no residue (not liquid
expanded on it in the following sections, the or paste). Experimental and not discussed in
design of the electrolyte (any by extension all detail here.
support materials used around it) thus features
centrally in the classification of electrode types: These choices between these general design
approaches also create restrictions (Table 8.1) on
1. Sponge electrode: A sponge saturated with (1) the size of the electrode (e.g., small HD vs
the fluid electrolyte, typically saline, with a large sponge) which can impact ability to lever-
metal/rubber inside the sponge (sponge pocket age electrode arrays for targeting, (2) how much
design) or on the sponge surface opposite the preparation is required and need for headgear, and
skin. The sponge sets the electrolyte shape and (3) if the electrodes can be applied on the hair.
conductive path.
2. Self-adhesive integrated electrode: A hydrogel
electrolyte that has sufficient rigidity not to 8.3 tES Electrodes: Sponge
flow or spread and with the gel or material Electrode
around the gel including an adhesive compo-
nent. The sponge-based electrode is the most common
3. HD electrode: A stiff mechanical support type of electrode in some forms of tES such as
(short tube/cup) material that contains the tDCS, tACS, and tRNS (Fig. 8.3, [29]); notably in
electrolyte, typically gel, and also controls these techniques, electrode positions over hairline
position of the metal. Used for smaller is common for which the sponge electrode is
electrodes and so suitable for arrays. well suited [30]. Sponge electrode require a
4. Free electrolyte on handheld conductor: headgear to hold them in place (as opposed to
“Free” indicates application by the operator self-adhesive electrodes) which can take the form
without strict control of thickness by the of a headband. Sponge electrodes increase the
electrode assembly. Reused solid metal contact quality even in the areas of the scalp
electrode, covered per-use with a thin with thick hairs because the electrolyte (saline)
electrolyte layer, and an operator handle to penetrates under the hair and saturates the skin
manually press down. Used in some forms of surface skin [31]. A related concern of using
ECT and not considered further here. sponges is that sponge is prone to leaking which
5. Free paste on conductive rubber electrode: distorts the “effective” electrode size making
The paste may also provide adhesion. Used in stimulation not reproducible [27] – for this
some investigational forms of tDCS/tACS and reason, the volume of saline added to the sponges
not considered in detail here. should be carefully calibrated (to the sponge
model, size, and application), and caps (e.g.,
Fig. 8.3 Architecture of sponge electrode and its varia- (b) For sponges without the metal rivet, a wire needs to be
tions. (a) Example of electrodes positioned on the scalp inserted inside the sponges to connect to the conductive
with the intention to stimulation transcranially the brain. rubber electrodes. A rubber band is then used to hold the
(A1a, A1b) Examples (CAD) of minor variations in electrodes to the scalp. (c) For sponges with a metal rivet,
sponge electrode design that can make significant differ- a lead with a snap connector may be used. In this case,
ences in usage. Both 5 × 5 cm2 . In both cases, a conductive the snap connector can be integrated into a head gear. This
rubber electrode is placed between saline-soaked sponges example is intended to show how seemingly small changes
(top sponge for illustration), but in one case, a metal snap in electrode deign can have significant impact on overall
is attached to the conductive rubber electrode. (A2a, A2b) usability
Renders of same sponges positioned over the skin surface.
neoprene) may be avoided since it both obscures the NeuroConn sponge electrode (neuroCare,
and supports fluid spread. There are important Munich, Germany), the rubber sheath is inserted
methodological and design details in sponge into a sown rectangular sponge pocket. In both
electrode design and preparation [27]. cases, the rubber electrode is smaller than the
As used in tDCS, tACS, and tRNS protocols, outer dimensions of the sponge. In the Amrex-
sponge electrode pads have a rectangular skin style sponge electrode (Caputron, NY, USA) a
contact area of 25 cm2 .The contact area is the metal electrode is placed behind the rectangular
interface between electrolyte-saturated sponge sponge, and an insulating rubber encases the
and skin. For sponge electrodes, selection and metal and sponge, except on the skin contact
positioning of the conductive carbon rubber side. These reusable conductive rubber electrodes
sheath or metal can be varied. For example, typically include a female port which is connected
Soterix Medical (EasyPad, Soterix Medical Inc., to a male banana clip or pin-terminated wire
NY, USA) provides rubber electrode embedded from the stimulator. CES devices can use circular
inside a rectangular sponge pocket and uses sponges soaked in tap water (Fisher Wallace
plastic rivets to hold the rubber in place. In electrode, New York, USA). Relatively small
Fig. 8.4 Example of sponge electrode headgear for auto- (d–f) Different views of head-strap placement on a subject
matic electrode positioning. (a) The components include head. The headgear with fixed-position sponge locations
the headgear with integrated snap leads and two snap ensure the electrodes are placed in the desired positions.
sponge electrodes. (b) The two snap sponge electrodes are Using different headgear electrodes can be placed in dif-
connected to the two available positions on the headgear. ferent locations. Having one position per headgear reduces
(c) The headgear assembly can then be placed on the head. the possibility for setup errors
disposable felt electrodes that are saturated no preparation (Soterix EasyPad-2, Fig. 8.4).
in saline are used in some CES devices with A further variation is a more rigid sponge with
ear clip electrodes (Alpha-Stim, Texas, USA). bristles that enhances penetration through hairs
Nonsalinized water is less common and for some and sponge materials embedded with salt in
applications like tDCS, it is contraindicated [27]. a manner that only water can be added over
When water is used, residual electrolyte must multiple uses (Halo Neuroscience, San Francisco,
be present either as impurities (tap) or absorbed CA). Along with new types of associated head-
from the skin. gear (e.g., home use) [32] and connectors (e.g.,
There are updated variants on the sponge magnetic), these examples illustrate that even
electrode design. The conductive rubber may with the conventional sponge electrode paradigm,
be semipermanently embedded into a circular there is an ongoing innovation often focused on
(Sponstim, Neuroelectrics, Spain) or rectangular ease of use (e.g., preassembled and saturated) or
(EasyPad-2, Soterix Medical Inc., NY, USA) reliability (e.g., sponge surface shape).
sponge with a male metallic connector attached
to the rubber and emerging through the sponge
(on the side opposite the skin contact). The male 8.4 tES Electrodes: Self-Adhesive
connector can be affixed to a female connector Electrode
on the headgear directly. As with other sponge
electrodes, the electrodes can be reused or are Self-adhesive electrodes adhere to the skin sur-
single use – for a single use, electrodes are face and typically require minimal preparation –
further available as pre-saturated so requiring this makes them easy to use at locations without
Fig. 8.5 Illustration of adhesive hydrogel electrode (a, b) midline on the back of the neck. (c, e) Representation
placement of rectangular anode on the subject’s right of analogous electrode positioning as a and b on a head
temples. Generally, adhesive electrodes or restricted to model. (d) Image of the adhesive electrode is in the middle
placement below the hairline. In this case, a square cathode column. The bottom of the electrode has an adhesive
electrode positioned about 1 cm to the right of the subject’s hydrogel for adherence with the skin, whereas at the top,
there is electrochemical metal mesh electrode
significant hair [33] but do not work well on to the connector on the electrodes. When the de-
hairline. Self-adhesive electrodes are often used vice is “wearable,” it may connect directly to the
with tPCS waveforms (Brainpod, Caputron, NY, adhesive electrode, and the adhesion may, in some
USA) and also with ECT (Thymapad, Somatics, cases, be sufficient to hold the device to the head.
FL, USA). In their simplest design, the bottom Because DC stimulation is electrochemically
of the electrode has a layer of conductive hy- demanding [5], adhesive electrodes have been
drogel along with an adhesive material; over this used only in a limited number of tDCS trials
layer is a conductive wire, rubber, or metal; and [33] and devices (Zendo E-Meditation), but self-
over either of them is a layer of insulation (see adhesive electrodes are common in other applica-
Fig. 8.5D2). In some designs, the metal may be tions where biphasic pulses and AC stimulation
connected to a short cable with a female pin are used such as cranial nerve electrical stimula-
connection (the cable from the stimulator can be tion [34]. Self-adhesive electrodes designed and
connected to this female pin), or the metal may validated for one stimulation dose may not be
be connected to a snap connector that protrudes tolerated for other doses.
through the insulation layer. When the device is Many approaches that use adhesive electrodes
handheld, the lead wire from the device extends for head stimulation are intended to activate cra-
nial nerves (or peripheral nerves) so as such are (what nervous system element is activated and
not “transcranial” and are, therefore, outside the correlated with outcomes), the targets of these de-
scope of this chapter. Still, insights from cranial vices can be speculative. For CES devices which
stimulation devices can inform tES devices. Cra- include models of adhesive electrodes (Caputron,
nial nerve stimulation devices have used hand- Mindgear, NY, USA), there may be indefinite
held device form factors (Monarch, NeuroSigma, target engagement (cranial nerve, brain [36], or
CA, USA) but also compact device that snap a combination of both).
directly the adhered electrodes (Thync pad, CA,
USA, and Cefaly, CT, USA) – making the en-
tire system wearable. Technologies intended to 8.5 tES Electrodes:
stimulate cranial nerves can have electrodes of High-Definition Electrode
varied separation, ranging from distant electrodes (HD Electrode)
across the head to proximal (adjacent) electrodes.
The latter case produces local superficial current High-definition (HD) electrodes are electrode as-
flow-suited stimulation of cranial nerves at the sembly with a skin contact area of less than 5 cm2 .
skin, but not transcranial. In the former case, the The HD electrode includes a cup that sits on the
two distant electrodes are presumably stimulat- skin and determines the skin contact area. The
ing two targets – though this is also increased cup is filled with conductive gel or paste [5]. Sus-
current through the head (transcranial). For this pended inside the gel is a metal ring, disk, or pellet
reason, transcranial systems with adhesive elec- made from Ag/AgCl. The gel and metal are thus
trodes avoid adjacent electrode placement (e.g., positioned by the interior dimensions of HD cup.
placed as a distance across the forehead) [33]. The design of the HD cup controls the important
These last points relate to a broader debate within factors of gel contact area with the skin and the
the noninvasive neuromodulation [35]; regard- distance between the metal and the skin (Fig. 8.6).
less of whether a system is called “transcranial” As with conventional tDCS using sponge elec-
or claimed to target cranial nerves, there can trodes, there are different montages of HD-tDCS,
be significant overlap in dosage between such but HD electrodes, by the virtue of being smaller,
systems. With verification of target engagement can be deployed in significantly higher number
Fig. 8.6 High-definition (HD) electrodes. (a) In contrast four electrodes of matched polarity are positioned around
to other types of tES electrode, HD electrodes are rela- a central electrode of opposite polarity. The render shows
tively small. (Render) An HD cup is placed on the skin and placement of the electrodes over the targeted brain region.
contain the metal electrodes (Ag/Agcl) and the electrolyte (c) Image of 4 × 1 HD electrode assembly on a subject
gel. (b) Because HD electrodes are smaller, they can be head. Electrodes are secured in a 4 × 1 configuration using
arranged in variation configurations on the head. Shown is a specialized head cap
the 4 × 1 ring configuration of electrode placement where
and/or precise placement [9, 37, 38]. A com- resistance and, conversely, in some cases (e.g.,
mon HD montage is the 4 × 1 ring montage subjects with high-resistance scalp), good contact
where a ring/circular fashion using four “return” may be associated with a moderately high resis-
(cathode) disk electrodes is arranged around an tance. Skin irritation and discomfort may be as-
“active” (anode) electrode at the center [6, 7, 39, sociated with high resistance but not necessarily.
40]. The active electrode is positioned over the Thus, monitoring of resistance is an adjunct tool
scalp (coinciding with the center of the active to detect not only ideal conditions at the electrode
tES sponge pad) and surrounded by four return skin interface but also a substitute for quality
electrodes: each at a disk distance (from center to electrode design and strict protocol adherence
center of the disk) of 3 cm from the active elec- [27, 49].
trode. The HD electrodes are held in place using The resistance measured by the device will
a cap headgear, and a conductive electrolytic gel be the sum of both electrodes including the un-
is filled into the electrode holders. Note that in derlying electrode-skin resistance and the body
contrast to sponge electrodes, here a cap does not resistance. Body resistance is typically a few K
introduce issues related to electrolyte spread since but will vary depending on electrode position on
the gel is well confined by the HD cup. the body and the conditions of the skin (e.g., cal-
Various waveforms can be applied in HD-tES. loused skin). Electrode-skin resistance will vary
HD-tDCS uses tDCS waveforms [37, 38, 41, 42]. depending on the electrode design and waveform
HD-tACS uses AC waveforms [8]. Still other applied [50]. For any given tES device, there
waveforms are specific to the use or arrays such will therefore be a specific total resistance range
as interferential stimulation [43] or high-intensity that is considered typical, and a resistance above
pulses [44]. Multiple brain regions can be targeted this range may suggest not ideal electrode setup,
with HD-tES [8]. in which case the operator may adjust the elec-
The form factor of HD-tES cups superficially trode setup to reduce the skin-electrode resis-
resembles EEG electrodes (though EEG elec- tance. Some device will deactivate if the resis-
trodes cannot be reliably used for stimulation), tance is atypically high.
and indeed it is possible to combine HD-tES and
EEG systems. However, while EEG recording
before HD-tES (e.g., to measure baseline state of 8.7 Current Control, Voltage
inform stimulation strategy; [45]) or after HD-tES Limits
(to measure outcomes; [46]) is valuable, record-
ing of EEG during tES is confounded by artifacts Electrodes play a central role in why current con-
[47, 48]. trol (as opposed to voltage controlled) is broadly
preferred across electrical stimulation applica-
tions [26], including tES. Voltage limits, and pro-
8.6 Electrode Resistance tocols to address voltage compliance, and settings
then reflect device specifications. When stimu-
Monitoring of electrode resistance before and lation is applied to a body from a tES device,
during tES is considered important for repro- the current must pass through electrodes before
ducibility and tolerability [29, 49], specifically reaching the body; therefore, the electrodes are
around issues related to electrode setup. An un- always in series between the device output and the
usually high electrode resistance can indicate un- body. For the simplest case of two electrodes, the
desired electrochemical changes and/or poor skin total impedance is the sum of the impedance of
contact conditions. tES devices will therefore in- the two electrodes and the impedance of the body.
clude a resistance measurement circuit. However, The impedance of each electrode is unknown,
monitoring of electrode impedance in no way variable over time, and changes with current ap-
reduces the need and importance of proper elec- plied [51] and can be significant compared to
trode selection and setup in the sense that poor body impedance [26].
electrode conditions may be associated with a low
First, we consider why voltage control is not the voltage, that kills you.” While the stimula-
preferred: If one used voltage-controlled stimu- tion intensities used in neuromodulation are much
lation, the total voltage provided by the device lower than hazardous accidental exposure, and
will be distributed across the two electrodes and electrodes are designed to be conductive (met-
the body. But since the electrode impedances are al/rubber and electrolyte), the analogy is valid in
unknown and changing, the voltage across the the sense that they dampen the voltage at the body
body is unknown and changing. The total current under voltage-controlled stimulation.
(which reflects the voltage divide by impedance) Since under current control, the voltage will
is also unspecified and changing. Though in tES increase with total path resistance, under situ-
we’re not aware of modern devices that use volt- ations of unusually high resistance, the voltage
age control in other brain stimulation applica- may increase to the limit of the current control
tions, there may be situations where voltage con- device, also called device voltage compliance.
trol is practical such as stimulation of the vagus For limit intensity tES devices, this voltage com-
nerve through electrode on the neck (Gamma- pliance is typically on the scales of tens of volts
Core, Electrocore, NJ, USA) or traditional inva- (e.g., 40 V).
sive stimulation technologies such as SCS and The voltage compliance is conventionally set
DBS (Medtronic, Fridley, MN, USA). to accommodate passing the maximum target
We can now contrast this with current concurrent under expected maximum resistance
trolled stimulation. Here the current output of the (e.g., with a target of 2 mA, and maximum
device is controlled. The current is passed through resistance of 20 K , 40 V is sufficient). In
the two electrodes and body, all in series, so the practice, the impedance may increase outside
current across the body is controlled. The voltage of expected or desired ranges, for example, as a
output of the device is therefore adjusted to keep result of poor electrode setup (see Resistance).
the current controlled at the target level. This In such cases the device output may reach
voltage divided by the current is the impedance voltage compliance, and the device will not be
of the system – also called dynamic impedance able to provide the desired current. Depending
to specify impedance during stimulation as op- on design, devices may respond to voltage
posed to static impedance prior to stimulation compliance in different ways. Some devices may
(see resistance below). Current control therefore simple abort stimulation, while other devices
accommodates for the unknown, variable, and may continue to stimulate with reduced current.
significant impedance presented by electrodes. Because current passage itself reduces current,
Arguably with current control, one does not know maximum impedances are often encountered
the voltage generated across the body, but this at the start of stimulation. Therefore, voltage
can be predicted knowing the body’s resistive compliances are often increased to accommodate
properties (see modeling). Moreover, the voltage this higher initial impedance. However, given that
across the body will not depend on electrode impedance would drop, one proposal for limited
impedances during current control and rather will voltage stimulation was to provide output with
be set by the controlled applied current times the moderate voltages, expecting voltage compliance
body impedance. to be reached at the start of stimulation, but for
The analogy for why current control provides gradual impedance reduction to then reduce
more specificity can be extended to accidental voltage, allowing target current to be reached
electrical exposure. An individual contacting a [50]. There are various reasons to minimize
high-voltage line but wearing insulative rubber voltage from simplifying circuitry or power
gloves would be protected, since the gloves pro- requirements, reducing stimulation energy, or
vide a high resistance path in series with the providing redundant tolerability measures in
body, hence the expression “it’s the current, not susceptible populations or use cases [52].
8.8 Indications for tES Use tES dose [65]. tES dose, along with head
anatomy, determines the resulting current flow
tES spans many clinical and behavioral (intensity and spatial pattern) in the brain [66,
interventions, and as noted, many sub-techniques 67] and so resulting neurophysiological and
[53], such as transcranial direct current stimu- behavioral changes [68]. However, the current
lation (tDCS), transcranial alternating current flow pattern in the head is complex and is not
stimulation (tACS), and transcranial pulsed simply “under” the electrodes and will vary
current stimulation (tPCS). What relates these across individuals. The task of current flow
different techniques is that they apply current models is to relate dose (as controlled by the
through electrodes on the scalp with the intention device) and the resulting brain current flow
of directly stimulating the cerebrum, rather than intensity and spatial pattern. While dose is what
the periphery [27, 36, 54]. Research that uses is specified, it is brain current flow that underpins
tES focuses on direct cortical modulation as an interpretation of outcomes.
explanation for changes in behavior, cognition, For current flow models, also known as vol-
neurophysiology, and imaging studies [6, 55]. ume conduction models, to be accurate, they must
From the perspective of the device, the dose is correctly represent the shape and resistivity of
deigned and selected to achieve specific changes head tissues (e.g., skin, skull, CSF, brain). The
in brain function and so clinical or cognitive out- physics governing volume conduction models of
comes. As described above, while this is a large tES mirror those used in electroencephalography,
parameter space, it can be reduced to features though more anatomically detailed variants have
of the electrode montage (e.g., how many, what been developed over time. Computational models
size, where) and features of the waveform (e.g., have been developed [9, 11, 69–74] and repeat-
intensity, frequency). The electrode montage is edly validated [66, 75–78] over a decade. Ap-
generally considered to determine which brain proaches invented using computational models,
regions are influenced, whereas waveform de- such as HD-tDCS, have been validated [6, 44, 54,
termines how they are influenced – though in 75, 77] and applied [8, 41, 42].
practice, montage and waveform will integrate to Models support the optimization of montages
determine where and how the brain is influenced). to target specific brain regions [9, 79] which can
For example, tDCS is applied as a possible be done at the population average or individual
treatment for major depressive disorder (MDD). level [80]. Different montages and electrode de-
A brain region of interest in MDD research is the signs can be tested [81–83]. The effect of invasive
dorsolateral prefrontal cortex (DLPFC), which is scenarios such as skull burr holes, lesions, or
targeted with tDCS by placing electrodes bilat- weight gain on brain current flow can be tested
erally on the forehead [20, 56–59]. tES clinical hypothetically [70, 84, 85]. Because the same
trials intending to treat pain disorders – e.g., mi- dose will produce different brain current flow
graine [60], fibromyalgia [61], craniofacial pain patterns across subjects, models can also support
[62, 63]) – often target the motor cortex (M1) with individual analysis [44, 86, 87]. The intensity of
an “active” electrode, while the “return” electrode brain current flow can also vary across individu-
is placed on the contralateral forehead (called the als, susceptible populations (e.g., age, stroke, tu-
“supraorbital” or SO position) (Fig. 8.4) [64]. mor), or species in the case of animal experiments
[88]. Current flow models can be used to compare
the effect of stimulation protocols. Current flow
8.9 Current Flow Modeling models can also be compared with imaging data
Informs Device/Electrode [89].
Design and Setup Thus, computational models are ancillary soft-
ware used to inform the design, setup, and pro-
Electrode size and position on the scalp along gramming of tES devices. Device specifications
with the current applied to each electrode define limit the dose range that can be explored by a
model, while conversely, models can encourage then positioned over the brain target (e.g., a
the creation of new device technology. As exam- 35 cm2 scalp contact area electrode positioned
ples, a home-based system relying on adhesive over inferior frontal gyrus (Fig. 8.7)) and meshed
electrodes would restrict explorable electrode lo- at different mesh densities using appropriate
cations in models to locations below the hairline mesh refinement procedures (e.g. Simpleware
[90], which in turn simulate the development Synopsys, CA, USA). The final volumetric
of simple-to-use electrodes that can go over the mesh of the head with electrodes comprising
hairline [91]. The potential for focal transcranial >10,000,000 degree of freedom (DOF) and
stimulation was suggested first by models [71], >12,000,000 tetrahedral elements, specific to this
but it was not until practical HD electrodes were exemplary head model (DOF and no. of elements
developed [5] that approaches to optimize tran- are inter-individual variant), is then imported into
scranial stimulation using HD arrays could be an FEM solver (i.e., COMSOL Multiphysics
tested. 5.1 MA, USA). For electrical stimulation, a
Some important aspects of computational quasistatic approximation [67] (steady-state
models are to investigate the role of parameters solution method) is implemented and solved for
such as electrode assembly, current directionally, electric current physics. The boundary conditions
and polarity of tES and use them to optimize are applied as normal current density at the top
therapeutic interventions for improving their exposed surface of the anode and ground (0 V)
risk/benefit ratio. A computational modeling at the top exposed surface of the return electrode
pipeline of tES starts with segmentation of (cathode). The remaining other external surfaces
an exemplary magnetic resonance imaging of electrode are electrically insulated, and the
(MRI) scan of a head into multiple tissue model is solved. Predicted results are represented
compartments, namely, scalp/skin, fat, skull, csf, as electric field/current density streamlines
gray matter, white matter, and air, to develop to show the current flow trajectories across
a high-resolution (<1 mm) MRI-derived finite different brain regions or volume plot of field
element method (FEM) model. Electrodes of intensity/current density at desired brain tissue
variant shapes, dimensions, and materials are (Fig. 8.7).
Fig. 8.7 Computational FEM head models and predicted streamlines inside the head tissue layers during tES. (A3)
field intensity of dual-hemisphere tES montage. (A1) 3D Volume plot of predicted field intensity and different views
image of a segmented brain generated from an MRI scan of brain under stimulation conditions. Predicted results
of a healthy adult and different views (F, L, R) of electrode plotted at same color range (peak = 0.3 V/m) indicated
placement over the inferior frontal gyrus. (A2) repre- comparable field intensity under both anode and cathode
sent an orientation of magnitude controlled electric field
8.10 tES Biophysics/Mechanisms modulation mechanism of low-intensity tES

technique such as tDCS [94] and tACS [97–99].
Neurons in the brain have a potential across their The neurophysiological and so behavioral con-
membranes (polarization) where changes in this sequences of tES will depend on how this next
polarization (most dramatically action potentials) polarization (across neurons and their compart-
underpin brain function. Given the brain is an ments) influences excitability and plasticity [94].
“electrical organ,” it is not surprising that brain Because low-intensity tES produces only incre-
function is responsive to tES. While there are mental membrane polarization, the cellular ef-
open questions about the mechanisms and effi- fects of low-intensity tES on brain function will
cacy of tES for varied indications, the biophysics further depend on ongoing activity [99–102] and
of tES related to current delivery to the brain (see may be amplified over time (tens of minutes
current flow modeling) and the resulting polariza- [103–105]). The organization of neurons in ac-
tion of neuronal membranes are well established tive networks with emergent properties like os-
[92, 93]. The polarization produced by tES is the cillations will influence the aggregate effects of
initial mechanism of action, with subsequently tES [99, 106–110]. The ultimate consequences
more complex changes in excitability and plas- of low-intensity tES on macroscopic measures of
ticity secondary to this polarization [94, 95]. neurophysiology (e.g., TMS) and behavior (e.g.,
Current that is passed through tES electrodes therapy) will be complex, but ongoing research
takes a path through the head determined by the [80, 111, 112] about such changes should not be
head anatomy and the resistivity of each tissue conflated with the well-established biophysics of
type. A fraction of the current never crosses the current flow and resulting membrane polarization
resistive skull (cranium) instead shunting across of low-intensity tES. As with any single aspect
the relativity conductive (lower resistivity) scalp of brain function and disease, and every inter-
[77]. Of the current fraction that crosses the skull, vention, “open questions” remain – and, again,
a further portion of this is shunted by the highly open questions should not be conflated with the
conductive cerebrospinal fluid. The remaining lack of scientific basis for tES. Specifically, there
current component that reaches the brain and is currently enough basic science supporting tES
crosses the gray and then white matter. As current to inform how devices can be designed and pro-
crosses brain tissue, it generates an electric field grammed in order to test hypothesis related to
on the local tissue. Neurons are exposed to and brain function and therapy.
so stimulated by local electric field. For low-
intensity tES, the current intensity is not uniform
across the brain, and so the electric field intensity 8.11 Tolerability of tES Devices
is also distributed. For conventional tES using two
large pad electrodes, this peak may be in a brain The tolerability of any intervention depends not
region between electrodes [20]. simply on the device and dose but on protocol
The peak electric field in the brain during 2 mA including subject inclusion/exclusion (e.g., age,
tES is 0.5–1 V/m based on intracranial recording preexisting condition), operator training and cer-
in subjects and validated current flow models tification, ongoing monitoring, and parallel inter-
[66, 75, 78]. In contrast ECT applies 700 mA ventions. For example, the scientific consensus
or current producing electric field of 300 V/m that tDCS is safe and tolerated [12, 33, 113–116]
[96]. This contrast is important. Whereas ECT is explicitly limited to those protocols tested. In
and most invasive brain stimulation techniques the same vain, human trials of tDCS in the USA
produce high-intensity electric fields in the are almost always considered nonsignificant risk
brain (>100 V/m), low-intensity tES approaches (risk comparable to daily activities). But this risk
produce weak electric fields (<1 V/m). This is designation – whether made by the FDA or by an
well known and directly support a “subthreshold” institutional IRB – must be made on a protocol-
specific basis, emphasizing that recommendation Homework

on safety and tolerability cannot be made on any
device but must also specify the methods of use. 1. What stimulation parameters define tES
tES device design may be considered to min- dose?
imize risk to the extent that they reliably control 2. Name at least three tES device types. What
dose and allow consistent electrode setup, when distinguishes them from each other?
used within the limits of established protocols. 3. A tES device provides a DC current through
Medical-grade tES devices and accessories that three electrodes. One electrode is an anode
are designed and manufactured to internation- and provides 2 mA. A second electrode is
ally recognized medical standards – regardless a cathode and collects 0.5 mA. Is the third
of region-specific approval for treatment [2, 114, electrode an anode or cathode? How much
117] – provide the highest standard of control in current does it provide or collect?
regard to reliability. 4. Approximately how much current is used to
Tingling is a common adverse effect reported produce a seizure during ECT? How much
in low-intensity tES studies [118, 119]. For low- current is used in techniques such as tDCS
intensity techniques like tDCS, the severity of and tACS?
adverse events is low across all conditions [59]; 5. In a tES setup, the body resistance is 2 kOhm,
however, the frequency of tingling is significantly one electrode-skin resistance is 1 kOhm, and
higher under thin vs. thick sponge stimulation the second electrode-skin resistance is 10
(88% vs. 64% incidence, respectively) [5]. As kOhm. What is the total impedance measured
discussed above, electrode size and salinity of by the tES device? If the second electrode is
sponge electrodes may influence sensation [120]. adjusted such that the second electrode-skin
In principle, electrode design must be optimized resistance is now 1 kOhm, what is the new
to reduce the frequency and intensity of tingling total impedance measured by the tES device?
and related sensations in clinical trials, which 6. In tES with two electrodes, what is the rea-
enhances blinding effectiveness. For this same son for not placing the electrodes proximal
reason, studies which have focused on the ef- (almost touching) each other? For what kind
fectiveness of tES (tDCS) blinding technique but of head electrical stimulation devices is prox-
provide little attention to the electrode design and imal placement rational?
preparation techniques (including document op- 7. A tES electrode assembly is made from a
erator training) are of limited generalized value. cylindrical gel compartment contacting the
There is a dissociation between erythema and skin with a circle interface of 1 cm radius.
tingling – tingling being higher under thin sponge The gel is encased in a hard plastic material
stimulation than thick electrodes [121]. A poten- of 0.5 cm thickness and held inside a cap
tial reason may be that the thick sponge produces with a circumference of the head. The side of
more uniform current density at the skin surface, the gel opposite from the skin makes contact
resulting in evenly diffused erythema distribution with a metal disk of 0.5 cm2 radius. When
and, hence, lower tingling sensation. this is used in a tES publication, what is the
“electrode area” that is practically reported in
Conflict of Interest The City University of New York describing the stimulation dose?
has patents on brain stimulation with MB and NK as 8. If one knows the dose and head anatomy,
inventor. MB advises Boston Scientific, GlaxoSmithKline,
and Mecta. MB has equity in Soterix Medical Inc. DQT what is the use of computational models (e.g.,
has no conflict to declare.
what aspects of brain current flow are models increases focality and intensity at target. J. Neural
used to predict)? Eng. 8, 046011 (2011)
10. A. Fertonani, C. Ferrari, C. Miniussi, What do
9. What are the two essential functions of the
you feel if I apply transcranial electric stimulation?
electrolyte used in tES? Safety, sensations and secondary induced effects.
10. Only a fraction of current reaches the brain in Clin. Neurophysiol 126, 2181–2188 (2015)
tES. Given that current is conserved, where 11. A. Opitz, W. Paulus, S. Will, A. Antunes, A.
Thielscher, Determinants of the electric field during
(what tissues) does the remainder of the cur-
transcranial direct current stimulation. NeuroImage
rent go? 109, 140–150 (2015)
12. A.J. Woods, A. Antal, M. Bikson, P.S. Boggio,
A.R. Brunoni, P. Celnik, L.G. Cohen, F. Fregni,
C.S. Herrmann, E.S. Kappenman, H. Knotkova, D.
References Liebetanz, C. Miniussi, P.C. Miranda, W. Paulus,
A. Priori, D. Reato, C. Stagg, N. Wenderoth, M.A.
1. M.S. George, Z. Nahas, X. Li, F.A. Kozel, B. An- Nitsche, A technical guide to tDCS, and related non-
derson, K. Yamanaka, J.-H. Chae, M.J. Foust, Novel invasive brain stimulation tools. Clin. Neurophysiol.
treatments of mood disorders based on brain cir- 127, 1031–1048 (2016)
cuitry (ECT, MST, TMS, VNS, DBS). Semin. Clin. 13. M. Bikson, A. Datta, A. Rahman, J. Scaturro, Elec-
Neuropsychiatry 7, 293–304 (2002) trode montages for tDCS and weak transcranial
2. M. Bikson, B. Paneri, A. Mourdoukoutas, Z. Es- electrical stimulation: Role of “return” electrode’s
maeilpour, B.W. Badran, R. Azzam, D. Adair, position and size. Clin. Neurophysiol. 121, 1976–
A. Datta, X.H. Fang, B. Wingeier, D. Chao, M. 1978 (2010)
Alonso-Alonso, K. Lee, H. Knotkova, A.J. Woods, 14. M. Kasschau, K. Sherman, L. Haider, A. Frontario,
D. Hagedorn, D. Jeffery, J. Giordano, W.J. Tyler, M. Shaw, A. Datta, M. Bikson, L. Charvet, A pro-
Limited output transcranial electrical stimulation tocol for the use of remotely-supervised transcranial
(LOTES-2017): Engineering principles, regulatory direct current stimulation (tDCS) in multiple sclero-
statutes, and industry standards for wellness, over- sis (MS). J. Vis. Exp. 106, e53542 (2015)
the-counter, or prescription devices with low risk. 15. A.F. DaSilva, M.S. Volz, M. Bikson, F. Fregni, Elec-
Brain Stimul. 11, 134–157 (2018) trode positioning and montage in transcranial direct
3. D.R. Merrill, M. Bikson, J.G.R. Jefferys, Electrical current stimulation. J. Vis. Exp. 51, 1 (2011)
stimulation of excitable tissue: Design of efficacious 16. M. Teichmann, C. Lesoil, J. Godard, M. Vernet, A.
and safe protocols. J. Neurosci. Methods 141, 171– Bertrand, R. Levy, B. Dubois, L. Lemoine, D.Q.
198 (2005) Truong, M. Bikson, A. Kas, A. Valero-Cabré, Direct
4. N. Khadka, H. Borges, T. Kauffman, A. Pascal, current stimulation over the anterior temporal areas
B. Paneri, E. Nassis, Y. Shin, H. Choi, S. Kim, boosts semantic processing in primary progressive
K. Lee, M. Bikson, Abstract #109: Tolerability of aphasia. Ann. Neurol. 80, 693–707 (2016)
an adaptive-tDCS upto 4 mA using subject assess- 17. M. Parazzini, S. Fiocchi, A. Cancelli, C. Cottone, I.
ment and machine-learning to optimize dose. Brain Liorni, P. Ravazzani, F. Tecchio, A computational
Stimul. 12, e37–e38 (2019) model of the electric field distribution due to re-
5. P. Minhas, V. Bansal, J. Patel, J.S. Ho, J. Diaz, gional personalized or nonpersonalized electrodes
A. Datta, M. Bikson, Electrodes for high-definition to select transcranial electric stimulation target.
transcutaneous DC stimulation for applications in I.E.E.E. Trans. Biomed. Eng. 64, 184–195 (2017)
drug delivery and electrotherapy, including tDCS. J. 18. J. Richardson, A. Datta, J. Dmochowski, L.C. Parra,
Neurosci. Methods 190, 188–197 (2010) J. Fridriksson, Feasibility of using high-definition
6. M. Alam, D.Q. Truong, N. Khadka, M. Bikson, transcranial direct current stimulation (HD-tDCS) to
Spatial and polarity precision of concentric high- enhance treatment outcomes in persons with apha-
definition transcranial direct current stimulation sia. NeuroRehabilitation 36, 115–126 (2015)
(HD-tDCS). Phys. Med. Biol. 61, 4506 (2016) 19. T.L. Rich, J.S. Menk, K.D. Rudser, M. Chen, G.D.
7. A. Datta, V. Bansal, J. Diaz, J. Patel, D. Reato, Meekins, E. Peña, T. Feyma, K. Bawroski, C. Bush,
M. Bikson, Gyri-precise head model of transcranial B.T. Gillick, Determining electrode placement for
direct current stimulation: Improved spatial focality transcranial direct current stimulation: A compar-
using a ring electrode versus conventional rectangu- ison of EEG- versus TMS-guided methods. Clin.
lar pad. Brain Stimul. 2, 201–207 (2009) EEG Neurosci. 48, 367–375 (2017)
8. R.M.G. Reinhart, J.A. Nguyen, Working memory 20. O. Seibt, A.R. Brunoni, Y. Huang, M. Bikson,
revived in older adults by synchronizing rhythmic The pursuit of DLPFC: Non-neuronavigated meth-
brain circuits. Nat. Neurosci. 22, 820–827 (2019) ods to target the left dorsolateral pre-frontal cortex
9. J.P. Dmochowski, A. Datta, M. Bikson, Y. Su, with symmetric bicephalic transcranial direct cur-
L.C. Parra, Optimized multi-electrode stimulation rent stimulation (tDCS). Brain Stimul. 8, 590–602
(2015)
21. J.P. Dmochowski, L. Koessler, A.M. Norcia, M. Bik- tocol for the use of remotely-supervised transcranial
son, L.C. Parra, Optimal use of EEG recordings to direct current stimulation (tDCS) in multiple sclero-
target active brain areas with transcranial electrical sis (MS). J Vis Exp 106, e53542 (2015)
stimulation. NeuroImage 157, 69–80 (2017) 33. B. Paneri, D. Adair, C. Thomas, N. Khadka, V.
22. G. Kronberg, M. Bikson, Electrode assembly design Patel, W.J. Tyler, L. Parra, M. Bikson, Tolerability of
for transcranial Direct Current Stimulation: A FEM repeated application of transcranial electrical stimu-
modeling study. 2012 Annual International Confer- lation with limited outputs to healthy subjects. Brain
ence of the IEEE Engineering in Medicine and Biol- Stimul. 9, 740–754 (2016)
ogy Society. 2012 Annual International Conference 34. J.D. Feusner, S. Madsen, T.D. Moody, C. Bohon, E.
of the IEEE Engineering in Medicine and Biology Hembacher, S.Y. Bookheimer, A. Bystritsky, Effects
Society (2012), pp. 891–895 of cranial electrotherapy stimulation on resting state
23. J.E. Dundas, G.W. Thickbroom, F.L. Mastaglia, Per- brain activity. Brain Behav 2, 211–220 (2012)
ception of comfort during transcranial DC stimu- 35. B. Asamoah, A. Khatoun, M. Mc Laughlin, tACS
lation: Effect of NaCl solution concentration ap- motor system effects can be caused by transcuta-
plied to sponge electrodes. Clin. Neurophysiol. 118, neous stimulation of peripheral nerves. Nat. Com-
1166–1170 (2007) mun. 10, 266 (2019)
24. P. Minhas, A. Datta, M. Bikson, Cutaneous per- 36. A. Datta, J.P. Dmochowski, B. Guleyupoglu, M.
ception during tDCS: Role of electrode shape and Bikson, F. Fregni, Cranial electrotherapy stimulation
sponge salinity. Clin. Neurophysiol. 122, 637–638 and transcranial pulsed current stimulation: A com-
(2011) puter based high-resolution modeling study. Neu-
25. Z. Turi, G.G. Ambrus, K.-A. Ho, T. Sengupta, W. roImage 65, 280–287 (2013)
Paulus, A. Antal, When size matters: Large elec- 37. J.J. Borckardt, M. Bikson, H. Frohman, S.T. Reeves,
trodes induce greater stimulation-related cutaneous A. Datta, V. Bansal, A. Madan, K. Barth, M.S.
discomfort than smaller electrodes at equivalent cur- George, A pilot study of the tolerability and effects
rent density. Brain Stimul. 7, 460–467 (2014) of high-definition transcranial direct current stim-
26. D.R. Merrill, M. Bikson, J.G.R. Jefferys, Electrical ulation (HD-tDCS) on pain perception. J Pain 13,
stimulation of excitable tissue: Design of efficacious 112–120 (2012)
and safe protocols. J. Neurosci. Methods 141, 171– 38. H.I. Kuo, M. Bikson, A. Datta, P. Minhas, W. Paulus,
198 (2005) M.F. Kuo, M.A. Nitsche, Comparing cortical plas-
27. A.J. Woods, A. Antal, M. Bikson, P.S. Boggio, A.R. ticity induced by conventional and high-definition
Brunoni, P. Celnik, L.G. Cohen, F. Fregni, C.S. Her- 4 × 1 ring tDCS: A neurophysiological study. Brain
rmann, E. Kappenman, H. Knotkova, D. Liebetanz, Stimul. 6, 644–648 (2013)
C. Miniussi, P.C. Miranda, W. Paulus, A. Priori, D. 39. B. Shen, Y. Yin, J. Wang, X. Zhou, S.M. McClure,
Reato, C. Stagg, N. Wenderoth, M.A. Nitsche, A J. Li, High-definition tDCS alters impulsivity in a
technical guide to tDCS, and related non-invasive baseline-dependent manner. NeuroImage 143, 343–
brain stimulation tools. Clin. Neurophysiol. 127, 352 (2016)
1031–1048 (2016) 40. A.T. Hill, N.C. Rogasch, P.B. Fitzgerald, K.E. Hoy,
28. N. Khadka, A.J. Woods, M. Bikson, Transcranial Effects of prefrontal bipolar and high-definition
direct current stimulation electrodes, in Practical transcranial direct current stimulation on cortical
Guide to Transcranial Direct Current Stimulation: reactivity and working memory in healthy adults.
Principles, Procedures and Applications, ed. by H. NeuroImage 152, 142–157 (2017)
Knotkova, M. A. Nitsche, M. Bikson, A. J. Woods, 41. E.M. Caparelli-Daquer, T.J. Zimmermann, E.
(Springer, Cham, 2019), pp. 263–291 Mooshagian, L.C. Parra, J.K. Rice, A. Datta, M.
29. A.F. DaSilva, M.S. Volz, M. Bikson, F. Fregni, Elec- Bikson, E.M. Wassermann, A pilot study on effects
trode positioning and montage in transcranial direct of 4×1 high-definition tDCS on motor cortex
current stimulation. J Vis Exp 51, e2744 (2011) excitability. Conf. Proc. IEEE Eng. Med. Biol. Soc.
30. M.A. Nitsche, M.A. Nitsche, W. Paulus, W. Paulus, 2012, 735–738 (2012)
Excitability changes induced in the human motor 42. M.F. Villamar, P. Wivatvongvana, J. Patumanond,
cortex by weak transcranial direct current stimula- M. Bikson, D.Q. Truong, A. Datta, F. Fregni, Fo-
tion. J. Physiol. 527(Pt 3), 633–639 (2000) cal modulation of the primary motor cortex in fi-
31. G. Kronberg, M. Bikson, Electrode assembly design bromyalgia using 4×1-ring high-definition transcra-
for transcranial direct current stimulation: A FEM nial direct current stimulation (HD-tDCS): Imme-
modeling study. Conference proceedings: . . . An- diate and delayed analgesic effects of cathodal and
nual International Conference of the IEEE Engineer- anodal stimulation. J. Pain 14, 371–383 (2013)
ing in Medicine and Biology Society. IEEE Engi- 43. N. Grossman, D. Bono, N. Dedic, S.B. Kodandara-
neering in Medicine and Biology Society. Annual maiah, A. Rudenko, H.-J. Suk, A.M. Cassara, E.
Conference 2012, 891–895 (2012) Neufeld, N. Kuster, L.-H. Tsai, A. Pascual-Leone,
32. M. Kasschau, K. Sherman, L. Haider, A. Frontario, E.S. Boyden, Noninvasive deep brain stimulation
M. Shaw, A. Datta, M. Bikson, L. Charvet, A pro- via temporally interfering electric fields. Cell 169,
1029–1041.e16 (2017)
44. D. Edwards, M. Cortes, A. Datta, P. Minhas, E.M. memory in patients with Parkinson’s disease. J. Neu-
Wassermann, M. Bikson, Physiological and model- rol. Sci. 249, 31–38 (2006)
ing evidence for focal transcranial electrical brain 56. A. McGirr, M.T. Berlim, Clinical usefulness of
stimulation in humans: A basis for high-definition therapeutic neuromodulation for major depression:
tDCS. NeuroImage 74, 266–275 (2013) A systematic meta-review of recent meta-analyses.
45. G. Thut, T.O. Bergmann, F. Fröhlich, S.R. Soekadar, Psychiatr. Clin. North Am. 41, 485–503 (2018)
J.-S. Brittain, A. Valero-Cabré, A.T. Sack, C. 57. L. Borrione, A.H. Moffa, D. Martin, C.K. Loo, A.R.
Miniussi, A. Antal, H.R. Siebner, U. Ziemann, C.S. Brunoni, Transcranial direct current stimulation in
Herrmann, Guiding transcranial brain stimulation by the acute depressive episode: A systematic review
EEG/MEG to interact with ongoing brain activity of current knowledge. J. ECT 34, 153–163 (2018)
and associated functions: A position paper. Clin. 58. J. Leite, Ó.F. Gonçalves, P. Pereira, N. Khadka,
Neurophysiol. 128, 843–857 (2017) M. Bikson, F. Fregni, S. Carvalho, The differential
46. A.T. Hill, N.C. Rogasch, P.B. Fitzgerald, K.E. Hoy, effects of unihemispheric and bihemispheric tDCS
Effects of single versus dual-site high-definition over the inferior frontal gyrus on proactive control.
transcranial direct current stimulation (HD-tDCS) Neurosci. Res. 130, 39–46 (2018)
on cortical reactivity and working memory perfor- 59. A.R. Brunoni, M.A. Nitsche, N. Bolognini, M.
mance in healthy subjects. Brain Stimul. 11, 1033– Bikson, T. Wagner, L. Merabet, D.J. Edwards, A.
1043 (2018) Valero-Cabre, A. Rotenberg, A. Pascual-Leone, R.
47. N. Gebodh, Z. Esmaeilpour, D. Adair, K. Chelette, Ferrucci, A. Priori, P.S. Boggio, F. Fregni, Clinical
J. Dmochowski, A.J. Woods, E.S. Kappenman, L.C. research with transcranial direct current stimula-
Parra, M. Bikson, Inherent physiological artifacts tion (tDCS): Challenges and future directions. Brain
in EEG during tDCS. NeuroImage 185, 408–424 Stimul. 5, 175–195 (2012)
(2019) 60. A.F. Dasilva, M.E. Mendonca, S. Zaghi, M. Lopes,
48. N. Noury, J.F. Hipp, M. Siegel, Physiological M.F. Dossantos, E.L. Spierings, Z. Bajwa, A. Datta,
processes non-linearly affect electrophysiological M. Bikson, F. Fregni, tDCS-induced analgesia and
recordings during transcranial electric stimulation. electrical fields in pain-related neural networks in
NeuroImage 140, 99–109 (2016) chronic migraine. Headache 52, 1283–1295 (2012)
49. N. Khadka, A.L. Zannou, F. Zunara, D.Q. Truong, 61. F. Fregni, R. Gimenes, A.C. Valle, M.J.L. Ferreira,
J. Dmochowski, M. Bikson, Minimal heating at the R.R. Rocha, L. Natalle, R. Bravo, S.P. Rigonatti,
skin surface during transcranial direct current stim- S.D. Freedman, M.A. Nitsche, A. Pascual-Leone,
ulation. Neuromodulation 21, 334–339 (2018) P.S. Boggio, A randomized, sham-controlled, proof
50. C. Hahn, J. Rice, S. Macuff, P. Minhas, A. Rahman, of principle study of transcranial direct current stim-
M. Bikson, Methods for extra-low voltage transcra- ulation for the treatment of pain in fibromyalgia.
nial direct current stimulation: Current and time de- Arthritis Rheum. 54, 3988–3998 (2006)
pendent impedance decreases. Clin. Neurophysiol. 62. T. Hagenacker, V. Bude, S. Naegel, D. Holle, Z.
124, 551–556 (2013) Katsarava, H.-C. Diener, M. Obermann, Patient-
51. N. Khadka, A. Rahman, C. Sarantos, D.Q. Truong, conducted anodal transcranial direct current stimula-
M. Bikson, Methods for specific electrode resis- tion of the motor cortex alleviates pain in trigeminal
tance measurement during Transcranial direct cur- neuralgia. J. Headache Pain 15, 78 (2014)
rent stimulation. Brain Stimul. 8, 150–159 (2015) 63. N. Hansen, M. Obermann, F. Poitz, D. Holle, H.-
52. B.T. Gillick, T. Feyma, J. Menk, M. Usset, A. Vaith, C. Diener, A. Antal, W. Paulus, Z. Katsarava, Mod-
T.J. Wood, R. Worthington, L.E. Krach, Safety and ulation of human trigeminal and extracranial no-
feasibility of transcranial direct current stimulation ciceptive processing by transcranial direct current
in pediatric hemiparesis: Randomized controlled stimulation of the motor cortex. Cephalalgia 31,
preliminary study. Phys. Ther. 95, 337–349 (2015) 661–670 (2011)
53. B. Guleyupoglu, P. Schestatsky, D. Edwards, F. 64. M.A. Nitsche, W. Paulus, Excitability changes in-
Fregni, M. Bikson, Classification of methods in duced in the human motor cortex by weak tran-
transcranial electrical stimulation (tES) and evolving scranial direct current stimulation. J. Physiol. Lond.
strategy from historical approaches to contemporary 527(Pt 3), 633–639 (2000)
innovations. J. Neurosci. Methods 219, 297–311 65. A.V. Peterchev, T.A. Wagner, P.C. Miranda, M.A.
(2013) Nitsche, W. Paulus, S.H. Lisanby, A. Pascual-Leone,
54. M.V. Jog, R.X. Smith, K. Jann, W. Dunn, B. Lafon, M. Bikson, Fundamentals of transcranial electric
D. Truong, A. Wu, L. Parra, M. Bikson, D.J.J. Wang, and magnetic stimulation dose: Definition, selec-
In-vivo imaging of magnetic fields induced by tran- tion, and reporting practices. Brain Stimul. 5, 435–
scranial direct current stimulation (tDCS) in human 453 (2012)
brain using MRI. Sci. Rep. 6, 34385 (2016) 66. A. Opitz, A. Falchier, C.-G. Yan, E.M. Yeagle,
55. P.S. Boggio, R. Ferrucci, S.P. Rigonatti, P. Covre, G.S. Linn, P. Megevand, A. Thielscher, A.R. Deb-
M. Nitsche, A. Pascual-Leone, F. Fregni, Effects of orah, M.P. Milham, A.D. Mehta, C.E. Schroeder,
transcranial direct current stimulation on working Spatiotemporal structure of intracranial electric
fields induced by transcranial electric stimulation stimulation. Conf. Proc. IEEE Eng. Med. Biol. Soc.
in humans and nonhuman primates. Sci. Rep. 6, 2016, 1774–1777 (2016)
srep31236 (2016) 79. R.J. Sadleir, T.D. Vannorsdall, D.J. Schretlen, B.
67. M. Bikson, D.Q. Truong, A.P. Mourdoukoutas, M. Gordon, Target optimization in transcranial direct
Aboseria, N. Khadka, D. Adair, A. Rahman, Model- current stimulation. Front. Psych. 3, 90–90 (2012)
ing sequence and quasi-uniform assumption in com- 80. M. Bikson, A.R. Brunoni, L.E. Charvet, V.P. Clark,
putational neurostimulation. Prog. Brain Res. 222, L.G. Cohen, Z.-D. Deng, J. Dmochowski, D.J. Ed-
1–23 (2015) wards, F. Frohlich, E.S. Kappenman, K.O. Lim, C.
68. K.-A. Ho, J.L. Taylor, T. Chew, V. Gálvez, A. Loo, A. Mantovani, D.P. McMullen, L.C. Parra, M.
Alonzo, S. Bai, S. Dokos, C.K. Loo, The effect of Pearson, J.D. Richardson, J.M. Rumsey, P. Sehat-
transcranial direct current stimulation (tDCS) elec- pour, D. Sommers, G. Unal, E.M. Wassermann, A.J.
trode size and current intensity on motor cortical Woods, S.H. Lisanby, Rigor and reproducibility in
excitability: Evidence from single and repeated ses- research with transcranial electrical stimulation: An
sions. Brain Stimul. 9, 1–7 (2016) NIMH-sponsored workshop. Brain Stimul 11, 465–
69. P.C. Miranda, M. Lomarev, M. Hallett, Modeling 480 (2018)
the current distribution during transcranial direct 81. A. Datta, M. Elwassif, F. Battaglia, M. Bikson,
current stimulation. Clin. Neurophysiol. 117, 1623– Transcranial current stimulation focality using disc
1629 (2006) and ring electrode configurations: FEM analysis. J.
70. T. Wagner, F. Fregni, S. Fecteau, A. Grodzinsky, M. Neural Eng. 5, 163 (2008)
Zahn, A. Pascual-Leone, Transcranial direct current 82. D.Q. Truong, A. Datta, J. Xu, F. Fregni, M. Bikson,
stimulation: A computer-based human model study. Prefrontal cortex transcranial direct current stimula-
NeuroImage 35, 1113–1124 (2007) tion via a combined high definition and conventional
71. A. Datta, V. Bansal, J. Diaz, J. Patel, D. Reato, electrode montage: A FEM modeling studying. 34th
M. Bikson, Gyri-precise head model of transcranial Annual International Conference of the IEEE En-
direct current stimulation: Improved spatial focality gineering in Medicine and Biology Society, 2012.
using a ring electrode versus conventional rectangu- EMBS ’12. 34th Annual International Conference
lar pad. Brain Stimul. 2, 201–207 (2009)., 207.e1 of the IEEE Engineering in Medicine and Biology
72. C.H. Im, H.H. Jung, J.D. Choi, S.Y. Lee, K.Y. Jung, Society, 2012. EMBS ’12 (2012), pp. 6608–6611
Determination of optimal electrode positions for 83. G.B. Saturnino, A. Antunes, A. Thielscher, On the
transcranial direct current stimulation (tDCS). Phys. importance of electrode parameters for shaping elec-
Med. Biol. 53, N219–N225 (2008) tric field patterns generated by tDCS. NeuroImage
73. G. Ruffini, M.D. Fox, O. Ripolles, P.C. Miranda, A. 120, 25–35 (2015)
Pascual-Leone, Optimization of multifocal transcra- 84. A. Datta, M. Bikson, F. Fregni, Transcranial direct
nial current stimulation for weighted cortical pattern current stimulation in patients with skull defects and
targeting from realistic modeling of electric fields. skull plates: High-resolution computational FEM
NeuroImage 89, 216–225 (2014) study of factors altering cortical current flow. Neu-
74. D.Q. Truong, M. Hüber, X. Xie, A. Datta, A. Rah- roImage 52, 1268–1278 (2010)
man, L.C. Parra, J.P. Dmochowski, M. Bikson, Clin- 85. D.Q. Truong, G. Magerowski, G.L. Blackburn, M.
ician accessible tools for GUI computational models Bikson, M. Alonso-Alonso, Computational model-
of transcranial electrical stimulation: BONSAI and ing of transcranial direct current stimulation (tDCS)
SPHERES. Brain Stimul. 7, 521–524 (2014) in obesity: Impact of head fat and dose guidelines.
75. Y. Huang, A.A. Liu, B. Lafon, D. Friedman, M. NeuroImage 2, 759–766 (2013)
Dayan, X. Wang, M. Bikson, W.K. Doyle, O. Devin- 86. I. Laakso, M. Mikkonen, S. Koyama, A. Hirata, S.
sky, L.C. Parra, Measurements and models of elec- Tanaka, Can electric fields explain inter-individual
tric fields in the in vivo human brain during transcra- variability in transcranial direct current stimulation
nial electric stimulation. elife 6, e18834 (2017) of the motor cortex? Sci. Rep. 9, 626 (2019)
76. A. Antal, M. Bikson, A. Datta, B. Lafon, P. Dechent, 87. M. Mikkonen, I. Laakso, M. Sumiya, S. Koyama, A.
L.C. Parra, W. Paulus, Imaging artifacts induced by Hirata, S. Tanaka, TMS motor thresholds correlate
electrical stimulation during conventional fMRI of with TDCS electric field strengths in hand motor
the brain. NeuroImage 85(Pt 3), 1040–1047 (2014) area. Front. Neurosci. 12, 426 (2018)
77. A. Datta, X. Zhou, Y. Su, L.C. Parra, M. Bikson, 88. M. Bikson, P. Grossman, C. Thomas, A.L. Zan-
Validation of finite element model of transcranial nou, J. Jiang, T. Adnan, A.P. Mourdoukoutas, G.
electrical stimulation using scalp potentials: Impli- Kronberg, D. Truong, P. Boggio, A.R. Brunoni,
cations for clinical dose. J. Neural Eng. 10, 036018 L. Charvet, F. Fregni, B. Fritsch, B. Gillick, R.H.
(2013) Hamilton, B.M. Hampstead, R. Jankord, A. Kirton,
78. A. Datta, M.R. Krause, P.K. Pilly, J. Choe, T.P. H. Knotkova, D. Liebetanz, A. Liu, C. Loo, M.A.
Zanos, C. Thomas, C.C. Pack, On comparing in Nitsche, J. Reis, J.D. Richardson, A. Rotenberg, P.E.
vivo intracranial recordings in non-human primates Turkeltaub, A.J. Woods, Safety of transcranial direct
to predictions of optimized transcranial electrical current stimulation: Evidence based update 2016.
Brain Stimul. 9, 641–661 (2016)
89. M.A. Halko, A. Datta, E.B. Plow, J. Scaturro, M. 102. M.R. Krause, T.P. Zanos, B.A. Csorba, P.K. Pilly, J.
Bikson, L.B. Merabet, Neuroplastic changes fol- Choe, M.E. Phillips, A. Datta, C.C. Pack, Transcra-
lowing rehabilitative training correlate with regional nial direct current stimulation facilitates associative
electrical field induced with tDCS. Neuroimage 57, learning and alters functional connectivity in the
885–891 (2011) primate brain. Curr. Biol. 27, 3086–3096.e3 (2017)
90. W.J. Tyler, A.M. Boasso, H.M. Mortimore, R.S. 103. L.J. Bindman, O.C. Lippold, J.W. Redfearn, Long-
Silva, J.D. Charlesworth, M.A. Marlin, K. Aeber- lasting changes in the level of the electrical activity
sold, L. Aven, D.Z. Wetmore, S.K. Pal, Transder- of the cerebral cortex produced bypolarizing cur-
mal neuromodulation of noradrenergic activity sup- rents. Nature 196, 584–585 (1962)
presses psychophysiological and biochemical stress 104. S.J. Pelletier, F. Cicchetti, Cellular and molecular
responses in humans. Sci. Rep. 5, 13865 (2015) mechanisms of action of transcranial direct current
91. M.T. Shaw, M. Kasschau, B. Dobbs, N. Pawlak, stimulation: Evidence from in vitro and in vivo
W. Pau, K. Sherman, M. Bikson, A. Datta, L.E. models. Int. J. Neuropsychopharmacol. 18 (2014)
Charvet, Remotely supervised transcranial direct 105. D. Reato, M. Bikson, L.C. Parra, Lasting modulation
current stimulation: An update on safety and toler- of in vitro oscillatory activity with weak direct cur-
ability. J. Vis. Exp. 2017 (2017) rent stimulation. J. Neurophysiol. 113, 1334–1341
92. P.C. Miranda, Physics of effects of transcranial brain (2015)
stimulation. Handb. Clin. Neurol. 116, 353–366 106. S.L. Schmidt, A.K. Iyengar, A.A. Foulser, M.R.
(2013) Boyle, F. Fröhlich, Endogenous cortical oscillations
93. A. Rahman, B. Lafon, M. Bikson, Multilevel com- constrain neuromodulation by weak electric fields.
putational models for predicting the cellular effects Brain Stimul. 7, 878–889 (2014)
of noninvasive brain stimulation. Prog. Brain Res. 107. J.J. Bonaiuto, S. Bestmann, Understanding the
222, 25–40 (2015) nonlinear physiological and behavioral effects of
94. M.P. Jackson, A. Rahman, B. Lafon, G. Kronberg, tDCS through computational neurostimulation.
D. Ling, L.C. Parra, M. Bikson, Animal models of Prog. Brain Res. 222, 75–103 (2015)
transcranial direct current stimulation: Methods and 108. D. Reato, F. Gasca, A. Datta, M. Bikson, L. Mar-
mechanisms. Clin. Neurophysiol. 127, 3425–3454 shall, L.C. Parra, Transcranial electrical stimulation
(2016) accelerates human sleep homeostasis. PLoS Com-
95. J. Modolo, Y. Denoyer, F. Wendling, P. Ben- put. Biol. 9, e1002898 (2013)
quet, Physiological effects of low-magnitude elec- 109. D. Reato, A. Rahman, M. Bikson, L.C. Parra, Low-
tric fields on brain activity: Advances from in vitro, intensity electrical stimulation affects network dy-
in vivo and in silico models. Curr. Opin. Biomed. namics by modulating population rate and spike
Eng. 8, 38–44 (2018) timing. J. Neurosci. 30, 15067–15079 (2010)
96. S. Bai, C. Loo, S. Dokos, A computational model of 110. M.M. Ali, K.K. Sellers, F. Fröhlich, Transcranial al-
direct brain stimulation by electroconvulsive therapy ternating current stimulation modulates large-scale
2010 Annual International Conference of the IEEE cortical network activity by network resonance. J.
Engineering in Medicine and Biology 2010 Annual Neurosci. 33, 11262–11275 (2013)
International Conference of the IEEE Engineering in 111. T. Reed, R.K. Cohen, Transcranial electrical stimu-
Medicine and Biology (2010), pp. 2069–2072 lation (tES) mechanisms and its effects on cortical
97. F. Fröhlich, Experiments and models of cortical excitability and connectivity. J. Inherit. Metab. Dis
oscillations as a target for noninvasive brain stim- 41, 1123–1130 (2018)
ulation. Prog. Brain Res. 222, 41–73 (2015) 112. A.B. De, M. Bikson, S. Bestmann, Predicting the be-
98. J.G.R. Jefferys, J. Deans, M. Bikson, J. Fox, Effects havioral impact of transcranial direct current stimu-
of weak electric fields on the activity of neurons and lation: Issues and limitations. Front. Hum. Neurosci.
neuronal networks. Radiat. Prot. Dosim. 106, 321– 7, 613–613 (2013)
323 (2003) 113. C. Poreisz, K. Boros, A. Antal, W. Paulus, Safety
99. D. Reato, A. Rahman, M. Bikson, L.C. Parra, Effects aspects of transcranial direct current stimulation
of weak transcranial alternating current stimulation concerning healthy subjects and patients. Brain Res.
on brain activity-a review of known mechanisms Bull. 72, 208–214 (2007)
from animal studies. Front. Hum. Neurosci. 7, 687 114. A. Antal, I. Alekseichuk, M. Bikson, J. Brockmöller,
(2013) A.R. Brunoni, R. Chen, L.G. Cohen, G. Dowthwaite,
100. A. Rahman, B. Lafon, L.C. Parra, M. Bikson, Direct J. Ellrich, A. Flöel, F. Fregni, M.S. George, R.
current stimulation boosts synaptic gain and coop- Hamilton, J. Haueisen, C.S. Herrmann, F.C. Hum-
erativity in vitro. J. Physiol. Lond. 595, 3535–3547 mel, J.P. Lefaucheur, D. Liebetanz, C.K. Loo, C.D.
(2017) McCaig, C. Miniussi, P.C. Miranda, V. Moliadze,
101. M. Bikson, A. Name, A. Rahman, Origins of speci- M.A. Nitsche, R. Nowak, F. Padberg, A. Pascual-
ficity during tDCS: Anatomical, activity-selective, Leone, W. Poppendieck, A. Priori, S. Rossi, P.M.
and input-bias mechanisms. Front. Hum. Neurosci. Rossini, J. Rothwell, M.A. Rueger, G. Ruffini, K.
7, 688 (2013) Schellhorn, H.R. Siebner, Y. Ugawa, A. Wexler, U.
Ziemann, M. Hallett, W. Paulus, Low intensity tran- and research use of transcranial direct current stim-
scranial electric stimulation: Safety, ethical, legal ulation (tDCS): Review and recommendations from
regulatory and application guidelines. Clin. Neuro- an expert panel. Clin. Res. Regul. Aff. 32, 22–35
physiol. 128, 1774–1809 (2017) (2015)
115. M. Bikson, P. Grossman, C. Thomas, A.L. Zan- 118. S.K. Kessler, P.E. Turkeltaub, J.G. Benson, R.H.
nou, J. Jiang, T. Adnan, A.P. Mourdoukoutas, G. Hamilton, Differences in the experience of active
Kronberg, D. Truong, P. Boggio, A.R. Brunoni, and sham transcranial direct current stimulation.
L. Charvet, F. Fregni, B. Fritsch, B. Gillick, R.H. Brain Stimul. 5, 155–162 (2012)
Hamilton, B.M. Hampstead, R. Jankord, A. Kirton, 119. C. Poreisz, K. Boros, A. Antal, W. Paulus, Safety
H. Knotkova, D. Liebetanz, A. Liu, C. Loo, M.A. aspects of transcranial direct current stimulation
Nitsche, J. Reis, J.D. Richardson, A. Rotenberg, concerning healthy subjects and patients. Brain Res.
P.E. Turkeltaub, A.J. Woods, Safety of Transcranial Bull. 72, 208–214 (2007)
direct current stimulation: Evidence based update 120. J.E. Dundas, G.W. Thickbroom, F.L. Mastaglia, Per-
2016. Brain Stimul. 9, 641–661 (2016) ception of comfort during transcranial DC stimu-
116. S. Nikolin, C. Huggins, D. Martin, A. Alonzo, C.K. lation: Effect of NaCl solution concentration ap-
Loo, Safety of repeated sessions of transcranial di- plied to sponge electrodes. Clin. Neurophysiol. 118,
rect current stimulation: A systematic review. Brain 1166–1170 (2007)
Stimul. 11, 278–288 (2018) 121. F. Ezquerro, A.H. Moffa, M. Bikson, N. Khadka,
117. F. Fregni, M.A. Nitsche, C.K. Loo, A.R. Brunoni, P. L.V.M. Aparicio, B. de Sampaio-Junior, F. Fregni,
Marangolo, J. Leite, S. Carvalho, N. Bolognini, W. I.M. Bensenor, P.A. Lotufo, A.C. Pereira, A.R.
Caumo, N.J. Paik, M. Simis, K. Ueda, H. Ekhitari, P. Brunoni, The influence of skin redness on blinding
Luu, D.M. Tucker, W.J. Tyler, J. Brunelin, A. Datta, in transcranial direct current stimulation studies:
C.H. Juan, G. Venkatasubramanian, P.S. Boggio, M. A crossover trial. Neuromodulation 20, 248–255
Bikson, Regulatory considerations for the clinical (2017)
Optogenetics: Novel Brain Interface
Technology That Originates 9
in Bioprospecting
Ramin Pashaie
Abstract technique, and some potential directions for

future research.
Optogenetics is a brain stimulation technique
in which the activity of stimulable cells, such Keywords
as neurons or astrocytes, is modulated by
exposing target cells to pulses of appropriate Optogenetics · Optical brain interface ·
wavelengths. Prior to optogenetic experi- Opsins · Neuromodulation · Optical neural
ments, we deliver specific genetic constructs stimulation
to target cells to express light-activated ion
channels or ion pumps and produce light
sensitivity. Once these proteins are produced,
we can precisely modulate cellular activity 9.1 Introduction
by exposing such cells to sequences of
light pulses. Optogenetics, combined with Understanding the dynamics of the brain and the
recording methods, is widely used to develop mechanisms that this biological computational
complex brain computer interface platforms. paradigm uses to process high-throughput data
Unique features of optogenetic stimulation, and generate complex outputs, such as perception
including specific cell-type targeting or or cognition, has brought scientists from a wide
bidirectional control of cellular activity, have range of backgrounds into this field. In recent
allowed researchers to use the method in the years, many mathematicians, physicists, and en-
study of brain networks, finding projections, gineers have concentrated on neuroscience prob-
or in dissociating circuitries of neurological lems and used their computational mindsets and
and psychiatric disorders. Here, we briefly system development skills to implement novel
review the essence of the technology, pros and brain interface platforms. This generation of neu-
cons of the method, major applications of the roscientists contributed to the development of
new treatments for mental disorders based on
interfering with the brain networks by speaking
Electronic Supplementary Material: The online version the electrical language of neurons. This approach,
of this chapter (https://doi.org/10.1007/978-3-030-43395- currently known as the interventional psychia-
6_9) contains supplementary material, which is available try, was in contrast to the traditional point of
to authorized users. view which assumed that any mental disease is
R. Pashaie () the result of some form of chemical imbalance
Electrical Engineering and Computer Science in the brain. Therefore, such disorders can be
Department, University of Wisconsin-Milwaukee, cured by an appropriate dosage of medicine to
Milwaukee, WI, USA
restore the chemical balance in the central ner-

294 R. Pashaie
vous system. The first step for the development generate only simple stimulation patterns, and
of interventional procedures was to learn how imposing complex modes of activity is almost
healthy neurons encode information and commu- out of reach. On the recording side, the limited
nicate to achieve certain goals. For this purpose, number of electrodes makes source identification
a wide range of recording technologies were in- or source localization problems (mathematical
vented which includes the development and test- inverse problems) highly ill-posed. Implanting
ing of multiple penetrating or surface-mounted arrays that embed a large number of penetrating
electrode array configurations. Once better un- electrodes is not practical in many cases since
derstanding of neural codes was achieved, re- larger arrays can cause irreversible damage to
searchers initiated the effort to reverse the process the sensitive brain tissue. Further integration will
and use recording devices this time to stimulate not necessarily help either since recorded data
neurons with pulse sequences, similar to recorded from electrodes that are geometrically closer than
patterns, to establish two-way communications. certain distances is highly correlated and the ac-
Improvements along these lines led to the devel- quired data is redundant. A third limitation of
opment of brain stimulation devices which soon electrode arrays is that such interface platforms
found bold clinical applications helping many pa- cannot bidirectionally (increase or suppress) con-
tients suffering from brain diseases or injuries that trol cellular activity. In most applications, elec-
disrupted the nervous system’s normal functions. trodes are used only to stimulate neurons, whether
Good examples are the deep brain stimulation excitatory or inhibitory.
(DBS) used in Parkinson’s or electrode arrays im- These technological limitations encouraged
planted on the visual cortex or retina to partially scientists to search for more potent brain
restore vision for blind people [1]. stimulation methods. One technology that
Despite immense enthusiasm in early days, emerged and addressed these restrictions was
most interventional therapeutic procedures faced optogenetics [2–8].
major limitations that restricted the application of In the early 2000s, a group of researchers
such technologies. Electrode-based stimulation introduced a new brain stimulation technology by
systems particularly suffer from multiple inherent combining tools of optics with advanced methods
limitations. Brain networks are complex mixtures of molecular genetics. Prior to optogenetics, for
of many different interconnected cell populations several decades, biochemists studied the struc-
where each cell type has a set of functions. Elec- ture and dynamics of light-sensitive proteins for
trode arrays which are designed for stimulation applications like energy harvesting [9]. Through
application are not capable of targeting any spe- these studies, they learned that in many micro-
cific cell population, and all networks in the vicin- scopic species, including certain types of bac-
ity of each electrode are influenced whenever teria or algae, light-sensitive proteins that form
a current pulse is injected [2]. Consequently, in membrane ion channels or ion pumps provide
many cases where brain stimulation is used for essential tools that help these species survive in
treatment, patients report mild, moderate, or even harsh environments and satisfy their basic vital
severe side effects. For instance, implanted elec- needs. These studies proved that the structure
trodes in Parkinsonian patients cause side effects and dynamics of such proteins are similar to ion
including depression, mood alteration, or sensory channels and ion pumps that we find in the mem-
and motor control problems. All such side effects brane of mammalian neurons. These proteins are
disappear the moment the stimulation is disabled. the most essential elements in the generation and
A second limitation of electrode arrays stems conduction of electric signals between excitable
from the significant difference between the num- cells including neurons [6–8]. By introducing
ber of electrodes in each array and the number these proteins to cultured neurons, which were
of cells that contribute to recorded signals. With harvested from mammalian brain cells, one could
such a small number of electrodes in each array, effectively stimulate the cells simply by exposing
in comparison with the cell numbers, we can them to appropriate wavelengths. It was shown
9 Optogenetics: Novel Brain Interface Technology That Originates in Bioprospecting 295
that the sequence of action potentials in such pressed, one can modulate the activity simply by
excitable cells closely follow the sequence of exposing target cells to appropriate wavelengths.
exposing light pulses. First, photosensitive optogenetic proteins were
The genetic element of this neuro-modulation the naturally occurring channelrhodopsin (ChR)
technology provided an effective mechanism to and halorhodopsin (HR) [12]. ChR and HR were
target cell populations of interest and potentially used to induce excitation or inhibition in neurons
avoid common side effects of interventional harvested from small rodents.
procedures [10]. Furthermore, the inherent In the late 1970s, researchers discovered
parallelism of optics makes it possible to target proteins of the microbial rhodopsin family
a larger population of cells for stimulation. By which are single-component units that transform
combining optogenetics with optical recording pulses of light into electric currents. Some
techniques, like calcium imaging or voltage- single-celled microorganisms, including Archaea
sensitive dye recording, we can implement Halobacterium salinarum, take advantage of
all-optical brain interface platforms that are microbial rhodopsins, e.g., bacteriorhodopsin
more versatile in their applications. It was also (BR) and HR, to harvest the energy of photons
shown that bidirectional control of cellular [6]. These spices use the acquired energy, for
activity can be achieved by co-expressing example, to generate the electrochemical gradient
light-activated proteins that depolarize or required for anaerobic glycolysis which is a vital
hyperpolarize excitable cells [2]. Then, switching process for surviving in a high osmatic pressure
the wavelength of the exposure modulates cellular environment. Another example is the HR found
activity in each direction. Optogenetics addressed in Archaea which functions as a light-activated
the main challenges of electrode-based brain ion pump and transports chloride across the
interface systems and the technology transformed membrane against the electrochemical gradient.
into an ever-growing field of research [11]. Later, in the early 2000s, a new form of microbial
In the following, we discuss some widely rhodopsin, aka ChRs, was isolated from the
used tools of optogenetics and salient features freshwater algae Chlamydomonas reinhardtii.
of the dynamics of these light-activated proteins. These proteins function as selective light-gated
We summarize major advances in the field of cation channels. When exposed to light, the pore
gene delivery and light delivery for optogenetics. of the channel opens to let specific cations (in
Next, we review examples of hybrid brain this case, sodium ion) pass across the membrane
interface platforms that combine optogenetics along the direction of the diffusion force.
with other brain interface or imaging modalities, The light sensitivity of all microbial rhodopsins
like electrophysiology, two-photon microscopy, comes from the isomerization of the retinal
coherence tomography, and magnetic resonance molecule which embeds within the protein
imaging. Finally, we discuss potential applica- structure (see Fig. 9.1). When a retinal molecule
tions and pitfalls of optogenetics in therapeutic absorbs light energy, it transforms from all-trans
applications. conformation to the higher-energy 13-cis state.
This conformational change of retinal applies
force to the protein structure and opens the
9.2 Tools of Optogenetics pore. Channel opening can happen in less than
1 millisecond. However, soon after and with no
9.2.1 Opsins for Excitation other photon interaction, the molecule returns to
and Inhibition its low-energy stable state, and the channel closes.
In this sense, microbial rhodopsin has mono-
In optogenetic brain stimulation, we first deliver stable dynamics. The time constant of closing
the genetic constructs to express light activated is usually in the order of tens of milliseconds.
ion channels or ion pumps in target cells of re- Therefore, when the exposure is sufficient, we
cipient species [2, 3, 8]. Once proteins are ex- can modulate the activity of excitable cells, like
296 R. Pashaie
Fig. 9.1 ChR2 protein functions as a light-gated cation from the all-trans to 13-cis isomer, and this process opens
channel. The electron-photon interaction in one chemical the pore of the channel. Under no further photon interac-
bond of retinal changes the conformation of the molecule tion, the molecule returns to its low-energy state, and the
channel closes
neurons, or heart and muscle cells, and force sensitivity was problematic when researchers
them to follow the sequence of pulses produced tried to use optogenetic stimulation in in vivo
by a light source like a computer-controlled laser experiments. High-energy blue wavelengths
module (see Fig. 9.2). within this spectral bandwidth cannot penetrate
Successful initial experiments led to a new deep into the tissue due to the high absorption
effort of discovering and/or engineering other rate of these wavelengths. The penetration depth,
photosensitive molecules. Two approaches were for effective optogenetic stimulation at these
considered in this endeavor. The first approach wavelengths, is limited to 100–200 um [13]. This
was based on bioprospecting and searching limited penetration makes it difficult to reach
nature for new forms of similar photosensitive deeper layers of the cortex without implanting
proteins. In the second approach, researchers light guides. Implanting a light guide in the
tried to use tools of molecular genetics to engineer brain is an invasive process. Targeting a large
photosensitive proteins with different parameters neural network requires implanting multiple
(such as opening/closing time constants, light light guides which is not acceptable in many
amplitude sensitivity, spectral sensitivity, and experiments. The need to find opsins which
even ionic selectivity) to expand the scope of offer red-shifted spectral sensitivity led to the
this new method [12]. For example, spectral discovery of other optogenetic tools such as
sensitivity of the first excitatory opsin, e.g., Volvox ChR which was isolated from spheroidal
ChR2, was mostly around shorter wavelengths algae Volvox carteri and offered about 50 nm
from 400 nm to 500 nm with the peak sensitivity shift in spectral response compared to ChR2 [14].
occurring around 445 nm [2]. This spectral Several novel approaches were tested to make the
Fig. 9.2 Example of in vitro optogenetic stimulation: generate optical stimulation patterns including individual
First cultured hippocampal neurons, harvested from new- 10 ms light pulses (top) or train of 10 ms pulses with the
born rat pups, were transfected with lentivirus vector to ex- repetition rates of 5 pulses (middle) and 15 pulses (bottom)
press ChR2 proteins. Seven days after transfection, whole- per second. Sufficient depolarization was achieved, and
cell current clamp experiment was conducted. Beam of the stimulated cell generates action potentials in response
a computer-controlled 473 nm laser diode was used to to every laser pulse
spectral sensitivity closer to the ideal form. For penetration depth of sunlight varies as a function
example, it was hypothesized that algae living of wavelength.
at different depths in open waters may provide To address the penetration depth problem
opsins of different spectral sensitivity since the for in vivo applications, researchers even tried
to discover new ion channels that respond to
298 R. Pashaie
other forms of energy such as heat [15] or also shown that certain mutations can change
acoustic waves. Genetic engineering methods the ion selectivity of optogenetic proteins. For
were also used to artificially produce opsins with example, conduction of divalent cations, e.g.,
red-shifted spectral sensitivity. For example, a Ca2+, significantly increases in the ChR2
new class of opsins called SFO (step-function L132C/T159C mutants [20]. Therefore, in this
opsin) was produced that offers long closing case, we can use optics to manipulate intracellular
time constants going well beyond 1 minute [16]. concentration of Ca2+ and trigger signaling
While the temporal response is compromised in cascades that are modulated by this cation.
this class of opsins, light sensitivity of the host Crystallography of the ChR2 molecule has
cell is improved. The reason is that we need provided valuable information about the structure
to open fewer channels to generate the same of the protein [21]. This information can help ge-
depolarization level since each SFO channel netic engineers make informative decisions when
remains open for a longer period. One benefit selecting sites to impose point mutations. With
of achieving better light sensitivity is improving this information, they can engineer proteins that
the depth of stimulation. Interestingly, it was offer kinetic parameters closer to desired values
also shown that SFO channels can be closed for arbitrary applications.
at any point simply by exposing the protein to One main advantage of optogenetics, com-
green light [16]. In this sense, SFO proteins pared to other optical stimulation paradigms, is
have bi-stable dynamics. Discovery of SFO that optogenetic tools are genetically encoded
had a major impact in neuroscience research photoactuators. As a result, specific cell-type tar-
since SFOs opened the path for two-photon geting can be achieved by choosing the right
optogenetic stimulation [17]. It was also shown promoter and incorporating that in the genetic
that by inducing point mutations in genetic codes construct that we deliver to the host [10].
of opsins, we could produce channels with much Optogenetics is also successfully used to ac-
faster kinetics. For example, the opsin ChETA tivate astrocytes [22]. Since astrocytes play a
(ChR2/E123T) allows targeted mammalian crucial role in the coupling of neural and vascular
neurons to generate bursts of action potentials networks in the brain, optogenetic stimulation can
that closely follow laser pulses up to 200 Hz [18]. significantly contribute to our understanding of
Nonetheless, light sensitivity is less in ChETA the dynamics of neurovascular units.
compared to SFO or even ChR2 proteins. A good
compromise between temporal response and light
sensitivity is the opsin ChR2/ H134R which is 9.2.2 Mechanisms of Gene Delivery
two to three times more sensitive compared to
the wild-type ChR2, and it is widely used in For gene delivery, researchers have successfully
optogenetic experiments in vivo or in vitro. tested several strategies. For in vitro applications,
We can also improve the light sensitivity by when a high rate of successful transfection is
overexpressing opsins in host cells. Nonetheless, not required, simple gene delivery procedures,
further research showed that overexpression can like the calcium phosphate protocol or electro-
cause intracellular accumulation of molecules poration, are reasonable choices. For in vivo ap-
which potentially leads to membrane trafficking plications, genetically engineered viral vectors,
complications. A good example is the light- such as the lentivirus vectors (LV) or the adeno-
activated electrogenic Cl − pump known as associated viruses (AAV), are the most popular
microbial halorhodopsin NpHR. Molecular gene delivery vehicles. LVs integrate into the
engineering of NpHR led to the development host’s genome which leads to the permanent ex-
of the enhanced version of this protein, known pression of the protein [23, 24]. On the other
as eNpHR, which offers potent optical inhibition hand, AAVs are less immunogenetic, and they can
without the aggregation and toxicity [19]. It was deliver the genetic construct to larger tissue vol-
umes. Both methods offer high expression rates tem. Small rodents, specifically mice and rats, are
and minimal side effects. Since AAVs do not inte- perhaps the most frequent subjects of optogenetic
grate into the host’s genome, they are considered experiments. Many brain circuitries, from corti-
as safer choices in regular lab applications and cal tissue to thalamus, amygdala, or hippocam-
used more frequently in optogenetic experiments pus, of small rodents were selected for optoge-
[25]. netic experiments to study fear, anxiety, sleep, de-
Several lines of optogenetic transgenic pression, Parkinson’s, etc. Transgenic mice were
animals are also developed [26]. A good example even used for their light sensitivity in peripheral
is the optogenetic transgenic mice in which ChR2 nerves. For instance, optogenetic stimulation of
protein is expressed under the control of the sciatic nerve was successfully tested to achieve
mouse thymus cell antigen 1 (Thy1) promoter. enhanced and physiologically more natural mus-
These animals are widely used in studies where cle recruitment compared to conventional electri-
uniform light sensitivity is needed in large- cal stimulation [27].
scale networks of cortex or even in optogenetic Some of the most interesting optogenetic
stimulation of nerves in the peripheral nervous experiments with rats are related to the integration
system [27]. of optogenetics with functional magnetic
resonance imaging (fMRI). It was shown
that optical stimulation of ChR2 proteins is
9.2.3 Target Species for sufficient to trigger blood oxygenation level-
Optogenetic Experiments dependent (BOLD) signals which are detectable
by high-resolution animal fMRI machines. These
Optogenetic tools are used in many species to experiments specifically opened a new era of
study the basic function of neural circuits or to un- combining advanced technologies for brain
derstand dynamics of diseases that affect the ner- mapping or finding projections in the brain
vous system. Simple creatures such as transgenic noninvasively. Optogenetics was also tested in
optogenetic Caenorhabditis elegans (C. elegans) nonhuman primates to activate specific circuits
are good candidates to study neural networks [31]. However, up to this point, no change in
that control animal motion. By expressing both behavior as a result of optogenetic interference
inhibitory and excitatory optogenetic proteins and has been reported in primates. As discussed
using spatial light modulators (SLM) to pattern later in this article, a main target for optogenetic
light over the body, researchers demonstrated the stimulation in primates, including humans, is the
power of optogenetics in controlling and even retina with the goal of retaining visual perception
programming the movement of the worm [28]. for those who have lost vision as a result of
Flies, such as Drosophila, are selected spices photoreceptor deterioration.
to investigate the neural basis of nociceptive re-
sponses and pain receptors [29]. In optogenetic
experiments with worms and flies, we should 9.3 Mechanisms of Light
deliver sufficient amount of retinal to cells, for in- Delivery
stance via food supplements, to retain the normal
function of optogenetic proteins. Unlike mam- 9.3.1 Light-Tissue Interaction
mals, the level of endogenous retinal is not suf-
ficient in these species. In addition to gene delivery, the second con-
Zebrafish are used widely in optogenetic ex- trol mechanism in optogenetic stimulation comes
periments [30]. Due to their transparency, this from the way we deliver light to brain tissue.
animal has provided the opportunity to test op- Depending on the application, the target area of
togenetics together with optical recording meth- the brain can change in depth or size. We also
ods, such as calcium imaging, to implement all- occasionally change the stimulation pattern based
optical interfaces with the animal’s nervous sys- on our experimental protocol and/or the feedback
300 R. Pashaie
we receive in real time from cells or the animal tering coefficient defines the number of times a
behavior in closed-loop control procedures [32, given photon is scattered in a unit of length, while
33]. The light delivery mechanism that we choose the absorption coefficient determines the chance
in each case should address needs of that specific of a photon to be absorbed in a unit of length along
application. its optical path [34]. Both parameters are highly
When light enters tissue, injected photons wavelength-dependent and can change signifi-
bounce from one molecule to another until cantly from one tissue to another. Many research
they get absorbed or find a way out of the groups have conducted experiments to extract
tissue [34]. Using electromagnetic theory to these two parameters for different tissue sam-
model light propagation in a complex random ples over a wide range of the optical spectrum.
environment, such as tissue, is overwhelmingly The advent of optogenetics also inspired many to
complicated and not practical. As a result, develop more detailed databases of brain tissue
most mathematical models of light-tissue optical properties, particularly for small rodents
interaction are phenomenological models. These [35–37]. While none of these models provides
models attempt to formulate an equation that a high-precision estimation for light distribution,
determines the collective destiny of photons it is proven experimentally that predictions of
that enter the tissue, without concentrating these models are reasonable and beneficial. For
on details like the exact phase of photons example, we can use these models to predict the
and diffraction/interference effects. A good required intensity of light to achieve effective
example of this approach is the radiative transport stimulation within a given volume and yet avoid
equation (RTE) which is used as an acceptable heating the tissue to unhealthy levels.
mathematical framework for the development
of several optical tomography scanners [34].
For instance, it is shown that under given 9.3.2 Light-Guiding Systems
conditions, the RTE equation transforms to a
less complex format known as the diffusion The most common method of light delivery
equation. Diffusion of photons is conceptually for optogenetic stimulation, particularly when
similar to the well-known heat transfer process, the target is a deep brain object, is implanting
and diffusion approximation (DA) has become optical fibers in the tissue (see Fig. 9.3) [13, 38].
the mathematical framework for the development Optical fibers act as guiding structures for light
of diffuse optical tomography (DOT) systems. and provide a reliable mechanism to steer the
Statistical methods are also used to find rea- beam from the source to the stimulation site.
sonable estimations of how the light emitted from Optical fibers are highly stable and relatively
a given source propagates inside tissue. In these flexible, and the power loss along the fiber, after
statistical methods (e.g., Monte Carlo simula- the coupling, is negligible. In most experiments,
tion), we use our computational power to trace the electronically controlled laser diodes (LD)
optical path of a large number of photons to find or super-luminescence light-emitting diodes
an approximation for the distribution of light in a (SLED) are sources of light. Usually a simple
given tissue with specified parameters. lens mechanism couples the beam of a light
For all models, we need to parameterize the source to the fiber, and the achieved coupling
medium in which the light is propagating. When efficiency, depending on the source parameters
biological tissue is the substrate for light propaga- and the numerical aperture (NA) of the fiber, is
tion, we usually assign two parameters to define more than 50%.
the complexity of such a random medium. These A simple yet practical formulation was devel-
two parameters are the reduced scattering coef- oped to estimate how deep the beam of light that
ficient and the absorption coefficient. The scat- leaves the tip of an optical fiber penetrates into
Fig. 9.3 (Left) Optical fiber fixed on the skull while the tip is delivering optical power to a target area inside the brain;
(right) the optical fiber assembly installed on the head of a rat
the brain tissue. This equation follows the simple Here, ∅(mm) is the diameter
of the core of
approximation known as Kubelka-Munk model. the optical fiber, NA = n1 − n2 is the fiber’s
2 2
Parameters of the model were adjusted for blue numerical aperture, and n1 and n2 are the refrac-
light and the scattering in the brain of small ro- tive indices of the core and clad of the fiber,
dents [13]. In this model, we assume that the brain respectively. These variables are available in the
is a highly scattering homogeneous medium with manual of any optical fiber. For example, if we
no absorption. The no-absorption assumption is choose a multimode optical fiber of core diameter
quite a stretch for blue wavelengths, but the model and numerical aperture of ∅ = 200 um = 0.2 mm,
still provides acceptable numbers. In this model, NA = 0.22, to guide 10 mW of optical power of a
the intensity of light at depth z(mm) from the 473 nm blue laser to the brain of a Dawley rat,
surface of the fiber and along the fiber’s optical in about 300 um from the fiber tip, the optical
axis, I(z), is estimated by the following equation: power drops to 1 mW which is close to the thresh-
old for effective stimulation of ChR2 proteins.
302 R. Pashaie
Obviously, in case we use the more sensitive light guides where the tip of each light guide
ChR2(H134R) opsin instead, we can stimulate a delivers light at a different depth [42, 43]. In a
larger volume perhaps up to 500 um from the fiber different approach, researchers have considered
tip. using the wide spectral sensitivity of optogenetic
Figure 9.4 shows curves of normalized light proteins to build a light guide which radiates
intensity as a function of penetration depth for different wavelengths of light at different depths
different values of fiber numerical aperture and along the fiber axis. For example, by integrat-
fiber core diameter. Increasing the fiber numerical ing tilted gratings inside the fiber, and tuning
aperture or fiber core diameter causes some each grating to radiate over a small portion of
reduction in the penetration depth since light the spectrum, we can stimulate opsins along the
spreads more horizontally inside the tissue. fiber by switching the wavelength from outside
Therefore, if the light intensity remains the [44]. The use of micro-electromechanical sys-
same, by changing the numerical aperture and tems (MEMS), electro-optic, and acousto-optic
fiber core diameter, it is possible to achieve light switching methods was also studied for op-
the same volume of activation (VoA) with togenetic applications. In such configurations, the
shorter penetration length [39]. To achieve longer user controls the delivery of light to different
penetration depths, we need an optical fiber with depths simply by sending electronic commands to
smaller core diameter and numerical aperture. an array of switches. Considering the complexity
In many experiments, we need to dynami- of such assemblies, it is usually the best practice
cally change the stimulation site and deliver light to choose the simplest technology that is just
at different depths. Researchers have developed enough for a given application.
several devices to address this need. For exam-
ple, it is possible to use a micro-actuator and
move the fiber up and down as needed inside 9.3.3 Spatial Light Modulators
the brain tissue [40, 41]. We can insert a thin
glass-made capillary into the tissue and move the When the cortex is targeted for optogenetic stim-
fiber to minimize damage. Instead of moving one ulation, it is possible to use more complex optics
single fiber, we can use an array of integrated to stimulate large-scaled networks with spatial-
Fig. 9.4 Normalized light intensity as a function of depth for different fiber numerical apertures (a) and different fiber
core diameters (b)
temporal patterns of light delivered from surface. as the source. Nevertheless, in two specific
For such applications, we can use matrices of applications, phase modulation SLMs are useful
light sources, such as LED arrays, or we can use for optogenetic experiments. By combining a
high-resolution spatial light modulators (SLM). phase modulation SLM with a femto-second
Spatial light modulators are optoelectronic de- laser, we can produce holograms in the cortex
vices that spatially and temporally modulate a to generate three-dimensional optogenetic
uniform beam of light. By using SLMs, we can stimulation patterns [47]. The two-photon effect
modulate the amplitude or phase of the beam. A is necessary to increase the wavelength of
good example of amplitude modulation SLM is optogenetic stimulation and reach deeper into the
the digital micro-mirror device (DMD) which is tissue. We use computer-generated holography
a MEMS composed of a two-dimensional array of (CGH) to calculate the two-dimensional phase
bistable programmable micro-mirrors [45]. The modulation pattern that we need to upload on
state of each mirror is independently determined the SLM. When we illuminate the SLM with
to reflect the incident beam toward a target or an the collimated beam of a coherence laser, the
absorber. Therefore, DMD can spatially modulate reflected (or transmitted) beam produces a
a uniform beam of light to project binary patterns three-dimensional distribution of light. Such
on a sample. Since the mirrors are electronically three-dimensional light distribution patterns
controlled, a user can apply pulse-width modu- allow us to stimulate neurons with specific
lation to program the period of time that each spatial-temporal patterns which are essential
mirror projects light on the corresponding sample when we intend to produce perception through
area to generate gray-scaled patterns. In other optogenetic stimulation. To generate perception,
words, a DMD does not change the amplitude of we need to precisely control the way we stimulate
light but instead modulates the exposure which is individual cells within the network. Therefore,
what we need in optogenetic experiments. DMD phase modulation SLMs can open a new era
systems are fast, reliable, and relatively easy to in the study of brain circuits. Unfortunately,
assemble. Nonetheless, the DMD chip plus the most phase modulation SLMs that are available
controlling electronics are bulky, and it is hard to today use liquid crystals, and as a result, they
install such systems directly on the head of small cannot deliver stimulation patterns with temporal
rodents. However, in head-fixed in vivo experi- resolution that we need for many neuroscience
ments, DMDs are good choices to pattern light on experiments. Also, generating three-dimensional
the brain surface and generate complex stimula- patterns of light with a two-dimensional device
tion patterns [32, 46]. Obviously, a pair of orthog- is inherently an ill-posed mathematical problem.
onal mirrors, aka galvanometers, that rapidly scan Therefore, we cannot produce high-resolution
the beam of a collimated light source over the tis- holograms with this approach. Moreover,
sue is another mechanism of patterning light over computer-generated holograms are in many cases
the brain. Once synchronized with the source, different in reality than what computers predict.
the system can project arbitrary spatial-temporal The reason is once again the ill-posedness of the
patterns on the tissue. Many fast, miniaturized problem of CGH, and uniqueness of the solution
and even MEMS-based galvanometers were deis almost never guaranteed. Development of more
veloped in recent years. These devices are great advanced phase modulation SLMs is currently an
choices for the development of complex optoge- active area of research in many micro- and nano-
netic stimulation platforms for freely behaving fabrication labs.
experiments. The second application of phase-modulating
Phase modulation SLMs are another attractive SLMs is in the recently explored approach in
choice for optogenetic stimulation. When which the exposing light is phase modulated in
photons go through multiple random scatterings a certain way to compensate the scattering effect
inside tissue, phase information is almost of the tissue. In this method, which is known
lost even if we use highly coherent lasers as digital optical phase conjugation (DOPC), we
304 R. Pashaie
first scan the sample to learn how light is scattered useful, for example, in treating mental disorders
by particles of the tissue [48–50]. This is usually such as depression.
achieved by using a combination of acousto-optic
and interferometry methods. What we learn from
this step is then used to calculate a phase mod- 9.4 Hybrid Platforms
ulation pattern. A coherent beam that is modu-
lated by this pattern follows the preplanned tra- Optogenetic stimulation is occasionally com-
jectory inside that tissue sample. For example, bined with other optical or nonoptical recording
DOPC can be used to focus light much deeper or imaging methods which help better understand
within the brain tissue. Currently, researchers are the effect of induced stimulations or to dissociate
trying to advance this methodology, for exam- neural circuitries and study their dynamics in
ple, to stimulate hippocampus of a mouse with health or disease condition. Good examples
the phase-modulated beam that is delivered from of such hybrid brain interface platforms are
the surface. While this technology has brought applications where optogenetics is combined
significant excitement to the community, several with electrophysiology, two-photon microscopy,
advances are necessary before considering this optical coherence tomography, or functional
approach as a powerful tool for optogenetic stim- magnetic resonance imaging.
ulation. Reading the scattering of the sample in
DOPC is quite complicated. In addition, the setup
needs meticulous effort for precise calibration. 9.4.1 Optogenetic Neural Probes
Any vibration or misplacement can make the sys-
tem completely dysfunctional. Therefore, using Electrophysiology is perhaps the most popular
this method in vivo, even in head-fixed experi- technique for implementation of brain interface
ments, is currently a major challenge. platforms. Whether we use penetrating electrodes
or surface-mounted arrays, electrophysiology
provides the means to stimulate and record from
9.3.4 Biological Sources for Light the cells. When researchers started developing
optogenetic tools for in vivo applications, they
A novel idea to solve the problem of light de- immediately considered combining optical fibers
livery in optogenetic applications is to use light- or other light-guiding structures with electrodes.
producing proteins that already exist in nature. Electrodes are mainly included to simultaneously
We can use the same gene delivery and target- record from the cells and monitor the effect of
ing method that we use in optogenetics to coex- laser pulses on the activity of cells. Electrodes
press light-producing proteins together with op- that are attached to the fiber can also help
togenetic opsins in the same stimulable cell. A guide the fiber inside the tissue and position
good example of light-producing protein is the the tip to expose the intended brain target.
firefly luciferase which emits yellow light in the For example, in the optogenetic mouse study
presence of its substrate, Luciferin. Fortunately, of Parkinson’s disease, one target area for
mammalian cells can tolerate luciferase with no stimulation was the subthalamic nucleus (STN)
noticeable side effect. The light emitted from [52]. STN is surrounded by the silent zona incerta
luciferase can stimulate NpHR pumps and hyper- (ZI) and internal capsule (IC). Therefore, for
polarize target cells without using any external cannula placement, virus injection, and fiber
source of light [51]. Obviously, in this approach implantation, simultaneous electrophysiology
we are not stimulating or inhibiting cells with was performed. When they attached the electrode
any engineered light pulse train. Therefore, no to the fiber, they specifically measured the
temporal neural coding can be applied in this distance between the fiber tip and the electrode
method. However, this approach is potentially recording site so that when the electrode was
reaching the silence area, the fiber tip was right of transparent electrodes. An example of this
on top of the STN. This is a great example of fabrication procedure is the ultra-flexible carbon-
using electrophysiology as a guidance to deliver layered electrode array ECoG (CLEAR-ECoG)
optogenetic pulses effectively to the preplanned which remains fully transparent over a wide range
coordination. from ultraviolet (UV) to IR wavelengths [54,
Many different configurations of fiber- 55]. Usually, these transparent ECoG devices are
electrode assemblies, aka optorodes, were implanted epidurally on the brain, and the bone is
fabricated and verified. For example, invention of replaced by a cranial window. This combination
lithography methods for curved surfaces allowed has provided the opportunity to combine ECoG
researchers to micro-fabricate electrode arrays recording with optogenetic stimulation and opti-
right on the body of optical fibers. cal imaging techniques.
Development of transparent electrocorticog- Graphene electrodes have been used success-
raphy (ECoG) devices also provided new op- fully for stimulation applications as well, and it
portunities to combine optogenetics and optical is shown that current pulses delivered by these
imaging with electrophysiology. In this attempt, electrodes can trigger calcium waves in the un-
first the substrate of the ECoG device was re- derneath neural circuit [56]. While graphene elec-
placed by the biocompatible transparent polymer trodes solved the transparency problem, the con-
parylene C, while platinum and gold were used ductance of these electrodes is far from optimal.
to fabricate electrodes (see Fig. 9.5) [53]. Such As a result, the signal-to-noise ratio of recordings
platforms achieve close to 90% transparency. The is compromised when the data is recorded by
next step was the development of fully transparent graphene electrodes. Development of better trans-
ECoGs. For this purpose, the main candidate for parent electrode arrays that can effectively com-
electrode fabrication was indium tin oxide (ITO) bine optogenetic stimulation and optical imaging
which is widely used in industry and fabrica- with electrophysiology is currently an ongoing
tion of transparent electronic circuits. ITO does line of research.
not remain fully transparent over a wide range
of spectrum and particularly in the near-infrared
(NIR) and infrared (IR) range, the transparency 9.4.2 Two-Photon Optogenetic
of ITO drops. These wavelengths are important Stimulation
in neuroscience and many optical imaging se-
tups, such as multiphoton microscopy systems Reaching deeper into the brain tissue noninva-
or optical coherence tomography scanners which sively for optical stimulation was a necessity par-
exclusively use these wavelengths. To solve this ticularly in the study of the cortical tissue. The
problem, graphene was used for the fabrication advent of two-photon microscopy (TPM) opened
Fig. 9.5 Transparent ECoG implanted epidurally under cranial window to provide optical access to the tissue for
stimulation and imaging parallel to electrophysiology recording
306 R. Pashaie
new opportunities to record from different lay- stimulation of step function opsins (SFO). Soon
ers of cortex and provided the opportunity to after, it was shown that the chimeric red-shifted
study the dynamics of data manipulation in these opsin C1V1 is a better option for two-photon
circuits or to find projections from one cortical stimulation experiments [17]. Usually, 10–15 ms
region to another. Therefore, from early days after the onset of such laser pulses, the cells under
after the development of optogenetic stimulation, test generate action potentials. It is possible to use
many researchers tried to use their TPM setups more effective scanning methods, as discussed,
and laser pulses of near-infrared femto-second to further reduce the energy of laser pulses or
lasers to stimulate ChR2-positive neurons. Most decrease the average exposure time-before-spike
efforts at that time were not successful since periods. By integrating liquid crystal SLMs, peo-
the cross-section of the retinal molecule for NIR ple have used CGH to generate three-dimensional
wavelengths is too small to form an effective patterns of light produced by femo-second lasers
electron-photon interaction. As a result, pulses of to target multiple cells following some prede-
these lasers, statistically speaking, could not open fined spatial-temporal stimulation patterns. Two-
enough number of channels to depolarize the photon optogenetic tools for inhibition were also
cell and generate action potentials. Some people developed later and added to this toolbox.
tried to increase the energy of laser pulses to see
whether it is possible to trigger action potentials
by exposing cells to light with larger flux of 9.4.3 Optogenetic Stimulation
photons which increases the chance of electron- and Coherence Tomography
photon interaction. Increasing the laser power
was helpful, and action potentials were generated Optical coherence tomography (OCT) is another
in close correlation with light pulses. Nonethe- optical imaging method that is used for brain
less, the required optical energy was so high that imaging. The main application of OCT in brain
the normal function of cells was disrupted by the imaging is perhaps in imaging the vascular net-
generated heat or phototoxicity effects. To find work of the brain and generating high-resolution
a viable method for in vivo experiments, some micro-angiograms [57]. OCT systems can mea-
researchers tested different scanning techniques sure dynamic changes in the physical dimensions
and showed that when the laser follows spiral of vessels, including dilation or contraction, and
trajectories on the cells, the chance of generating estimating the velocity of blood in vessels and
action potentials increases, while the energy of capillaries via Doppler coherence tomography
the laser pulses was reduced marginally. (DCT). Therefore, OCT systems are useful plat-
A better solution to this problem was using forms for measuring cerebral blood flow (CBF)
molecular genetic methods to change kinetic pa- noninvasively [39, 58].
rameters of the opsins. The basic idea in this A combination of optogenetic stimulation
approach was relatively simple. By reducing the and coherence tomography is highly useful for
closing time constant of the channel, it was pos- many studies where the feedback from the hemo-
sible to depolarize the cell further, even when dynamic signals is necessary in experiments.
the cell is exposed to less intense laser pulses A good example of such applications is the
of longer wavelengths. In other words, the tem- study of neurovascular coupling. We can use
poral resolution of optogenetic stimulation was optogenetics to stimulate or suppress the activity
compromised to achieve better light sensitivity. of neurons or even astrocytes as we record the
When each open channel remains open for longer, dynamics of the vascular network and changes in
more ions can pass through the pore to contribute the CBF in real time. Since in many diseases (e.g.,
to the depolarization. Hence, we need to open a stroke, hypertension, or Alzheimer’s) the normal
smaller number of channels to generate an action process of neurovascular coupling is disrupted, by
potential, and light pulses can be less intense. The combining optogenetics with OCT angiography,
first successful test came from the two-photon we can study the source and extent of changes
under these disease conditions in unprecedented genetic and fMRI can become more important in
detail. the future.
Multiple signaling pathways are involved in Optogenetic fMRI has significant potential in
the process of coupling the dynamics of the vas- neuroscience research. Nonetheless, some obsta-
cular networks to neural circuits. One main path- cles and practical challenges have limited the
way starts at neurons and goes to astrocytes and application of this new technology. For example,
then the smooth muscle cells or pericytes that it was shown that the BOLD signal can appear in
control the dilation of vessels and capillaries. fMRI recordings at the site of light delivery (tip of
With optogenetics, we can stimulate neurons as the optical fiber) in animals with no optogenetic
we record from astrocytes (e.g., via two-photon protein expression. Some researchers believe that
calcium imaging) and monitor vascular dynam- such signals are artifacts that are the side effect of
ics by an OCT scanner. We can directly acti- the heat we generate in the tissue when we inject
vate astrocytes, without interfering with neurons, significant amount of optical power. Observations
and measure the difference between the vascular of this nature make it clear that recorded fMRI
dynamics in this condition versus the previous data in response to optogenetic stimulations needs
scenario. We can even use optogenetics to sup- to be interpreted cautiously.
press astrocytes as we stimulate neurons while It is possible to add the power of optical
monitoring vascular dynamics via coherence to- coherence tomography or multispectral [61,
mography. This test shows to which extent the 62] imaging as parallel technologies to this
vascular dynamics is suppressed as a result of combination to better understand details of
astrocyte silencing. This example clearly displays the hemodynamics responses that are recorded
the advantage of optogenetics in clarifying the by the fMRI machines following optogenetic
role of astrocytes in the mechanisms that mediate stimulations. Coherence tomography and
the coupling between the neural and the vascular multispectral imaging can demonstrate the
networks. hemodynamics and metabolic responses of the
tissue in fine details and noninvasively (through
thinned skull in small rodents) in the superficial
9.4.4 Optogenetic Functional areas of the brain. It is possible to arrange
Magnetic Resonance Imaging experiments in which cortical areas of the brain
(ofMRI) is stimulated by optogenetics and imaged by
coherence tomography and multispectral imaging
By combining the cellular control capability of technique, and then use the same animal to repeat
optogenetics with the whole brain imaging power
of fMRI, we can form a new setup which offers
remarkable potential in finding brain long-range
projections [59, 60]. It is possible to stimulate the
cortical tissue via optogenetics and look at the
induced activity in thalamus or hippocampus and
vice versa. Since optical fibers are magnetically
inert, distortion of the field remains negligible
when we install the fiber in the brain of the animal
prepared for fMRI imaging (see Fig. 9.6). Also,
the effect of the intense magnetic field within the
core of the MRI machine on the performance of
optical fibers is negligible. As a result, optogenet-
ics and fMRI are compatible technologies. Since Fig. 9.6 A rat placed in the MRI machine with the optical
fiber assembly to stimulate the brain during the functional
the applications of optogenetics is moving from
imaging session
small rodents to primates, combination of opto-
308 R. Pashaie
the experiment with fMRI. The comparison Another troubling feature of gene delivery for
between these two datasets can help researchers optogenetic applications is that the expression
better understand the nature and origin of the of the protein usually remains limited to the
BOLD signal. immediate area surrounding the injection site.
Laser pulses that are delivered for optogenetic Therefore, to target a large area for effectively
stimulation generate heat in the tissue. The intervention in human brain, the virus should
material that absorbs light and generates heat be injected at multiple locations, which adds
the most in the tissue is blood. However, blood to the invasiveness of the method. Therefore,
moves, and this motion helps dissipate the heat we need appropriate gene delivery mechanisms
at the same time. Finding some reasonable to make the method of optogenetics more
estimation of heat generation and dissipation in acceptable for human trials. Fortunately, the rate
the brain is necessary and even essential for many of progress in the field of molecular genetics is
applications. Ongoing studies that concentrate significant nowadays, and many other genetic-
on light absorption and heat generation in the based methods with potentials for therapeutic
brain tissue will ultimately help unravel some applications are introduced every year. A
of these mysteries and will open the field for good example of such methods is the gene-
more advanced optogenetic fMRI applications in editing technology known as CRISPR (clustered
future. regularly interspaced short palindromic repeats).
CRISPR and other parallel technologies will help
in opening the field for human trial of optogenetic
9.5 Optogenetic Stimulation for stimulation in future.
Therapy A third challenge is the need for advanced light
delivery mechanisms. In many applications of
An important question is whether optogenetics optogenetics, we only need to stimulate one area
has the potential for application in clinics and of the brain without any major concern regarding
treatment of mental diseases. Optogenetics of- the exact spatial-temporal stimulation pattern. An
fers some unique features that make the method example of that is the treatment of depression or
highly desirable for therapeutic procedures. For anxiety. However, if we want to use optogenetic
example, the cell-type targeting aspect of op- stimulation in the visual cortex to generate arti-
togenetics can be used to minimize some po- ficial vision, which is currently one of the most
tential side effects of many interventional ther- exciting applications of the technology, we need
apeutic procedures in which we currently use to develop methods to stimulate a large number of
electrodes that stimulate all cells in the region neurons by precisely engineered spatial-temporal
without specificity. In theory, optogenetics can stimulation patterns. In other words, we need to
replace electrode-based brain interface platforms generate high-resolution and dynamically chang-
similar to the deep brain stimulation assemblies ing holograms within the visual cortex which is
used in Parkinsonian patients. Nonetheless, the a major engineering challenge. It is possible to
main obstacle for pushing optogenetics toward target less complicated neural networks to restore
clinical translations is the requirement for gene vision with optogenetics and avoid the complex-
delivery. Most patients, quite understandably, do ity of the brain stimulation if the patient condition
not want to be the recipient of viral injections in allows. For example, currently the main target for
their brain or other parts of the nervous system. the first application of optogenetics in humans is
Before pushing for clinical applications, side ef- the retina. Blindness in many eye diseases is the
fects of such gene therapy procedures should be result of losing a large number of photoreceptor
studied extensively so that healthcare providers cells. Since three other major cell populations
know when to avoid such procedures or provide (ganglion, bipolar, and Amacrine cells) relay the
some assurance for patients when the condition signal from photoreceptors to the optic nerve, one
allows. possibility is to express ChR2 in one of these
other cell populations and make them function as (4) Compensating or bypassing dys-
receptors of light. We can use the rest of the visual functional brain circuits by utilizing
system to generate the perception. Currently, we prosthetic devices
use electronic retinal implants in humans, and 2. What item is not a main limitation in
the number of photodetectors in such implants electrode-based stimulation?
is very limited. The acquired vision remains less (1) Targeting specific cell population(s)
than 10% of the normal state. The new approach, (2) Stimulating without causing significant
based on optogenetics, can potentially generate side effect(s)
vision with much better quality. This is an area (3) Spatial resolution for generating com-
of ongoing high-risk yet high-reward research. plex stimulation pattern(s)
Optogenetics can also contribute to medicine (4) Stimulate inhibitory neurons
indirectly by helping researchers study neurolog- 3. Which statement is accurate about optoge-
ical and psychiatric disorders and dissociate the netic technology?
circuitry involved in such diseases and better un- (1) Optogenetic stimulation can only gener-
derstand the contribution of each cell population ate low-frequency firing patterns
in the dynamics of each disease. Since human (2) Better light sensitivity for optogenetic
trials are complicated and need to be conducted stimulation was achieved mainly by sac-
with ultimate attention to details and following rificing the temporal resolution of well-
many scientific and ethical rules, studying the known optogenetic proteins
neural circuitry of diseases in animal models is (3) Optogenetics does not offer any strategy
the area that is moving forward rapidly these days. to reduce side effects of interventional
Ultimately, when safer gene and light delivery procedures
methods are invented, the result of these research (4) Optogenetics is a method in which op-
endeavors will contribute to the development of tics and molecular genetics are combined
new procedures in clinics. to develop a versatile method for neural
stimulation and recording
Acknowledgments The author’s work is currently sup- 4. In in vivo optogenetic stimulation:
ported by the Army Research Office (ARO) grant# 71227- (1) Shorter visible wavelengths (e.g., blue
LS, the National Science Foundation (NSF) CAREER
Award grant# 1454300, the NSF grant# 1830145, and the
light) can penetrate deeper into the brain
University of Wisconsin Research Growth Initiative (RGI) tissue compared to longer wavelengths
grant# 101X378. (e.g., red or infrared light)
(2) It is not possible to coexpress excitatory
and inhibitory opsins in the same cell
Homework population for in vivo experiments
(3) The heat generated by light absorption is
1. What is the main aspect of interventional negligible and cannot affect the outcome
psychiatry? of experiments
(1) The science and practice of using neuro- (4) Gene delivery is one challenge that has
technologies to identify dysfunctional limited the application of optogenetics in
brain networks and utilizing neuro- primates
modulation methods to help restore 5. Which statement is not correct?
normal functionality of the circuits (1) By overexpressing opsins, we can
(2) Prescribing appropriate dosage of achieve better light sensitivity
medicine or chemicals to help restore (2) Engineering opsins with red-shifted
normal function of brain circuits spectral sensitivity can allow stimulating
(3) Using personalized medicine technology deeper areas of cortex optogenetically
to make interventional procedures more without implanting light guides into the
effective tissue
310 R. Pashaie
(3) Expression of opsins always leads to 9. For effective two-photon optogenetic stimu-
membrane trafficking complications in lation, the acceptable solution comes from:
host cells (1) Producing opsins with longer closing
(4) ChR2 mutants are introduced which al- time constants
low influx of Ca2+, and one can use (2) Increasing the optical power of the light
these proteins to trigger signaling path- pulses
ways optically (3) Use spatial light modulators to illuminate
6. What is the main advantage of adeno- cells with complex patterns
associate virus (AAV) compared to Lentivirus (4) All the above
(LV) for gene delivery in optogenetics? 10. What are some of the motivations for pro-
(1) Permanent expression of protein ducing or discovering opsins with diverse
(2) Integration to the host genome spectral sensitivity?
(3) AAV is less immunogenetic (1) Combining optogenetic stimulation with
(4) Higher expression rate optical recording methods such as cal-
7. How can we increase light penetration depth cium imaging.
in the cortex for optogenetic stimulation? (2) Expressing opsins of different spectral
(1) Increase fiber’s numerical aperture but sensitivity in different cell populations
reduce fiber’s core diameter and using the wavelength diversity to
(2) Reduce fiber’s numerical aperture and individually target each population for
reduce fiber’s core diameter stimulation
(3) Increase fiber’s numerical aperture and (3) Producing opsins with sensitivity to
increase fiber’s core diameter longer wavelengths to minimize the
(4) Reduce fiber’s numerical aperture but in- effect of light scattering inside the tissue
crease fiber’s core diameter (4) All the above
8. In in vivo light delivery:
(1) Holographic optogenetic stimulation
provides the opportunity to generate References
precise spatial-temporal stimulation
patterns which is an important step 1. J. Gardner, A history of deep brain stimulation: Tech-
nological innovation and the role of clinical assess-
toward generating perception via ment tools. Soc. Stud. Sci. 43(5), 707–728 (2013)
optogenetic stimulation 2. F. Zhang, A.M. Aravanis, A. Adamantidis, L. de
(2) A single optical fiber can deliver many Lecea, K. Deisseroth, Circuit-breakers: Optical tech-
wavelengths but only one wavelength at nologies for probing neural signals and systems. Nat.
Rev. Neurosci. 8, 577–581 (2007a)
a time 3. F. Zhang, L.-P. Wang, M. Brauner, J.F. Liewald, K.
(3) Amplitude-modulated spatial light Kay, N. Watzke, P.G. Wood, E. Bamberg, G. Nagel,
modulators combined with femto-second A. Gottschalk, K. Deisseroth, Multimodal fast optical
lasers are used to generate holograms interrogation of neural circuitry. Nature 446, 633–639
(2007b)
inside cortical tissue 4. E.S. Boyden, F. Zhang, E. Bamberg, G. Nagel, K.
(4) Amplitude and phase modulation spa- Deisseroth, Millisecond-timescale, genetically tar-
tial light modulators are used to gener- geted optical control of neural activity. Nat. Neurosci.
ate complex stimulation patterns on the 8, 1263–1268 (2005)
5. K. Deisseroth, Optogenetics. Nat. Methods 8(1), 26–
surface of the brain, and the spatial reso- 29 (2011)
lution of the pattern is preserved as light 6. L. Fenno, O. Yizhar, K. Deisseroth, The development
penetrates into the tissue and application of optogenetics. Ann. Rev. Neurosci.
34, 389–412 (2011)
7. O. Yizhar, L.E. Fenno, T.J. Davidson, M. Mogri, K. 21. H.E. Kato, F. Zhang, O. Yizhar, C. Ramakrishnan, T.
Deisseroth, Optogenetics in neural systems. Neuron Nishizawa, K. Hirata, J. Ito, Y. Aita, T. Tsukazaki, S.
71(1), 9–34 (2011) Hayashi, P. Hegemann, A.D. Maturana, R. Ishitani,
8. R. Pashaie, Distributed light delivery and detection K. Deisseroth, O. Nureki, Crystal structure of the
via single optical fiber and tilted grating. J. Mod. Opt. channelrhodopsin light-gated cation channel. Nature
16(6), 518–529 (2014) 482(7385), 369–374 (2012)
9. J.E. Brody, A Strange Bacteria’s Purple Pigment, 22. M. Figueiredo, S. Lane, F. Tang, B.H. Liu, J. Hewin-
Which Uses Light to Generate Energy, may Yield son, N. Marina, V. Kasymov, E.A. Souslova, D.M.
Scientific Goldmine, New York Times, (5 Jul 1977), Chudakov, A.V. Gourine, A.G. Teschemacher, S. Kas-
p. 17 parov, Optogenetic experimentation on astrocytes.
10. V. Gradinaru, K.R. Thompson, F. Zhang, M. Mogri, Exp. Physiol. 96(1), 40–50 (2011)
K. Kenneth, M. Bret Schneider, K. Deisseroth, Tar- 23. L. Naldini, U. Blömer, P. Gallay, D. Ory, R. Mulli-
geting and readout strategies for fast optical neural gan, F.H. Gage, I.M. Verma, D. Trono, In vivo gene
control in vitro and in vivo. J. Neurosci. 27(52), delivery and stable transduction of nondividing cells
14231–14238 (2007) by a lentiviral vector. Science 272(5259), 263–267
11. K. Deisseroth, Optogenetics: 10 years of microbial (1996)
opsins in neuroscience. Nat. Neurosci. 18(9), 1213– 24. T. Dittgen, A. Nimmerjahn, S. Komai, P. Licznerski,
1225 (2015) J. Waters, T.W. Margrie, F. Helmchen, W. Denk, M.
12. J. Mattis, K.M. Tye, E.A. Ferenczi, C. Ramakrish- Brecht, P. Osten, Lentivirus-based genetic manipula-
nan, D.J. O’Shea, R. Prakash, L.A. Gunaydin, M. tions of cortical neurons and their optical and electro-
Hyun, L.E. Fenno, V. Gradinaru, O. Yizhar, K. Deis- physiological monitoring in vivo. Proc. Natl. Acad.
seroth, Principles for applying optogenetic tools de- Sci. U. S. A. 101(52), 18206–18211 (2004)
rived from direct comparative analysis of microbial 25. P.E. Monahan, R.J. Samulski, AAV vectors: Is clin-
opsins. Nat. Methods 9(2), 159–172 (2011) ical success on the horizon? Gene Ther. 7(1), 24–30
13. A.M. Aravanis, L.-P. Wang, F. Zhang, L.A. Meltzer, (2000)
M.Z. Mogri, M.B. Schneider, K. Deisseroth, An opti- 26. H. Zeng, L. Madisen, Mouse transgenic approaches in
cal neural interface: In vivo control of rodent motor optogenetics. Prog. Brain Res. 196, 193–213 (2012)
cortex with integrated fiber optic and optogenetic 27. M.E. Llewellyn, K.R. Thompson, K. Deisseroth, S.L.
technology. J. Neural Eng. 4, S143–S156 (2007) Delp, Orderly recruitment of motor units under opti-
14. F. Zhang, M. Prigge, F. Beyrière, S.P. Tsunoda, J. cal control in vivo. Nat. Med. 16, 1161–1165 (2010)
Mattis, O. Yizhar, P. Hegemann, K. Deisseroth, Red- 28. A.M. Leifer, C. Fang-Yen, M. Gershow, M.J. Alkema,
shifted optogenetic excitation: A tool for fast neural A.D.T. Samuel, Optogenetic manipulation of neural
control derived from Volvox carteri. Nat. Neurosci. activity in freely moving Caenorhabditis elegans. Nat.
11(6), 631–633 (2008) Methods 8(2), 147–152 (2011)
15. J.G. Bernstein, P.A. Garrity, E.S. Boyden, Optogenet- 29. J.H. Simpson, L.L. Looger, Functional imaging and
ics and thermogenetics: Technologies for controlling optogenetics in drosophila. Genetics 208(4), 1291–
the activity of targeted cells within intact neural cir- 1309 (2018)
cuits. Curr. Opin. Neurobiol. 22(1), 61–71 (2012) 30. J. Simmich, E. Staykov, E. Scott, Zebrafish as an
16. A. Berndt, O. Yizhar, L.A. Gunaydin, P. Hegemann, appealing model for optogenetic studies. Prog. Brain
K. Deisseroth, Bi-stable neural state switches. Nat. Res. 196, 145–162 (2012)
Neurosci. 12(2), 229–234 (2009) 31. I. Diester, M.T. Kaufman, M. Mogri, P. Ramin, W.
17. R. Prakash, O. Yizhar, B. Grewe, C. Ramakrishnan, Goo, O. Yizhar, C. Ramakrishnan, D. Karl, K.V.
N. Wang, I. Goshen, A.M. Packer, D.S. Peterka, R. Shenoy, An optogenetic toolbox designed for pri-
Yuste, M.J. Schnitzer, K. Deisseroth, Two-photon mates. Nat. Neurosci. 14(3), 387–397 (2011)
optogenetic toolbox for fast inhibition, excitation 32. R. Pashaie, T. Richner, R. Baumgartner, S. Brodnick,
and bistable modulation. Nat. Methods 9, 1171–1179 M. Azimipour, K. Eliceiri, J. Williams, Closed-loop
(2012) optogenetic brain Interface. IEEE Trans. Biomed.
18. L.A. Gunaydin, O. Yizhar, A. Berndt, V.S. Sohal, Eng. 62(10), 2327–2337 (2015)
K. Deisseroth, P. Hegemann, Ultrafast optogenetic 33. L. Grosenick, J.H. Marshel, K. Deisseroth, Closed-
control. Nat. Neurosci. 13(3), 387–392 (2010) loop and activity-guided optogenetic control. Neuron
19. V. Gradinaru, K.R. Thompson, K. Deisseroth, eN- 86(1), 106–139 (2015)
pHR: A Natronomonas halorhodopsin enhanced for 34. T. Vo-Dinh, Biomedical Photonics Handbook (CRC
optogenetic applications. Brain Cell Biol. 36(1–4), Press, ISBN-10: 0849311160, 2003)
129–139 (2008) 35. M. Azimipour, R. Baumgartner, Y. Liu, S. Jacques, K.
20. F. Schneider, D. Gradmann, P. Hegemann, Ion se- Eliceiri, R. Pashaie, Extraction of optical properties
lectivity and competition in channelrhodopsins. Bio- and prediction of light distribution in rat brain tissue.
phys. J. 105(1), 91–100 (2013) J. Biomed. Opt. 19(7), 075001 (2014)
312 R. Pashaie
36. M. Azimipour, F. Atry, R. Pashaie, Effect of blood 50. M. Azimipour, F. Atry, R. Pashaie, Calibration of
vessels on light distribution in optogenetic stimula- digital optical phase conjugation setups based on
tion of cortex. Opt. Lett. 40(10), 2173–2176 (2015) Zernike decomposition analysis. Appl. Opt. 55(11),
37. Y. Liu, S. Jacques, M. Azimipour, J. Rogers, R. 2873–2880 (2016)
Pashaie, K. Eliceiri, OptogenSIM: A 3D Monte Carlo 51. B.B. Land, C.E. Brayton, K.E. Furman, Z. LaPalom-
simulation platform for light delivery design in op- bara, R.J. DiLeone, Optogenetic inhibition of neurons
togenetics. Biomed. Opt. Express 6(12), 4859–4870 by internal light production. Front. Behav. Neurosci.
(2015) 8(108) (2014)
38. C. Pawela, E. DeYoe, R. Pashaie, Intracranial in- 52. V. Gradinaru, M. Mogri, K.R. Thompson, J.M. Hen-
jection of an optogenetics viral vector followed by derson, K. Deisseroth, Optical deconstruction of
optical cannula implantation for neural stimulation in Parkinsonian neural circuitry. Science 324(5925),
rat brain cortex. In: Kianianmomeni A. (eds) Optoge- 354–359 (2009)
netics. Methods in Molecular Biology, vol 1408, pp 53. T.J. Richner, S. Thongpang, S.K. Brodnick,
227–241, Humana Press, New York, NY (2016) A.A. Schendel, R.W. Falk, L.A. Krugner-Higby,
39. F. Atry, R. Chen, J. Pisaniello, S. Brodnik, A. Sum- R. Pashaie, J.C. Williams, Optogenetic micro-
inski, J. Novello, J. Ness, J. Williams, R. Pashaie, electrocorticography for modulating and localizing
Optogenetic interrogation and manipulation of vascu- cerebral cortex activity. J. Neural Eng. 11, 016010
lar blood flow in cortex. J. Neural Eng. 15, 056033 (2014)
(2018b) 54. A.A. Schendel, D.-W. Park, S. Mikael, T.J. Richner,
40. R. Pashaie, R. Falk, Single optical fiber probe for S.K. Brodnick, F. Atry, S. Frye, R. Pashaie, S. Thong-
fluorescence detection and optogenetic stimulation. pang, J.C. Williams, Z. Ma, Carbon layered electrode
IEEE Trans. Biomed. Eng. 60(2), 268–280 (2013) array (CLEAR) technology for neural imaging and
41. I. De La Rosa, M. Azimipour, P. Cullen, F. Helm- optogenetic applications. Nat. Commun. 5(5258), 1–
stetter, R. Pashaie, Angularly resolved deep brain 11 (2014)
fluorescence imaging using a single optical fiber. Int. 55. D.-W. Park, S. Brodnick, J. Ness, F. Atry, L. Krugner-
J. Optics, 10 (2018)., Article ID 8903413 Higby, A. Sandberg, S. Mikael, T. Richner, J. Novello,
42. A. Zorzos, E. Boyden, C. Fonstad, Multiwaveguide H. Kim, D.-H. Baek, J. Bong, C. Nguyen, S. Frye,
implantable probe for light delivery to sets of dis- S. Thongpang, K. Swanson, W. Lake, R. Pashaie, J.
tributed brain targets. Opt. Lett. 35(24), 4133–4135 Williams, Z. Ma, Fabrication and utility of a transpar-
(2010) ent graphene neural electrode array for electrophysi-
43. A. Zorzos, J. Scholvin, E. Boyden, C. Fonstad, Three- ology, in vivo imaging, and optogenetics. Nat. Protoc.
dimensional multiwaveguide probe array for light de- (2016). https://doi.org/10.1038/nprot.2016.127
livery to distributed brain circuits. Opt. Lett. 37(23), 56. D.-W. Park, J.P. Ness, S.K. Brodnick, C. Esquibel,
4841–4843 (2012) J. Novello, F. Atry, D.-H. Baek, H. Kim, J. Bong,
44. R. Pashaie, P. Anikeeva, J.H. Lee, R. Prakash, K.I. Swanson, A.J. Suminski, K.J. Otto, R. Pashaie,
O. Yizhar, M. Prigge, D. Chander, T. Richner, J. J.C. Williams, M.A. Zhenqiang, Electrical neural
Williams, Optogenetic brain interfaces. IEEE Rev. stimulation and simultaneous in vivo monitoring
Biomed. Engg. 7, 3–30 (2014) with transparent graphene electrode arrays implanted
45. D. Dudley, W. Duncan, J. Slaughter, Emerging digital in GCaMP6f mice. ACS Nano 12(1), 148–157
micromirror device (DMD) applications. Proc. SPIE (2018)
4985, 14–25 (2003) 57. F. Atry, I. De La Rosa, K. Rarick, R. Pashaie, De-
46. T.J. Richner, R. Baumgartner, S.K. Brodnick, K.W. sign and implementation guidelines for a modular
Eliceiri, J.C. Williams, R. Pashaie, Patterned opto- spectral-domain optical coherence tomography scan-
genetic modulation of neurovascular and metabolic ner. Int. J. Optics, 22 (2018a)., Article ID 3726207 22
signals. J. Cereb. Blood Flow Metab. 35, 140–147 pages https://doi.org/10.1155/2018/3726207
(2015) 58. F. Atry, S. Frye, T.J. Richner, S.K. Brodnick, A. Soe-
47. V. Nikolenko, B.O. Watson, R. Araya, A. Woodruff, hartono, J. Williams, R. Pashaie, Monitoring cerebral
D.S. Peterka, R. Yuste, SLM microscopy: Scanless hemodynamics following optogenetic stimulation via
two-photon imaging and photostimulation with spa- optical coherence tomography. IEEE Trans. Biomed.
tial light modulators. Front Neural Circuits 2, 5 Eng. 62(2), 766–773 (2015)
(2008) 59. J.H. Lee, R. Durand, V. Gradinaru, F. Zhang, I.
48. Y.M. Wang, B. Judkewitz, C.A. Dimarzio, C. Yang, Goshen, D.-S. Kim, L.E. Fenno, C. Ramakrishnan, K.
Deep-tissue focal fluorescence imaging with digitally Deisseroth, Global and local fMRI signals driven by
time-reversed ultrasound encoded light. Nat. Com- neurons defined optogenetically by type and wiring.
mun. 3, 928 (2012) Nature 465(7299), 788–792 (2010)
49. X. Xu, H. Liu, L.V. Wang, Time-reversed ultrasoni- 60. J.H. Lee, Tracing activity across the whole brain
cally encoded optical focusing into scattering media. neural network with optogenetic functional magnetic
Nat. Photonics 5, 154–157 (2011) resonance imaging. Front. Neuroinform. 5, 21 (2011)
61. A. Dunn, A. Devor, H. Bolay, M. Andermann, M. 62. A. Dunn, A. Devor, A. Dale, D. Boas, Spatial extent
Moskowitz, A. Dale, D. Boas, Simultaneous imaging of oxygen metabolism and hemodynamic changes
of total cerebral hemoglobin concentration, oxygena- during functional activation of the rat somatosensory
tion, and blood flow during functional activation. Opt. cortex. NeuroImage 27, 279–290 (2005)
Lett. 28(1), 28–30 (2003)
Selective Chronic Recording in the
Peripheral Nervous System 10
Dominique M. Durand and Thomas Eggers
Abstract (FINE), which has been shown to be reliable.

It has been implanted in human patients for
Reliable interfacing with the peripheral ner-
several years and can provide safe nerve stimu-
vous system has been and still remains a dif-
lation for sensory substitution and has not been
ficult problem to solve. Yet the ability to ob-
shown to record signals useful for functional
tain signals from peripheral nerves would have
recovery.
significant benefits such as detection of motor
There are many issues specific to chronic
intent in patients with amputation. Similarly
recordings that will be discussed. One major
decoding signals from the autonomic nervous
problem is EMG contamination, and several
system would allow continuous monitoring of
approaches to deal with this high amplitude
organ function.
signal will be discussed. Another major is-
However, there are many problems that pre-
sue is the signal-to-noise ratio. The design
vent reliable signal detection in chronic ani-
of ultralow-noise amplifiers particularly well
mals and human patients. One of the problems
suited for ENG recording will be discussed.
is that axons are arranged in tightly packed
Another important issue deals with the recov-
bundles surrounded by membranes that are
ery of fascicular signals from mixed signals
difficult to penetrate. Therefore, access to the
generated by multiple fascicles active simul-
signals is challenging, and neural engineers
taneously. Various algorithms capable of ex-
have designed many types of electrodes to ad-
tracting and separating fascicular signals will
dress this issue. In this chapter, we will review
be discussed.
the various types of neural interfaces such
Finally, the combination of finite element
as cuff electrodes, intra- and extrafascicular
modeling and computation neuroscience al-
electrodes, as well as regeneration electrodes.
lows accurate models of nerve bundles and
We will focus in one particular type of
recording electrodes. These models can pro-
electrode, the flat interface nerve electrode
vide important information about the band-
width required for accurate ENG detection
and the effect of the axonal diameter on the
Electronic Supplementary Material: The online version recorded signals and can lead to the design of
of this chapter (https://doi.org/10.1007/978-3-030-43395- improved peripheral nerve interfaces.
Keywords
D. M. Durand () · T. Eggers
Neural Engineering Center, Department of Biomedical Neural interfacing · Cuff electrodes · Neural
Engineering, Case Western Reserve University, recording · Separation algorithm · Low-noise
Cleveland, OH, USA
amplifier · Computer modeling

316 D. M. Durand and T. Eggers
Abbreviations tween the recording electrode and the neural tis-

sue. However, it is not known if extrafascicular
Beam Beamforming cuff electrodes can produce neural recordings
BF Biceps femoris that are highly selective for multiple fascicles
BSFE Bayesian spatial filter extraction with high SNR because they detect weak, local
BW Bandwidth field potentials outside the electrically insulating
CC Correlation coefficient perineurium layer [6]. Intrafascicular techniques
DoFs Degrees of freedom could also provide additional capability if shown
EEG Electroencephalogram to be safe and reliable. The three most preva-
EMG Electromyogram lent designs for intrafascicular interface are as
ENG Electroneurogram follows: (1) the longitudinal intrafascicular elec-
FEM Finite element model trode (LIFE) [7], which is sewn into the nerve
FINE Flat interface nerve electrode along its length; (2) the transverse intrafascic-
GN Gastrocnemius ular multichannel electrode (TIME) [8], which
HBSE Hybrid Bayesian signal extraction penetrates through the nerve’s cross-section; and
ICA Independent component analysis (3) the Utah Slanted Electrode Array (USEA)
PNS Peripheral nervous system [9], which is an array of multiple, cone-shaped,
SIR Signal-to-interference ratio needle structures that penetrate the fascicles. Al-
SNR Signal-to-noise ratio though these intrafascicular electrodes can ini-
ST Semitendinosus tially record high SNR neural signals [10, 11]
TA Tibialis anterior and can be quite selective [12], they have yet
to demonstrate chronic implant reliability, possi-
bly because the electrode flexural rigidity is at
10.1 Introduction least five orders of magnitude greater than the
surrounding neural tissue [13, 14]. In particular,
Interfacing directly with the nervous system of- the TIME implanted through the epineurium does
fers the ability to recover motor intent as well as not easily penetrate the perineurium of the fasci-
restore sensation with a single device to people cles and settles between fascicles [15]. It can be
who have amputated limbs. Early work in the implanted inside fascicles but requires a needle
field showed that residual nerves (and thus cor- insertion method similar to the LIFE electrode
tical areas) retained functional motor and sensory [12, 16]. Another approach to neural interfacing is
connections even years after amputation [1]. Di- to take advantage of the regeneration potential of
rect cortical interfacing has been explored, but it cut axons. Axons are induced to regrow near the
is mostly limited to para- and quadriplegics due electrodes placed within a nerve, and the system
to its surgically invasive nature. Peripheral nerve is designed to provide intimate contact between
interfacing is less invasive and has been studied many axons for recording and stimulation [17].
extensively [2]. Extrafascicular approaches such Several groups have designed electrode interfaces
as nerve cuffs have demonstrated long-term sta- with severed nerves with so far promising but lim-
bility and have been used in human trials [3], ited success. Two such approaches are the TEENI
most notably for stimulation to restore sensation [18] and microchannels [19]. These approaches
[4]. Cuff electrodes are the most widely used are quite promising as they provide access to
neural interface type based on their safety com- many axons simultaneously. Another promising
pared to other neural interface implementations, approach is the reinnervation of a nerve or nerve
ease of implantation, and clinical results from fascicle into a muscle (targeted muscle reinner-
nerve stimulation trials [4, 5]. A true, feedback- vation) [20] or small parts of muscle (regenera-
based neuromodulation system must be able to tive peripheral nerve interfaces) [21]. Although it
successfully record and process neural activity can be difficult to control which and how many
with a high signal-to-noise ratio (SNR), which fibers will innervate successfully and make func-
requires a stable, low electrical impedance be- tional connections to the muscle, the SNR of the
10 Selective Chronic Recording in the Peripheral Nervous System 317
Fig. 10.1 Interfacing with peripheral nerves: Peripheral or stimulate. Interfascicular electrodes can be placed either
nerves are made up of large numbers of axons (ax) grouped longitudinally (b) or transversely (c). (d) The flat interface
into fascicles each surrounded by a perineurium mem- nerve electrode is a cuff with multiple contact that main-
brane (p) and bundled by epineurium (epi) membrane. (a) tains the nerve into a flat shape. (d) Silicon probe can be
Cuff electrodes can be placed around the nerve to record placed directly into the nerve
recorded signal is high since it is obtained from and computer modeling of the nerve-electrode
the muscle fibers. interface.
One of the goals of the development of these Severe limb injuries have highlighted the need
technologies is to detect motor intent in order to for more robust upper limb prosthetic systems.
control motor prostheses (Fig. 10.1). Despite having developed advanced upper arm
prostheses capable of replicating most of the
arm’s natural range of movement, many patients
10.2 Movement Intent Recovery today still use the traditional passive or cosmetic
prostheses [23]. The reason the currently adopted
Recording from peripheral nerves has long been prostheses lag behind the state of the art is that
motivated by the field of prosthetic control, i.e., a robust control mechanism for such limbs is yet
recovering motor signals from residual nerves to be established [24]. In order to control such
in order to control a prosthetic limb. For the prostheses, a neural interface must be able to
remainder of this chapter, we will focus on the estimate the motor intent of the patient reliably
cuff electrode approach since it has generated the over long periods of time (i.e., years to decades).
longest chronic results for both selective stimula- The most advanced commercial interface for
tion and selective recording [4, 22]. In particular, upper limb prostheses is the EMG-controlled or
we will review below some of the advances for a myoelectric device. This technique records mus-
specific type of neural interface, the flat interface cle activity of residual muscles which drives the
nerve electrode (FINE), as used to chronically prosthesis. This approach works well for trans-
record this information. We will focus on the ap- radial amputees, as many of the muscles which
plication of extracting motor signals from nerves once innervated the missing hand are still intact in
from the FINE with special emphasis on low- the forearm. However, as the level of amputation
noise amplification, signal processing algorithms, increases, the system must replace more functions
EMG contamination, bandwidth consideration,
with fewer available control sources. Researchers lation/recordings by spatially separating the fasci-
have attempted to use advanced signal process- cles. Previous research has demonstrated the abil-
ing techniques, such as pattern recognition and ity of the FINE to selectively record from differ-
machine learning, to map the low number of ent fascicles in in silico studies and in acute prepa-
muscles to the higher number of degrees of free- rations in rabbits, as well as two recent chronic
dom (DoF) of the prosthesis [25]. While showing studies in canines in which successful binary
some promise, these control schemes have proved classification of fascicle activity was achieved
unsuccessful in clinical acceptance, as they gener- [22]. The information transfer of the peripheral
ally only produce state-based control and may not interface was also measured [34]. The cuff ability
be robust enough for at-home use [26]. The more to record selectively over long periods of time
widely utilized control scheme for such high- has been established and can be modeled to pro-
level amputee patients involves using the biceps vide an improved understanding of its capabil-
and triceps to sequentially control each DoF; ity. However, the combination of low-amplitude
this scheme is highly robust but produces slow, signals and higher electrode source resistance
disjointed movements. Thus specifically for the requires novel amplifier designs.
trans-humeral amputee population, an alternative
control source is desirable.
The ideal motor interface would be intuitive 10.3 High SNR Amplification
to learn and allow simultaneous and proportional of Neural Signals
control of many DoFs. The brain naturally gen-
erates these types of signals and sends them to Peripheral nerves carry neural signals that could
the arm via the peripheral nerves. However, the be used to control hybrid bionic systems. Cuff
peripheral nervous system carries the processed electrodes provide a robust and stable interface,
motor commands and is easier and less risky to but the recorded signal amplitude is small
surgically access. Previous studies have shown (<3 μVRMS 700 Hz–7 kHz), thereby requiring
that both the natural sensory and motor fibers a baseline noise of less than 1 μVRMS for a
which once innervated the amputated limb re- useful signal-to-noise ratio. Flat interface nerve
main intact and functional, even many years after electrode (FINE) contacts alone generate thermal
the amputation [1]. Several studies since have noise of at least 0.5 μVRMS; therefore, the
demonstrated the ability to restore sensation of amplifier should add as little noise as possible
the missing hand as well as control a three-grip [22]. Since mainstream neural amplifiers have
hand prosthesis [4, 27, 28]. a baseline noise of 2 μVRMS or higher, novel
The least invasive technique for directly inter- designs are required. The concept of hardware
facing with peripheral nerves involves wrapping averaging to nerve recordings was applied
electrodes around the nerve without penetrating to obtain high SNRs from cuff electrodes.
any neural tissue. Standard cuff electrodes have Optimization procedures have been developed
shown great stability over time with both record- to minimize noise and power simultaneously
ing and stimulating in humans [3, 4, 29, 30], (Fig. 10.2).
although they offer limited selectivity and small The novel design implementation is based on
signal amplitudes for recording. The flat interface existing neural amplifiers (Intan Technologies,
nerve electrode (FINE) is a nerve cuff devel- LLC) and has been validated with signals ob-
oped to address these issues [31–33]. The FINE tained from the FINE in chronic dog experiments.
aims to reshape or maintain the nerves in a flat It was shown that hardware averaging leads to a
configuration, increasing the surface area to vol- reduction in the total recording noise by a factor
√
ume ratio. The decreased distance between fibers of 1/ N or less depending on the source resis-
and contacts with this configuration increases the tance. Chronic recording of physiological activity
recorded SNR as well as the selectivity of stimu- with FINE using this design showed significant
Fig. 10.2 Noise model of hardware averaging for FINE to N parallel instrumentation amplifiers. Averaging the
recording. (a) A cartoon of FINE structure and placement outputs can be performed by a cascaded averaging stage
around a flattened nerve. The potential difference between or off-line. Zwe is much smaller than Zin (<10 k in com-
the center contact (we) and the shorted outer reference parison to 13 M for the investigated devices); hence, the
contacts (re) corresponds to the activity generated inside portion of in passing throw Zin is negligible, and virtually
the cuff. (b) Equivalent noise model of FINE connection all of it will pass through Zwe and Zre
improvement on the recorded baseline noise with EMG signals, and its removal relies on different
at least two parallel operation transconductance techniques.
amplifiers (OTAs) leading to a 46.1% reduction
at N = 8 [35]. The functionality of these record-
ings was quantified by the signal-to-noise ra- 10.4 EMG Interference
tio improvement and shown to be significant for and Rejection
N = 3 or more. This design was shown to be
capable of generating <1.5 μVRMS total record- One of the common problems with ENG record-
ing baseline noise when connected to a FINE ings in awake animals is the contamination of
placed on the sciatic nerve of an awake animal the recorded signals with EMG. EMG can be
[22, 35]. An algorithm was introduced to find several orders of magnitude larger than ENG,
the value of N that can minimize both the power and so the electrode was specifically designed to
consumption and the noise in order to design reject external signals. This EMG rejection was
a miniaturized ultralow-noise neural amplifier. accomplished by a combination of the tripolar
These results demonstrate the efficacy of hard- recording setup and external shielding. Ideally,
ware averaging on noise improvement for neural the tripolar design alone eliminates any interfer-
recording with cuff electrodes and can accommo- ence [36], but the precise conditions for perfect
date the presence of high source impedances that rejection are never met in practice [37]. To further
are associated with the miniaturized contacts and reduce interference, an external shielding scheme
the high channel count in electrode arrays. This was utilized [38], in which a conductive surface
technique can be adopted for other applications (gold) is exposed on the outer surface of the cuff
where miniaturized and implantable multichannel in order to create an equipotential field around the
acquisition systems with ultra-low noise and low cuff. In simulation as well as benchtop testing,
power are required [35]. Another major source this shielding has been shown to reduce EMG
of noise is biological and arises from the large interference by 80% [39]. It is important to note
that rejection also likely depends on the fit of the of phase with the neural signals, with near-zero
cuff around the nerve and the degree of closure correlation coefficients. A “dummy” cuff was im-
or poorly closed cuffs being potentially more planted and placed near the implant site. Another
susceptible to interference. In this study, the cuffs single channel FINE was implanted on the tibial
were designed to fit closely to the nerve, with the nerve alongside the dummy cuff. Figure 10.3 also
cuff perimeter to nerve circumference ratio from shows the open BW for these two signals. The
approximately 1.1 to 1.5. dummy cuff recorded some activity, although it
To investigate the possibility that EMG con- was out of phase with the tibial signal. Post-
tributes to the ENG recordings, two methods were processing revealed that a 2 kHz (not 1 kHz) high-
applied: recordings from large muscle near the pass filter was necessary to remove all signals
ENG recording site and recording from a dummy from the dummy cuff. Subsequent recordings (not
cuff positioned near the recording site. Potential shown) show a similar trend. To determine po-
EMG interference could come from the muscles tential interference in the 16 channel FINEs, the
directly surrounding the cuff, the biceps femoris correlation coefficients between the raw neural
(BF), and/or semitendinosus (ST) muscles and and BF/ST EMG were calculated for two band-
would correlate in time with the neural record- widths, with a high-pass filter at 200 Hz and
ing (the semimembranosus [SM] is also near the 2 kHz. The correlation coefficients for the two
sciatic nerve, although the bifurcation level in bandwidths are shown in Fig. 10.3c, d. These
the animals implanted occurred between only the traces represent two different animals, as only a
BF and ST). Neural activity is shown (Fig. 10.3) single hamstring EMG was recorded in each of
in two bandwidths, one high-pass filtered above these two animals due to limitations in the number
1 kHz (“classic BW”) and again with a high- of percutaneous connections. Neither correlation
pass filter above 200 Hz (“open BW”). The neural was statistically greater than 0 nor was there
signals were acquired from 100 to 9000 Hz, and any difference between the two (paired t-test,
additional filtering was performed with a zero- p > 0.05) indicating that there is no interference
phase digital filter. The EMGs are largely out between the large muscle (hamstring surrounding
Fig. 10.3 Interfering EMG/ENG shows no correlation. recording with two different high-pass filters used, 200
(a). Example recording during gait, showing ENG with and 2000 Hz. The dummy cuff shows a similar out-of-
the classic and open BW as well as the nearby bicep phase trend as the recorded EMG and disappears once
femoris (BF) and semitendinosus (ST) EMG. BF EMG the lower bandwidth (200–2 kHz) is removed. (c) and (d)
is clearly out of phase with both neural plots, while ST Correlations between the ENG and recorded EMG (BF
EMG shows both in-phase and out-of-phase activity. (b) and ST, respectively) with the two bandwidths over the
Example of a dummy cuff recording with a tibial nerve first 2 months of recordings
Comparing Raw SNR for Different HPFs electroencephalogram (EEG) source localization,
**
in that the inverse problem can be solved to es-
** * timate source activity [40]. Various algorithms
10 *p<0.05 from the source localization literature, including
**p<0.001 sLORETA and FOCUSS, have been investigated
SNR (dB)
by other researchers to solve this problem in rat

5 nerves [41], utilizing the known cuff/nerve geom-
etry to create a lead field matrix [42]. Another
early study investigating the FINE in rabbits used
beamforming, in which a transformation matrix
0
0.2kHz HPF 1kHz HPF 2kHz HPf was created by inverting the lead field matrix [43].
A Bayesian algorithm (BSFE), based on an MEG
Fig. 10.4 SNR of neural signals over the first 2 months source localization method [44] which utilized
in three animals (SNR calculated over the same datasets).
knowledge of baseline and interfering sources,
The low-pass filter is set at 8 kHz for all three scenarios.
Increasing the bandwidth below 2 kHz increases the raw was then implemented [45] to create improved
recorded SNR for both the 1–8 kHz and 0.2–8 kHz band- spatial filters. Both methods were tested on acute
widths data and showed promising results, and in a recent
the cuff electrode) and the neural signals recorded chronic study, binary state gait classification was
from the cuff located within the muscle. achieved [22]. However, an end goal of the sep-
aration is to predict proportional motor activity
in order to control a prosthetic limb. The BSFE
10.4.1 Increasing SNR with Open
algorithm did improve the SNR sufficiently for
Bandwidth
accurate classification of ankle movement but
could not predict dynamic activation.
With low levels of interference from EMG, one
To address this shortcoming, a novel hybrid
can then ask the question of the optimal band-
algorithm was recently developed to extract the
width for the ENG. In particular, can the band-
command signals from the noisy, mixed neural
width be extended to improve the SNR? To char-
recordings from the cuff. This algorithm com-
acterize the effect of increasing the BW on the
bines elements of two previous algorithms [31,
recorded signal’s SNR, the SNR of the raw neural
32]; briefly, recordings are first fit to the simple
signals was calculated for each recording for both
model of Eq. 10.1, which learns Ai for each source
BWs for all three legs in the first 2 months and is
Si , and then the cost function in Eq. 10.2 is itera-
shown in Fig. 10.4. A small although significant
tively estimated over a fixed time step to estimate
(paired t-test, p < 0.05) increase of 2.9 dB in SNR
neural activity after training:
is observed for the open vs. classic BW. SNR was
measured as the ratio of the average stance phase
Y = Ai ∗ Si + V (10.1)
RMS to a baseline (i.e., standing) RMS.
With multiple channels of low-noise neural
activity obtained, the various source signals from C(i) = 1/2(Y − L ∗ S)2 + PS ΣN (10.2)
each fascicle can be recovered using separation
algorithms. for each of i sources, where Y is the recorded
signal (16 channels), L is the lead field matrix
[31], S is the estimate of source activity (i.e.,
10.5 Separation Algorithm variable of interest), and PS is the power of S and
Derivation represents a penalty term for the complexity of
S; finally, N represents the noise and interfer-
Recovering fascicular activity with extrafascic- ence, i.e., N = BBT + V, where B represents
ular electrodes shares many characteristics with other sources (i.e., Aj = i ) [46]. The beamform-
ing algorithm created the initial estimate of Y, implanted on the sciatic nerve of canines just
while the BSFE iteratively improved upon this proximal to the bifurcation of the tibial and
estimate to generate dynamic source activity in peroneal fascicles. These fascicles control plantar
real time (<10 ms). This algorithm was compared flexion and dorsiflexion of the lower limb. These
directly to the other algorithms and was capable motions in turn are predominately controlled
of separating several sources within the nerve by the gastrocnemius (GN) and tibialis anterior
[39] and was subsequently tested in chronic dog (TA) muscles, respectively; EMG signals from
preparation. these muscles were simultaneously recorded
and compared to the extracted neural signals.
Figure 10.5 shows the experimental procedure.
10.6 Chronic Nerve Recordings Recordings were taken as the animal walked
freely on a treadmill at a moderate pace. Figure
Chronic electroneurogram (ENG) recordings 10.6 shows some raw data as well as the envelope
from the sciatic nerves of canines were of the average gait cycle from a single recording
conducted; the methods have previously been session from each of the three legs, 2–4 weeks
published [22]. Canines were chosen due to post-implant. Figure 10.6a shows approximately
their unique physiology in that the sciatic nerve 5 s of raw neural activity from one animal.
is largely composed of two large fascicles Amplitude differences between channels are used
instead of many smaller ones, simplifying the by the algorithm to localize activity for signal
task of localizing neural activity. Figure 10.5 extraction. In Fig. 10.6b, envelopes were created
demonstrates this setup, with an example of by taking the root mean square (RMS) of the
the FINE and picture showing an implanted signals over 100 ms segments. Gait cycles were
cuff on the sciatic nerve. Briefly, FINEs were normalized to allow direct comparison between
Fig. 10.5 Chronic recording experimental design. (a) proximal to bifurcation of tibial and peroneal fascicles.
Example of implanted FINE, with 16 channels arranged (c) After implant, canines were trained to walk on a
longitudinally across the cuff, with gold shielding attached treadmill while ENG and EMG recordings were taken
and exposed on the exterior of the cuff [28]. (b) Example (percutaneous connector/wires to amplifier not shown)
of previous FINE (no shielding) on sciatic nerve, just
Fig. 10.6 ENG from two electrodes obtained from a three study animals. Blue and green bars represent the
FINE implanted for 9 months. (a) Raw recording from expected gastrocnemius/tibialis anterior activations, while
two channels on opposite sides of the cuff. Amplitude the vertical line represents foot strike. X-axis represents
differences between channels are used to localize activity normalized portion of the gait
within the cuff. (b) Average waveform envelope for all
animals and show the approximate time the foot significantly higher than using 1 bit (paired t-test,
lifts off the ground (vertical line) as well as the p < 0.05) [34].
activation times of the two target muscles, the
tibialis anterior (TA) and medial gastrocnemius
(GN). The SNR ranged from 4 to 7 dB across 10.7 Computer Model of Neural
animals and the impedance of each contact from Recording Properties
2 to 4 k , with no downward trend over the of a FINE
implant duration.
Figure 10.7 shows an example of signal re- Although is it clear that the FINE can record
covery from two fascicles in one animal (left activity from within the nerve and activity from
leg) 2 weeks post-implantation (the first record- different fascicles can be detected and recov-
ing from this animal). Figure 10.7b shows the ered, more information can be gained by com-
recovered neural signals (solid lines) alongside bining these experimental results with compu-
the corresponding EMG signals (dashed lines), tational modeling. ENG amplifiers normally re-
both normalized to 1. Figure 10.7c shows the move activity below 1 kHz to limit the possibil-
collective data from the three animals under study ity of EMG contamination. However, the results
for the first 2 months of recordings. The grouped in the previous sections show that neural activ-
recovery correlation coefficients for the GN were ity exists in this lower bandwidth. By modeling
0.76 ± 0.05 and 0.52 ± 0.09 for the TA. Over the electrode and neural signals, it is possible
this 2-month period, no downward trend in cor- to compare the recorded open bandwidth neural
relation coefficients is seen (t-test for regression spectrum with a computational simulation to de-
line, p > 0.05). The information transfer rate termine the origin of this low-frequency activity.
was also calculated for these neural signals [34]. This problem can be investigated by generating
This calculation was performed by converting a finite element model of the FINE/nerve and
the neural and muscle signals into discrete bits inserting nerve action potential currents to sim-
based on amplitude and calculating the informa- ulate the waveform generated by a single fiber
tion transfer between those signals. Using the action potential. Motor neural activity can then be
methodology described, the information transfer simulated with the firing of many fibers – specifi-
rate (ITR) can be calculated as 5.2 ± 1.3 bits cally alpha and gamma motor neurons to replicate
per second (bps) while encoding 2 bits of infor- the frequency spectrum from experimental neural
mation in the transmitted symbols, which was recordings.
Fig. 10.8 Finite element model of the nerve. The cuff

(blue) perfectly fits around the nerve, which consists of
three layers – epineurium, perineurium, and endoneurium,
each with unique conductive properties
Table 10.1 Tissue conductivities

Material Conductivity (1/ .m)
Epineurium 0.0826
Perineurium 0.0021
Endoneurium:
Transverse 0.5714
Longitudinal 0.0826
Fig. 10.7 Source localization. (a) Example of nerve with
overlaid source localization (3198). Hot/cool colors corre-
spond to peroneal/tibial nerves. Extra fascicles likely cor- to fit perfectly against the cuff, consisting of
respond to cutaneous sural nerves. (b) Example recording several layers. The outermost layer was the
showing recovered neural signals (solid) and correspond-
ing EMG (dash) during 10 s of gait. Data taken 2 weeks epineurium, which contained two fascicles.
post-implantation. (c) Combined correlation coefficients The larger fascicle used in this study measured
over 2 months for three animals. Week 9 has only one 1 × 2 mm with hexagonal shape and was
data point as the other two lost either EMG (3198) or used to generate the simulated recordings. Each
percutaneous connector (3219RL)
fascicle was surrounded by a perineurium layer
with a thickness of 50 μm which contained
10.7.1 Finite Element Model the endoneurial tissue. The conductance of the
electrode materials used the default values in
A three-dimensional model of the FINE and Maxwell 3D, while the conductivities used for
canine sciatic nerve was created in ANSYS each tissue type are detailed in Table 10.1 [47].
Maxwell 3D software. The internal transverse The entire model was surrounded by saline
dimensions of the cuff were 6 × 2 mm, with a solution with boundaries 250% of the nerve size.
hexagonal shape on the hinges as seen in Fig.
10.8. Sixteen contacts measuring 0.5 × 1 mm
were placed circumferentially around the nerve, 10.7.2 NEURON Model
and four large references measuring 5 × 1 mm
were placed on the ends of the cuff and externally NEURON was employed to simulate membrane
connected. The cuff was silicone, 200 μm thick, currents for single fiber action potentials (AP)
and 20 mm long. The nerve was designed of a myelinated axon. The currents were simu-
Fig. 10.9 Membrane Single Unit Membrane Current for Varying Axon Diameters
current for simulated 2
D = 2.7
currents in NEURON. D = 4.9
0
Amplitude (nA)
Current density was D = 7.2
calculated for an axon and
D = 9.4
scaled for various fiber –2 D = 11.7
sources, which correspond
to fiber diameters found in
–4
the dog sciatic nerve
–6
0 0.2 0.4 0.6 0.8 1 1.2 1.4
Time (ms)
Table 10.2 Model parameters in NEURON Table 10.3 Parameters used to simulate propagating
action potential in MATLAB implementation
Parameter Value
Axon diameter 10 μm Parameter Value
Node length 1 μm Nodal gap width 1 μm
Nodal capacitance 2 μF/cm2 Internodal length to 100
fiber diameter ratio
G_Nap 0.01 S/cm2
AP speed to diameter 6
G_Naf 3.0 S/cm2
ratio
G_Ks 0.08 S/cm2
Firing rate Fiber dependent
G_l 0.007 S/cm2
E_Na 50 mV
E_K −90 mV
10.7.3 Single Fiber Action Potential
E_l −90 mV
Through FINE Simulation
Vrest −80 mV
To generate the recorded potential from a

lated using the model MRG axon [48], which has single AP, a transfer function of the cuff was
been shown to mimic mammalian motor fibers generated. A 10-μm-diameter cylindrical fiber
(original model can be found online). The AP was placed within the fascicle of the FEM, and
was initiated at the proximal end of the axon, an outward current was generated, with the return
and the membrane current was recorded for the current at both ends of the fiber. This active
propagating AP, 50 nodes downstream. Mem- site, mimicking a node of Ranvier, was then
brane currents were calculated as the sum of all moved along the length of the cuff in 1 mm
sodium, potassium, and leakage currents through steps to generate the recorded response to a
the membrane; these currents represented the cur- current source moving through the cuff. This
rent density of the membrane. Total currents for transfer function represents the response to a
axons of different sizes were calculated by mul- static current. To generate the recorded signal
tiplying this current density by the correspond- from an AP, this function was convolved with
ing area of the node of Ranvier, or active mem- the membrane current signal from the NEURON
brane, for each size. Figure 10.9 shows an ex- simulation. Figure 10.10 shows the resulting
ample of the nodal current for axons of vary- tripolar recording of a single fiber action potential
ing sizes. The parameters for modeling the fiber traveling through the cuff, which matches
current with the NEURON model are shown in previously simulated AP [47]. Table 10.3 shows
Table 10.2. the parameters used for the propagating action
potential.
Tripolar Recording of Single Axon Axon Size Distribution

Fiber Distribution
2 0.2 Gamma
Distribution
Relative
Amplitude (uV)
0.15 Alpha
0 0.1
0.05
–2 0
2 5 8 11 14 17
–4 Axon Diameter
0.2 0.4 0.6 0.8 1
Time (ms) Fig. 10.11 Fiber distribution of sciatic nerve. High-
lighted portions correspond to the sizes associated with
two aspects of the efferent motor system, the alpha and
gamma fibers, which were used to simulated neural motor
activity
fibers [49, 50]; Gaussian noise was added to the

inter-pulse interval to introduce a more realistic
stochastic firing rate. The offset of the nodes
of Ranvier was also randomly varied for each
1 µV
fiber; the transfer function from the FINE was
100 µs
interpolated from 1 mm to 125 μm steps to allow
placement of nodes for the smallest fiber (2 μm
Fig. 10.10 Comparing SFAP to previous model. (a) Sim-
ulate SFAP with FEM and NEURON in this work. (b) diameter) and random offsets for larger fibers.
Previously simulated SFAP using NEURON and FEM, Final simulated neural recordings were created
which looks similar to the simulation in this work and by adding the pulse trains of all fibers together,
validates this modeling work
and the resulting spectrum is shown below in Fig.
10.12 along with an example spectrum from a
10.7.4 Generating Representative chronic recording trial.
Spectra of ENG Both spectra exhibit peaks in activity in the
lower-frequency band, 0.5–1 kHz, with activity
To generate realistic neural recordings, many dif- trailing off beyond 5 kHz. The peak in the lower
ferent fibers of different sizes were simulated band represents the contribution from the smaller
simultaneously and combined. The distribution fibers, as can be seen below in Fig. 10.13. These
and number of fibers in the cuff were based on frequency differences are due to the speed of the
the actual distribution of fibers in the dog sci- AP through the tripolar cuff electrode, with faster
atic nerve, as measured via histology [22]. Fig- fibers creating a more distributed “wideband”
ure 10.11 shows the measured distribution from spectrum. The “zeros” seen on the left-hand side
several nerves which was used as a representa- of Fig. 10.13 are also related to fiber diameter
tive fiber population. The highlighted areas cor- and specifically are due to the cuff acting as a
respond to the fiber diameters associated with spatial filter. This relationship can be mathemat-
gamma and alpha fibers, which comprise the ef- ically predicted using the cuff transfer function
ferent motor drive to the muscle. To simulate mo- displayed in the following equation:
tor activity, these fibers were simulated together
to create the spectra. A train of pulses about 1 s d
H (s) = 2 − 2 cos ω ∗ (10.3)
long was created for each fiber size. The firing v
rate was 50 Hz and 20 Hz for the gamma/alpha
fibers with a 1 ms refractory period for each fiber These zeros occur when the term ω ∗ dv equals
size, which is based on previous studies of these 0, 2π, etc., where ω is the frequency, d the dis-
Fig. 10.12 Simulated and (A) Power Spectral Density of Simulated ENG
recorded spectra. (a) –20
Simulated motor neural
Magnitude (dB)
spectra using alpha and
–40
gamma fibers. (b)
Recorded spectrum from
sciatic recordings in –60
canines
–80
0 1000 2000 3000 4000 5000 6000 7000

Frequency (Hz)
(B) Power Spectral Density of Recorded ENG

–20
Magnitude (dB)
–40
–60
–80
0 1000 2000 3000 4000 5000 6000 7000

Frequency (Hz)
Simulated Single Fiber Spectra

0.06 0.06
D=3.0 D=15.0
0.04 0.04
|Y (f)|
0.02 0.02
0 0
0 2 4 6 8 0 2 4 6 8
Freq (kHz)
Fig. 10.13 Simulated spectra from two fibers, 3 μm in conduction velocity through the FINE. The “dips” seen
and 15 μm, which correspond to the approximated sizes in the 3 μm are due to filtering effects of the FINE and can
of gamma and alpha motor neurons, respectively. These be predicted using Eq. 10.3
fibers exhibit different spectra largely due the difference
tance between contacts, and v the velocity of the is present in the lower bandwidth, which validates
AP; the first zero thus occurs when the frequency the claim that increasing the bandwidth by adding
equals 100∗ v since d = 0.01 m for our cuff lower-frequency power in the chronic recordings
(1.8 kHz for the 3 μm fiber traveling 18 m/s) [37]. can improve the representation of the neural ac-
This computational study does have some limita- tivity in the nerve.
tions. Most importantly, fibers not associated with
the motor system, which make up a large portion
of the total fiber count, are ignored. The simulated 10.8 Summary and Conclusions
ENG thus represents pure efferent activity, which
is not realistic. However, this simulation does Experiments using chronic animal experimen-
provide evidence that significant neural activity tation show the ability to reliably recover dy-
namic motor information from an intact nerve for 40 mm with a single recording electrode
in a freely moving animal using an extraneural placed 1 mm above the line in the middle.
electrode for long periods of time. The technol- 7. Simulate a monopolar recording of an action
ogy requires ultralow-noise amplifier [35] and potential using the current calculated in Ques-
appropriate algorithms [34] to extract the high- tion 5 and the potentials found in equation six.
fidelity motor intent from individual fascicles to 8. Simulate an electroneurogram signal by
obtain independent signals. A hybrid algorithm, summing a large number of the units signals
utilizing two previously developed approaches, obtained from the solution of Question
improved recovery metrics compared to previous 7) with a random delay. Compare your
methods. Analysis of the recorded signals showed results to real signals shown in Fig. 10.6.
that two neural signals were present using si- Note that hundreds to thousands of units
multaneous recordings of potentially interfering are required to generate a realistic ENG
muscles. Subsequent analysis demonstrated that signal.
dynamic motor signals which matched the target 9. Calculate the power spectra of the simulated
EMG could be reliably recovered. The informa- signal in Question 8. Are there any differ-
tion transfer within a nerve can also be calculated ences? Can you explain why?
and allows for the quantitative assessment of in-
formation recovery with the FINE.
References
Homework 1. G.S. Dhillon, S.M. Lawrence, D.T. Hutchinson, K.W.
Horch, Residual function in peripheral nerve stumps
1. Review and compare the various methods of amputees: Implications for neural control of artifi-
cial limbs. J. Hand Surg. Am. 29, 605–615 (2004)
for interfacing with the peripheral nervous 2. X. Navarro et al., A critical review of interfaces with
system. Provide a comparative list of the peripheral nervous system for the control of neu-
advantages and disadvantages of these various roprostheses and hybrid bionic systems. J. Peripher.
techniques. Nerv. Syst. 10, 229–258 (2005)
3. M.K. Haugland, T. Sinkjaer, Cutaneous whole nerve
2. Simulate the effect of hardware averaging recordings used for correction of footdrop in hemi-
for input thermal noise reduction in Multisim plegic man. IEEE Trans. Rehabil. Eng. 3, 307–317
or other circuit simulation software (see Fig. (1995)
10.2). Determine the relationship between the 4. D. Tan, M. Schiefer, M.W. Keith, R. Anderson, D.J.
Tyler, Stability and selectivity of a chronic, multi-
noise amplitude and the number of amplifiers. contact cuff electrode for sensory stimulation in a
3. List three other methods to decrease the input- human amputee. Int. IEEE/EMBS Conf. Neural Eng.
referred noise of a bioamplifier. Compare to NER 12, 859–862 (2013)
hardware averaging using simulation. 5. G.E. Loeb, R.A. Peck, Cuff electrodes for chronic
stimulation and recording of peripheral nerve activity.
4. Choose three of the neural interfacing methods J. Neurosci. Methods 64, 95–103 (1996)
in Question 1. From the literature, determine 6. B. Wodlinger, D.M. Durand, Localization and recov-
the value of the source resistance of the elec- ery of peripheral neural sources with beamforming
trode in each case and find the amplifier design algorithms. IEEE Trans. Neural Syst. Rehabil. Eng.
17, 461 (2009)
with the optimum S/R. 7. S. Micera et al., Decoding of grasping information
5. Simulate in NEURON (or other simulation from neural signals recorded using peripheral in-
software) a single axon action potential and trafascicular interfaces. J. Neuroeng. Rehabil. 8, 53
calculate the transmembrane current along the (2011)
8. A. Kundu, M. Wirenfeldt, K.R. Harreby, W. Jensen,
axon (compare your results to Fig. 10.9). Biosafety assessment of an intra-neural electrode
6. Calculate the voltage generated by a point (TIME) following sub-chronic implantation in the
current source to simulate an axon. Assume median nerve of göttingen minipigs. Int. J. Artif.
the current source moves in a straight line Organs 37, 466–476 (2014)
9. N.M. Ledbetter et al., Intrafascicular stimulation of 23. P.F. Pasquina et al., Advances in amputee care. Arch.
monkey arm nerves evokes coordinated grasp and Phys. Med. Rehabil. 87, 34–43 (2006)
sensory responses. J. Neurophysiol. 109, 580–590 24. F. Cordella et al., Literature review on needs of up-
(2013) per limb prosthesis users. Front. Neurosci. 10, 1–14
10. Y. Aoyagi, R.B. Stein, A. Branner, K.G. Pearson, R.A. (2016)
Normann, Capabilities of a penetrating microelec- 25. M. Ortiz-catalan, B. Håkansson, R. Brånemark, Real-
trode array for recording single units in dorsal root time and simultaneous control of artificial limbs
ganglia of the cat. J. Neurosci. Methods 128, 9–20 based on pattern recognition algorithms. IEEE Trans.
(2003) Neural Syst. Rehabil. Eng. 22, 756–764 (2014)
11. K. Yoshida, D. Farina, M. Akay, W. Jensen, Multi- 26. D. Farina et al., The extraction of neural information
channel intraneural and intramuscular techniques for from the surface EMG for the control of upper-limb
multiunit recording and use in active prostheses. Proc. prostheses: Emerging avenues and challenges. IEEE
IEEE 98, 432–449 (2010) Trans. Neural Syst. Rehabil. Eng. 22, 797–809 (2014)
12. J. Badia et al., Comparative analysis of transverse 27. S. Micera et al., On the use of longitudinal intrafascic-
intrafascicular multichannel, longitudinal intrafasci- ular peripheral interfaces for the control of cybernetic
cular and multipolar cuff electrodes for the selec- hand prostheses in amputees. IEEE Trans. Neural
tive stimulation of nerve fascicles. J. Neural Eng. 8, Syst. Rehabil. Eng. 16, 453–472 (2008)
036023 (2011) 28. P.M. Rossini et al., Double nerve intraneural interface
13. G.A. McCallum, X. Sui, C. Qiu, J. Marmerstein, Y. implant on a human amputee for robotic hand control.
Zheng, T.E. Eggers, C. Hu, L. Dai, D.M. Durand, Clin. Neurophysiol. 121, 777–783 (2010)
Chronic interfacing with the autonomic nervous sys- 29. E.A. Pohlmeyer et al., Toward the restoration of hand
tem using carbon nanotube (CNT) yarn electrodes. use to a paralyzed monkey: Brain-controlled func-
Sci. Rep. 7(1), 11723 (2017) tional electrical stimulation of forearm muscles. PLoS
14. T.D. Yoshida Kozai et al., Ultrasmall implantable One 4, 1–8 (2009)
composite microelectrodes with bioactive surfaces for 30. M.K. Haugland, A. Hoffer, T. Sinkjaer, Skin contact
chronic neural interfaces. Nat. Mater. 11, 1065–1073 force information in sensory nerve signals recorded
(2012) by implanted cuff electrodes. IEEE Trans. Rehabil.
15. K.R. Harreby et al., Subchronic stimulation per- Eng. 2, 18–28 (1994)
formance of transverse intrafascicular multichannel 31. B. Wodlinger, D.M. Durand, Localization and recov-
electrodes in the median nerve of the Göttingen ery of peripheral neural sources with beamforming
Minipig. Artif. Organs 39, E36–E48 (2015) algorithms. IEEE Trans. Neural Syst. Rehabil. Eng.
16. J. Badia, S. Raspopovic, J. Carpaneto, S. Micera, 17, 461–468 (2009)
X. Navarro, Spatial and functional selectivity of pe- 32. Y. Tang, B. Wodlinger, D.M. Durand, Bayesian spa-
ripheral nerve signal recording with the transversal tial filters for source signal extraction: A study in the
intrafascicular multichannel electrode (TIME). IEEE peripheral nerve. IEEE Trans. Neural Syst. Rehabil.
Trans. Neural Syst. Rehabil. Eng. 24, 20–27 (2016) Eng. 22, 302–311 (2014)
17. B.S. Spearman et al., Tissue-engineered peripheral 33. D.J. Tyler, D.M. Durand, Functionally selective pe-
nerve interfaces. Adv. Funct. Mater. 28, 1701713 ripheral nerve stimulation with a flat interface nerve
(2018) electrode. IEEE Trans. Neural Syst. Rehabil. Eng. 10,
18. V.H. Desai, et al., Design, fabrication, and 294–303 (2002)
characterization of a scalable tissue-engineered- 34. T.E. Eggers, Y.M. Dweiri, G.A. McCallum, D.M.
electronic-nerve-interface (TEENI) device, in Durand, Recovering motor activation with chronic pe-
International IEEE/EMBS Conference on Neural ripheral nerve computer interface. Sci. Rep. 8, 14149
Engineering, NER 203–206, 2017. https://doi.org/ (2018)
10.1109/NER.2017.8008326 35. Y.M. Dweiri, T. Eggers, G. McCallum, D.M. Du-
19. I.R. Minev, D.J. Chew, E. Delivopoulos, J.W. Fawcett, rand, Ultra-low noise miniaturized neural amplifier
S.P. Lacour, High sensitivity recording of afferent with hardware averaging. J. Neural Eng. 12, 046024
nerve activity using ultra-compliant microchannel (2015)
electrodes: An acute in vivo validation. J. Neural Eng. 36. A. Demosthenous, I.F. Triantis, An adaptive ENG
9, 026005 (2012) amplifier for tripolar cuff electrodes. IEEE J. Solid
20. T.A. Kuiken et al., Targeted muscle reinnervation for State Circuits 40, 412–420 (2005)
real-time myoelectric control of multifunction artifi- 37. J.J. Struijk, M. Thomsen, Tripolar nerve cuff record-
cial arms. J. Am. Med. Assoc. 301, 619–628 (2016) ing: Stimulus artifact, EMG and the recorded nerve
21. P.P. Vu et al., A regenerative peripheral nerve interface signal. Proc. 17th Int. Conf. Eng. Med. Biol. Soc. 2,
allows real-time control of an artificial hand in upper 1105–1106 (1995)
limb amputees. Sci. Transl. Med. (2020) 38. B. Sadeghlo, P.B. Yoo, Enhanced electrode design for
22. Y.M. Dweiri et al., Stable detection of movement peripheral nerve recording. Int. IEEE/EMBS Conf.
intent from peripheral nerves: Chronic study in dogs. Neural Eng. NER, 1453–1456 (2013). https://doi.org/
Proc. IEEE 105, 50–65 (2017) 10.1109/NER.2013.6696218
39. T.E. Eggers, Y.M. Dweiri, G.A. McCallum, D.M. Du- 45. Y. Tang, B. Wodlinger, D.M. Durand, An algorithm
rand, Model-based Bayesian signal extraction algo- for source signal extraction from the peripheral nerve,
rithm for peripheral nerves. J. Neural Eng. 14, 056009 in Proceedings of Annual International Conference of
(2017) the IEEE Engineering in Medicine and Biology So-
40. R. Grech et al., Review on solving the inverse problem ciety, p. 4251–4254 (2011). https://doi.org/10.1109/
in EEG source analysis. J. Neuroeng. Rehabil. 5, 25 IEMBS.2011.6091055
(2008) 46. T. Eggers, D.M. Durand, Chronic PNS recordings
41. J. Zariffa, A review of source separation and source with flexible carbon nanotube fibers, in Abstr. BMES
localization approaches in peripheral nerves, in Con- Annu. Conf. (2017)
ference Record - Asilomar Conference on Signals, 47. A.Q. Choi, J.K. Cavanaugh, D.M. Durand, Selectivity
Systems and Computers 2015–April, p. 293–298 of multiple-contact nerve cuff electrodes: A simula-
(2015) tion analysis. IEEE Trans. Biomed. Eng. 48(2) (2000)
42. H. Hallez et al., Review on solving the forward prob- 48. C.C. McIntyre, A.G. Richardson, W.M. Grill, Mod-
lem in EEG source analysis. J. Neuroeng. Rehabil. 4, eling the excitability of mammalian nerve fibers: In-
46 (2007) fluence of after potentials on the recovery cycle. J.
43. B. Wodlinger, D.M. Durand, Selective recovery of Neurophysiol. 87, 995–1006 (2002)
fascicular activity in peripheral nerves. J. Neural Eng. 49. P.H. Ellaway, A. Taylor, R. Durbaba, Muscle spindle
8, 56005 (2011) and fusimotor activity in locomotion. J. Anat. 227,
44. D.P. Wipf, J.P. Owen, H.T. Attias, K. Sekihara, S.S. 157–166 (2015)
Nagarajan, Robust Bayesian estimation of the loca- 50. M.N. Abdelghani, J.J. Abbas, K.W. Horch, R. Jung, A
tion, orientation, and time course of multiple cor- functional model and simulation of spinal motor pools
related neural sources using MEG. NeuroImage 49, and intrafascicular recordings of motoneuron activity
641–655 (2010) in peripheral nerve. Front. Neurosci. 8, 1–14 (2014)
Functional Magnetic Resonance
Imaging 11
Zhongming Liu and Jiayue Cao
Abstract coding, and characterize brain networks

while humans are engaged in a realistic
Functional magnetic resonance imaging has
and dynamic environment similar to daily
become a primary tool for psychological
life experiences. In this chapter, we discuss
and cognitive studies or preclinical brain
the principles, methods, and applications of
research. As a technique to map brain function,
fMRI, with emphasis on its biophysical and
fMRI measures the blood oxygenation level–
physiological basis, experimental designs and
dependent signal as a collective effect of
analysis methods, and applications to human
changes in cerebral blood flow, cerebral blood
and animal studies. Example data or results
volume, and cerebral metabolic rate of oxygen
from empirical studies are presented to help
following changes in neural activity. The
illustrate methods or support scientific views.
use of fMRI in combination with carefully
designed task paradigms has enabled scientists
Keywords
to map perceptual, cognitive, or behavioral
functions onto brain regions and networks. Nuclear magnetic resonance · Longitudinal
Spontaneous activity observed with fMRI in relaxation · Transverse relaxation ·
task-free resting states has been used to reveal Hemoglobin · Magnetic susceptibility ·
intrinsic functional networks that collectively Blood oxygenation level dependent ·
depict the brain’s functional architecture Neurovascular coupling · Hemodynamic
or connectome. Naturalistic paradigms for response function · Statistical parametric
fMRI are increasingly used to map brain mapping · Resting state networks ·
activation, address neural representation and Functional connectivity · Naturalistic
paradigm
Electronic Supplementary Material: The online ver-

sion of this chapter (https://doi.org/10.1007/978-3-030-
43395-6_11) contains supplementary material, which is
available to authorized users. 11.1 Introduction
Z. Liu ()
Department of Biomedical Engineering, University of
Michigan, Ann Arbor, MI, USA To study how the brain works, it is desirable to
image neural activity throughout the brain while
Department of Electrical Engineering and Computer
Science, University of Michigan, Ann Arbor, MI, USA being able to see every neuron and detect every
e-mail: [email protected] neuronal spike. This requires an imaging tech-
J. Cao nique to have high spatial resolution, high tem-
Department of Biomedical Engineering, University of poral resolution, and whole-brain coverage. Un-
Michigan, Ann Arbor, MI, USA fortunately, such a technique is unavailable to

332 Z. Liu and J. Cao
date for human brain imaging, in part because for

human applications it is also desirable to refrain
from using any invasive procedure.
Every technique has its pros and cons and
continues to progress on its own or in combina-
tion with other techniques [1]. Among all that
is currently available, functional magnetic reso-
nance imaging (fMRI) stands out with notable
advantages by providing sub-millimeter spatial
resolution, sub-second temporal resolution, and Fig. 11.1 Hydrogen protons in the absence of external
whole-brain coverage. In the past three decades, magnetic field. (a) The spin of a single 1 H generates a
small magnetic field −
→
μ . (b) Spins are randomly oriented
fMRI has undergone rapid development and has
become a primary tool for human psychological
and cognitive studies, as well as preclinical (ani- tiny magnet (Fig. 11.1a). Spins in a spatial ele-
mal) brain research. ment, typically referred to as a voxel, form a spin
In this chapter, we discuss fMRI with special system. In the absence of any external magnetic
emphasis on its biophysical and physiological ba- field, the spins in a spin system behave like many
sis, experimental designs and analysis methods, small magnets oriented in random directions (Fig.
and applications to human and animal studies. 11.1b). The magnetic fields generated by individ-
To facilitate discussions, we include, as relevant ual spins sum to zero.
examples, some data from our prior studies. In When an external magnetic field B0 is present
addition, this chapter is not intended to serve as or applied, the spins in a spin system exhibit
a comprehensive literature review, and the ref- a weak tendency to precess (i.e., a gyroscopic
erence list is only meant to provide additional, motion) along the direction of the external field,
but not inclusive, materials to guide interested giving rise to very small net magnetization in a
readers. direction parallel to B0 (Fig. 11.2). Such preces-
sion has a characteristic frequency ω0 , called the
Larmor frequency, which is proportional to B0 by
11.2 Magnetic Resonance a fixed ratio γ , called the gyromagnetic ratio. For
Imaging 1
H, the gyromagnetic ratio is 42.58 MHz/Tesla.
The precession (or Larmor) frequency is about
As the name suggests, fMRI uses magnetic res- 128 MHz for an MRI system operating under
onance imaging (MRI) to measure brain activity a 3 Tesla static magnetic field, and it is about
and map brain functions. The physics underlying 300 MHz for 7 Tesla MRI. Under typical field
MRI is nuclear magnetic resonance (NMR). It strengths, the precession frequency is always in
describes the magnetic behavior of any atomic the radio frequency (RF) range. As a spin is pre-
nucleus that has an odd number of protons. Be- cessing, it creates a rotating magnetic field in the
cause the human brain has high water content, the transverse plane, which is perpendicular to B0 .
hydrogen atom (1 H), which consists of a single However, spins precess with random phases; as a
proton and carries a positive charge, is the most result, the sum of their transverse magnetization
abundant nucleus for MRI. Although other nu- is still equal to zero (Fig. 11.2).
clei, e.g., 13 C, 31 P, 23 Na, and 19 F, are also visible For a spin system to generate detectable mag-
to MRI, fMRI is nearly all based on 1 H MRI. netic fields in the transverse plane, it needs to
Hereafter, we refer to hydrogen atoms simply receive a brief RF excitation – a rotating magnetic
as protons, unless they are explicitly specified field applied to the transverse plane by using an
otherwise. RF transmitter (Fig. 11.3). When the RF excita-
A proton spins about itself and creates a mag- tion uses the same frequency as the precession
netic moment. It is perhaps convenient to think frequency, i.e., on resonance, the spin system
of a spinning proton (or a spin in short) as a
11 Functional Magnetic Resonance Imaging 333
Fig. 11.2 Spins in an external magnetic field B0 . Every B0 . Of the bulk magnetization, the longitudinal component
spin precesses at the same frequency but with a different is non-zero, but the transverse component is zero
phase. More spins align parallel to B0 than antiparallel to
is also governed by a time constant, namely, T2

when the magnetic field is homogeneous within
the spin system or T2 * when it is inhomoge-
neous. A shorter T2 or T2 * means a faster trans-
verse relaxation (Fig. 11.4). As T1 , T2 , and T2 *
are all tissue-specific properties, the signals de-
tected by the RF receiver can report various tissue
contrasts, depending on the pulse sequence used
Fig. 11.3 Spins given on resonance excitation. An RF to transmit, encode, and receive the RF signals
excitation B1 is an applied magnetic field rotating in to/from spins in the brain.
the transverse plane (left). The excitation flips the bulk For the scope of this chapter, it only covers the
magnetization toward the transverse plane by a flip angle θ
basic physics of MRI as the prerequisite for learn-
ing fMRI. For more comprehensive discussions
effectively absorbs the energy from the excita- about MRI, we refer the readers to other materials
tion and progressively synchronizes the phases [2–4].
of individual spins while progressively reduc-
ing the longitudinal magnetic field. During the
course of the excitation, the bulk magnetization 11.3 Blood Oxygenation
that arises from the spin system is flipped down Level–Dependent Contrast
toward the transverse plane while rotating about
B0 (Fig. 11.3). One type of tissue contrast observable with MRI
Once the excitation is off, the longitudinal is blood oxygenation level dependent (BOLD)
magnetization progressively returns to its thermal [5]. In the brain, cerebral vasculature circulates
equilibrium, showing an approximately exponen- blood to supply oxygen, glucose, and nutrients.
tial recovery, namely, the longitudinal relaxation. Hemoglobin (i.e., red blood cell) is the primary
The recovery of longitudinal magnetization is carrier of oxygen. Arteries deliver oxygenated
governed by a time constant T1 . A shorter T1 hemoglobin. Oxygen is extracted from capillar-
means a faster longitudinal relaxation (Fig. 11.4). ies and consumed by brain tissues, creating de-
In the meantime, the spin system progressively oxygenated hemoglobin. Veins drain the deoxy-
desynchronizes the phases of individual spins. genated hemoglobin, along with other metabolic
During dephasing, the transverse component of products.
the bulk magnetization can be detected by an RF Hemoglobin is diamagnetic when oxygenated
receiver. The detected signal shows an approx- but paramagnetic when deoxygenated [6]. With
imately exponential decay to zero, namely, the only oxygenated hemoglobin, the magnetic
transverse relaxation. The transverse relaxation susceptibility of blood is nearly identical to that of
Fig. 11.4 Longitudinal

(left) or transverse (right)
relaxation as an
exponential recovery or
decay characterized by a
time constant T1 , T2 , or
T2 *
Fig. 11.5 Deoxygenated blood distorts the static mag- Large circles indicate blood vessels. Small circles indicate
netic field, but oxygenated blood does not. Deoxygenated oxygenated (red) or deoxygenated (blue) hemoglobin
blood causes a shorter T2 * than does oxygenated blood.
brain tissue. As a result, a voxel that contains hemoglobin increases or when the concentration
blood and tissue tends to experience a homoge- of deoxygenated hemoglobin decreases [8].
neous magnetic field. With only deoxygenated
hemoglobin, the blood susceptibility mismatches
the tissue susceptibility, distorting the magnetic 11.4 BOLD Response to Neural
field into an inhomogeneous distribution. Activity
The blood in a voxel includes a varying
mixture of oxygenated and deoxygenated Functional MRI uses the BOLD signal to localize
hemoglobin. Higher concentration of deoxy- neural activity [9–11]. The precise relationship
genated hemoglobin results in greater distortion between the BOLD signal and neural activity is
to the magnetic field and causes the spins in not fully understood and still under active re-
the voxel to experience more distinct magnetic search. Nevertheless, evidence from prior studies
fields such that they precess with more distinct has shed light onto the biophysical basis and
frequencies [7]. As a result, spins tend to run out physiological origin of the BOLD signal [12,
of synchronization faster, while faster dephasing 13]. Here, we discuss the understanding that has
results in shorter T2 * and faster transverse received general consensus.
relaxation (Fig. 11.5). Neural activity requires energy and consumes
When reading out the T2 *-weighted signal at oxygen. Elevation of neural activity (or activa-
an echo time close to T2 *, the signal magni- tion) triggers a cascade of metabolic and hemo-
tude is negatively dependent on the concentra- dynamic events, collectively contributing to the
tion of deoxygenated hemoglobin or positively BOLD signal. As illustrated in Fig. 11.6, at an
dependent on the concentration of oxygenated activated region, more oxygen is extracted and
hemoglobin. The latter has been more commonly consumed, giving rise to an increase in the cere-
used for interpretation of fMRI and has been con- bral metabolic rate of oxygen (CMRO2 ). The
ventionally termed as BOLD since a seminal pa- brain reacts to the increasing demand and con-
per by Ogawa et al. [5]. In short, the BOLD signal sumption of oxygen by actively dilating arterioles
is higher when the concentration of oxygenated and capillaries to allow more oxygenated blood
than in the white matter, although findings from

recent studies suggest the feasibility of using
BOLD fMRI to map white matter functions
[24–26].
11.5 Hemodynamic Response

Function
Fig. 11.6 Neural activation causes oxygen consumption
to increase, arterioles to dilate, capillaries and veins to Although the physiological mechanism of neu-
expand, both blood flow and volume to increase, and rovascular coupling is not fully clear, models have
the BOLD signal to rise. Solid and dashed lines indicate
the blood vessels before and after neural activation, re- been derived from empirical data in an attempt to
spectively. OEF, oxygen extraction fraction, indicates the mathematically describe the relationship between
fraction of oxygen extracted from the bloodstream and neural activity and its resulting hemodynamic
supplied to brain tissue response [23, 27, 28]. Neurovascular coupling is
considered as a linear time-invariant system, for
to flow into the activated region. The upstream which neural activity is the input and vascular
increase in blood flow causes a passive expansion response is the output. In line with the estab-
of the downstream venioles or veins, similar to lished notion of linear systems, the model of
draining water through a balloon [14]. As such, neurovascular coupling is often described as a
both cerebral blood volume (CBV) and cerebral hemodynamic impulse response function, which
blood flow (CBF) increase given elevated neural describes the system’s output given an impulse
activity. input, i.e., a delta function. In literature, the hemo-
Importantly, the increase in CBF overcompen- dynamic impulse response function is called the
sates for the increase in CMRO2 [15]. The brain hemodynamic response function (HRF). Despite
supplies more blood to deliver more oxygen than the omission of “impulse”, it is worth empha-
is consumed by neural activity. This effect leads sizing that the HRF should be interpreted as the
to increase in the concentration of oxygenated vascular response to an impulse neural input.
hemoglobin or decrease in the concentration of The HRF is typically modeled as the sum of
deoxygenated hemoglobin, causing the BOLD two gamma functions. Many software tools for
signal to increase. To capture this signal with fMRI analysis have implemented the HRF, e.g.,
MRI, such pulse sequences as echo-planar imag- the MATLAB-based SPM software. Parameters
ing [16] or spiral imaging [17, 18] are often used that control the shape of the HRF include the
for fast imaging with T2 *-weighted contrast. Re- latencies and durations. With the default param-
cent advances in RF coils, pulse sequences, and eters, the HRF is called the canonical HRF (Fig.
image reconstruction have contributed to further 11.7), in which the impulse response reaches the
acceleration to enable whole-brain fMRI within positive peak at 5 s, returns to baseline (zero)
1 s [19, 20]. at 12 s, undershoots at about 15 s, and again
How neural activity drives hemodynamic returns to baseline at about 25 s. Clearly, the HRF
changes (i.e., neurovascular coupling) is not is very slow and behaves like a low-pass filter
precisely understood [21]. Evidence suggests with an (arguably conservative) cutoff frequency
that active dilation of blood vessels is not directly at 0.2 Hz. For this reason, the BOLD signal is
controlled by neurons but mediated through not rapid enough to follow fast modulation of
astrocytes [22]. Hemodynamic responses are neural activity – a notable limitation of fMRI.
more driven by and coupled to synaptic inputs to That said, the bandwidth of the BOLD signal
neurons, rather than spiking output from neurons is still debatable, as recent findings demonstrate
[23]. In part for this reason, the BOLD signal the feasibility of detecting neurogenic BOLD re-
is more observable in the brain’s gray matter sponses as fast as 0.8 Hz [29, 30].
noise and only yields the event-related response.

Since the BOLD response is slow, the response
to one event may overlap with the response to the
next event, if they are not adequately separated in
time. To avoid overlapping responses, the events
should be repeated with an interval greater than
25 s or longer. This design, however, is inefficient
because it has to prolong the experiment in order
to include a sufficient number of events to obtain
the event-related response with a high signal
to noise ratio. An alternative design is to repeat
events at random times such that the event-related
response can be obtained by deconvolution, while
the interval between adjacent events does not
have to be long [28, 32]. For this type of event-
Fig. 11.7 Canonical hemodynamic response function related design, a specific strategy is to set the
event timing according to an M-sequence [33],
which is a pseudorandom sequence of ones and
11.6 Event-Related and Block zeros with one indicating the presence of an
Design event and zero indicating the absence of an
event. The fact that the M-sequence has zero
The HRF can be used to predict how the BOLD autocorrelation at any (non-zero) integer time
response should look like at a location engaged in shift prepares a nice precondition for the ease of
processing a stimulus (e.g., a flash of light) or per- deconvolution.
forming a task (e.g., tapping a finger). The BOLD More common than the event-related design
signal at each voxel can be compared with the is the block design. In a block-design paradigm,
HRF predicted response. If they are similar, the stimuli are typically presented for a sustained pe-
voxel is considered “activated”; otherwise, not. riod (or block), followed by a resting (or control)
This strategy to localize brain activations, known period that contains no stimulus (or only control
as statistical parametric mapping [31], requires a stimuli). The stimulus-on block alternates with
careful experimental design alongside a rigorous the stimulus-off block for multiple cycles. The
statistical analysis. This section is focused on ON block is often designed to have the same
the experimental design, and the next section is duration as the OFF block (but not always or nec-
focused on statistical analysis. We further confine essarily the case), such that the paradigm can be
the context of discussion to stimulus processing, described by a periodic boxcar function in which
while the same notion is readily generalizable to 1 means stimulus-on and 0 means stimulus-off.
task performance. Prediction of the BOLD response is derived by
Recall that HRF is the BOLD response to an convolving the boxcar function with the HRF.
impulse neural input. In other words, HRF itself Typically, the on or off block lasts around 30 s
is the prediction of the BOLD response given an such that the boxcar function has a characteristic
impulse stimulus. A stimulus that lasts no more frequency close to the peak frequency of the
than 2 s is brief enough to be considered as an HRF in order to elicit the BOLD response with a
“event” or impulse, because HRF is slow and lasts relatively high signal to noise ratio. If the boxcar
over 25 s. To measure the event-related response, function has a much shorter period (i.e., a shorter
it is intuitive to design an experiment that includes ON or OFF block), the spectral characteristics of
discrete stimuli applied for many repetitions. HRF limit the response amplitude and lower the
Averaging the BOLD signal across the repeated signal to noise ratio.
events excludes the event-unrelated signal or
11.7 BOLD Time Series Analysis The response model is simply a linear regres-
sion model, in which the same regressor is used to
It follows that mapping brain activations is often explain the BOLD signal at every voxel, denoted
achieved by analyzing the time series of each as yi (t) where i is the voxel index.
voxel with a model that depends on the experi-
mental paradigm and HRF [31]. Specifically, the yi (t) = βi x(t) + bi (11.2)
problem of activation mapping is formulated as
a hypothesis test addressed separately for each The bias term, bi , can be eliminated, if prepro-
voxel. For a given voxel, the null hypothesis is cessing is applied such that both yi (t) and x(t) have
that the voxel is not activated by the stimulus/task their average (over time) equal to zero.
paradigm; the alternative hypothesis is that it is To address how well yi (t) is predictable by
activated. Herein, “activation” means the ability x(t), one can simply evaluate the temporal corre-
to predict the voxel time series with a response lation between yi (t) and x(t) and test its statistical
model (or design matrix), while the predictability significance. The use of this simple correlation-
is evaluated for statistical significance. based method can be dated back to a seminal
To elaborate, let us use a simple example in paper by Bandettini et al., who were among the
which visual stimuli are presented in a block- first to use fMRI to map activations (with a motor
design paradigm (Fig. 11.8). This paradigm in- task) [9]. Although it is simple and effective, the
cludes four resting blocks interleaved with three correlation-based activation assessment is not ap-
stimulation blocks; the resting and stimulation plicable when the response model includes mul-
blocks are both 30 s. Given this paradigm, the tiple regressors.
response model sets up a prediction as to how The need to use multiple regressors may arise
the BOLD response should look like at a voxel in several scenarios. Perhaps the most intuitive
activated by the stimulation. As aforementioned, one is when the experimental paradigm includes
the response model assumes that neurovascular more than one type of stimuli. For example, we
coupling is a linear time-invariant system that can use visual stimuli selective for the magno-
can be described by the canonical HRF for every cellular (M) visual pathway in one stimulation
voxel in the brain. Given this assumption, the block but the parvocellular (P) visual pathway in
predicted response should look like a time series another stimulation block (Fig. 11.9). Then we
that results from temporal convolution of the 30 s- need to include two regressors: one for M, x1 (t),
off-30 s-on boxcar function with the HRF. and the other for P, x2 (t). The response model
Mathematically, let s(t) be the function with should be rewritten as below (note that the bias
which stimulation is applied, h(t) be the HRF, term is eliminated by preprocessing as mentioned
and x(t) be the regressor used to predict the fMRI earlier).
response to the stimulation.
x1 (t) = s1 (t) ∗ h(t) (11.3)
x(t) = s(t) ∗ h(t) (11.1)
Fig. 11.8 Typical

response model (bottom)
given an ON-OFF
block-design paradigm
(top)
Fig. 11.9 Mapping magnocellular and parvocellular pathways in the visual system [34]. (b) The response
pathways. (a) A block-design paradigm involves two types contrast between the M- and P-selective stimuli is shown
of visual stimulation: one with high spatial frequency at both on the cortical surface and in the brain volume.
10 cycles per degree (c/deg) and low temporal frequency The contrast map segregates the M and P divisions in
at 1 Hz and the other with low spatial frequency at 1 c/deg lateral geniculate nuclei (LGN) and their extensions onto
and high temporal frequency at 10 Hz, to selectively (presumably) the dorsal and ventral streams on the cortex
activate the magnocellular (M) and parvocellular (P)
x2 (t) = s2 (t) ∗ h(t) (11.4) for an example of using this strategy to sep-
arate the magnocellular and parvocellular divi-
yi (t) = βi1 x1 (t) + βi2 x2 (t) (11.5) sions of the visual thalamus (i.e., lateral genic-
ulate nuclei) and their extension onto the visual
The regression parameters (or beta values) can cortex.
be estimated by least-squares estimation – an In another scenario, one may use multiple re-
established method that has been implemented gressors even when the experimental paradigm
in many statistical analysis tools and has been only includes one type of stimulation. Let us
elaborated in many statistical textbooks. Herein, revisit how a regressor is defined. As in Eq.
we skip the details about the least-squares esti- (11.1), the regressor results from convolving the
mation and refer the readers interested to existing boxcar function and the HRF, and it is assumed
literature [31]. The estimated beta values, β̂i1 to be identical across voxel. What if the HRF
and β̂i2 , can be further divided is different from one location to another? This
by their standard
errors, SE β̂i1 and SE β̂i2 , yielding the t question is valid because the HRF is heuristic and
neurovascular coupling may indeed vary across
statistics and the p values used to evaluate sta-
different brain regions (between the gray matter
tistical significance regarding the BOLD activa-
and the white matter, or between regions with
tion associated with each stimulation condition
distinct vascular density).
or the contrast between conditions. See Fig. 11.9
To accommodate the possible variation in Note that the unknown coefficients, α i1 and
HRF, one can express the HRF at each voxel as a α i2 , are considered as parts of the unknown beta
Taylor’s series of the canonical HRF. values, which can be estimated from data by us-
ing least-squares estimation as discussed earlier.
hi (t) = h(t) + αi1 h (t) + αi2 h (t) (11.6) To evaluate the statistical significance, one only
needs to evaluate the significance of the model
Here, hi (t) is considered as a model of neu- as a whole (based on the F statistic) instead of
rovascular coupling specific to the i-th voxel, and the significance of each regressor (based on the t

h (t) and h (t) are the first and second derivative of statistic), because all three regressors correspond
the canonical HRF. The coefficients, α i1 and α i2 , to a single stimulus condition.
are unknown and assumed to be variable across Lastly, let us place the problem in a differ-
locations in order to account for spatial variation ent scenario, which has been overlooked in most
in the HRF. Convolving the boxcar function with fMRI studies but should, arguably, be considered.
this voxel-wise HRF generates three regressors in To understand this problem, let us recall that
the response model, as shown in Fig. 11.10. the HRF is a model of neurovascular coupling,
describing how the neural response transforms to
x1 (t) = s(t) ∗ h(t) (11.7) the BOLD response, given the external stimuli.
Regressors used to predict the BOLD response
should consider how neurons may differentially
x2 (t) = s(t) ∗ h (t) (11.8) respond to the stimuli. In fact, evidence from neu-
rophysiological studies suggests neural responses
to a sustained period (e.g., 30 s) of stimulation
x3 (t) = s(t) ∗ h (t) (11.9) may manifest themselves as a transient response
at the onset of stimulation, a sustained response
across the entire period of stimulation, a slow
yi (t) = βi1 x1 (t) + βi2 x2 (t) + βi3 x3 (t) (11.10) adaptation over the course of the stimulation pe-
Fig. 11.10 Multiple regressors resulting from convolving the boxcar function with the canonical HRF, its first-order
derivative, and its second-order derivative
Fig. 11.11 Multiple

regressors resulting from
convolving sustained,
onset, offset, and
adaptation neural responses
with the HRF, given a
block-design paradigm
riod, a transient response at the offset of stimula- yi (t) = βi1 x1 (t) + βi2 x2 (t) + βi3 x3 (t) + βi4 x4 (t)
tion, or a mixture of these response features [35] (11.15)
(Fig. 11.11).
Equation (11.1) implies that we only consider Although this model is more complex, it ac-
the possibility of sustained neural response, commodates the variation of neural response and
which can be modeled as a boxcar function likely localizes different types of responses to
identical to the function that describes the different brain regions. See the example in Fig.
stimulation. If we consider all four types of neural 11.12.
response that reflect the aforementioned onset, The models discussed in this section are ap-
sustained, adaptation, and offset response, the plicable to univariate analysis of BOLD time
response model should be revised as below. series. Being univariate means that the model is
used to explain or predict the time series signal
x1 (t) = s(t) ∗ h(t) (11.11) observed at each single voxel. It is in contrast to
multivariate pattern analysis [36], for which the
focus is on the activity pattern that spans mul-
x2 (t) = s(t)δ (t − onset) ∗ h(t) (11.12)
tiple (typically neighboring) voxels. Moreover,
the analysis discussed above is also limited to
x3 (t) = s(t)δ (t − offset) ∗ h(t) (11.13) the signal observed from each single subject. The
subject-level statistic can be used as the input to
a second-level statistical test for evaluating the
t − onset group-level significance.
x4 (t) = s(t) 1 − ∗ h(t)
offset − onset
(11.14)
Fig. 11.12 BOLD activations corresponding to sustained, onset, offset, and adaptive responses to full-screen checker-
board stimulation in a block-design paradigm
This provides a way to localize the neural

11.8 Task fMRI for Functional representation of each specific body part (see Fig.
Mapping 11.13 for an example) and to further reveal the
somatotopic organization. This strategy has been
Based on the aforementioned experimental highly effective in mapping sensory, motor, and
design and model-based analysis, fMRI has cognitive functions. The neuroscientific impact of
been widely used for functional mapping or task-based fMRI is significant, perhaps surpass-
localization. A function is exemplified by a ing any prior method for functional mapping or
specific task or stimulus. For example, pictures localization for its noninvasiveness, ease of use,
of human faces can be used as the visual and high resolution. Clinical applications of task
stimulus presented in an event-related or block- fMRI, however, remain challenging and limited,
design paradigm. The model-based time series in part because interpretation of fMRI activations,
analysis can be used to map the stimulus-evoked although established in terms of statistics, is not
activation. The activation presumably highlights straightforward as to how fMRI provides quan-
the brain regions involved in face recognition. titative evidence to support neuropathological
One may also stimulate different body parts diagnosis or treatment planning.
and map the resulting activations in the brain.
Fig. 11.13 In rats, BOLD fMRI activations with electrical stimulation applied to the forepaw
The value of fMRI has been most recognized a simple functional interpretation. The statistical
for human psychological and cognitive sciences. parametric mapping as described earlier is not
It is worth noting that fMRI is also increasingly readily applicable to resting state fMRI.
used in preclinical animal models. Animal fMRI For resting state fMRI, an established method
has its unique value. With animals, it is much is so-called the seed-based correlation. It begins
easier (and of less ethical concern) to combine with selecting a region as the seed region and then
fMRI with other invasive procedures. Combining calculating the correlation between the BOLD
fMRI with invasive neural recording or stimula- signal extracted from the seed region and the
tion is desirable to reveal the relationship between BOLD signal from every other voxel in the brain.
fMRI and neurophysiology, uncover the physio- The distribution of the resulting correlation coef-
logical mechanism of fMRI, guide neuromodu- ficients highlights where in the brain spontaneous
lation techniques for optimal effects on the brain, activity is temporally correlated with that at the
and evaluate the interaction between the brain and seed region. Since temporal correlation is simply
visceral organs, e.g., the gut. Therefore, animal interpreted as functional connectivity, a term orig-
fMRI plays an important role in brain research inally coined by Biswal et al. in his seminal paper
much beyond a backward translation from human [38], the map of correlation to a seed region is in-
fMRI and continues to be an area under active terpreted as a network, which includes all the re-
research. gions that interact with the seed region. This seed-
based correlation method is simple and effective
and has been widely used to evaluate functional
11.9 Resting State fMRI brain networks. To use this method, however, it
requires one to select a seed region, presumably
Even in the absence of any overt task, the brain is based on a specific question or hypothesis of
still active with spontaneous activity observable interest. What if one does not have any question or
with BOLD fMRI [37]. The use of fMRI to mea- hypothesis in mind? In this case, the seed region
sure and characterize brain activity in the resting is likely subject to a somewhat arbitrary choice.
state is referred to as resting state fMRI. As the In this case, such tools as InstaCorr implemented
name suggests, the resting state is not controlled in AFNI would be helpful, since it allows one to
by any task. Spontaneous activity is not driven by explore any location as the seed region, while it
any predefined experimental paradigm, and it is calculates and visualizes the seed-based network
thus not predictable by any task model that bears nearly in real time.
Alternatively, an entirely data-driven method be applied to data concatenated across a group of

can be applied to uncover functional networks subjects, yielding group-level independent com-
all at once without necessarily selecting any seed ponents that reflect functional network patterns
region as driven by a predefined hypothesis. Per- consistent across subjects [42].
haps the most established method of this type is Although seed-based correlation and ICA are
independent component analysis (ICA), or spatial seemingly distinctive methods, they often end up
ICA to be more specific [39]. For ICA, all voxels with showing comparable spatial patterns [43].
are considered altogether as elements of a high- These patterns are generally referred to as rest-
dimensional input variable (or vector), and every ing state networks and collectively depict the
time point is considered as a sample of this vari- brain’s functional organization, or brain connec-
able. The goal of ICA is to identify around tens tome [44]. It is worth noting the resting state
of components, which are high-dimensional vec- networks arising from spontaneous activity are
tors that are mutually independent, while pushing consistent with patterns of brain activation with
their linear combination to be able to explain any various tasks [45]. This consistency lends sup-
sample of the input variable. In other words, any port to the functional relevance of resting state
spatial pattern of resting state activity reflects an networks. However, the specific function of a
unknown but linear mixture of some fixed spa- resting state network is not always easy to elu-
tial patterns, each represented by an independent cidate, since it may or may not be associated
component. Collectively, these independent com- with exteroceptive processes or human conscious
ponents capture the networks onto which voxels cognition, for which one may design a relevant
are organized (see Fig. 11.14 for an example). task for fMRI experiments. Some networks, such
Learning algorithms to identify independent com- as the default-mode network [46], are intrinsic
ponents from data, e.g., the Infomax algorithm and preserved across brain states (wakefulness,
[41], have been implemented in software tools, sleep, anesthetized) and across many species (rat,
e.g., MELODIC in FSL. When ICA is applied monkey, human). Mapping resting state networks
to resting state fMRI data from a single sub- has become a mainstream focus for fMRI and
ject, the resulting independent components can holds the unique promise to facilitate further un-
be inspected to identify and remove artifacts from derstanding and effective diagnosis of neurologi-
signals in order to denoise the data. ICA can also cal disorders.
Fig. 11.14 Resting state networks obtained by applying ICA to fMRI data in rats. (Data are from Cao et al. [40])
subject reproducibility of the fMRI signal. The

11.10 Naturalistic Paradigm
high reproducibility is unique to the fMRI signal
during natural stimuli and not observable given
Functional MRI is also increasingly used with
seemingly complex and irregular stimuli that are
naturalistic paradigms. Unlike event-related or
perceptually meaningless [50]. This method is
block-design paradigms as discussed above, nat-
robust, effective, and model-free, extending the
uralistic paradigms set up a behavioral context
application of fMRI to ecologically relevant sce-
much closer to our daily life experiences. For
narios.
example, a natural visual paradigm may require a
The fact that brain responses to natural
subject to watch a movie typically for 5–10 min-
stimuli are significantly reproducible within and
utes. The movie often includes realistic scenes
across subjects can be utilized to map functional
and objects, human or animal activity, and so-
networks engaged in processing natural stimuli.
cial content. Such a movie attracts attention, en-
For this purpose, we may use inter-session or
gages cognition, and activates the brain to a much
inter-subject functional connectivity analysis
greater extent than otherwise overly simplified vi-
[51]. Specifically, when a human subject watches
sual stimuli as used in conventional fMRI studies.
the same movie twice, we can choose a seed
Similarly, a natural auditory paradigm may use
region, extract its signal from the first session,
continuous music or speech as stimuli [47, 48].
and calculate its correlation with the signal
Natural visual or auditory stimulation is
from every voxel in the second session [52].
complex, because it involves many elements or
The inter-session functional connectivity is only
features entangled in space, time, and frequency.
attributable to stimulus-driven responses instead
Disentangling such features is seemingly
of spontaneous activity, which is unrelated to
unapproachable. The model-based analysis as
stimuli and thus unlikely to be correlated between
used for event-related or block-design fMRI is
two separate sessions. Similarly, one may apply
not readily applicable to fMRI data obtained with
this analysis to data from two subjects watching
naturalistic paradigms. Given natural stimuli,
the same movie, yielding inter-subject functional
the fMRI signal appears nearly as irregular
connectivity also indicative of stimulus-driven
as spontaneous activity observed with resting
functional networks.
state fMRI. Seemingly, no tangible clue is easily
The methods described above are compelling
accessible to separate stimulus-driven responses
because of their simplicity. Nevertheless, it
from spontaneous activity.
should be noted that the correlation-based mea-
The dilemma is resolved, partially, with the
sures of inter-subject/inter-session reproducibil-
finding first reported by Hasson et al. In their
ity reveal only where in the brain is involved
seminal paper [49], evidence reported suggests
in processing natural stimuli, but not how an
that naturalistic stimuli, either a movie or an
activated voxel is involved or what information
audio story, elicit highly reproducible responses
it encodes. To answer these questions, we should
within and across subjects. When a human sub-
address the stimulus-response relationship (or
ject watches the same movie twice (in two re-
neural coding) at each voxel [53]. As mentioned
peated sessions), the responses observed in the
earlier, the complexity of natural stimuli requires
first session and the second session are signifi-
a model to be able to unpack the stimuli into can-
cantly correlated for each voxel involved in pro-
didate features, which are individually or collec-
cessing the information from the movie. When
tively represented by a voxel. Although it remains
two subjects watch the same movie, the responses
to be fully developed, a promising method, as ad-
observed from the first and second subjects are
vocated in recent studies [54–56], is to use brain-
also highly correlated for each activated voxel.
inspired deep neural networks as feature models
This finding lends support to a simple method
to address neural coding with natural stimuli.
that allows us to map brain activation with natural
The use of natural visual or auditory stimuli is
stimuli by evaluating the voxel-wise intra/inter-
emerging as a new paradigm for fMRI. It places
new challenges and opportunities for using fMRI • Seed-based correlation or independent compo-
not only to assign functions to regions (functional nent analysis can be used to map resting state
localization) but also to uncover the computa- networks.
tional role of individual locations, regions, or • Brain activation evoked by naturalistic stim-
networks (i.e., neural coding or representation) in uli can be mapped by evaluating intra/inter-
a realistic and dynamic condition being one step subject reproducibility of the BOLD signal
closer to our daily life. observed during two separate sessions of the
same stimuli.
• Brain networks evoked by naturalistic stim-
11.11 Summary uli can be mapped by evaluating intra/inter-
subject functional connectivity with the BOLD
To recap, we list the following points in sum- signal observed during two separate sessions
mary. of the same stimuli.
• fMRI is based on MRI of hydrogen protons.

• Transverse relaxation following on-resonance Homework
RF excitation depends on T2 *.
• Hemoglobin is diamagnetic when oxygenated Please mark all the correct answers for each of
but paramagnetic when deoxygenated. the following questions. Note that each ques-
• Increase in the concentration of deoxygenated tion may have one or more than one correct
hemoglobin shortens T2 *. answer.
• The bold oxygenation level–dependent signal
is the vascular response to neural activity. 1. Which of the following nuclei is the most
• Neural activation increases regional cerebral abundant for functional magnetic resonance
metabolic rate of oxygen (CMRO2 ), triggers imaging?
vessel dilation, and increases regional cerebral (A) 1 H
blood flow (CBF) and cerebral blood volume (B) 13 C
(CBV). (C) 31 P
• Increase in CBF overcompensates CMRO2 (D) 19 F
and raises the blood oxygenation level. 2. Hydrogen protons spin at about 300 MHz in a
• Neural activation increases the BOLD signal. 7 Tesla MRI system. Which of the following
• fMRI uses the BOLD signal to localize neu- is close to the gyromagnetic ratio (MHz T−1 )
ral activations, despite the incomplete under- of 1 H spins?
standing of the BOLD mechanism. (A) 42.6
• HRF describes neurovascular coupling as a (B) 6.53
linear time-invariant system. (C) 40.1
• HRF reflects the BOLD response to an im- (D) 11.3
pulse neural response, or an impulse stimulus. 3. Which of the following are true about on-
• Task fMRI typically uses event-related or resonance RF excitation?
block-design paradigms. (A) It transmits an oscillating magnetic field
• To map or localize activation, the BOLD signal along the longitudinal direction.
is compared against a response model derived (B) It transmits an oscillating magnetic field
from the experimental paradigm and the HRF. in the transverse plane.
• A voxel is activated by a stimulus or task, if (C) It transmits an oscillating magnetic field
its time series is predictable by the response with a frequency that matches the Lar-
model. mor frequency of the target spin sys-
• Resting state fMRI measures spontaneous tems.
brain activity in the absence of any overt task.
(D) It transmits energy to be effectively ab- 10. In the block design, what would be a typical
sorbed by the target spin systems. block duration?
4. Which of the following contribute to (A) 30 seconds ON vs. 30 seconds OFF
the blood oxygenation level-dependent (B) 30 milliseconds ON vs. 30 milliseconds
contrast? OFF
(A) Cerebral blood flow (C) 30 minutes ON vs. 30 minutes OFF
(B) Cerebral blood volume (D) None of the above
(C) Cerebral metabolic rate of oxygen 11. Which of the following are TRUE about rest-
(D) Myelin density ing state fMRI?
5. Which of the following regional changes oc- (A) It is used to report instrumental noises
cur accompanying local neural activation? from the MRI scanner
(A) Arterioles dilate (B) It is used to measure spontaneous brain
(B) Blood flow increases activity in the absence of overt tasks or
(C) Oxygen consumption increases stimuli
(D) Blood oxygenation level increases (C) It is used to measure fluctuations in
6. What happens when the concentration of membrane potentials around −70 mV
deoxy-hemoglobin increases? (D) None of the above
(A) Transverse relaxation becomes faster 12. Functional connectivity as observed with
(B) Transverse magnetization decays faster resting state fMRI refers to?
(C) Longitudinal relaxation becomes faster (A) Temporal correlations between the sig-
(D) Longitudinal magnetization recovers nals observed from different brain loca-
faster tions
7. Which of the following are TRUE about the (B) Anatomical connections between differ-
hemodynamic response function (HRF)? ent brain locations
(A) It indicates the hemodynamic response (C) The relationship between neural and
given an impulse input stimulus vascular signals in the brain
(B) It indicates the hemodynamic response (D) None of the above
given a sustained block of input stimulus 13. When applied to resting state fMRI data,
(C) In HRF, the peak response delays from independent component analysis
the time zero (A) Separates brain networks without speci-
(D) In HRF, the peak response precedes the fying a seed location
time zero (B) Assumes brain networks are spatially
8. How fast is the fMRI signal typically sam- independent of one another
pled? (C) Shows where in the brain is at rest
(A) Every millisecond (D) None of the above
(B) Every second 14. To map brain activations with a continuous
(C) Every minute period of naturalistic stimuli, one can
(D) Every hour (A) Calculate the voxel-wise correlation be-
9. To derive the response model (or design ma- tween the fMRI signals from a subject
trix) for the BOLD time series analysis, one during two repeated sessions of the same
needs to stimuli
(A) Convolve the stimulus paradigm with (B) Calculate the voxel-wise correlation be-
the hemodynamic response function tween the fMRI signals from two sub-
(B) Multiply the stimulus paradigm with the jects during the same stimuli
hemodynamic response function (C) Use a response model derived from
(C) Add the stimulus paradigm with the convolving a boxcar function with the
hemodynamic response function canonical HRF
(D) None of the above
(D) Calculate the seed-based correlation 14. R.B. Buxton, E.C. Wong, L.R. Frank, Dynamics of
based on the fMRI signals recorded blood flow and oxygenation changes during brain
activation: The balloon model. Magn. Reson. Med.
from a single session
39(6), 855–864 (1998)
15. P.T. Fox, M.E. Raichle, Focal physiological uncou-
pling of cerebral blood flow and oxidative metabolism
References during somatosensory stimulation in human subjects.
PNAS 83(4), 1140–1144 (1986)
16. M.K. Stehling, R. Turner, P. Mansfield, Echo-planar
1. B. He, Z. Liu, Multimodal functional neuroimaging:
imaging: Magnetic resonance imaging in a fraction of
Integrating functional MRI and EEG/MEG. IEEE
a second. Science 254(5028), 43–50 (1991)
Rev. Biomed. Eng. 1, 23–40 (2008)
17. G.H. Glover, Spiral imaging in fMRI. NeuroImage
2. R.W. Brown, Y.-C. Cheng, M.E. Haack, M.R. Thomp-
62(2), 706–712 (2012)
son, R. Venkatesan, Magnetic Resonance Imaging:
18. D.C. Noll, J.D. Cohen, C.H. Meyer, W. Schneider,
Physical Principles and Sequence Design, 2nd edn.
Spiral k-space MR imaging of cortical activation. J.
(Wiley, Hoboken, New Jersey, 2014)
Magn. Reson. Imaging 5(1), 49–56 (1995)
3. S.A. Huettel, A.W. Song, G. McCarthy, Functional
19. S. Moeller, E. Yacoub, C.A. Olman, E. Auerbach, J.
Magnetic Resonance Imaging, 3rd edn. (Oxford Uni-
Strupp, N. Harel, K. Ugurbil, Multiband multislice
versity Press, 2014)
GE-EPI at 7 tesla, with 16-fold acceleration using par-
4. Z.-P. Liang, P.C. Lauterbur, Principles of Magnetic
tial parallel imaging with application to high spatial
Resonance Imaging: A Signal Processing Perspective
and temporal whole-brain fMRI. Magn. Reson. Med.
(IEEE Press, 1999)
63(5), 1144–1153 (2010)
5. S. Ogawa, T.M. Lee, A.R. Kay, D.W. Tank, Brain
20. K. Setsompop, B.A. Gagoski, J.R. Polimeni, T.
magnetic resonance imaging with contrast depen-
Witzel, V.J. Wedeen, L.L. Wald, Blipped-controlled
dent on blood oxygenation. PNAS 87(24), 9868–9872
aliasing in parallel imaging for simultaneous mul-
(1990)
tislice echo planar imaging with reduced g-factor
6. L. Pauling, C.D. Coryell, The magnetic properties and
penalty. Magn. Reson. Med. 67(5), 1210–1224
structure of hemoglobin, oxyhemoglobin and carbon-
(2012)
monoxyhemoglobin. PNAS 22(4), 210–216 (1936)
21. N.K. Logothetis, What we can do and what we cannot
7. K.R. Thulborn, J.C. Waterton, P.M. Matthews, G.K.
do with fMRI. Nature 453(12), 869–883 (2008)
Radda, Oxygenation dependence of the transverse
22. G.C. Petzold, V.N. Murthy, Role of astrocytes in neu-
relaxation time of water protons in whole blood at
rovascular coupling. Neuron 71(5), 782–797 (2011)
high field. Biochim. Biophys. Acta 714(2), 265–270
23. N.K. Logothetis, J. Pauls, M. Augath, T. Trinath, A.
(1982)
Oeltermann, Neurophysiological investigation of the
8. R. Turner, D. Le Bihan, C.T.W. Moonen, D. De-
basis of the fMRI signal. Nature 412, 150–157 (2001)
spres, J. Frank, Echo-planar time course MRI of
24. Z. Ding, Y. Huang, S.K. Bailey, Y. Gao, L.E. Cutting,
cat brain oxygenation changes. Magn. Reson. Med.
B.P. Roger, A.T. Newton, J.C. Gore, Detection of
22(1), 159–166 (1991)
synchronous brain activity in white matter tracts at
9. P.A. Bandettini, E.C. Wong, S.R. Hinks, R.S. Tikof-
rest and under functional loading. PNAS 115(3), 595–
sky, J.S. Hyde, Time course EPI of human brain
600 (2018)
function during task activation. Magn. Reson. Med.
25. J.R. Gawryluk, E.L. Mazerolle, R.C. D’Arcy, Does
25(2), 390–397 (1992)
functional MRI detect activation in white matter?
10. K.K. Kwong, J.W. Belliveau, D.A. Chesler, I.E. Gold-
A review of emerging evidence, issues, and future
berg, R.M. Weisskoff, B.P. Poncelet, D.N. Kennedy,
directions. Front. Neurosci. 8, 239 (2014)
B.E. Hoppel, M.S. Cohen, R. Turner, Dynamic mag-
26. L. Marussich, K.-H. Lu, H. Wen, Z. Liu, Mapping
netic resonance imaging of human brain activity dur-
white-matter functional organization at rest and dur-
ing primary sensory stimulation. Proc. Natl. Acad.
ing naturalistic visual perception. NeuroImage 146,
Sci. U S A 89(12), 5675–5679 (1992)
1128–1141 (2017)
11. S. Ogawa, D.W. Tank, R. Menon, J.M. Ellermann,
27. G.M. Boynton, S.A. Engel, G.H. Glover, D.J. Heeger,
S.-G. Kim, H. Merkle, K. Ugurbil, Intrinsic signal
Linear systems analysis of functional magnetic res-
changes accompanying sensory stimulation: Func-
onance imaging in human V1. J. Neurosci. 16(13),
tional brain mapping with magnetic resonance imag-
4207–4221 (1996)
ing. PNAS 89(13), 5951–5955 (1992)
28. G.H. Glover, Deconvolution of impulse response in
12. R.B. Buxton, The physics of functional magnetic
event-related BOLD fMRI. NeuroImage 9(4), 416–
resonance imaging. Rep. Prog. Phys. 76(9), 096601
429 (1999)
(2013)
29. J. Cao, K.-H. Lu, S.T. Oleson, R.J. Phillips, D. Jaffey,
13. S.-G. Kim, S. Ogawa, Biophysical and physiological
C.L. Hendren, T.L. Powley, Z. Liu, Gastric stimu-
origins of blood oxygenation level dependent fMRI
lation drives fast BOLD responses of neural origin.
signals. J. Cereb. Blood Flow Metab. 32(7), 1188–
NeuroImage 197, 200–211 (2019)
1206 (2012)
30. L.D. Lewis, K. Setsompop, B.R. Rosen, J.R. Poli- 44. D.C. Van Essen, S.M. Smith, D.M. Barch, T.E.J.
meni, Fast fMRI can detect oscillatory neural activity Behren, E. Yacoub, K. Ugurbil, The WU-Minn hu-
in humans. PNAS 113(43), E6679–E6685 (2016) man connectome project: An overview. NeuroImage
31. K.L. Friston, A.P. Holmes, K.J. Worsley, J.-P. Poline, 80, 62–79 (2013)
C.D. Frith, R.S.J. Frackowiak, Statistical parametric 45. S.M. Smith, P.T. Fox, K.L. Miller, D.C. Glahn, M.P.
maps in functional imaging: A general linear ap- Fox, C.E. Mackay, N. Filippini, K.E. Watkins, R.
proach. Hum. Brain Mapp. 2(4), 189–210 (1994) Toro, A.R. Laird, C.F. Beckmann, Correspondence of
32. T.T. Liu, The development of event-related fMRI the brain’s functional architecture during activation
designs. NeuroImage 62(2), 1157–1162 (2012) and rest. PNAS 106(31), 13040–13045 (2009)
33. G.T. Buracas, G.M. Boynton, Efficient design of 46. M.E. Raichle, The Brain’s default mode network.
event-related fMRI experiments using M-sequences. Annu. Rev. Neurosci. 38, 433–447 (2015)
NeuroImage 16, 801–813 (2002) 47. Y. Zhang, G. Chen, H. Wen, K.-H. Lu, Z. Liu, Musical
34. A.M. Derrington, P. Lennie, Spatial and temporal imagery involves Wernicke’s area in bilateral and
contrast sensitivities of neurones in lateral geniculate anti-correlated network interactions in musicians. Sci.
nucleus of macaque. J. Physiol. 357, 219–240 (1984) Rep. 7(17066), 2017 (2017)
35. E. Duff, J. Xiong, B. Wang, R. Cunnington, P.T. 48. Y. Zhang, K. Han, R.M. Worth, Z. Liu, Connecting
Fox, G. Egan, Complex spatio-temporal dynamics of concepts in the brain by mapping cortical represen-
fMRI BOLD: A study of motor learning. NeuroImage tations of semantic relations. biorxiv, https://doi.org/
34(1), 156–168 (2007) 10.1101/649939 (2019)
36. N. Kriegeskorte, R. Goebel, P.A. Bandettini, 49. U. Hasson, Y. Nir, I. Levy, G. Fuhrmann, R. Malach,
Information-based functional brain mapping. PNAS Intersubject synchronization of cortical activity dur-
103(10), 3863–3868 (2006) ing natural vision. Science 303(5664), 1634–1640
37. M.D. Fox, M.E. Raichle, Spontaneous fluctuations (2004)
in brain activity observed with functional magnetic 50. K.-H. Lu, S. Hung, H. Wen, L. Marussich, Z. Liu,
resonance imaging. Nat. Rev. Neurosci. 8, 700–711 Mapping white-matter functional organization at rest
(2007) and during naturalistic visual perception. PLoS One
38. B. Biswal, F.Z. Yetkin, V.M. Haughton, J.S. Hyde, 11(8), e0161797 (2016)
Functional connectivity in the motor cortex of resting 51. E. Simony, C.J. Joney, J. Chen, O. Losiksky, Y. Yeshu-
human brain using echo-planar MRI. Magn. Reson. run, A. Wiesel, U. Hasson, Dynamic reconfiguration
Med. 34(4), 537–541 (1995) of the default mode network during narrative compre-
39. C.F. Bechmann, M. DeLuca, J. Devlin, S.M. Smith, hension. Nat. Commun. 7, 12141 (2016)
Investigation into resting-state connectivity using in- 52. L.K. Lynch, K.-H. Lu, H. Wen, Y. Zhang, A.J. Saykin,
dependent component analysis. Philos. Trans. R. Soc. Z. Liu, Task-evoked functional connectivity does not
B 360(1457), 1001–1013 (2005) explain functional connectivity differences between
40. J. Cao, K.-H. Lu, T.L. Powley, Z. Liu, Vagal nerve rest and task conditions. Hum. Brain Mapp. 39(12),
stimulation triggers widespread responses and alters 4939–4948 (2018)
large-scale functional connectivity in the rat brain. 53. T. Naselaris, K.N. Kay, S. Nishimoto, J.L. Gallant,
PLoS One 12(12), e0189518 (2017) Encoding and decoding in fMRI. NeuroImage 56(2),
41. A.J. Bell, T.J. Sejnowski, An information- 400–410 (2011)
maximization approach to blind separation and 54. M. Eickenberg, V.G. Gramfort, B. Thirion, Seeing it
blind deconvolution. Neural Comput. 7(6), 1129– all: Convolutional network layers map the function of
1159 (1995) the human visual system. NeuroImage 152, 184–194
42. V.D. Calhoun, J. Liu, T. Adali, A review of group ICA (2017)
for fMRI data and ICA for joint inference of imaging, 55. U. Güçlü, M.A.J. van Gerven, Deep neural networks
genetic, and ERP data. NeuroImage 45, S163–S172 reveal a gradient in the complexity of neural represen-
(2009) tations across the ventral stream. J. Neurosci. 35(27),
43. K.R.A. Van Dijk, T. Hedden, A. Venkataraman, K.C. 100005–110014 (2015)
Evans, S.W. Lazar, R.L. Buckner, Intrinsic functional 56. H. Wen, J. Shi, Y. Zhang, K.-H. Lu, J. Cao, Z. Liu,
connectivity as a tool for human connectomics: The- Neural encoding and decoding with deep learning for
ory, properties, and optimization. J. Neurophysiol. dynamic natural vision. Cereb. Cortex 28(12), 4136–
103(1), 297–321 (2010) 4160 (2018)
Photoacoustic Tomography of Neural
Systems 12
Lei Li, Junjie Yao, and Lihong V. Wang
Abstract review the principles of PAT, present the

major implementations, and summarize the
Neuroscience has become one of the most
representative neuroscience applications. We
exciting contemporary research areas with
also discuss challenges in translating PAT to
major breakthroughs expected in the coming
human brain imaging and envision its potential
decades. Modern imaging techniques have
promise.
enabled scientific understanding of the
neural system by revealing anatomical,
Keywords
functional, metabolic, and molecular in-
formation about the brain. Among these Brain · Neuroimaging · Neural activities ·
techniques, photoacoustic tomography (PAT), Photoacoustic tomography · Multiscale
drawing more and more attention, is playing imaging · Whole brain imaging · Functional
an increasingly important role in brain imaging · Optical contrasts
studies, thanks to its rich optical absorption
contrast, high spatiotemporal resolution, and
deep penetration. More importantly, PAT’s
unique scalability empowers neuroscientists
12.1 Introduction
to examine the brain at multiple spatial
to Photoacoustic
scales using the same contrast mechanism,
Tomography
bridging microscopic insights to macroscopic
observations of the brain. In this chapter, we
Photoacoustic tomography (PAT), also known as
optoacoustic tomography (OAT), refers to cross-
sectional or three-dimensional (3D) imaging of
a target, based on the photoacoustic (PA) effect.
6_12) contains supplementary material, which is available Although Alexander Graham Bell firstly reported
to authorized users. the PA effect in 1880 [1], the development of PAT
took off in the early 2000s following the advances
L. Li · L. V. Wang ()
Caltech Optical Imaging Laboratory, Department of in ultrasonic transducers, computers, and lasers.
Electrical Engineering and Andrew and Peggy Cherng Typically, in PAT, non-ionizing laser pulses (ps–
Department of Medical Engineering, California Institute ns pulse width) are directed to the target (when
of Technology, Pasadena, CA, USA
microwave or radio-frequency pulses are used,
the technology is referred to as thermoacoustic to-
J. Yao
mography). Some of the delivered optical energy
Photoacoustic Imaging Laboratory, Department of
Biomedical Engineering, Duke University, Durham, NC, is absorbed by the target and converted into heat.
USA The heat then induces a pressure rise through

350 L. Li et al.
Table 12.1 Comparison of PAT with deep-tissue (>2 mm) imaging modalitiesa [27–29]
Sensitivity
(moles of
Temporal Spatial detected Soft-tissue Functional Ionizing
Modalityb resolution resolution Throughput substance) contrast contrast radioactivity
X-ray CT 0.1 s 30– Low 10−6 Low Low Yes
100 μm
MRI Seconds to 50– Low 10−9 –10−6 High Moderate None
minutes 200 μm
PET/SPECT 0.3 s 1–2 mm Low 10−15 – NA High Yes
10−14
US ms 100– High 10−8 Moderate Moderate None
(operating 200 μm
at 5 MHz)
DOT ms Poor: High 10−12 Low High None
One-third
of imaging
depth
PAT 50 μs 100– High 10−12 High High None
(operating 200 μm
at 5 MHz)
a The high-resolution optical imaging techniques, such as multiphoton microscopy and optical coherence tomography,
cannot penetrate more than 2 mm and thus are not listed in the table
b X-ray CT X-ray computed tomography, MRI magnetic resonance imaging, PET positron emission tomography, SPECT
single-photon emission computed tomography, US ultrasound, DOT diffuse optical tomography
thermoelastic expansion. The pressure rise propa- and histologic imaging [2–13]. By exploiting
gates as an ultrasonic wave, which is referred to as exogenous contrasts, including organic dyes,
a PA wave. The PA wave is detected by ultrasonic proteins, and nanoparticles, PAT can perform
transducers to form an image by a computer. molecular imaging [14–26]. Second, PAT directly
PAT is a hybrid imaging technique that relies detects acoustic waves induced by the optical
on two forms of energy—optical and acoustic excitation, regardless of whether the photons
energy. PAT combines the rich contrasts of op- are ballistic or scattered/diffused; thus PAT
tical absorption with the high spatial resolution achieves far greater penetration than pure optical
of ultrasound detection for deep imaging in the microscopy. More importantly, acoustic waves
optical quasidiffusive and diffusive regimes. In are much less scattered inside biological tissue
Table 12.1, PAT is compared with other major (about three orders of magnitude weaker than
biomedical imaging modalities, including both optical scattering on a per unit path length basis);
optical and non-optical approaches. therefore, PAT can provide orders of magnitude
PAT inherits the advantages of both optical higher spatial resolution in deep tissue (>2 mm)
imaging and ultrasound imaging. First, PAT is than pure optical imaging technology. In addition,
sensitive to the optical absorption of molecules. the image resolution and imaging depth of
By preferentially exciting different molecules PAT are scalable with the ultrasonic frequency
with carefully selected optical wavelengths, within reach of diffuse photons. As the ultrasonic
PAT reveals optical contrasts based on the central frequency and bandwidth increase, spatial
chemical compositions. Taking advantage of resolution improves at the expense of penetration.
the endogenous absorption of hemoglobin, PAT has demonstrated high-resolution imaging
cytochrome, and DNA/RNA, for example, at scales ranging from organelles to small-animal
PAT offers anatomical,functional, metabolic, whole bodies and human organs.
12 Photoacoustic Tomography of Neural Systems 351
12.2 Photoacoustic Generation dV

= −κp + βT , (12.3)
and Propagation V
12.2.1 Initial Photoacoustic Pressure where κ is the isothermal compressibility

(5 × 10−10 Pa−1 for soft tissue or water); β
In optical excitation, there are two important is the thermal coefficient of volume expansion
timescales—thermal relaxation time and stress (4 × 10−4 K−1 for muscle); p is the pressure
relaxation time. The thermal relaxation time change in Pa; and T is the temperature change in
characterizes the heat dissipation of the absorbed K.
optical energy by thermal conduction, which can When both the thermal and stress confinement
be approximated by conditions are satisfied, the fractional volume ex-
pansion is negligible and the initial pressure rise
dc2 p0 can be derived from Eq. (12.3):
τth = , (12.1)
αth βT
p0 = . (12.4)
where α th is the thermal diffusivity of the material κ
(1.4 × 10−3 cm2 s−1 for soft tissue) and dc is The local temperature rise can be calculated as
the targeted spatial resolution. If the laser pulse
width is much shorter than τ th , the excitation ηth Ae
is said to be in thermal confinement, where the T = , (12.5)
ρCV
heat conduction is negligible during the optical
excitation. For example, for a laser pulse width where Ae is the local energy deposition density
of 20 ns, the thermal diffusion length during (J m−3 ), ηth is the percentage that is converted
the pulse period is less than 0.1 μm, which is to heat, ρ is the mass density (1000 kg m−3 for
much less than the spatial resolution that most soft tissue and water), and CV is the specific heat
PAT systems can achieve. Therefore, the thermal capacity at a constant volume (4000 J (K Kg)−1
confinement condition is easily satisfied in PAT. for muscle). The isothermal compressibility κ can
The stress relaxation time characterizes the be expressed as
pressure propagation, which can be estimated by
CP
κ= , (12.6)
dc ρvs2 CV
τs = , (12.2)
vs
where CP is the specific heat capacity at a constant
where vs is the speed of sound (1480 m s−1 in pressure in J (K Kg)−1 . We define the Grüneisen
water). Similarly, if the laser pulse width is much parameter (dimensionless) as
shorter than τ s , the excitation is said to be in stress
confinement, where the stress propagation is neg- β βvs2
Γ = = . (12.7)
ligible during the laser pulse. Under the stress κρCV CP
confinement condition, thermoelastic pressure in
the object can build up rapidly [30]. For example, Then Eq. (12.4) can be rewritten as
for PAT with a spatial resolution of 150 μm, τ th is
0.16 s and τ s is 100 ns. Typically, the laser pulse p0 = Γ ηth Ae , (12.8)
width in PAT is 1–20 ns, and thus both the thermal
confinement and stress confinement are satisfied. or
Upon a laser pulse excitation, the fractional
volume expansion of the object dV/V can be p0 = Γ ηth μa F, (12.9)
expressed as
352 L. Li et al.
where μa is the optical absorption coefficient 12.2.3 General Forward Solution

(cm−1 ) and F is the local optical fluence (J cm−2 ).
The forward solution for the general photoacous-
tic equation, shown in Eq. (12.10), can be ob-
12.2.2 General Photoacoustic tained through the Green function approach. In
Equation general, the solution to Eq. (12.10) can be ex-
pressed as

The acoustic pressure, p r , t , at location r and
t+

time t, in an acoustically homogenous medium is p r,t = dt d r G r , t; r , t
described by the following general photoacoustic −∞
equation:
2
β ∂ T r,t
,

2 ∂ 2
T r,t κvs2 ∂t
1 ∂ β
∇2 − 2 2 p r , t = − 2 ,
(12.13)
vs ∂t κvs ∂t where r and t are the source location and
(12.10) time, respectively. The Green function, in infinite
space, is given by
where T is the temperature rise. The left-hand
side of Eq. (12.10) describes the wave propaga-

r −r

tion, and the right-hand side represents the source δ t −t −
vs
term. G r , t; r , t = ,
In thermal confinement, the temperature
4π r − r
change caused by a heating source, H r , t , (12.14)
follows the thermal equation
which describes an impulse diverging spherical
∂T r,t wave. Please note that a temporal delta function
ρCV = H r,t , (12.11) is translated to a step heating function, because
∂t
the source term of the photoacoustic equation

is proportional to the first time derivative of
where H r , t is the heating function defined
the heating function, as shown in Eq. (12.12).
as the converted thermal energy per unit volume
A spatial delta function in the source term
per unit time; thus it is related to the local optical
simply represents a point acoustic source. In
power deposition Ap by H = ηth Ap and to the local
other words, the Green function describes the
optical fluence rate ∅ by H = ηth μa ∅.
response of a point absorber to a step heating
Substituting Eq. (12.11) into Eq. (12.10), we
function.
can get the photoacoustic equation below:
In thermal confinement, substituting Eqs.
(12.11) and (12.14) into Eq. (12.13) yields
2 ∂H r,t
1 ∂ β
∇2 − 2 2 p r , t = − . β ∂ 1
vs ∂t CP ∂t p r,t = d r
4π CP ∂t
(12.12) r − r
⎛ ⎞ (12.15)
The source term on the right-hand side of Eq.
⎜ r − r ⎟
(12.12) is related tothe first
time derivative of the H ⎝r ,t − ⎠.
vs
heating source, H r , t ; thus only time-variant
heating produces a pressure wave, whereas time-
invariant heating does not. The heating function can be written as the
product of a spatial absorption function and a tem-
poral illumination function under the condition of than the piezoelectric transducers. However, the
thermal confinement as in Eq. (12.16): sensitivity of the piezoelectric transducer drops as
its element size decreases, whereas the sensitivity
H r , t = Hs r Ht t . (12.16) of an optical detector is generally independent of
the element size. The optical sensors can offer
higher sensitivity when the element size is below
If Ht (t ) = δ(t ), the delta heating response of
a breaking-even point. For example, given the
an arbitrary absorbing object can be expressed as
sensitivity of Fabry-Perot sensors and PVDF-
based transducers reported in the literature, it
1 ∂ 1
p r,t = d r p0 r shows that the breaking-even point lies at 1 mm
4π vs2 ∂t vs t
⎛ diameter for a 20 MHz bandwidth.
⎞⎤

⎜ r − r ⎟⎥
δt ⎝ t − ⎠⎦ .
vs 12.3.2 General Image Reconstruction
(12.17)
According to the PA generation theory, the ini-
tial PA pressure at position −
→ r , excited− by
a
temporal delta pulse δ(t)H −
→r , is p →r =
→ − 0
Γ − r H →
12.3 Photoacoustic Detection r . Then Eq. (12.12) can be written
and Image Reconstruction as
→
12.3.1 Photoacoustic Detection 1 ∂2
p0 − r dδ(t)
∇ − 2 2 p r,t = −
2
2
.
vs ∂t vs dt
The propagating PA waves can be detected by
(12.18)
an ultrasonic transducer or transducer array for
image reconstruction. Because the ultrasonic As shown in Fig. 12.1, here we use the
transducer serves only as an acoustic receiver spherical geometry (ultrasonic detectors are
while the transmission efficiency is not important, arranged on a spherical shell) as an example.
the detector for PA measurement can be specially The pressure received by the ultrasonic detector
→
designed for optimized detection sensitivity and at −
→
r0 is p − r0 , t . For three common imaging
bandwidth. Till now, a variety of piezoelectric geometries—spherical, planar, and cylindrical

ultrasonic transducers and optical-acoustic surfaces—the
− initial pressure p0 − →r =
detectors have been used for PA measurement. p → r , t = 0 can be recovered using a universal
The piezoelectric-based detectors, which are back-projection (UBP) formula:
most widely used in PAT, have low thermal noise

and good sensitivity and provide a wide range of → → → − dΩ0
frequency selection ranging from low megahertz p0 −
r = r0 , t = −
b − r −→
r0 ,
Ω0 Ω0
to hundreds of megahertz. The piezoelectric (12.19)
transducers are also flexible for fabrication
into arrays with different geometry, including → → ∂p −
→
where b − r0 , t = 2p − r0 , t − 2t (∂t0 ) is the
r ,t
linear, planar, circular, and spherical shapes.
back-projection term related to the measurement
Optical-acoustic detectors are often based on
at position −
→r0 , 0 is the solid angle of the whole
PA-pressure-induced displacement or refractive
surface S, and d 0 is the solid angle subtended
index changes. Optical-acoustic detectors, such
by the detection element. A rigorous proof of Eq.
as Fabry-Perot ultrasound sensors and microring
(12.19) for the three common geometries can be
resonators, are easy to be miniaturized for
found in Ref. [31].
endoscopic PA applications. Typically, optical
sensors have lower sensitivity per unit area
354 L. Li et al.
by a total solid angle as shown in Eq. (12.23). A

detailed study on the reconstructions in limited-
view PAT can be found in Ref. [32].
Equation (12.19) is a unified and exact time-
domain back-projection algorithm for the three
common measurement geometries with the as-
sumption of constant speed of sound (SOS) prop-
agation from the sources to the detectors. It has
to be pointed out that significant acoustic inho-
mogeneity in the acoustic propagation path may
introduce reconstruction artifacts. To date, many
approaches, such as iterative SOS corrections and
reconstruction with two different speeds or a mea-
sured SOS map, have been developed to address
the SOS heterogeneity. Details regarding these
Fig. 12.1 In the measurement, an ultrasonic transducer methods can be found in Refs. [29, 33, 34].
at position −
→
r0 on the surface S receives PA signals emitted Other inverse reconstruction methods, includ-
from the source at −
→
r . In reconstruction, a quantity related ing time-reversal reconstruction and iterative re-
to the measurement at − →
r0 projects backward on a spherical construction, have also been widely used, and the
surface with respect to position −→
r0
details can be found in Refs. [35–46].
The Eq. (12.19)

→ reconstruction
back projects
the quantity b − r0 , t to a spherical surface cen- 12.4 Implementations
tered at position − →r0 . The first time derivative of Photoacoustic
−
→
term 2t (∂t0 ) is a ramp filter k in the frequency
∂p r ,t Tomography
domain, which suppresses the low-frequency sig-
PAT system can be classified according to
nals and amplifies the high-frequency signals. In
→ − ∂p −
→ different attributes, as shown in Fig. 12.2 [47].
practice, when k − r −→ r0 1, t (∂t0 )
r ,t
− Based on the image formation methods, PAT has

∂p −→
p → r , t ; thus we have b − →
r , t ≈ −2t ( 0 ) .
r ,t
0 0 ∂t two primary incarnations: inverse-reconstruction-
In other words, the high-frequency components based photoacoustic computed tomography
of the PA signals are the major components in (PACT) and focused-scanning-based photoacous-
the reconstruction of the initial acoustic pressure tic microscopy (PAM). Initially, single-element
inside the tissue. ultrasonic transducers were used in both PACT
In practice, the space around the tissue sample and PAM [48–50]; later multi-element ultrasonic
is sometimes limited for ultrasound detection, i.e., transducer arrays were introduced to improve
limited-view PAT. For example, we can only use a the system performance [51–55]. PAT has
half-spherical coverage to image a human breast, demonstrated anatomical, functional, metabolic,
where the solid angle for each detector on the molecular, and histologic contrasts of the
half-spherical surface with respect to a location vasculature, hemodynamics, oxygen metabolism,
inside the breast is less than 4π and may also neural activities, biomarkers, and gene expression
vary at different positions. For sources at different [9, 12, 56–65]. PAT can be implemented with
positions but with the same initial pressures, the various footprints, including benchtop, handheld,
reconstructed signal amplitude may vary at differ- endoscopic, and intravascular systems [66–72].
ent positions, resulting in reconstruction distor- PAT offers multi-dimensional imaging, covering
tion. A straightforward way to compensate for the space, time and excitation wavelengths [73–75].
reconstruction distortion due to the limited view Moreover, in most PAT implementations, the
is to normalize the reconstruction at each position spatial resolution is determined by the central
Fig. 12.2 Classification of PAT systems [47]
Fig. 12.3 PAT scales its spatial resolution with the desired penetration depth [28, 47]
frequency and bandwidth of the acoustic detec- the better the spatial resolution but the shallower
tion, which are selected mainly according to the the penetration. Thus, as shown in Fig. 12.3,
desired penetration and hence the expected fre- such high scalability enables PAT to scale spatial
quency range of the PA signals that have survived resolutions with the desired penetration depths
the tissue’s acoustic attenuation. The higher the in tissue, while a high can be maintained. As
central frequency and the broader the bandwidth, a rule of thumb, the desired depth-to-resolution
356 L. Li et al.
ratio is on the order of 200. PAT has achieved a blood vessels and background tissue in the
penetration up to 7 cm in depth [76]. The finest visible light region. Hemoglobin has two forms:
resolution that PAT has demonstrated is 90 nm oxy-hemoglobin and deoxy-hemoglobin, which
[77]. have different absorption spectra (Fig. 12.4). By
PAT provides multiscale imaging from measuring the optical absorption of hemoglobin
organelles to human organs with consistent at two wavelengths, we can estimate the relative
optical absorption contrast. By detecting the concentrations of the two forms of hemoglobin
optical absorption of biomolecules, PAT has in blood based on the following equations:
revealed versatile contrasts including both
the endogenous and exogenous contrasts. μa (λ1 ) = ln 10 [εox (λ1 ) Cox + εde (λ1 ) Cde ] ,
Endogenous contrasts are naturally presented (12.20)
inside the tissue, which do not perturb the
original microenvironment and are nontoxic.
μa (λ2 ) = ln 10 [εox (λ2 ) Cox + εde (λ2 ) Cde ] ,
The absorption spectra of the major endogenous
(12.21)
absorbers are summarized in Fig. 12.4 [47].
Thanks to their unique absorption features, where μa is the measured optical absorption; λ1
label-free PAT has so far successfully imaged and λ2 are the two wavelengths used in the
DNA/RNA, cytochromes, bilirubin, hemoglobin, measurement; εox and εde are the molar extinction
myoglobin, melanin, lipid, water, and glucose. coefficients of oxy-hemoglobin and deoxy-
Hemoglobin is the most important and most hemoglobin, respectively; and Cox and Cde are
commonly used endogenous contrast in PAT, the molar concentrations of oxy-hemoglobin and
which provides more than 100:1 contrast between deoxy-hemoglobin, respectively.
Then the concentrations of oxy-hemoglobin
and deoxy-hemoglobin can be computed as
1 εde (λ2 ) μa (λ1 ) − εde (λ1 ) μa (λ2 )

Cox = ,
ln 10 εde (λ2 ) εox (λ1 ) − εde (λ1 ) εox (λ2 )
(12.22)
1 εox (λ2 ) μa (λ1 ) − εox (λ1 ) μa (λ2 )

Cde = ,
ln 10 εde (λ2 ) εox (λ1 ) − εde (λ1 ) εox (λ2 )
(12.23)
Thus, the oxygen saturation of hemoglobin

and total hemoglobin concentration are
Cox
sO2 = , (12.24)
Cox + Cde
Fig. 12.4 Absorption spectra of common endogenous CHb = Cox + Cde , (12.25)
contrast agents in biological tissue at normal concentra-
tions [47]. DNA and RNA, 1 g L−1 in cell nuclei; bilirubin, By imaging the hemoglobin in red blood cells,
12 mg L−1 in blood; oxy-hemoglobin (HbO2) and deoxy-
PAT has measured important hemodynamic pa-
hemoglobin (HbR), 2.3 mM in blood; oxy-myoglobin
(MbO2) and reduced myoglobin (MbR), mass concentra- rameters, such as total hemoglobin concentration,
tion 0.5% in skeletal muscle; lipid, volume concentration the oxygen saturation of hemoglobin, blood flow
20% in tissue; water, 80% volume concentration in tissue;
melanin, 14.3 g L−1 in medium human skin
velocity, and metabolic rate of oxygen [56, 78– 12.4.1 Photoacoustic Computed
89]. Therefore, PAT is capable of functional and Tomography
metabolic imaging.
The near-infrared (NIR) light (from 700 to When PAT is implemented in the form of com-
1100 nm, the tissue’s “NIR optical window”) is puted tomography, a broadened laser beam illu-
least attenuated by biological tissues, because minates the tissue surface. An ultrasonic trans-
of the relatively low optical absorption of ducer array is typically placed outside the tissue to
hemoglobin, melanin, and water. Optical scat- receive the emitted acoustic waves. The received
tering in biological tissues decreases with longer PA signals are then amplified and digitized by
wavelengths. Therefore, NIR light provides PAT a data acquisition system. Finally, inverse re-
with the deepest penetration. According to Eq. construction yields a tomographic image, which
(12.9), the low optical absorption reduces PA maps the original optical energy deposition of the
signals if the optical fluence is held constant, tissue. PACT has been primarily configured in
resulting in a low detection sensitivity at depths. four detection geometries: linear, circular, spheri-
However, exogenous contrast agents, such as cal, and planar geometry, or their scanning equiv-
micro/nanoparticles, have much larger absorption alents (Fig. 12.5).
cross sections in the NIR region, which leads to Linear array-based PACT (LA-PACT) is
stronger optical absorption and thus stronger widely used for pre-clinical imaging and clinical
PA signals. So far, a great variety of exogenous translations [93–97]. The linear ultrasonic trans-
contrast agents have been explored by PAT, ducer array is relatively low cost, commercially
including dye-loaded microbubbles, organic available with a wide bandwidth selection, and
dyes, micro/nanoparticles, and reporter gene convenient to use with handheld operations. The
products. These agents have been used in PAT for pre-clinical or clinical ultrasound imaging system
molecular, genetic, and chemical imaging. Com- can be converted into LA-PACT by adding a
pared with endogenous molecules, exogenous laser excitation source, and then it can provide
contrast agents provide several advantages [90]. both ultrasonic and optical contrasts. LA-PACT
First, the structures and chemical and optical (1–5 MHz frequency range) has imaged tissues
properties of exogenous contrast agents can deep to 7 cm [76] (Fig. 12.6a) and noninvasively
be specifically engineered to enhance imaging detected sentinel lymph nodes in breast cancer
contrast and detection sensitivity and to suppress patients [98] (Fig. 12.6b). A key drawback of the
the background signal with optimal excitation. LA-PACT is the limited detection view, which
Second, conjugated with targeting agents (e.g., could result in missing detecting features that are
antibodies), exogenous contrast agents can perpendicular to the linear array. This problem
selectively bind to cell surface receptors for can be addressed by either adding acoustic
tumor cell detection. Third, exogenous contrast reflectors [99–101] or rotationally scanning the
agents can be engineered to be light or ultrasound linear array to increase the view angle coverage
sensitive for targeted drug/chemical delivery and [102, 103].
therapy. When selecting a contrast agent for a Circular/ring array-based PACT (RA-PACT)
specific application, one must carefully consider or its scanning equivalent was first explored in
its absorption spectrum, toxicity, optical stability, 2003, which demonstrated the functional PA
size, shape, composition, surface chemistry, and imaging of the rodent brain functions through an
targeting moieties [91]. intact scalp for the first time [48]. Following
this first functional PACT, the PA field has
experienced rapid growth. RA-PACT provides 2π
358 L. Li et al.
Fig. 12.5 Implementations of PACT [92]. (a) A linear laterally with 2π in-plane coverage. (c) A hemispherical
array-based PACT system, where optical fiber bundles array-based PACT system. (d) A planar array-based PACT
flank a linear ultrasonic array for light delivery. (b) A system, where a 2D Fabry-Perot interferometer is used as
circular/ring array-based PACT system, where the laser the planar ultrasonic detector array. The PA signals are
beam is broadened and homogenized by an engineered detected by raster scanning an interrogation beam over the
diffuser for illumination and the PA waves are received sensing plane of the interferometer
Fig. 12.6 (a) PA imaging of a blood-containing tube ultrasound imaging for noninvasive sentinel lymph node
in chicken tissue, where the overlaid PA and ultrasound (SLN) detection in patients with breast cancer, where the
image reveals the tube at 7 cm depth. Laser fluence, co-registered PA-ultrasound image shows the SLN and
19 mJ cm−2 at 650 nm [76]. (b) Dual-modality PA and biopsy needle. Laser fluence, 10 mJ cm−2 at 650 nm [98]
angular in-plane coverage, effectively eliminating Spherical array-based PACT (SA-PACT) or

the limited-view artifacts [51, 104–109]. The its scanning equivalent has also been developed
state-of-the-art RA-PACT, equipped with a 512- for imaging both small animals [111, 112] (Fig.
element full-ring ultrasonic transducer array, one- 12.8a, b) and human organs [113, 114] (Fig.
to-one mapped amplification and digitization, 12.8c, d). The key advantage of RA-PACT is
and advanced reconstruction algorithm, yields that it can provide near isotropic resolution in
superior performance with deep penetration, high all directions within the field of view (FOV), if
spatiotemporal resolution, and full-view fidelity dense spatial sampling is satisfied [114–119].
[29]. It has imaged in vivo whole-body dynamics Due to either the limited number of elements or
of small animals (Fig. 12.7a) and revealed 3D limited view coverage of the array itself, SA-
angiographic structures and tumors in human PACT scans the array around the tissue object
breasts within a single breath hold (Fig. 12.7b) to achieve dense spatial sampling, sacrificing the
[29, 110]. temporal resolution [119, 120]. Typically, SA-
Fig. 12.7 (a) Small-animal whole-body PACT of the mouse. Scale bar, 5 mm [29]. (b) Single-breath-hold PACT of
cancerous breasts, where the tumors are identified by dashed circles. Scale bar, 1 cm [110]
Fig. 12.8 (a) In vivo whole-body SA-PACT of a mouse. [112]. (c) SA-PACT of a human palm. Scale bar, 2 cm
Scale bar, 3 mm [111]. (b) In vivo whole-body SA-PACT [113]. (d) SA-PACT of human breasts (back-to-back im-
showing the internal organs of a mouse. Scale bar, 5 mm ages, left breast on the right). Scale bar, 5 cm [114]
PACT requires an open aperture on the spherical the PA waves reaching the sensing plane. This
detection surface for light delivery; thus, the configuration is equivalent to scanning a single-
solid angle of detection is slightly less than element transducer over the detection plane with a
4π . Particularly for human breast imaging, the sensing area equaling the size of the interrogation
detection surface is in a hemispherical form with beam. The FP sensor is transparent and can be
a solid angle of at most 2π . placed directly above the tissue without blocking
Planar array-based PACT (PA-PACT) has the excitation laser. The frequency spectrum of an
been implemented using a 2D Fabry-Perot (FP) FP sensor is primarily determined by the sensor
interferometer as the acoustic sensor [121–128]. thickness. Moreover, the FP sensor has a much
The focused interrogation beam raster scans over higher sensitivity to measure the low-frequency
the surface of the FP interferometer to record PA signals than that of the resonant piezoelectric
360 L. Li et al.
Fig. 12.9 (a) PA-PACT images of a tyrosinase- inferior cerebral vein. Top, x-y projection image; bottom,
expressing K562 tumor-bearing mouse. The tumor is y-z projection image [130]. (c) PA-PACT images of human
shown in yellow, and the blood vessels are shown in gray. peripheral limb vessels. Top left, y-z projection image;
Top, x-y projection image; bottom, y-z projection image top right, x-y projection image; bottom, slice image as
[127]. (b) PA images of the vasculature in the mouse indicated by the dashed line in the top right panel [129]
brain. A, superior sagittal sinus; B, transverse sinus; C,
detectors. For planar geometry implementation, 2.5 2.5

WFWHM = 2 × =2× λc ≈ 0.8λc ,
the FP sensor is preferred over a 2D piezoelectric kc 2π
transducer array, providing a larger number of (12.27)
elements and higher detection sensitivity. The
state-of-the-art PA-PACT has demonstrated high- For the planar and spherical geometry, the
quality imaging of both small animals [127] resolutions are nearly isotropic at the center of the
(Fig. 12.9a, b) and human extremities [129] (Fig. FOV, which can be estimated using Eq. (12.27).
12.9c). For the linear and circular geometry, the axial
Spatial resolution of PACT with an ideal full- and lateral resolutions in the imaging plane can
view detection configuration is bandwidth lim- be derived from Eq. (12.27). And the elevational
ited. Assuming that a system has a rectangular- resolution for linear and circular geometry, deter-
shaped bandwidth with a cutoff frequency fc , the mined typically by cylindrical acoustic focusing,
corresponding point spread function (PSF) can be can be written as
expressed as [131]
0.71λ0
FWHMele ≈ , (12.28)
k 3 j1 (kc R) NA
PSF(R) = c 2 , (12.26)
2π kc R where λ0 is the acoustic wavelength at the central
frequency and NA is the numerical aperture of the
where R is the radial coordinate from the point
acoustic lens. Typically, the NA of the acoustic
of observation, kc = 2πf vs
c
= 2π
λc
, λc is the cor-
lens is small (0.1–0.2) to offer large enough depth
responding wavelength at the cutoff frequency,
of focus. The elevational resolution for linear
and j1 is the spherical Bessel function of the first
and circular geometry is worse than the in-plane
kind. The full width at half maximum (FWHM)
resolution.
of the PSF is typically used to quantify the spatial
Spatial sampling in PACT should satisfy
resolution. It can be obtained that 3j1x(x) = 0.5,
the spatial Nyquist sampling theorem for
when x = 2.5. Then the FWHM can be computed
reconstruction. Nyquist sampling theorem
as
requires that the spatial sampling frequency
Fig. 12.10 Reconstructed

images with different
numbers of spatial
sampling channels. The
images show a mouse liver
acquired using a circular
array-based PACT system
should be at least twice of the highest detected 4π D 2

acoustic frequency. In other words, the sampling Nmin = , (12.30)
λ2c
at the edge of the FOV should be dense enough to
guarantee that the spatial sampling interval is no where D is the diameter of the FOV in 3D and
more than half of the acoustic wavelength. Figure λc is the corresponding wavelength at the cut-
12.10 shows, for the same FOV, how the number off frequency fc . For example, if D = 24 mm,
of spatial samples affects the reconstruction. fc = 5 MHz, then the minimum number of chan-
For the circular geometry, if the designed FOV nels for spatial sampling is Nmin ≈ 8 × 104 . Due to
is D in diameter and the central frequency of the large number of samples, spherical geometry
the circular/ring transducer is f0 , the minimum PACT typically rotates the transducer array for
number of channels for in-plane sampling can be dense spatial sampling.
computed as
πD 12.4.2 Photoacoustic Microscopy

Nmin = , (12.29)
λc /2
In PAM, typically, a focused (spherical or cylin-
where λc is the corresponding wavelength at the drical) transducer is employed to receive PA sig-
cutoff frequency fc and λc = fvsc . For example, nals primarily from the transducer’s focal zone.
if D = 24 mm, fc = 5 MHz, then the mini- The focused transducer is used for analog image
mum number of channels for in-plane sampling is reconstruction, by directly projecting the received
Nmin = 502. For an array-based PACT system, if time-domain PA signals back into the space do-
the number of element is fewer than Nmin , spatial main [50, 83, 132, 133]. To maximize the detec-
scanning is necessary to provide a dense spatial tion sensitivity, a confocal design of optical illu-
sampling. mination and acoustic detection is preferred (Fig.
For the spherical geometry, the minimum 12.11). The acoustic focusing can be achieved
number of channels for spatial sampling can either by an acoustic lens (spherical or cylindri-
be estimated by cal) affixed to a flat ultrasonic transducer or by
a curved ultrasonic element surface. With each
362 L. Li et al.
Fig. 12.11 Schematics of representative (a) AR-PAM and (b) OR-PAM
laser pulse, PAM receives a depth-resolved 1D OR-PAM was first developed in 2008 [132],
(A-line) image. Linear scanning across the tissue showing single-cell resolution with 1 mm
surface yields a 2D (B-scan) image. Raster scan- penetration. As shown in Fig. 12.11b, the
ning over the tissue yields a 3D image [28, 92, reflection-mode OR-PAM uses tightly focused
134, 135]. The axial resolution (depth-resolved laser beam and confocally aligned acoustic
resolution) is determined acoustically. Based on detection, where the optical focus is much tighter
the lateral resolution, PAM can be further clas- than the acoustic focus. The light beam passing
sified into acoustic-resolution PAM (AR-PAM, through the optical-acoustic combiner forms a
Fig. 12.11a) and optical-resolution PAM (OR- diffraction-limited spot on the tissue surface.
PAM, Fig. 12.11b). The generated acoustic waves are reflected by
AR-PAM was first developed in 2005 [49], the optical-acoustic combiner to the ultrasonic
providing tens of microns resolution over a transducer. Different configurations of OR-
3 mm penetration. As shown in Fig. 12.11a, the PAM have been implemented for improved
reflection-mode AR-PAM has been implemented detection sensitivity and imaging speed [74, 83,
using dark-field illumination and tightly focused 138, 139]. OR-PAM is capable of quantitative
ultrasonic detection. The laser light is firstly imaging of total hemoglobin concentration,
expanded by a conical lens and then focused blood oxygen saturation, blood flow velocity,
through an optical condenser. The light focus and metabolic rate of oxygen using hemoglobin
overlaps with the focal spot of the ultrasonic as the endogenous contrast (Fig. 12.13) [6, 56,
transducer, forming a confocal configuration. 85]. OR-PAM has demonstrated anatomical,
The laser beam on the tissue surface has a functional, molecular, and histological imaging
donut-shaped profile, which effectively reduces (Fig. 12.13), using endogenous contrasts (e.g.,
the PA signals from the superficial paraxial hemoglobin, DNA/RNA, melanoma, etc.) and
areas. In AR-PAM, the diffused light beam exogenous contrasts (e.g., organic dyes, proteins,
is broader than the acoustic focus, which and nanoparticles).
defines the lateral resolution of AR-PAM. AR- Spatial resolution in the axial direction (depth
APM has demonstrated functional, anatomical, direction) for both AR-PAM and OR-PAM is
and molecular imaging on small animals and determined by the acoustic bandwidth of the ul-
subcutaneous microvasculature imaging on trasonic detector, which can be expressed as [141]
humans (Fig. 12.12).
Fig. 12.12 In vivo AR-PAM image of (a) the cortical melanoma in a nude mouse [50], and (c) the vasculature
vasculature of an adult mouse with both the scalp and of a human palm [137]. Scale bar, 1 mm
skull intact [136], (b) a subcutaneously inoculated B16-
Fig. 12.13 OR-PAM images of (a) the total concentra- image of a thin slice of mouse connective tissue, showing
tion of hemoglobin, scale bar, 500 μm, (b) the oxygen the erythrocytes (bright red), cytoplasm (pinkish purple),
saturation of the hemoglobin in the area indicated by the and collagen (blue) [140]. (e) Label-free histology-like
dashed box in (a), and (c) the blood flow in the area OR-PAM image of a thin slice of cancerous breast tissue
indicated by the dashed box in (b) [56]. (d) OR-PAM [62]
0.88vs 0.51λ
Raxial ≈ , (12.31) RORlateral ≈ , (12.33)
B NA
where vs is the speed of sound of the tissue and where λ is the excitation laser wavelength and NA
B is the one-way acoustic detection bandwidth. is the numerical aperture of the optical focusing
Here we assume that the frequency response of lens.
the ultrasonic detector has a Gaussian profile. The
lateral resolution of AR-PAM is defined by the
acoustic focal spot size, which can be estimated as 12.5 Photoacoustic Tomography
for Neural Imaging
0.71λ0
RARlateral ≈ , (12.32)
NA Studying how the brain works is a grand chal-
lenge, which will not only benefit fundamental
where λ0 is the central acoustic wavelength and science but also provide the key to understand-
NA is the numerical aperture of the focused ultra- ing and treating neurological diseases, such as
sonic detector. The tight optical focus determines Alzheimer’s and Parkinson’s disease. Optical mi-
the lateral resolution of OR-PAM. If illuminated croscopy can only penetrate the first 1–2 mm even
with a diffraction-limited optical focus, the lateral in a mouse brain and face a grand challenge for
resolution of OR-PAM can be expressed as deep brain imaging. To date, most deep brain
364 L. Li et al.
studies have been based on non-optical imaging ing. With ultraviolet (UV) illumination (266 nm),
modalities, such as functional MRI (fMRI) and label-free PAM has provided histology-like im-
Doppler ultrasound. With acoustic penetration ages of the brain structures, resolving single cell
and optical contrast, PAT opens a new possibility nuclei, single capillaries, and single axons based
for multiscale brain imaging. on the DNA/RNA, hemoglobin, and lipid absorp-
tion, respectively (Fig. 12.15) [4, 63]. With NIR
illumination (1210 nm), PAM has imaged pe-
12.5.1 Photoacoustic Tomography ripheral nerves without any labeling (Fig. 12.15)
of the Brain [144].
12.5.1.1 PAM of the Brain Vasculature 12.5.1.3 Label-Free PACT of the Mouse
at Single-Cell Resolution Brain Structures
Leveraging the high spatial resolution and high Based on the endogenous contrast of hemoglobin,
sensitivity, PAM has provided multi-parametric cytochrome, lipid, and DNA/RNA, label-free
images of the mouse brain cortex at single red PAT has revealed detailed brain vasculature and
blood cell resolution [84, 139, 142]. With the other structures [145]. Taking advantages of the
ultrasonically assisted extraction of the mouse strong optical absorption of hemoglobin and
skull contour, PAM can dynamically focus on the deep penetration of low-frequency ultrasound
cortical vessels when raster scanning across the waves, PACT has imaged the whole mouse
uneven brain surface. Thus, by one raster scan, brain vasculature (coronal plane) at 50 μm
PAM achieves simultaneous quantification of to- spatial resolution in vivo (Fig. 12.16a) [102].
tal hemoglobin concentration (CHb ), oxygen sat- After removing the blood from the brain via
uration of hemoglobin (sO2 ), and cerebral blood saline perfusion, label-free PACT has revealed
flow (CBF) at the microvascular level through the detailed whole brain structures with MR image
intact skull (Fig. 12.14) [143]. quality. Spectral PACT, using visible light,
has clearly identified different brain structures
12.5.1.2 Label-Free Histology-Like (horizontal plane), including the olfactory bulb,
PAM of the Mouse Brain neocortex, corpus callosum, hippocampus,
and Peripheral Nerves inferior colliculus, and cerebellum, based on the
PAM has 100% sensitivity to optical absorption. cytochrome and lipid absorption contrast (Fig.
By utilizing the peak absorption wavelengths, 12.16b) [2]. Using NIR light, a wealth of detailed
PAM can image biomolecules of interest with brain structures have been revealed in 3D ex vivo
good sensitivity and specificity without label- (Fig. 12.16c) [146].
Fig. 12.14 Simultaneously acquired multi-parametric blue arrows in (c) represent the blood flow directions along
PAM images of the mouse cortical vasculature through the B-scan axis, and the white arrow shows an arteriole
the intact skull, which map (a) CHb , (b) sO2 , and (c) whose orientation is nearly perpendicular to the B-scan
CBF (both speed and direction) at high spatial resolution, axis. Imaging wavelengths, 532 nm and 559 nm. Scale bar,
respectively. The arrows in (b) show a pair of cortical 0.5 mm [143]
arteriole and venule with distinct sO2 values. The red and
Fig. 12.15 Image gallery of features from label-free hemoglobin absorption at UV light, (c) lipid absorption
PAM images of the mouse brain and peripheral nerves, at UV light, (d) lipid absorption at NIR light, and (e) both
based on (a) DNA/RNA absorption at UV light, (b) DNA/RNA and lipid absorption at UV light [63, 144]
Fig. 12.16 (a) PACT image of the whole mouse brain the adult mouse brain ex vivo. Illumination wavelength,
vasculature (coronal plane, bregma, −1.0 mm). Scale bar, 740 nm; scale bar, 1 mm; V2MM, secondary visual cortex,
2 mm [102]. (b) PACT image of the saline-perfused mouse medio-medial; CA1, hippocampal CA1 area; DG, dentate
brain (horizontal plane) at 2.8 mm depth, clearly resolv- gyrus; D3V, dorsal third ventricle; ZID, zona incerta dor-
ing the structures of the brain. Illumination wavelength, sal; SNr, substantia nigra reticulata; VTA, ventral tegmen-
620 nm; scale bar, 1 mm; OB, olfactory bulb; Nc, neocor- tal area; IFN, inter-fascicular nucleus; M1, motor cortex
tex; CC, corpus callosum; Hp, hippocampus; IC, inferior 1; CG, cingulum; MCLH, magnocellular lateral hypotha-
colliculus; Cb, cerebellum [2]. (c) 3D PACT images of lamus; MCPO, magnocellular preoptic nucleus [146]
366 L. Li et al.
12.5.1.4 Spectral PAT for Advanced OR-PAM with a head-restrained an-

Neuroimaging imal imaging platform has been implemented to
In PAT, employing exogenous contrast agents assess cortical hemodynamics in the awake ro-
can effectively improve the detection sensitivity dent brain at the microscopic level. OR-PAM can
and specificity and enhance the imaging contrast quantitatively characterize the hemodynamic re-
by suppressing the background signals. Labeled sponses of the mouse brain to isoflurane, includ-
by Congo red, amyloid plaques in an Alzheimer ing diameter-dependent arterial dilation, elevated
diseased mouse brain can be visualized by OR- blood flow, and reduced oxygen extraction (Fig.
PAM in vivo through intact skull (Fig. 12.17a, 12.19) [58].
b) [147]. Assisted by immunochemistry staining
of Tuj1, OR-PAM has imaged densely packed 12.5.2.2 Mapping Large-Scale Neural
neurons (Fig. 12.17c) and resolved single neuron Activities
cell bodies, dendrites, and nuclei (Fig. 12.17d) The high spatiotemporal resolution and whole
[148]. PACT has identified U87 glioblastoma brain imaging capability enable PACT to capture
tumors in the rodent brains labeled either by iRFP large brain compartments of small animals to
or by IRDye800 and peptide (Fig. 12.17e–h) [21, assess large-scale neural activities. PACT has im-
146, 149]. Combining differential imaging or aged the resting-state functional connectivity of
frequency lock-in detection with photoswitchable the mouse cortex to map the distributed activity of
proteins can significantly improve PACT’s cortical circuits (Fig. 12.20a, b) [154]. PACT has
detection sensitivity (Fig. 12.17i, j) [150–152]. monitored the whole rat brain spontaneous hemo-
Further, using two different photoswitching dynamic responses and studied the whole brain
proteins, PACT also achieved quantitative multi- functional connectivity (Fig. 12.20c–e) [29]. This
contrast molecular imaging in the brain (Fig. demonstrated the potential of PACT as a high-
12.17j) [150]. resolution imaging tool for studying large-scale
networks of the entire brain.
12.5.2 Photoacoustic Tomography 12.5.2.3 Imaging Brain Diseases

of Neural Activities at the Whole Brain Level
Despite the remarkable progress in visualizing
12.5.2.1 Monitoring Brain cellular and molecular processes, neuroscience is
Hemodynamic Responses still looking for a general theory about how brain
at Multiple Scales circuit dysfunction can lead to neurological and
First published in 2003, functional PACT im- neuropsychiatric diseases. With the capability of
aged brain functions of rats through intact scalp monitoring neuronal activity and hemodynamics
(Fig. 12.18a). Figure 12.18a shows that one-sided at multiple scales, photoacoustic neuroimaging
whisker movement activated the hemodynamic has shown encouraging results in studying a va-
responses in the contralateral side of the brain riety of brain disorders and diseases, including
[48]. The advanced OR-PAM with a high volu- brain tumors [18, 149–151], traumatic disorders
metric imaging speed has imaged the brain re- [155], stroke [156–158], and seizures of various
sponses to the electrical hind paw stimulation etiologies [102, 159–161]. For example, PACT
at single capillary resolution in real time (Fig. has visualized the propagation of the epileptic
12.18b) [139]. Employing two different wave- waves during a seizure (Fig. 12.21a–d) [102] and
lengths, functional OR-PAM has measured the simultaneously monitored the blood oxygenation
transient responses to a single visual stimulation responses at multiple locations inside the brain
and visualized the process of single RBCs re- (Fig. 12.21e, f) [159].
leasing oxygen in the mouse brain (Fig. 12.18c)
[153].
Fig. 12.17 (a) OR-PAM image of a mouse brain with U87 glioblastoma is highlighted by the dashed circle. (h)
Alzheimer’s disease, acquired at 570 nm. (b) The pro- PACT image of the tumor-bearing mouse brain showing
cessed dual-contrast OR-PAM image showing the amyloid the blood vessels in red and U87 glioblastoma in yellow.
plaques in green and the blood vessels in red. Scale bar, Scale bar, 1 mm [149]. (i) PACT images of the RpBphP1-
100 μm [147]. (c) OR-PAM image of embryoid body- expressing U87 tumor in the mouse brain. By subtracting
derived neurons, stained with anti-neurofilament/HRP- the OFF image (photoswitchable protein is in the OFF
secondary antibody/DAB. Scale bar, 200 μm. (d) The state) from the ON image (photoswitchable protein is in
close-up image of the dashed box region in (a) showing the ON state), the tumor is highlighted and background
clearly resolved nucleus and dendrite. Scale bar, 50 μm signals from blood have been suppressed. Scale bar, 2 mm
[148]. PACT of iRFP expressing U87 glioblastoma in [151]. (j) Two tumors are differentiated based on the
the subcortical brain overlaid on (e) histological and (f) photoswitching rates of two different photoswitchable
anatomical PA images. Scale bar, 1 mm [21]. (g) Photo- proteins. Scale bar, 2 mm [150]
graph of the excised tumor-bearing mouse brain, where
12.5.2.4 Imaging Brain Glucose a glucose analog, 2-NBDG, which can diffuse
Metabolism across the blood-brain barrier and provide exoge-
Similar to positron emission tomography (PET) nous PA contrast, spectral PACT has simultane-
that is widely used for imaging metabolic rate ously imaged both hemodynamic and glucose re-
of glucose in the brain, PACT can noninvasively sponses of a mouse brain to forepaw stimulations
measure the glucose consumption in the mouse (Fig. 12.22) [57]. Unlike PET, PACT can be used
brain at high spatiotemporal resolution. Using
368 L. Li et al.
Fig. 12.18 PAT of brain hemodynamic responses to stim- OR-PAM of single RBC responses to the visual stimu-
ulations. (a) PACT of rat hemodynamic responses to vi- lation in the visual cortex. A flashing LED was used to
brational stimulations on the whiskers, showing the in- stimulate the eye (left panel), and transient responses to
creased total hemoglobin concentration in the contralateral a single visual stimulation were measured. The magnitude
somatosensory cortex [48]. (b) OR-PAM of mouse brain of oxygen saturation gradient (∇sO2 ), blood flow speed
responses to electrical hind paw stimulations. LH, left (vf ), and oxygen uploading rate (rO2 ) increase upon stim-
hemisphere; RH, right hemisphere; LHS, left hind paw ulation (right panel). Scale bar in the middle panel, 10 μm
stimulation; RHS, right hind paw stimulation [139]. (c) [153]
to monitor brain metabolism over a long period of indicators, such as voltage-/calcium-sensitive dye
time without the ionizing radiation. or proteins, PAT can image the action potentials
that are direct reflections of neural activities.
12.5.2.5 Visualizing Neural Activities Using a voltage-sensitive dye, dipicrylamine
Using Voltage-/Calcium- (DPA), spectral OR-PAM has successfully
Sensitive Indicators separated the voltage responses from the
Previously demonstrated functional PAT of hemodynamic responses in a mouse brain
the brain responses and neural activities is to the electrical stimulations (Fig. 12.23a–c)
primarily based on the hemodynamics via the [162]. Combining PACT with transgenic animal
neurovascular coupling, which is an indirect models, such as GCaMP5G zebrafish, high-
measurement of the activities of brain neuron resolution imaging of Ca2+ dynamics in neural
circuits. Employing voltage-/calcium-sensitive
Fig. 12.19 (a) Schematic of functional OR-PAM with a (OFF) and presence (ON) of isoflurane have been mea-
head-restrained animal imaging platform. PBS, polarizing sured. The white arrows indicate the isoflurane-induced
beam splitter; NDF, neutral density filter; BS, beam sam- changes in sO2 and blood flow speed. Scale bar, 500 μm.
pler; SMF, single-mode fiber. (b) Hemodynamic and oxy- (c) Quantitative analysis of the isoflurane-induced changes
gen metabolic responses of the normoxic mouse brain to in the average CHb , sO2 , and flow speed of the feeding and
isoflurane. CHb , sO2 , and blood flow speed in the absence draining vessels [58]
370 L. Li et al.
Fig. 12.20 (a) PACT of the functional resting-state con- VI, visual cortex; TE, temporal cortex; RS, retrosplenial
nectivity maps in a mouse brain cortex, correlation maps of cortex [154]. (c) PACT of the functional resting-state
the main functional regions (top row), the four subregions connectivity maps in a rat whole brain (coronal plane). (d)
of the somatosensory cortex (bottom row, left panel), and Segmentations of different functional regions of the brain.
the three subregions of the visual cortex (bottom row, (e) Correlation matrix of the segmented functional regions
right panel). The white circles indicate the seed locations labeled in (d). Hip, hippocampus; M1, primary motor
for the correlation computation. (b) Correlation maps cortex; M2, secondary motor cortex; RSD, retrosplenial
of functional regions with the corresponding segmented dysgranular cortex; RSGc, retrosplenial granular cortex;
regions (inset). Dashed black lines are shown for added S1Sh, primary somatosensory-shoulder region; S1HL, pri-
visualization. OB, olfactory bulb; L, limbic cortex; M, mo- mary somatosensory cortex-hindlimb region; Thal, thala-
tor cortex; SS, somatosensory cortex; PA, parietal cortex; mus [29]
circuits across the entire zebrafish brain has been [164, 165]. Strong optical absorption of indicators
achieved (Fig. 12.23d, e) [163]. in the red and NIR spectral ranges is preferred.
In addition, the other characteristics of indica-
12.5.2.6 Outlook tors, such as the voltage-/calcium-sensing mech-
Overall, PAT is a powerful tool for imaging brain anism, lifetime, and response time, need to be
functions, complementary to other brain imaging tailored for optimized PA contrast. NIR bacterio-
modalities in its contrast mechanism, spatiotem- phytochromes could be candidates for reporting
poral resolution, speed, and penetration. The in- action potentials [2]. The major barrier of trans-
creasing applications of PAT in brain reserch and lating PAT to adult human brain imaging is the
clinical translations provide strong momentum skull. The adult human skull (5–11 mm thick)
for PAT’s development. The exciting research severely attenuates the excitation light and the
and translational capabilities of PAT come with emitted PA waves (beyond 1 MHz) and strongly
several technical challenges, but none are beyond distorts the PA waveforms, resulting in low image
reach [1]. For imaging neural activities in the quality. A potential solution is to combine PACT
deep brain, novel voltage-/calcium-sensitive in- with X-ray CT or MRI, which can provide accu-
dicators of action potentials need to be found rate skull information to correct for the wavefront
or engineered. Most of the currently available distortion [3]. NIR light is still attenuated strongly
voltage/calcium indicators operate in the visible by the skull, which hinders imaging the whole
wavelength range, which limits tissue penetration human brain using PACT. As another source of
Fig. 12.21 (a–d) PACT imaged epileptic activities of epileptic wave propagation directions. Scale bar, 2 mm
a mouse brain during a seizure at different times after [102]. (e) PACT images of a mouse brain in horizontal
the injection of 4-aminopyridine solution. The fractional and coronal planes, respectively. Colored outlines specify
changes in the PA amplitude are overlaid on the anatomic different brain regions. TA, thalamus; PC, parietal cor-
image (bregma −1.0 mm). The arrow indicates the in- tex; FL, frontal lobe. (f) Changes of the oxy-hemoglobin
jection site and the dashed green arrows indicate the and deoxy-hemoglobin concentrations from locations in-
dicated in (e) [159]
non-ionizing radiation, microwaves can also heat 5. Given dc = 1 mm or 0.01 mm, compute τ th
tissues by producing molecular rotations and tor- and τ s in muscles.
sions [166, 167]. The human skull is more trans- 6. Derive the photoacoustic equation shown in
parent to microwaves than to photons. Thermoa- Eq. (12.10).
coustic tomography (TAT) [168–171], utilizing 7. Show that the time reversal of the temporal
microwave pulses instead of laser pulses, can function is equivalent to the complex conju-
potentially extend the penetration depth beyond gation of the temporal spectrum.
10 cm and enable deep human brain imaging. 8. Use Eq. (12.17) to derive and plot the PA
pressure wave as a function of time observed
outside a sphere excited by (a) a delta pulse
Homework and (b) a Gaussian pulse.
9. Use Eq. (12.17) to derive and plot the pres-
1. Show that the units of the pressure and the sure wave as a function of time observed
energy density are the same. outside a line object excited by (a) a delta
2. Estimate the temperature and the initial pres- pulse and (b) a Gaussian pulse.
sure rises upon short-pulsed laser excitation 10. The line in Question 9 has a finite length;
of whole arterial blood at the body tempera- please simulate the PA pressure wave de-
ture, with an optical fluence of 10 mJ cm−2 tected by (a) a linear transducer array and
at 532 nm. (b) a ring array (see the geometry below).
3. Estimate the local initial pressure rise per Please reconstruct the PA image using the
one-degree local temperature rise at the body forward data from the linear array and the
temperature. ring array. Hint: please use the MATLAB k-
4. In water, estimate the fractional PA amplitude wave toolbox for both the forward and recon-
change upon one-degree local temperature struction simulations. Please download the k-
rise with the baseline temperature of (a) 20 ◦ C wave toolbox from http://www.k-wave.org/.
and (b) 37 ◦ C.
372 L. Li et al.
Fig. 12.22 (a) Chemical structure of 2-NBDG changes of 2-NBDG concentration (shown in blue) and
(C12 H14 N4 O8 ). (b) Molar extinction coefficients of total hemoglobin concentration (shown in red), overlaid
2-NBDG, oxy-hemoglobin, and deoxy-hemoglobin. on the resting-state image at 570 nm (shown in gray).
Fractional changes of the PA amplitudes (shown in color) (d) Relative changes of 2-NBDG and total hemoglobin
acquired at (c) 478 nm and (d) 570 nm in response to concentrations averaged over three mice. Error bars:
the right paw stimulation (RPS) and left paw stimulation standard deviation. The p values were computed from
(LPS), overlaid on the resting-state image (shown in gray). the paired Student’s t-test between stimulated states and
(e) Spectrally unmixed images showing the fractional resting states [57]
Fig. 12.23 (a) OR-PAM image of the DPA-stained spectra of calcium-bound and calcium-free GCaMP5G.
mouse brain. (b) Time sequence for electrical stimulation. (D) PACT (left) and planar epi-fluorescence (right) images
(c) Voltage (red) and hemodynamic (black) responses of of the zebrafish brain before (15.0 s) and after exposure to
the brain in the frequency domain. The signal-to-noise the neurostimulant agent (78.0 and 95.0 s), showing both
ratios (SNRs) of the voltage and hemodynamic response PA and fluorescence detected calcium responses [163]
peaks are quantified, respectively [162]. (d) Absorption
11. For the circular geometry, if the designed

imaging FOV is 25 mm in diameter, to sat- References
isfy the spatial Nyquist sampling require-
1. A.G. Bell, ART. XXXIV.–On the production and
ment, what is the minimum number of sam- reproduction of sound by light. Am. J. Sci. (1880–
pling channels for detection at a cutoff fre- 1910) 20(118), 305 (1880)
quency of (a) 2.25 MHz and (b) 15 MHz? 2. L. Li et al., Label-free photoacoustic tomography
of whole mouse brain structures ex vivo. NEUROW
12. Under the same conditions in Q.10, please
3(3), 035001 (2016)
calculate the minimum number of sampling 3. T.T. Wong et al., Use of a single xenon flash lamp for
channels for the full spherical geometry. photoacoustic computed tomography of multiple-
13. Derive Eq. (12.31), assuming the frequency centimeter-thick biological tissue ex vivo and a
whole mouse body in vivo. J. Biomed. Opt. 22(4),
response of the detector has a Gaussian pro-
041003 (2016)
file. 4. T. Imai et al., High-throughput ultraviolet photoa-
14. Assuming you are engineering a “perfect” PA coustic microscopy with multifocal excitation. J.
contrast agent for molecular imaging, please Biomed. Opt. 23(3), 036007 (2018)
5. Y. Qu et al., Dichroism-sensitive photoacous-
list all the desired key characteristics and
tic computed tomography. Optica 5(4), 495–501
explain the reasons. (2018)
374 L. Li et al.
6. J. Yao et al., In vivo photoacoustic imaging of trans- 22. C. Vinegoni et al., Transillumination fluorescence
verse blood flow by using Doppler broadening of imaging in mice using biocompatible upconverting
bandwidth. Opt. Lett. 35(9), 1419–1421 (2010) nanoparticles. Opt. Lett. 34(17), 2566–2568 (2009)
7. D.-K. Yao et al., Optimal ultraviolet wavelength 23. A.C. Stiel et al., High-contrast imaging of reversibly
for in vivo photoacoustic imaging of cell nuclei. J. switchable fluorescent proteins via temporally un-
Biomed. Opt. 17(5), 056004 (2012) mixed multispectral optoacoustic tomography. Opt.
8. D.-K. Yao et al., In vivo label-free photoacoustic Lett. 40(3), 367–370 (2015)
microscopy of cell nuclei by excitation of DNA and 24. G.S. Filonov et al., Deep-tissue photoacoustic to-
RNA. Opt. Lett. 35(24), 4139–4141 (2010) mography of a genetically encoded near-infrared
9. L. Lin et al., In vivo photoacoustic tomography fluorescent probe. Angew. Chem. Int. Ed. 51(6),
of myoglobin oxygen saturation. J. Biomed. Opt. 1448–1451 (2012)
21(6), 061002 (2015) 25. C. Kim et al., In vivo photoacoustic mapping of lym-
10. X.L. Deán-Ben, D. Razansky, Functional optoacous- phatic systems with plasmon-resonant nanostars. J.
tic human angiography with handheld video rate Mater. Chem. 21(9), 2841–2844 (2011)
three dimensional scanner. Photo-Dermatology 1(3– 26. J. Yao et al., Evans blue dye-enhanced capillary-
4), 68–73 (2013) resolution photoacoustic microscopy in vivo. J.
11. S. Gottschalk et al., Noninvasive real-time visualiza- Biomed. Opt. 14(5). 054049 (2009)
tion of multiple cerebral hemodynamic parameters 27. M. Baker, Whole-animal imaging: The whole pic-
in whole mouse brains using five-dimensional op- ture. Nature 463(7283), 977–980 (2010)
toacoustic tomography. J. Cereb. Blood Flow Metab. 28. L.H.V. Wang, S. Hu, Photoacoustic tomography: In
35(4), 531–535 (2015) vivo imaging from organelles to organs. Science
12. D. Razansky, A. Buehler, V. Ntziachristos, Volu- 335(6075), 1458–1462 (2012)
metric real-time multispectral optoacoustic tomog- 29. L. Li et al., Single-impulse panoramic photoacoustic
raphy of biomarkers. Nat. Protoc. 6(8), 1121–1129 computed tomography of small-animal whole-body
(2011) dynamics at high spatiotemporal resolution. Nat.
13. Y. Li et al., Snapshot photoacoustic topography Biomed. Eng. 1, 0071 (2017)
through an ergodic relay for high-throughput imag- 30. V.E. Gusev, A.A. Karabutov, Laser optoacoustics.
ing of optical absorption. Nat. Photonics 14(3), 1–7 NASA STI/Recon Technical Report A 93 (1991)
(2020) 31. M. Xu, L.V. Wang, Universal back-projection al-
14. V. Ntziachristos, D. Razansky, Molecular imaging gorithm for photoacoustic computed tomography.
by means of multispectral optoacoustic tomography Phys. Rev. E 71(1), 016706 (2005)
(MSOT). Chem. Rev. 110(5), 2783–2794 (2010) 32. Y. Xu et al., Reconstructions in limited-view ther-
15. D. Razansky, C. Vinegoni, V. Ntziachristos, Multi- moacoustic tomography. Med. Phys. 31(4), 724–733
spectral photoacoustic imaging of fluorochromes in (2004)
small animals. Opt. Lett. 32(19), 2891–2893 (2007) 33. Y. Xu, L.V. Wang, Effects of acoustic heterogeneity
16. A. Taruttis et al., Real-time imaging of cardio- in breast thermoacoustic tomography. IEEE Trans.
vascular dynamics and circulating gold nanorods Ultrason. Ferroelectr. Freq. Control 50(9), 1134–
with multispectral optoacoustic tomography. Opt. 1146 (2003)
Express 18(19), 19592–19602 (2010) 34. J. Xia et al., Enhancement of photoacoustic tomog-
17. E. Herzog et al., Optical imaging of cancer het- raphy by ultrasonic computed tomography based
erogeneity with multispectral optoacoustic tomogra- on optical excitation of elements of a full-ring
phy. Radiology 263(2), 461–468 (2012) transducer array. Opt. Lett. 38(16), 3140–3143
18. N.C. Burton et al., Multispectral Opto-Acoustic To- (2013)
mography (MSOT) of the brain and glioblastoma 35. K. Wang et al., Investigation of iterative image re-
characterization. NeuroImage 65, 522–528 (2013) construction in three-dimensional optoacoustic to-
19. J. Yao et al., Reversibly switchable photoacoustic tomography. Phys. Med. Biol. 57(17), 5399–5423
mography using a genetically encoded near-infrared (2012)
phytochrome, in Photons Plus Ultrasound: Imaging 36. C. Huang et al., Full-wave iterative image recon-
and Sensing 2016 97082U (2016) struction in photoacoustic tomography with acous-
20. L. Li et al., In vivo photoacoustic multi-contrast tically inhomogeneous media. IEEE Trans. Med.
imaging and detection of protein interactions using Imaging 32(6), 1097–1110 (2013)
a small near-infrared photochromic protein, in Pho- 37. Q. Sheng et al., A constrained variable projection
tons Plus Ultrasound: Imaging and Sensing 2019 reconstruction method for photoacoustic computed
1087818 (2019) tomography without accurate knowledge of trans-
21. N.C. Deliolanis et al., Deep-tissue reporter-gene ducer responses. IEEE Trans. Med. Imaging 34(12),
imaging with fluorescence and optoacoustic tomog- 2443–2458 (2015)
raphy: A performance overview. Mol. Imaging Biol. 38. J. Poudel et al., Mitigation of artifacts due to iso-
16(5), 652–660 (2014) lated acoustic heterogeneities in photoacoustic com-
puted tomography using a variable data truncation-
based reconstruction method. J. Biomed. Opt. 22(4), navigation in intestines in vivo. Sci. Robot. 4(32),
041018 (2017) eaax0613 (2019)
39. T.P. Matthews et al., Parameterized joint reconstruc- 56. J. Yao et al., Label-free oxygen-metabolic
tion of the initial pressure and sound speed dis- photoacoustic microscopy in vivo, SPIE
tributions for photoacoustic computed tomography. (2011)
SIAM J. Imag. Sci. 11(2), 1560–1588 (2018) 57. J. Yao et al., Noninvasive photoacoustic computed
40. B.E. Treeby, B.T. Cox, k-Wave: MATLAB toolbox tomography of mouse brain metabolism in vivo.
for the simulation and reconstruction of photoacous- NeuroImage 64, 257–266 (2013)
tic wave fields, SPIE (2010) 58. R. Cao et al., Functional and oxygen-metabolic pho-
41. B.E. Treeby, E.Z. Zhang, B.T. Cox, Photoacoustic toacoustic microscopy of the awake mouse brain.
tomography in absorbing acoustic media using time NeuroImage 150, 77–87 (2017)
reversal. Inverse Probl. 26(11), 115003 (2010) 59. L. Li et al., Photoacoustic imaging of lacZ gene
42. B.E. Treeby et al., Modeling nonlinear ultrasound expression in vivo. J. Biomed. Opt. 12(2), 020504
propagation in heterogeneous media with power law (2007)
absorption using a k-space pseudospectral method. 60. X. Cai et al., Multi-scale molecular photoacous-
J. Acoust. Soc. Am. 131(6), 4324–4336 (2012) tic tomography of gene expression. PloS one 7(8),
43. X.L. Dean-Ben et al., Accurate model-based re- e43999 (2012)
construction algorithm for three-dimensional op- 61. J. Aguirre et al., Precision assessment of label-free
toacoustic tomography. IEEE Trans. Med. Imaging psoriasis biomarkers with ultra-broadband optoa-
31(10), 1922–1928 (2012) coustic mesoscopy. Nat. Biomed. Eng. 1(5), 0068
44. A. Rosenthal, V. Ntziachristos, D. Razansky, Model- (2017)
based optoacoustic inversion with arbitrary-shape 62. T.T.W. Wong et al., Fast label-free multilayered
detectors. Med. Phys. 38(7), 4285–4295 (2011) histology-like imaging of human breast cancer by
45. A. Rosenthal, V. Ntziachristos, D. Razansky, Acous- photoacoustic microscopy. Sci. Adv. 3(5), e1602168
tic inversion in optoacoustic tomography: A review. (2017)
Curr. Med. Imaging Rev. 9(4), 318–336 (2013) 63. T.T.W. Wong et al., Label-free automated
46. T. Jetzfellner et al., Performance of iterative optoa- three-dimensional imaging of whole organs by
coustic tomography with experimental data. Appl. microtomy-assisted photoacoustic microscopy. Nat.
Phys. Lett. 95(1), 013703 (2009) Commun. 8(1), 1386 (2017)
47. J. Yao, L.V. Wang, Sensitivity of photoacoustic mi- 64. L. Lin et al., In vivo deep brain imaging of rats us-
croscopy. Photo-Dermatology 2(2), 87–101 (2014) ing oral-cavity illuminated photoacoustic computed
48. X. Wang et al., Noninvasive laser-induced photoa- tomography. J. Biomed. Opt. 20(1), 016019–016019
coustic tomography for structural and functional in (2015)
vivo imaging of the brain. Nat. Biotechnol. 21(7), 65. P. Zhang et al., In vivo superresolution photoacoustic
803 (2003) computed tomography by localization of single dyed
49. K. Maslov, G. Stoica, L.V. Wang, In vivo dark- droplets. Light-Sci. Appl. 8(1), 1–9 (2019)
field reflection-mode photoacoustic microscopy. 66. F. Knieling et al., Multispectral optoacoustic tomog-
Opt. Lett. 30(6), 625–627 (2005) raphy for assessment of Crohn’s disease activity. N.
50. H.F. Zhang et al., Functional photoacoustic mi- Engl. J. Med. 376(13), 1292–1294 (2017)
croscopy for high-resolution and noninvasive in vivo 67. J.-M. Yang et al., Photoacoustic endoscopy. Opt.
imaging. Nat. Biotechnol. 24(7), 848 (2006) Lett. 34(10), 1591–1593 (2009)
51. J. Gamelin et al., A real-time photoacoustic to- 68. J.-M. Yang et al., Simultaneous functional photoa-
mography system for small animals. Opt. Express coustic and ultrasonic endoscopy of internal organs
17(13), 10489–10498 (2009) in vivo. Nat. Med. 18(8), 1297 (2012)
52. L. Song, K. Maslov, L.V. Wang, Multifocal optical- 69. B. Dong et al., Photoacoustic probe using a mi-
resolution photoacoustic microscopy in vivo. Opt. croring resonator ultrasonic sensor for endoscopic
Lett. 36(7), 1236–1238 (2011) applications. Opt. Lett. 39(15), 4372–4375 (2014)
53. H.-P.F. Brecht et al., Whole-body three-dimensional 70. K. Jansen et al., Intravascular photoacoustic imaging
optoacoustic tomography system for small animals. of human coronary atherosclerosis. Opt. Lett. 36(5),
J. Biomed. Opt. 14(6), 064007 (2009) 597–599 (2011)
54. E. Z. Zhang, J. Laufer, P. Beard, Three-dimensional 71. S. Sethuraman et al., Spectroscopic intravascular
photoacoustic imaging of vascular anatomy in small photoacoustic imaging to differentiate atheroscle-
animals using an optical detection system, in Pho- rotic plaques. Opt. Express 16(5), 3362–3367
tons Plus Ultrasound: Imaging and Sensing 2007: (2008)
The Eighth Conference on Biomedical Thermoa- 72. L. Lin et al., Handheld optical-resolution photoa-
coustics, Optoacoustics, and Acousto-optics 64370S coustic microscopy. J. Biomed. Opt. 22(4), 041002
(2007) (2016)
55. Z. Wu et al., A microrobotic system guided by 73. J.-T. Oh et al., Three-dimensional imaging of skin
photoacoustic computed tomography for targeted melanoma in vivo by dual-wavelength photoacous-
376 L. Li et al.
tic microscopy. J. Biomed. Opt. 11(3), 034032 89. X. Wang et al., Noninvasive imaging of hemoglobin
(2006) concentration and oxygenation in the rat brain using
74. L. Li et al., Fully motorized optical-resolution pho- high-resolution photoacoustic tomography, SPIE
toacoustic microscopy. Opt. Lett. 39(7), 2117–2120 (2006)
(2014) 90. L. Li, J. Yao, L.V. Wang, Photoacoustic tomography
75. X. Luís Deán-Ben, D. Razansky, Adding fifth di- enhanced by nanoparticles, in Wiley Encyclopedia of
mension to optoacoustic imaging: Volumetric time- Electrical and Electronics Engineering, (2016), pp.
resolved spectrally enriched tomography. Light-Sci 1–14
Appl 3, e137 (2014) 91. G.P. Luke, D. Yeager, S.Y. Emelianov, Biomedical
76. H. Ke et al., Performance characterization of an inte- applications of photoacoustic imaging with exoge-
grated ultrasound, photoacoustic, and thermoacous- nous contrast agents. Ann. Biomed. Eng. 40(2),
tic imaging system. J. Biomed. Opt. 17(5), 056010 422–437 (2012)
(2012) 92. L.V. Wang, J. Yao, A practical guide to photoacous-
77. A. Danielli et al., Label-free photoacoustic tic tomography in the life sciences. Nat. Methods 13,
nanoscopy. J. Biomed. Opt. 19(8), 086006 (2014) 627 (2016)
78. V. Ermolayev et al., Simultaneous visualization of 93. Y. Zhou et al., Handheld photoacoustic probe to
tumour oxygenation, neovascularization and con- detect both melanoma depth and volume at high
trast agent perfusion by real-time three-dimensional speed in vivo. J. Biophotonics 8(11–12), 961–967
optoacoustic tomography. Eur. Radiol. 26(6), 1843– (2015)
1851 (2016) 94. C.H. Lee et al., Chemical imaging in vivo:
79. C. Lutzweiler et al., Real-time optoacoustic tomog- Photoacoustic-based 4-dimensional chemical analy-
raphy of indocyanine green perfusion and oxygena- sis. Anal. Chem. 91(4), 2561–2569 (2019)
tion parameters in human finger vasculature. Opt. 95. J. Jo et al., Detecting joint inflammation by an LED-
Lett. 39(14), 4061–4064 (2014) based photoacoustic imaging system: A feasibility
80. S. Gottschalk et al., Correlation between volumetric study, SPIE (2018)
oxygenation responses and electrophysiology iden- 96. J. Jo et al., Photoacoustic tomography for human
tifies deep thalamocortical activity during epileptic musculoskeletal imaging and inflammatory arthritis
seizures, SPIE (2016) detection. Photoacoustics 12, 82–89 (2018)
81. L. Wang et al., Ultrasonically encoded photoacoustic 97. Y. Zhu et al., Identifying intestinal fibrosis and in-
flowgraphy in biological tissue. Phys. Rev. Lett. flammation by spectroscopic photoacoustic imag-
111(20), 204301 (2013) ing: An animal study in vivo. Biomed. Opt. Express
82. J. Xia et al., Calibration-free quantification of ab- 9(4), 1590–1600 (2018)
solute oxygen saturation based on the dynamics of 98. A. Garcia-Uribe et al., Dual-modality photoacous-
photoacoustic signals. Opt. Lett. 38(15), 2800–2803 tic and ultrasound imaging system for noninvasive
(2013) sentinel lymph node detection in patients with breast
83. S. Hu, K. Maslov, L.V. Wang, Second-generation cancer. Sci. Rep. 5, 15748 (2015)
optical-resolution photoacoustic microscopy with 99. B. Huang et al., Improving limited-view photoa-
improved sensitivity and speed. Opt. Lett. 36(7), coustic tomography with an acoustic reflector. J.
1134–1136 (2011) Biomed. Opt. 18(11), 110505 (2013)
84. S. Hu et al., Functional transcranial brain imag- 100. R. Ellwood et al., Photoacoustic imaging using
ing by optical-resolution photoacoustic microscopy, acoustic reflectors to enhance planar arrays. J.
SPIE (2009) Biomed. Opt. 19(12), 126012 (2014)
85. S. Hu, K. Maslov, L.V. Wang, Noninvasive label- 101. G. Li et al., Tripling the detection view of high-
free imaging of microhemodynamics by optical- frequency linear-array-based photoacoustic com-
resolution photoacoustic microscopy. Opt. Express puted tomography by using two planar acoustic re-
17(9), 7688–7693 (2009) flectors. Quant. Imaging Med. Surg. 5(1), 57 (2015)
86. Z. Guo, S. Hu, L.V. Wang, Calibration-free absolute 102. P. Zhang et al., High-resolution deep functional
quantification of optical absorption coefficients us- imaging of the whole mouse brain by photoacous-
ing acoustic spectra in 3D photoacoustic microscopy tic computed tomography in vivo. J. Biophotonics
of biological tissue. Opt. Lett. 35(12), 2067–2069 11(1), e201700024 (2018)
(2010) 103. G. Li et al., Multiview Hilbert transformation for
87. Y. Zhou et al., Calibration-free in vivo transverse full-view photoacoustic computed tomography using
blood flowmetry based on cross correlation of slow a linear array, SPIE (2015)
time profiles from photoacoustic microscopy. Opt. 104. J. Xia et al., Whole-body ring-shaped confocal pho-
Lett. 38(19), 3882–3885 (2013) toacoustic computed tomography of small animals
88. S.N. Hennen et al., Photoacoustic tomography in vivo, SPIE (2012)
imaging and estimation of oxygen saturation of 105. C. Li et al., Real-time photoacoustic tomography
hemoglobin in ocular tissue of rabbits. Exp. Eye Res. of cortical hemodynamics in small animals, SPIE
138, 153–158 (2015) (2010)
106. J. Xia et al., Three-dimensional photoacoustic to- 124. E.Z.Y. Zhang, P.C. Beard, Ultrahigh-sensitivity
mography based on the focal-line concept, SPIE wideband Fabry-Perot ultrasound sensors as an
(2011) alternative to piezoelectric PVDF transducers for
107. M.R. Chatni et al., Tumor glucose metabolism im- biomedical photoacoustic detection, SPIE (2004)
aged in vivo in small animals with whole- 125. E.Z. Zhang et al., In vivo high-resolution 3D pho-
body photoacoustic computed tomography, SPIE toacoustic imaging of superficial vascular anatomy.
(2012) Phys. Med. Biol. 54(4), 1035–1046 (2009)
108. J. Xia et al., Retrospective respiration-gated whole- 126. J.G. Laufer et al., In vivo preclinical pho-
body photoacoustic computed tomography of mice, toacoustic imaging of tumor vasculature develop-
SPIE (2014) ment and therapy, SPIE (2012)
109. L. Li et al., Imaging small animal whole-body dy- 127. A.P. Jathoul et al., Deep in vivo photoacoustic imag-
namics by single-impulse panoramic photoacoustic ing of mammalian tissues using a tyrosinase-based
computed tomography, in Photons Plus Ultrasound: genetic reporter. Nat. Photonics 9, 239 (2015)
Imaging and Sensing 2017 100640M (2017) 128. R. Ellwood et al., Photoacoustic tomography using
110. L. Lin et al., Single-breath-hold photoacoustic com- orthogonal Fabry–Pérot sensors, SPIE (2016)
puted tomography of the breast. Nat. Commun. 9(1), 129. A.A. Plumb et al., Rapid volumetric photoacoustic
2352 (2018) tomographic imaging with a Fabry-Perot ultrasound
111. X.L. Deán-Ben et al., Spiral volumetric optoacous- sensor depicts peripheral arteries and microvascular
tic tomography visualizes multi-scale dynamics in vasomotor responses to thermal stimuli. Eur. Radiol.
mice. Light-Sci Appl 6(4), e16247 (2017) 28(3), 1037–1045 (2018)
112. H.-P.F. Brecht et al., Whole-body three-dimensional 130. J. Laufer et al., Three-dimensional noninvasive
optoacoustic tomography system for small animals, imaging of the vasculature in the mouse brain using
SPIE (2009) a high resolution photoacoustic scanner. Appl. Opt.
113. K. Nagae et al., Real-time 3D photoacoustic visual- 48(10), D299–D306 (2009)
ization system with a wide field of view for imag- 131. M. Xu, L.V. Wang, Analytic explanation of spatial
ing human limbs [version 2; referees: 2 approved]. resolution related to bandwidth and detector aperture
F1000Research 7(1813) (2019) size in thermoacoustic or photoacoustic reconstruc-
114. R.A. Kruger et al., Dedicated 3D photoacoustic tion. Phys. Rev. E 67(5), 056605 (2003)
breast imaging. Med. Phys. 40(11), 113301 (2013) 132. K. Maslov et al., Optical-resolution photoacoustic
115. R.A. Kruger et al., Photoacoustic angiography of the microscopy for in vivo imaging of single capillaries.
breast. Med. Phys. 37(11), 6096–6100 (2010) Opt. Lett. 33(9), 929–931 (2008)
116. I. Yamaga et al., Vascular branching point counts us- 133. J. Yao, L.V. Wang, Photoacoustic microscopy. Laser
ing photoacoustic imaging in the superficial layer of Photonics Rev. 7(5), 758–778 (2013)
the breast: A potential biomarker for breast cancer. 134. L.V. Wang, Multiscale photoacoustic microscopy
Photoacoustics 11, 6–13 (2018) and computed tomography. Nat. Photonics 3(9), 503
117. T. Shiina, M. Toi, T. Yagi, Development and clin- (2009)
ical translation of photoacoustic mammography. 135. L.V. Wang, Tutorial on photoacoustic microscopy
Biomed. Eng. Lett. 8(2), 157–165 (2018) and computed tomography. IEEE J. Sel. Top. Quan-
118. Y. Matsumoto et al., Visualising peripheral arteri- tum Electron. 14(1), 171–179 (2008)
oles and venules through high-resolution and large- 136. E.W. Stein, K.I. Maslov, L.V. Wang, Noninvasive, in
area photoacoustic imaging. Sci. Rep. 8(1), 14930 vivo imaging of blood-oxygenation dynamics within
(2018) the mouse brain using photoacoustic microscopy. J.
119. K. Nagae et al., Real-time 3D photoacoustic visual- Biomed. Opt. 14(2), 020502 (2009)
ization system with a wide field of view for imag- 137. C.P. Favazza et al., In vivo photoacoustic mi-
ing human limbs. F1000Research 7, 1813–1813 croscopy of human cutaneous microvasculature and
(2019) a nevus. J. Biomed. Opt. 16(1), 016015 (2011)
120. R. Su et al., Small animal optoacoustic tomography 138. L. Wang et al., Fast voice-coil scanning optical-
system for molecular imaging of contrast agents, resolution photoacoustic microscopy. Opt. Lett.
SPIE (2016) 36(2), 139–141 (2011)
121. O. Ogunlade et al., In vivo three-dimensional pho- 139. J. Yao et al., High-speed label-free functional pho-
toacoustic imaging of the renal vasculature in pre- toacoustic microscopy of mouse brain in action. Nat.
clinical rodent models. Am. J. Physiol. Renal Phys- Methods 12, 407 (2015)
iol. 314(6), F1145–F1153 (2018) 140. Y.S. Zhang et al., Optical-resolution photoacoustic
122. J. Buchmann et al., Evaluation of Fabry-Perot poly- microscopy for volumetric and spectral analysis of
mer film sensors made using hard dielectric mirror histological and immunochemical samples. Angew.
deposition, in Photons Plus Ultrasound: Imaging Chem. 126(31), 8237–8241 (2014)
and Sensing 2016 970856 (2016) 141. C. Zhang et al., In vivo photoacoustic microscopy
123. E. Zhang, J. Laufer, P. Beard, Backward-mode mul- with 7.6-μm axial resolution using a commercial
tiwavelength photoacoustic scanner using a pla- 125-MHz ultrasonic transducer. J. Biomed. Opt.
nar Fabry-Perot polymer film ultrasound sensor for 17(11), 116016–116016 (2012)
high-resolution three-dimensional imaging of bio-
logical tissues. Appl. Opt. 47(4), 561–577 (2008)
378 L. Li et al.
142. H.-C. Hsu et al., Dual-axis illumination for virtually 157. S. Hu et al., Optical-resolution photoacoustic mi-
augmenting the detection view of optical-resolution croscopy of ischemic stroke, in Photons Plus Ultra-
photoacoustic microscopy. J. Biomed. Opt. 23(7), sound: Imaging and Sensing 2011 789906 (2011)
076001 (2018) 158. R. Cao et al., Photoacoustic microscopy reveals
143. B. Ning et al., Ultrasound-aided multi-parametric the hemodynamic basis of sphingosine 1-phosphate-
photoacoustic microscopy of the mouse brain. Sci. induced neuroprotection against ischemic stroke.
Rep. 5, 18775 (2015) Theranostics 8(22), 6111 (2018)
144. T.P. Matthews et al., Label-free photoacoustic mi- 159. S. Gottschalk et al., Correlation between volumetric
croscopy of peripheral nerves, SPIE (2014) oxygenation responses and electrophysiology iden-
145. J. Shi et al., High-resolution, high-contrast mid- tifies deep thalamocortical activity during epileptic
infrared imaging of fresh biological samples with seizures. NEUROW 4(1), 011007 (2016)
ultraviolet-localized photoacoustic microscopy. Nat. 160. V. Tsytsarev et al., In vivo imaging of epileptic
Photonics 13(9), 609–615 (2019) activity using 2-NBDG, a fluorescent deoxyglu-
146. S.V. Ovsepian et al., Pushing the boundaries of neu- cose analog. J. Neurosci. Methods 203(1), 136–140
roimaging with optoacoustics. Neuron 96(5), 966– (2012)
988 (2017) 161. Q. Zhang et al., Non-invasive imaging of epileptic
147. S. Hu et al., Intravital imaging of amyloid plaques in seizures in vivo using photoacoustic tomography.
a transgenic mouse model using optical-resolution Phys. Med. Biol. 53(7), 1921–1931 (2008)
photoacoustic microscopy. Opt. Lett. 34(24), 3899– 162. B. Rao et al., Photoacoustic imaging of voltage
3901 (2009) responses beyond the optical diffusion limit. Sci.
148. J. Yao, L.V. Wang, Photoacoustic brain imaging: Rep. 7(1), 2560 (2017)
From microscopic to macroscopic scales. NEUROW 163. X.L. Deán-Ben et al., Functional optoacoustic
1(1), 011003 (2014) neuro-tomography for scalable whole-brain mon-
149. M.-L. Li et al., Simultaneous molecular and hypoxia itoring of calcium indicators. Light-Sci. Appl. 5,
imaging of brain tumors in vivo using spectroscopic e16201 (2016)
photoacoustic tomography. Proc. IEEE 96(3), 481– 164. T.-W. Chen et al., Ultrasensitive fluorescent pro-
489 (2008) teins for imaging neuronal activity. Nature 499, 295
150. L. Li et al., Small near-infrared photochromic pro- (2013)
tein for photoacoustic multi-contrast imaging and 165. Y. Gong et al., High-speed recording of neural spikes
detection of protein interactions in vivo. Nat. Com- in awake mice and flies with a fluorescent voltage
mun. 9(1), 2734 (2018) sensor. Science 350(6266), 1361–1366 (2015)
151. J. Yao et al., Multiscale photoacoustic tomography 166. G. Ku, L.V. Wang, Scanning microwave-induced
using reversibly switchable bacterial phytochrome thermoacoustic tomography: Signal, resolution, and
as a near-infrared photochromic probe. Nat. Meth- contrast. Med. Phys. 28(1), 4–10 (2001)
ods 13(1), 67 (2016) 167. R.A. Kruger, D.R. Reinecke, G.A. Kruger,
152. J. Yang et al., Focusing light inside live tissue using Thermoacoustic computed tomography–technical
reversibly switchable bacterial phytochrome as a considerations. Med. Phys. 26(9), 1832–1837
genetically encoded photochromic guide star. Sci. (1999)
Adv. 5(12), eaay1211 (2019) 168. M. Xu, L.V. Wang, Time-domain reconstruction for
153. L. Wang, K. Maslov, L.V. Wang, Single-cell label- thermoacoustic tomography in a spherical geom-
free photoacoustic flowoxigraphy in vivo. Proc. etry. IEEE Trans. Med. Imaging 21(7), 814–822
Natl. Acad. Sci. 110(15), 5759–5764 (2013) (2002)
154. M. Nasiriavanaki et al., High-resolution photoacous- 169. R.A. Kruger et al., Thermoacoustic CT with radio
tic tomography of resting-state functional connectiv- waves: A medical imaging paradigm. Radiology
ity in the mouse brain. Proc. Natl. Acad. Sci. 111(1), 211(1), 275–278 (1999)
21–26 (2014) 170. G. Ku et al., Thermoacoustic and photoacoustic to-
155. C. Rui et al., Comprehensive characterization of mography of thick biological tissues toward breast
cerebrovascular dysfunction in blast traumatic brain imaging. Technol. Cancer Res. Treat. 4(5), 559–565
injury using photoacoustic microscopy. J. Neuro- (2005)
trauma 0(0), null 171. Y. Xu, L.V. Wang, Rhesus monkey brain imaging
156. M. Kneipp et al., Functional real-time optoacoustic through intact skull with thermoacoustic tomogra-
imaging of middle cerebral artery occlusion in mice. phy. IEEE Trans. Ultrason. Ferroelectr. Freq. Con-
PloS one 9(4), e96118 (2014) trol 53(3), 542–548 (2006)
Electrophysiological Mapping
and Source Imaging 13
Bin He, Lei Ding, and Abbas Sohrabpour
Abstract We further continue to discuss methods that

attempt the reverse, i.e., solving for underlying
In this chapter, we present the physical and
brain sources given scalp measurements
physiological basics behind EEG and MEG
such as EEG and MEG, a process called
signal generation and propagation. We first
source imaging. We will provide various
start by presenting the biophysical principles
examples of how electrophysiological source
that explain how the coordinated movements
imaging techniques can help study the brain
of ions inside and outside neuronal cells
during its normal and pathological states.
result in macroscale phenomena at the scalp,
We will also briefly discuss how combining
such as electric potentials recorded by EEG
electrophysiological signals from EEG
and magnetic fields sensed by MEG. These
with hemodynamic signals from functional
physical principles enforce EEG and MEG
magnetic resonance imaging (fMRI) helps
signals to have specific spatial and temporal
improve the spatiotemporal resolution of
features, which can be used to study brain’s
estimates of the underlying brain sources,
response to internal and external stimuli.
which is critical for studying spatiotemporal
We continue our exploration by developing
processes within the brain. The goal of
a mathematical framework within which EEG
this chapter is to provide proper physical
and MEG signals can be computed if the
and physiological intuition and biophysical
distribution of underlying brain sources is
principles that explain EEG/MEG signal
known, a process called forward problem.
generation, its propagation from sources in
the brain to EEG/MEG sensors, and how
this process can be inverted using signal
processing and machine learning techniques
and algorithms.
of this chapter (https://doi.org/10.1007/978-3-030-43395- Keywords
to authorized users. Electroencephalogram (EEG) ·
Magnetoencephalography (MEG) ·
B. He () · A. Sohrabpour
Department of Biomedical Engineering, Carnegie Mellon Electrophysiological source imaging (ESI) ·
University, Pittsburgh, PA, USA Event-related potential (ERP) · Forward
e-mail: [email protected]; [email protected] problem · Inverse problem · EEG/MEG
L. Ding mapping · Multimodal imaging
Stephenson School of Biomedical Engineering,
University of Oklahoma, Norman, OK, USA

380 B. He et al.
vicinity to the scalp where electrical or magnetic

13.1 Introduction
sensors are placed. Due to the separation of the
apical and basal dendrites in pyramidal cells, a
13.1.1 Generation and Measurement
considerable distance exists between the current
of EEG and MEG
sources and sinks, resulting in strong current
dipoles as perceived by EEG and MEG [5].
Although electrical activity recorded from the
Additionally, these cells are arranged in parallel
exposed cerebral cortex of a monkey was re-
to each other and perpendicular to the cortical
ported in 1875 [1], it was not until 1929 that
surface, in an arrangement referred to as the
Hans Berger, a psychiatrist in Jena, Germany,
palisade which constructively adds the effect
first recorded noninvasively rhythmic electrical
of smaller current dipoles from individual cells
activity from the human scalp [2], which has
together, to effectively constitute a strong current
subsequently known as electroencephalography
dipole [6]. It is thus believed that EEG and MEG
(EEG). Since then, EEG has become an important
predominantly detect synchronized current flows
tool for probing brain electrical activity and aid-
in the cortical pyramidal neurons, which are laid
ing in clinical diagnosis of neurological disorders,
out perpendicularly to the convoluted cortical
due to its excellent temporal resolution in the
sheet of gray matter [7]. This is schematically
order of millisecond. The first recording of mag-
shown in Fig. 13.1.
netic fields from the human brain was reported in
Dipole models are used more frequently (com-
1972 by David Cohen at the Massachusetts Insti-
pared to monopoles and multipoles [7, 8]) to
tute of Technology [3], which led to the develop-
describe the underlying biophysics of neural ac-
ment of magnetoencephalography (MEG). Like
tivity, as they provide an easier physical interpre-
EEG, MEG also enjoys high temporal resolution
tation of the underlying phenomenon and can be
in detecting brain electrical activity. EEG and
viewed as an approximate discrete representation
MEG have become two prominent methods for
of current density at a mesoscopic level. Fur-
noninvasive assessment of brain electrical activ-
thermore, the electromagnetic fields generated by
ity, providing unsurpassed temporal resolution, in
multipoles attenuate much faster with distance,
neuroscience research and clinical applications.
compared to dipoles, inadvertently resulting in
EEG and MEG are considered to originate
dipole fields dominating EEG and MEG mea-
from, in principle, the same brain electrical activ-
surements [8, 9]. This is supported by the fact
ity, which are current flows caused by neuronal
that the distance between current sources and
excitation. The discharge of a single neuron or
sinks is almost neglectable as compared with their
single nerve fiber in the brain generates an ex-
distances to the locations where EEG and MEG
tremely small electric potential or magnetic field,
signals are being recorded.
which cannot be observed over the scalp due to
The intensities of the scalp EEG range from
the background noise.
0 to 200 μV, which fluctuate mainly in the fre-
Instead, the externally recorded EEG and
quency range of up to 50 Hz. The EEG recording
MEG represent the summation of the electric
involves the application of a set of electrodes to
potentials and magnetic fluxes generated by
standard positions on the scalp. The most com-
many thousands or even millions of neurons or
monly used electrode placement montage is the
fibers when they fire synchronously [4]. In other
international 10–20 system, which uses the dis-
words, the intensities of EEG and MEG signals
tances between bony landmarks of the head to
are determined mainly by the number of neurons
generate a system of lines which run across the
and fibers that fire in synchrony. An anatomic
head and intersect at intervals of 10% or 20%
structure in the human brain, which favors the
of their total length. Additional electrodes can
neuronal synchrony and summation of electric
also be introduced according to expanded 10–
potentials or magnetic fields from neuronal
20 systems as proposed by the American EEG
synchrony, is the cortex, which is also in the
Society. Most clinical EEG recordings are up to
13 Electrophysiological Mapping and Source Imaging 381
Fig. 13.1 Electrophysiological principles of current at MEG sensors, are due to the summation of many
dipoles. Macroscopic phenomena, such as the electric microscopic quantities such as postsynaptic potentials
potential recorded at scalp EEG or magnetic field recorded of pyramidal cells
32 electrodes, while merits of high-density EEG fully detected and analyzed in scalp recordings
recordings have been reported by multiple studies [12–14]. HFOs are typically observed in intracra-
[4, 10]. A recent recommendation from the work- nial recordings and can span a frequency band
ing group of the International Federation of Clini- of 30–600 Hz and are thought to be involved in
cal Neurophysiology suggests using 64 electrodes physiological processes such as attention, learn-
or more for source imaging and localization [11]. ing, and memory, as well as pathological pro-
The difficulty in recording magnetic fields cesses such as ictogenesis [13]. HFOs, typically
from the human brain is its strengths that are in the range of 80–250 Hz, have been ubiqui-
weaker than couple of pico-Tesla (pT), which is tously and reliably observed and reported in scalp
about 108 times less than the earth’s geomagnetic recordings, in the recent years [15, 16]. Not only
field. MEG recordings were made available due have these events been reliably detected in non-
to the invention of a sensitive magnetic flux invasive scalp measurements, but also HFOs are
detector, known as the superconducting quantum traced back to the source space and shown to
interference device (SQUID) [7] (Fig. 13.2). The correlate well with clinical findings determining
frequency range of MEG is similar to EEG, which the seizure onset in epilepsy patients [15], en-
is between 0 and 50 Hz. couraging researchers and clinicians to consider
While most analysis performed in EEG and HFOs as a potential biomarker for ictogenesis
MEG is within the 0–50 Hz band due to the high [17]. While this view has been modified recently
concentration of energy within these bands, high- [18], HFOs can be reliably detected in EEG and
frequency oscillations (HFOs) have been success- MEG, emphasizing that broad spectral informa-
382 B. He et al.
contributed by spontaneous brain electrical activ-

ity and potentials and/or magnetic fields evoked
by external stimuli/events, known as the evoked
potentials and/or (magnetic) fields or event-
related potentials and/or fields (ERPs/ERFs).
Since external stimuli/events can be specifically
designed to evoke targeted functional areas, such
as visual, auditory, and somatosensory cortices,
associated measurements have thus been widely
practiced to study the functions of these areas.
Correspondingly, evoked potentials and/or fields
are the visual evoked potential/field (VEP/VEF),
auditory evoked potential/field (AEP/AEF),
and the somatosensory evoked potential/field
(SEP/SEF).
Fig. 13.2 Schematics of MEG instrumentation. A single-

channel axial gradiometer and associated SQUID inside a
13.1.2 Spatial and Temporal
dewar filled with liquid helium. The bottom depicts the Resolution of EEG and MEG
sensor array of a 306-channel MEG helmet where each
sensor unit contains two orthogonal planar gradiometers Brain electrical activation is a spatiotemporal pro-
and one magnetometer. (From [20], licensed under CC BY
4.0)
cess, which means that its activity is distributed
over three dimensions and evolves in time. The
most important merit of EEG and MEG is their
tion can be extracted in noninvasive recordings as unsurpassed millisecond-scale temporal resolu-
well [6]. tion. This feature is essential for resolving rapid
In recording systems, while the number of change of neurophysiological process, consider-
MEG sensors used is usually different from EEG, ing the typical temporal scale of neuronal elec-
the spatial coverage and layout of MEG sensors trical events which is from one to several tens of
are similar to those for EEG, which are distributed milliseconds. However, both EEG and MEG are
over a surface in parallel to the scalp surface limited by their spatial resolutions.
(Fig. 13.2). MEG sensors are not necessary to The conventional EEG has limited spatial res-
touch the scalp due to the magnetic permeability olution mainly due to two factors. One factor is
of air, which is also different from EEG. On the the limited spatial sampling. A remarkable devel-
other hand, since the magnetic fields from the opment in the past decades is that high-resolution
human brain are extremely weak compared with EEG systems with 64–256 electrodes have been
ambient magnetic fields, MEG recording systems commercially available. For example, with up to
are much more complicated than EEG recording 124 scalp electrodes, the average inter-electrode
systems. The SQUID system is commonly de- distance can be reduced to about 2.5 cm [10, 21].
signed not to be sensitive to uniform background The multichannel SQUID system was challenged
magnetic fields using gradiometers, and MEG initially due to the complexity of superconduc-
recordings are usually conducted in a magneti- tive coils that were necessary to be sensitive to
cally shielded room. Recently the feasibility of a weak brain magnetic signals [7]. Nowadays, mul-
wearable MEG system was reported for human tichannel SQUID systems have been commer-
use [19], although this technology is still under cially available too. The second factor is the head
development and is currently quite expensive. volume conduction effect. The electric potentials
In both EEG and MEG signals recorded over generated from neural sources are attenuated and
the human head, the major constituents are those blurred as they pass through the neural tissue,
cerebrospinal fluid, meninges, low-conductivity mapping is to visualize potential values from

skull, and scalp [9]. While the magnetic fields different electrodes measured at the same time
are also suffered from the volume conduction instance on the scalp surface. Since EEG record-
effect as for its attenuation and spatial smooth- ings can only be obtained in locations where
ness, MEG is practically unaffected by the low- electrodes are placed, potential values in inter-
conductivity skull. electrodes areas are usually interpolated, mainly
Advanced EEG and MEG imaging techniques using linear methods, for higher-resolution vi-
are highly desired in order to compensate for sualization. The assumption behind linear inter-
the head volume conduction effect and enhance polations is the smooth transition of potential
the spatial resolution of scalp EEG and MEG. values among neighbored electrodes. However,
The solutions of two separate but closely related the accuracy of interpolations also depends on
problems, EEG/MEG forward problem and the number of electrodes. Figure 13.4a, b illus-
EEG/MEG inverse problem, are required for trates an example of scalp EEG maps interpolated
imaging of brain electric activity based on using measurements from 32 channels and 122
external potential and/or field measurements. channels, respectively. The scalp EEG map in
Fig. 13.4a is smoother with reduced peak values
and sharper transitions than the scalp EEG map
13.2 Electrophysiological from Fig. 13.4b. These problems are caused by
Mapping the low-density samples from a fewer number of
electrodes, which leads to large inter-electrode
13.2.1 EEG Mapping distances. Nonlinear interpolations can also be
used, such as spline interpolation [24]. An ex-
Due to the fast response of EEG/MEG to neu- ample of spline interpolation can be found in
ral events, a major use of EEG/MEG signals applications where a continuous function of an
is to make observations in their time courses EEG map is necessary, such as for the calculation
[22, 23]. Plenty of temporal components have of a surface Laplacian EEG map.
been well defined and widely accepted in various To illustrate EEG maps, two visualization
paradigms. For example, N100 component is a tools are usually used, contour lines, in which
negative-going deflection from baseline in AEPs each line connects isopotential points on
(its equivalent in MEG is the M100 [7]), which the scalp, or pseudo-colors (which are more
peaks at the latency of about 100 ms after the common), in which each color represents a
onset of an auditory stimulus. In VEP, multi- potential value. Figure 13.4 shows EEG maps
ple temporal (either positive- or negative-going) using pseudo-colors. Along the direction of
components at different latencies have been iden- current flow within the brain source area
tified in a sequence after a visual stimulus. The (indicated by the red arrow in figures), potentials
dynamics of these temporal components and their are positive. A symmetric negative pattern is
latencies indicate the important information about usually accompanied in the opposite direction
the timings and sequences of neuronal processes of current flow (Fig. 13.4a, b). Note that
in response to specific stimuli. EEG measurements are usually made against a
Other than time information, efforts have been reference. While the symmetric pattern along the
made to obtain spatial information with regard direction of current flow always exists, whether
to the underlying brain electrical activity. Figure potential values are positive or negative also
13.3 shows an example of scalp EEG maps during depends on the selection of reference.
a binocular rivalry paradigm [22]. Strong coun- The scalp EEG maps in both Fig. 13.4a, b are
terphase modulations are revealed in EEG maps generated by a simulated current dipole source
for attended rivalry, and the scalp EEG maps also (Fig. 13.4b, right column) via solving the forward
suggest occipital origin of sources responsible problem. A scalp EEG map generated by a small
for the scalp EEG during binocular rivalry. EEG brain source (modeled by a current dipole) can ex-
384 B. He et al.
Fig. 13.3 Time courses of scalp EEG power maps. These locked signal from the other eye. Inset line graphs show
scalp topographies show power at the tagged frequencies the results from occipital electrodes. Both line graphs
at each electrode, as averaged over a group of subjects. and topographies show strong counterphase modulations,
Seven maps were drawn for each 6 s epoch. In each of the except in the unattended rivalry condition. (From Zhang et
four panels, the upper row shows power for the aligned al. [22] with permission)
eye’s frequency, and lower row shows power for the time-
tend about centimeters in diameters over the scalp stead of EEG signals. In MEG, positive values
surface, which is caused by the so-called volume indicate the outflow of magnetic flux coming at
conductor effect. Although the head volume con- the recording sensor location and negative val-
ductor effort causes a smoothed version of spatial ues indicate the inflow of magnetic flux at that
distribution of EEG corresponding to the brain particular location. It is worthwhile to note that
electric sources, EEG mapping represents an easy MEG signals do not depend on references like
and fast tool to assess the global nature of brain EEG and have different sensitivity profiles [25]
electric activity (e.g., frontal lobe vs. occipital compared to EEG. Examples of MEG maps are
lobe, see also Fig. 13.3 for visual events). shown in Fig. 13.4b, c (the middle columns)
using the same simulated brain sources as for
EEG in the same figure. MEG maps also suffer
13.2.2 MEG Mapping from the volume conductor effect. However, since
the magnetic permeability of the skull is simi-
The concept of MEG mapping is similar to EEG lar to other brain tissues, the low-conductivity
mapping except that MEG signals are used in- skull layer affects MEG less. Another property
a EEG 32 channels EEG 122 channels b MEG 152 channels current sources
c
EEG MEG
20 uV 75 fT
–20 uV –75 fT
Fig. 13.4 Simulated EEG data and MEG data under dif- MEG (middle) from 151 sensors generated by a tangential
ferent conditions. (a) The scalp EEG map generated by dipole on the cortical surface (right). (c) The high-density
a tangential dipole using low-density 32 electrodes. (b) scalp EEG (left) from 122 electrodes and MEG (middle)
The high-density scalp EEG (left) from 122 electrodes and from 151 sensors generated by a radial dipole on the
cortical surface (right)
of MEG is that it is not sensitive to radially source separation techniques [28]. Additionally,
oriented cortical sources [7]. Figure 13.4 illus- signals reaching the surface measurements from
trates an example of MEG map generated by these deep sources can still be localized to these
a brain source on the ridge of a cortical fold subcortical structures using source imaging tech-
that is close to radial orientation. Its MEG sig- niques [27, 28].
nals are ten times less than MEG signals from It is important to understand the difference
a tangential source (Fig. 13.4b). Both EEG and between EEG and MEG maps since both reflect
MEG are less sensitive to deeper sources, with the common brain activity while each of them
MEG being notably insensitive to deeper sources has better sensitivity on different aspects of the
[26]. However, these structural limitations do not common brain activity. The electrical field gra-
necessarily mean EEG, and MEG cannot detect dient reaches the highest along the direction of
any deep sources. Recent studies with concurrent current flow of the brain source (indicated by the
intracranial and EEG/MEG recordings have pro- red arrow in figures), while the magnetic field has
vided evidence to the contrary; electromagnetic the highest gradient across the direction of current
activity from subcortical regions in the thala- flow. Thus, the symmetric field pattern of MEG is
mus, amygdala, and hippocampus was unequiv- on the both side of the arrow, while the symmetric
ocally recorded at EEG and MEG [27, 28]. See- field pattern of EEG appears on the tail and head
ber et al. showed that the envelope of alpha- of the arrow. It is therefore expected that the trans-
wave activity from sources as deep as the cen- verse features of brain sources are more precisely
tromedian nuclei of the thalamus (direct electri- estimated with MEG and the longitudinal features
cal recordings from deep brain stimulation elec- of brain sources are more precisely estimated with
trodes placed in these regions) can be recorded EEG. Furthermore, MEG is not sensitive to radial
in high-density scalp EEG recordings (256 chan- brain sources as discussed earlier, whereas EEG is
nels) [27]. Furthermore, it was shown that these sensitive to brain sources of all orientations (e.g.,
activities can be traced back to deep source re- comparing Fig. 13.4).
gions by solving the inverse problem. Addition- In summary, while the EEG and MEG map-
ally, Pizzo et al. showed that interictal spikes ping can provide spatial patterns about brain ac-
observed by stereo-EEG (sEEG) electrodes im- tivity on the scalp, they are limited by their inher-
planted near the amygdala and hippocampus can ited low spatial resolution. The spatial locations
be detected in MEG recordings by means of blind of those temporal components of interests can
386 B. He et al.
only be referred at the scalp surface according EEG-based brain-computer interface to improve
to beneath lobular or sublobular organizations. signal quality of measurements associated with
Significant improvement of spatial resolution of intentions.
EEG/MEG can be accomplished by source imag-
ing from scalp EEG or MEG.
13.2.4 Multivariate Pattern Analysis
of EEG and MEG Signals
13.2.3 Surface Laplacian Mapping
The brain encodes the information it receives and
In parallel to the development of the source imag- processes into neural codes, which, inadvertently,
ing methods to enhance spatial resolution of EEG manifest themselves as neural patterns of activity.
and MEG, another surface mapping technique, The neuronal activity, consequently, leaves an
surface Laplacian (SL), has been developed for electromagnetic footprint that gets picked up by
the similar purpose. The SL does not need to EEG and MEG [4]. A great deal of studies and
solve the inverse problem as discussed below, investigations are conducted to decode these pat-
nor does it require a forward volume conductor terns and extract such information. Multivariate
model. Instead, it applies a spatial Laplacian filter pattern analysis or MVPA is the general term
(second spatial derivative) to compensate for the used to describe the process of analyzing signals
head volume conduction effect and achieves high- gathered from many neurons and brain regions
resolution surface mapping directly over the scalp to differentiate between different brain states to
surface. ultimately understand how the brain encodes in-
The SL has been considered as an estimate formation [34]. MVPA can be thought of as su-
of the local current density flowing perpendic- pervised learning, to put in machine learning lan-
ular to the skull into the scalp; thus it has also guage, which learns spatial patterns of neuronal
been termed current source density or scalp cur- activity over different cognitive conditions or ex-
rent density [29]. The SL has also been con- ternal stimuli. This technique has been applied to
sidered as an equivalent surface charge density MEG and EEG measurements at scalp, prior to
corresponding to the surface potential [30]. Com- solving the inverse problem [35], and at source
pared to the EEG source imaging approaches, the space, after solving the inverse [36]. MVPA can
SL approach does not require exact knowledge be thought of as a systematic approach to map-
about the source models and the volume conduc- ping spatiotemporal neural activity to brain states
tor models and has unique advantage of reference and cognitive conditions, in continuation of what
independence. was discussed above.
Since Hjorth’s early exploration on scalp MVPA is capable of detecting complex spatial
Laplacian of EEG [31], many efforts have neural patterns as experimental conditions or ex-
been made to develop reliable and easy-to-use ternal stimuli can be repeated many times to en-
SL techniques. Of note are the developments sure the statistical integrity of the data. This tech-
of spherical spline SL [29] and the realistic nique has been applied recently to MEG and EEG
geometry spline SL [24, 32]. Bipolar or tripolar measurements for studying object recognition,
concentric electrodes have also been used to face perception, and memory [35]. These studies
measure the SL. He and colleagues proposed to not only benefited from spatially rich information
use the bipolar concentric electrode to record contained in EEG and MEG measurements but
the SL [30] under the assumption that the outer also took advantage of the high temporal resolu-
ring of the concentric electrode would provide tion of the aforementioned modalities to further
reasonable estimate of the averaged potential over understand when different brain processes occur
the surrounding ring [30]. A tripolar concentric in the brain with respect to each other; for in-
ring electrode has also been used to measure stance, Linde-Domingo et al. showed that during
SL [33]. The SL has been widely used in seeing objects, low-level visual features could be
decoded faster than high-level conceptual fea- an impressed current density J, which is driven by
tures [35]. The reverse was true for associative the electrochemical process of excitable cells in
memory recall. Therefore, applying MVPA to the brain. In other words, it is a nonconservative
EEG and MEG could provide a spatiotemporal current that arises from the bioelectric activity of
decoding scheme which ultimately leads to the nerve cells due to the conversion of energy from
better understanding of the brain. chemical to electrical form [37].
The simplest brain electric source model is
a monopole source. In the monopole model, a
13.3 EEG/MEG Forward Modeling volume source with ignorable size is considered
as a point current source of magnitude Iv lying
Given the known information on brain electric in a conducting medium, with its current flow
source distribution (i.e., source models) and head lines radially directed in all directions. However,
volume conduction properties (i.e., volume con- in a living system, only a collection of positive
ductor models), EEG and MEG forward prob- and negative monopole sources is physically re-
lems determine the source-generated electric po- alistic as the total sum of currents is zero due
tential and magnetic field (Fig. 13.4). Note that to electrical neutrality. The simplest collection
while the EEG forward solution mainly refers of monopole sources is a dipole, which consists
to electric potentials, such as the cortical poten- of two monopoles of opposite sign, but equal
tial or the scalp potential, it can also be other strength, separated by an infinitely small distance.
metrics, for example, the surface Laplacian. In In such a dipole model, its current flow lines start
MEG, the forward solution is usually referred from the positive pole of the source and end at the
to as magnetic fields. Since magnetic fields are associated negative pole. The dipole model is the
vector fields, the forward solution can be referred most commonly used model in EEG/MEG source
as a component of magnetic fields, such as ra- imaging techniques.
dial or tangential component. Furthermore, since Until now, we have only considered the equiv-
most MEG systems use gradiometers, the MEG alent source models for the impressed current
forward solution can be magnetic gradient fields density, which are generated by excitable cells.
or second-order gradient fields. Both EEG and In order to solve the EEG/MEG source imag-
MEG forward problems are well defined and have ing problems, a global equivalent source distri-
a unique solution, governed by the quasi-static bution model should also be determined which
approximations of Maxwell’s equations, that is, can account for the electric activity within the
Poisson’s equation [8, 9, 37]. entire brain. State-of-the-art source models usu-
By solving the EEG and MEG forward prob- ally consist of a source distribution to reflect the
lems, the relationship between neuronal sources distributed nature of electric sources associated
and external sensor measurements can be estab- with neuronal excitation. Once such a source
lished. In particular, for a given source distribu- distribution model is defined, the source imaging
tion, EEG and MEG measurements and underly- solutions can only be searched over the space con-
ing brain electric sources can be related by the so- fined to the distribution model, hence, also known
called transfer matrix or lead field matrix, which as the source space or solution space. Source
is only dependent on the geometry and electrical models, including the dipole model (which can be
properties of the head volume conductor. viewed as a special case of a source distribution)
and the source distribution model, are generally
used for both EEG/MEG forward and inverse
13.3.1 Source Models problems. There are mainly two types of source
models, i.e., parametric dipole models [38] and
Several source models have been proposed to distributed source models [39–41]. The paramet-
equivalently represent brain electric sources. The ric dipole models use the ideal equivalent dipole
primary bioelectric sources can be represented as model (ECD) to represent focal electrical activity.
388 B. He et al.
In parametric dipole models, multiple ECDs are geometry model [45, 46]. Here, the multiple
also used to model multiple focal sources over layers again refer to the interfaces between the
different brain regions. The distributed source skin, the skull, and the brain, which are similarly
models are more suitable in characterizing ex- represented in three-layer concentric spherical
tended sources in which the source space is rep- model, but of realistic geometries. The realistic
resented by continuously distributed dipole ele- geometries can be obtained by segmenting brain
ments over a volume (i.e., the brain) [39, 41] or a tissues from magnetic resonance (MR) structural
surface (i.e., the cortical surface) [40]. images. In addition to the boundary element
The source models are not limited to model method, the finite element method (FEM) has
electrical currents but may be electric potentials also been used to model the head volume
over the cortical surface [21, 42] or within the 3D conductor [47, 48] in which each finite element
brain volume [43]. can be assigned with a conductivity value or even
a conductivity tensor that represents different
conductivity values along different directions
13.3.2 Volume Conductor Models in a 3D space (known as the anisotropy)
[49, 50].
The volume conductor models are developed to While the aforementioned discussion applies
model the human head, which sits between brain to MEG, as well, in practice, the volume conduc-
sources and EEG/MEG sensors. In order to build tor models for MEG are much simpler than those
these models, the geometry and conductivity for EEG. The major reason is that the perme-
or permeability profiles are crucial for EEG ability profile for MEG is almost uniform for all
or MEG. Early works used spherical head brain tissues including the skull. Thus, a volume
models as closed solutions for EEG/MEG conductor model with realistic shape for the brain
forward problems. The single-sphere model may be sufficient for the forward calculation of
represents the simplest approximation of the head MEG signals [7]. In practice, one-sphere model
geometry. The three-layer concentric spherical with a similar size to the subject’s head is used,
model [44] has been well used to represent occasionally.
compartments of the skin, the skull, and the
brain in head volume conductor. Such a model
was essentially developed to consider the skull 13.3.3 Forward Solutions
layer since it has significant low-conductivity
layer as compared with the skin and the brain. Once the volume source model and volume
An important development in the field was to conductor model are selected, the forward
incorporate anatomic constraint into EEG/MEG solutions can be calculated uniquely. Here
source imaging by developing approaches which we discuss two cases of forward solutions:
could take the realistic head geometry into monopoles and dipoles in infinite homogeneous
consideration. He et al. proposed the use of medium. While these represent the simplest cases
realistic geometry head models for EEG source for the calculation of forward solutions, which
localization by applying the boundary element might not be quite realistic in real applications,
method (BEM) [38]. Hämäläinen and Sarvas it can help readers understand the concepts such
[45] further developed BEM-based approach to as the volume conductor effect. Other advanced
model the head volume conductor for MEG/EEG methods in calculating forward solutions, such
incorporating the low-conductivity skull layer as piece-wise homogeneous realistic geometry
in addition to the scalp and brain. Several BEM models or inhomogeneous realistic geometry
approaches have been developed to solve the head models, can be found in the literature [45, 48].
forward problem using a multiple layer realistic
If the volume conductor is infinite and homo- If the three compartments (the brain, skull,
geneous with conductivity of σ, the bioelectric scalp) are considered and their surfaces are of
potential obeys Poisson’s equation under quasi- realistic shapes, it becomes a realistic geometry
static conditions [37]: piecewise homogeneous model. This is a reason-
able approximation for the electrical conductivity

∇ · Ji Iv profile of the human head modeling the scalp,
∇ 2Φ = =− (13.1) skull, and brain. The forward solution becomes
σ σ
a sum of the electric potential/magnetic field in
Equation 13.1 is a partial differential equation the infinite homogeneous medium with a sec-
satisfied by the electric potential in which Iv is ond term that reflects the effect of conductivity
the source function. The solution of Eq. 13.1 for inhomogeneity between different compartments
the scalar function for a region that is uniform [8]. The piecewise homogeneous model and its
and infinite in extent is [37]: solution can be generalized to more complicated
inhomogeneous model since an inhomogeneous

1 1 volume conductor can be divided into a finite
Φ=− ∇ · J i dv (13.2)
4π σ V r number of homogeneous regions. A boundary
element method algorithm [45] has been intro-
where r refers to the distance from the source to duced to accurately calculate electrical potential
the observation point. Since the source element and magnetic fields in piecewise homogeneous

∇ · J i dv in Eq. 13.2 behaves like a point source, head volume conductor model.
in that it sets up a field that varies as 1/r, the

expression Iv = −∇ · J i can be considered as 13.4 EEG/MEG Source Imaging
an equivalent monopole source
[8, 37, 51].

Given the known electrical potential or magnetic
Using the identity ∇ · J i /r = ∇ (1/r) ·
field (e.g., scalp EEG or MEG measurement)

and head volume conductor properties, the
J i + (1/r) ∇ · J i and the divergence (or Gauss’s)
EEG/MEG source imaging reconstructs the
theorem, Eq. 13.2 can be transformed to [8]:
distribution of electric sources within the
brain (source space) corresponding to the
1 1
Φ= ∇ · J i dv (13.3) measured EEG/MEG (Fig. 13.5). A solution to
4π σ V r
the EEG/MEG source imaging problem thus
provides desirable information with regard to the
Here, the source element J i dv behaves like brain electric activity, such as locations or extent
a dipole source, with a field that varies as 1/r. of current sources, which can be directly related
Therefore, the impressed current density may to the underlying neural activation. Although
be interpreted as an equivalent dipole source. the EEG/MEG inverse problem is technically
Although higher-order equivalent source models challenging, work conducted in the past few
such as the quadrupole can also be studied to decades has indicated that the EEG/MEG source
represent the bioelectric sources, the dipole imaging problem can be solved with reasonable
model has been so far the most commonly used resolution and accuracy by incorporating various
brain electric source model. a priori information, such as anatomic constraints
Similar to electric potential, the magnetic field on the sources [40, 41], on volume conductor
due to a monopole or dipole current source in [38, 45], or functional constraints provided by
an infinite homogeneous medium can be derived other imaging modalities such as functional MRI
based on Poisson’s equation. Interested readers [52–54].
can consult the details in [8]. EEG/MEG source imaging solutions require
a source model. The choice of source models
390 B. He et al.
Fig. 13.5 Schematic diagram of EEG/MEG electrophys- such as BEM or FEM, i.e., forward problem. By solving
iological neuroimaging. The scalp EEG/MEG is recorded the inverse source imaging problem, brain electric sources
using multichannel data acquisition system. The realis- are estimated over the cortex or throughout the brain
tic geometry head volume conductor model can be con- volume with substantially enhanced spatial resolution as
structed from the structure MRI of the subject, and the lead compared with scalp EEG/MEG. (From He et al. [4] with
field matrix can be modeled using numerical techniques permission)
depends on particular applications, while the pri- estimates the center of gravity of brain electric
mary goal of EEG/MEG source imaging prob- activity, which can be informative for focal brain
lems remains the same: to find an equivalent activation, such as origin of focal epileptic activ-
representation of brain electric sources that can ity. The multiple dipole model includes several
account for external EEG/MEG measurements. dipoles, each representing a certain anatomical
region of the brain. These dipoles have vary-
ing magnitudes and varying orientations, while
13.4.1 Dipole Source Localization their locations could be either fixed or variable
(i.e., multiple moving ECD models). Due to finite
The most commonly used brain electric source signal-to-noise ratio of the EEG/MEG record-
model is the equivalent current dipole (ECD) ings, the number of multiple dipoles that can be
model, which assumes that the scalp EEG or reliably estimated is limited, usually no more than
MEG is generated by one or a few focal dipole two dipoles in moving dipoles model [55].
sources. Each of the focal sources can be modeled Given a specific dipole source model, the
by an ECD with six parameters: three location dipole source localization (DSL) solves the EEG
parameters and three dipole-moment parameters. or MEG inverse problem by using a nonlinear
In MEG, since it is less sensitive to radial sources, multidimensional minimization procedure, to
parameter for radial orientation might be omitted, estimate the dipole parameters that can best
which leads to five parameters for an ECD. explain observed scalp potential or magnetic
The simplest and representative ECD model field measurements in a least-square sense [38,
is the single moving dipole, which has varying 55–58]. Further improvement of the DSL can
magnitude and orientation, as well as variable be achieved by combining EEG with MEG data
location. The location of the single moving dipole which may increase information content and
improve the overall signal-to-noise ratio [59, known locations inside the brain during a certain
60]. Generally speaking, there are two DSL time interval, and the variations in scalp potentials
approaches. One approach is the single time- or magnetic fields are due only to variations in
slice source localization, in which the dipole the strengths and orientations of these sources.
parameters are fitted at a time instance, based The dipole sources S can be related to the scalp
on single time “snapshots” of measured scalp potentials or magnetic fields, denoted as , by
EEG or MEG data [38, 58]. For example, scalp the lead field matrix A, which is only dependent
potentials or magnetic fields at a single time-slice on the head volume conductor properties and the
could be controlled into column vector φ, each source-sensor configurations:
row of which is electric potential or magnetic

field data recorded from one sensor. The problem Φ = AS (13.5)
then is to find a column vector X, the collection of
potentials or magnetic fields at the same sensor Here, is the N channels by T time-slices
sites but generated by assumed sources inside the EEG/MEG data matrix, and S is the M dipoles
brain. In practice, an initial starting point (also by T time-slices source waveform matrix. The
termed seed point) is estimated, and then using an task of the spatiotemporal DSL is to determine the
iterative procedure, the assumed dipole sources locations of multiple dipoles [56], whose parame-
are moved around inside the brain (the source ters could best account for the spatial distribution
space) in an attempt to produce the best match as well as the temporal waveforms of the scalp
between φ (measured scalp potential/field) and EEG/MEG measurement. Similar to Eq. 13.4, an
ψ (scalp potential/field generated by X). This iterative procedure is needed to adjust source pa-
involves solving the forward problem repetitively rameters with the aim to minimize the following
and calculating the difference between measured objective function:
and estimated data vectors at each step. The most
$ $ $ $
commonly used measure is the squared distance $ $2 $ $2
between the two data vectors, which is given by: $
J = $Φ − A S $ = $ I − AA Φ $
$ $ +
$ (13.6)
$ $
$ $2
J =$ φ − ψ $ (13.4) where I is the identity matrix and A+ is the
$ $
pseudo-inverse of matrix A. At each iterative step,
locations and orientations of sources are updated
where J is the objective function which is to which subsequently causes the update of J. Once
be minimized. From Eq. 13.3, it can be known the product between A and its pseudo-inverse
that the relationship between the dipole loca- becomes close to I, optimal source locations and
tion (r) and electric potential is nonlinear, and orientations are found since the objective func-
thus the problem expressed in Eq. 13.4 needs to tion is minimized. With the incorporation of the
be solved via nonlinear optimization. Different EEG/MEG temporal information in the model
methods could be applied to solve this nonlin- fitting, the spatiotemporal DSL is more robust
ear optimization problem, such as the simplex against measurement noise and artifacts than the
method [38], due to its simplicity and relative single time-slice DSL.
robustness to local minima. The nonlinear nature All DSL algorithms need an a priori knowl-
of DSL holds for MEG source localization, as edge of the number and class of the underlying
well. dipole sources. If the number of dipoles is under-
Another approach is the multiple time-slice estimated for a given model, then the DSL inverse
source localization, also termed spatiotemporal solution is biased by the missing dipoles. On the
source localization, which incorporates both the other hand, if too many dipoles are specified, then
spatial and temporal components of the EEG spurious dipoles can be introduced, which maybe
in model fitting [56]. In this approach, multiple indiscernible from the true dipoles. Moreover,
dipole sources are assumed to be fixed on un-
392 B. He et al.
since the computational complexity of the least- presurgical evaluation of epilepsy patients, which
squares estimation problem is highly dependent is known as electrocorticography (ECoG). Work
on the number of nonlinear parameters that must on CPI has suggested the similarity between
be estimated, too many dipoles also add needless measured ECoG signals and noninvasively
computational burden and may not lead to reliable reconstructed cortical potentials [21, 42, 64]
solutions. which suggests the potential clinical application
In practice, the principal component analysis of CPI in providing a noninvasive alternative of
(PCA) and multiple signal classification (MU- ECoG. Correcting the smearing effect of the
SIC) algorithms have been used to approximately low-conductivity skull layer, CPI techniques
estimate the number of field patterns contained offer enhanced spatial resolution in assessing
in the scalp EEG/MEG data [61]. For example, the underlying brain activity as compared to the
the MUSIC algorithm scans through the 3D brain blurred scalp potentials. The CPI is also referred
volume (solution space) to identify sources that to as downward continuation [21], in which the
produce potential patterns that lie within the sig- electric potentials over the epicortical surface are
nal subspace of the EEG/MEG measurements reconstructed from the electrical potentials over
[61]. To localize brain electric sources, a linearly the scalp surface.
constrained minimum variance (LCMV) beam- State-of-the-art cortical potential imaging has
former approach [62] has been developed for used a multilayer boundary element method ap-
EEG/MEG source localization, by designing a proach which links the cortical potential and scalp
bank of narrow-band spatial filters where each potentials via a linear relationship with inclu-
filter passes signals originating from a specified sion of the low-conductivity skull layer. By solv-
location represented by a dipole within the brain ing the inverse problem, cortical potentials were
while attenuating signals from other locations. estimated during somatosensory evoked poten-
Furthermore, statistical parametric maps based on tials [42] and interictal spikes in epilepsy patients
beamformers can be created by looking at output [64], which illustrate the potential clinical ap-
changes of spatial filters’ comparing conditions, plication of CPI approach. The CPI approach to
such as between the resting and the task, over the estimate cortical potential maps can also be real-
entire brain. ized with the finite element method (FEM) rather
than BEM [21]. A benefit of using FEM is that
it can handle local inhomogeneity and anisotropy
13.4.2 Cortical Potential Imaging in electrical conductivity profile, which cannot be
handled by BEM. An example of such a tech-
The cortical potential imaging (CPI) technique nique has been implemented in Zhang et al. [48]
employs a distributed source model, in which to reconstruct cortical potential distributions in
the equivalent sources are distributed in two- the existence of low conductive ECoG grid pads
dimensional (2D) cortical surface, and no ad in a configuration of simultaneous scalp EEG
hoc assumption on the number of source dipoles and ECoG recordings. The reconstructed cortical
is needed as in dipole source localization. This potentials were directly compared with recorded
group of techniques is mostly deployed with EEG ECoG signals from the same session.
signals. Using an explicit biophysical model of
the passive conducting properties of a head, the
CPI attempts to deconvolve a measured scalp 13.4.3 Cortical Current Density
potential distribution into a distribution of the Source Imaging
electrical potential over the epicortical surface
[21, 42, 63, 64]. While dipole source localization has been
The CPI techniques are of clinical relevance demonstrated to be useful in locating a spatially
because cortical potentials are invasively restricted brain electric event, it has a major
recorded in current clinical routines for the limitation in that its simplified source model may
not adequately describe sources of significant Since the CCD model is formed by a num-
extent. Therefore, distributed current density ber of dipoles (usually several thousands), the
source imaging has been aggressively studied forward solution for the dipole is still applied
in the past decades. Cortical current density here. Assuming quasi-static conditions, and the
source imaging techniques are distinguished linear properties of the head volume conductor,
from cortical potential imaging techniques in two the brain electric sources and the scalp EEG/MEG
aspects: (1) it uses electrical current density as a measurements can be mathematically described
variable instead of electric potential; (2) the cor- by the following linear matrix equation:
tical surface is convoluted which is different from

the epicortical surface used in cortical potential φ = AX + n (13.7)
imaging.

13.4.3.1 Cortical Current Density where φ is the vector of scalp potential or mag-

Source Model netic field measurement, X is the vector of current

Unlike the point dipole source models, the source distribution, n is the vector of additive
distributed source models do not make any ad measurement noise, and A is the transfer matrix

hoc assumption on the number of brain electric relating φ and X. So the cortical current den-
sources. Instead, the sources are distributed in
two-dimensional (2D) sheet such as the cortical sity source imaging is to reconstruct X from φ
surface or 3D volume of the brain. In this with the known transfer function A, by solving
section, we will discuss the current sources the inverse problem from Eq. 13.7. The same
distributed over the convoluted cortical surface relationship is also applied to volume current
(Fig. 13.6), known as the cortical current density density source imaging techniques, which will be
(CCD) model [40, 53, 65–67]. The rationale in discussed later. Reconstruction problems in both
implementing the CCD model is based on the cortical current density and volume current den-
observation that scalp EEG and MEG signals are sity imaging techniques belong to distributed cur-
mainly contributed by electrical currents flowing rent density imaging and can be solved with simi-
through cortical pyramidal neurons along the lar mathematic algorithms and methods. Thus, the
normal direction of the cortical surface [68]. The imaging estimation algorithms discussed below
cortical surface is highly folded (Fig. 13.6) and apply to both cortical current density and volume
has to be represented numerically in order to current density source imaging problems in gen-
conduct computations, such as calculating the eral.
lead field matrix, over it. A common approach in
numerical representation of the cortical surface 13.4.3.2 Linear Inverse Filters
is to triangulate the surface into many small The aim of the distributed current density imaging
triangles, on which a current dipole is assumed is to reconstruct source distributions from the
representing the cortical patch. noninvasive scalp EEG/MEG measurements or,
mathematically speaking, to design an inverse
Fig. 13.6 An illustration of the cortical surface, segmented from MRI data of a human subject, in side, back, and top
views
394 B. He et al.
filter B, which can project the measured data into However, when the rank of A is less than the
the solution space: number of its rows, AAT is singular, and its inverse
does not exist. In such a case, the general inverse

X = Bφ (13.8) can be sought by the method of singular value
decomposition (SVD) [70]. For an m×n matrix
This linear inverse estimation approach, A, its SVD is given by:
however, is intrinsically underdetermined,
because the number of unknown distributed A = U ΣV T (13.11)
sources within the brain is usually much larger
than the limited number of electrodes/sensors where U = [u1 , u2 , . . . , um ], V = [v1 , v2 , . . . , vn ],
over the scalp. Additional constraints have to = diag (λ1 , λ2 , . . . λp ), λ1 > λ2 > . . . > λp ,
be imposed in order to obtain unique linear and p = min (m, n). The vectors ui and vi are
inverse solutions. Below we discuss different the orthonormal eigenvectors of AAT and AT A,
imaging estimation solutions based on the respectively. The λi are the singular values of
different selections of additional constraints or matrix A, and is a diagonal matrix with the
assumptions. Readers may skip the following singular values on its main diagonal. Based on the
detailed treatment of imaging estimation SVD of matrix A, the general inverse of matrix A
techniques till Sect. 13.4.4, without affecting can be solved by:
the understanding of the concepts. The interested
p
1
reader can also refer to He et al. for a detailed +
A = VΣ −1
U =
T
vi uTi (13.12)
treatment of various source imaging estimation i=1
λ i
algorithms [4].
where ()+ is also known as the Moore-Penrose in-
General Inverse verse or the pseudo-inverse. For the linear system
The general inverse, also termed the minimum- of Eq. 13.7, the inverse solution estimated by Eq.
norm least-squares (MNLS) inverse, minimizes 13.12 is given by:
the least-square error of the estimated inverse

p
solution X under the constraint φ = AX in
+

−1
1
X = A φ = VΣ U φ=
T T
vi ui φ
the absence of noise. In mathematical terms, the λ
i=1 i
MNLS inverse filter BMNLS is determined when (13.13)
the following objective function is minimized:
$ $2 Truncated SVD
$ →$
− Although the general inverse leads to a unique
JMNLS = $φ − A X $
$
$ (13.9)
inverse solution with smallest residual error giv-
ing constraint in Eq. 13.9, it is often impractical
For an underdetermined system, if AAT is non- for real applications due to the ill-posed nature of
singular, we have: the EEG/MEG source imaging problem. In other

T − words, the small measurement errors in φ can be
BMNLS = A AA T
(13.10)
amplified by the small or near-zero singular val-
ues, leading to large perturbations in the inverse
where ()T and ()− denote matrix transpose and
solution.
matrix inversion, respectively. The general in-
A technique called truncated singular value
verse solution is also a minimum-norm solution
decomposition (TSVD) can be used to address the
among the infinite set of solutions, which satisfy
issue of small single values in the general inverse,
the scalp potential or magnetic field measure-
which is simply carried out by truncating at an
ments [39, 69].
index k < p in the evaluation of A+ given by Eq.
13.13 or mathematically [71]:
−1
k MNLS is a special case of the filter described in
1
BTSVD = V UT = vi uTi (13.14) Eq. 13.16, when λ = 0, explaining why MNLS
λ i
k i=1 solutions are extremely sensitive to noise. In
summary, the Tikhonov regularization parameter
The effects of measurement noise on the in-
is used to balance the details in reconstructions
verse solution are reduced because the significant
(lost because of the emphasis on G) and influence
amplification effect from small singular values is
from noise.
removed by truncating the k + 1 small singular
values. Meanwhile, the high-frequency spatial in-
13.4.3.3 Regularization Parameters
formation contributed by the small singular val-
As noted earlier, in order to improve the stability
ues is also lost as a trade-off, which also leads
of the source imaging problem, a free regular-
to smooth reconstructions of source signals. The
ization parameter λ in TIK [Eq. 13.15] or k in
balance between the stability and accuracy of the
TSVD [Eq. 13.14] is introduced and should be
inverse solution is controlled by the truncation
determined. Proper selection of this parameter is
parameter k.
critical for the inverse problem to balance the
stability and accuracy of the inverse solution. In
Tikhonov Regularization
theory, optimal regularization parameters should
A common approach to overcome the numeri-
be determined by minimizing relative error (RE)
cal instability caused by the ill-posedness is the
or maximizing correlation coefficient (CC) be-
Tikhonov regularization (TIK), in which the in-
tween the true source Xtrue and the inversely re-
verse filter is designed to minimize an alternative
constructed source Xinv :
objective function [72]:
$ $
$ $ $ $ $ $
$ $2 $ $2 $X true − X inv $
$ $
JTIK = $ φ − AX$ + λ$G φ $
$ $ $
$ (13.15) RE = $ $ (13.17)
$ $
$X true $
$ $
where λ is a small positive number known as
the Tikhonov regularization parameter and G can
be identity, gradient, or Laplacian matrix, corre-
sponding to the zeroth-, first-, and second-order X true · X inv
CC = $ $ $ $ (13.18)
Tikhonov regularization, respectively. The under- $ $ $ $
$X true $ · $X inv $
lying concept of this approach is to minimize both $ $ $ $
the measurement residual error and the inverse
solution (either source distribution, gradient, or Unfortunately, in real applications, the true
curvature) together with a relative weighting pa- source distribution is unknown, and alternative
rameter λ, in order to suppress unwanted ampli- methods that do not depend on a priori knowl-
fication of noise on small singular values in the edge of Xtrue should be used. Here we introduce
inverse solution. The corresponding inverse filter two types of methods in estimating regularization
is given by [72]: parameters, while more methods can be found in
− the literature [73].
BTIK = AT AAT + λGGT (13.16)
L-Curve Method
It can be observed that large values of λ make Hansen [74] popularized the L-curve approach
the solution smoother because the second term to determine a regularization parameter. The L-
in Eq. 13.15 dominates, while for a small value curve approach involves a plot, using a log-log
of λ, the first term in Eq. 13.15 dominates, and scale, of the norm of the solution, on the ordinate
the influence from noise might not be sufficiently against the norm of the residual, on the abscissa,
suppressed if λ is too small. For instance, the with λ or k as a parameter along the resulting
396 B. He et al.
13.4.3.4 Interpretation of Linear

Inverse in Bayesian Theory
The linear solutions discussed earlier can also be
understood in a Bayesian perspective [78, 79].
Consider the forward problem in Eq. 13.7. From
Bayes’ theorem, the posterior probability for the
inverse solution x conditioned on the data φ is
given by:
Fig. 13.7 Illustration of the L-curve approach. By plot- P (φ|x) P (x)

P (x|φ) = (13.20)
ting the norm of the inverse solution versus the norm of P (φ)
the residual as functions of regularization parameter (λ or
k), an “L” shaped curve occurs, and the optimal parameter which one would like to maximize as the posterior
is placed near the “corner” of the curve
probability for the inverse solution given the data.
P(φ| x)is the conditional probability for the data
curve. In most cases, the shape of this curve is in given the inverse solution, and P(x) is a prior
the form of an “L,” and the λ or k value at the distribution reflecting the knowledge of the statis-
corner of the “L” is taken as the optimal regu- tical properties of the source model. To maximize
larization value (Fig. 13.7). At the corner, clearly the posterior probability, the cost function could
both ||X|| and φ − Ax attain simultaneous indi- be formulated, usually, using the log-posterior
vidual minima that intuitively suggests an optimal probability as:
solution. A numerical algorithm to automatically
compute the site of the L-curve corner, when it x̂ = argmaxP (φ|x) P (x)
x
exists, has been given by Hansen and O’Leary
[75]. The algorithm defines the corner as the point ≡ argmax (ln (P (φ|x)) + ln (P (x))) (13.21)
s
on the L-curve with maximum curvature.
If Gaussian white noise with variance of σ 2 is
Statistical Methods assumed, the likelihood is denoted by P (φ|x) ∝
Statistical methods have been proposed for the
e− 2σ 2 φ−Ax2 . If the prior distribution is given
1 2
regularization parameter determination. For ex-

by P(x) ∝ e−(θf (x)) , where θ is a scalar constant
ample, if the expectations of noise and measure-
and f (x) is a function of the inverse solution x,
ment are both available, the truncation parameter
by applying the log operation, the cost function
of TSVD in Eq. 13.14 could be determined by
yielding the maximum a posteriori estimate could
[71, 76]:
be written as:
⎧ $ $ ⎫
⎪
⎪ $$2 ⎪ ⎪
⎪
⎪ E $n$ ⎪ ⎪ C(x) = φ − Ax2 + λ · f (x) (13.22)
⎨ λ2 ⎬
k = max i| 2 ≥ $ $2
i
(13.19)
⎪ λ1
i ⎪ $$ ⎪ ⎪ where λ = 2θσ 2 . If f (x) = Gx22 , cost function
⎪
⎪ E $ $ ⎭ ⎪
⎩ $φ $ ⎪ here is exactly same as the objective function (Eq.
13.15) obtained through Tikhonov regularization.
Another popular method for choosing the reg- One benefit in discussing linear inverse solu-
ularization parameter is the generalized cross- tions in the Bayesian perspective is that the theory
validation (GCV) method proposed by Golub et can be extended to include the understanding of
al. [77]. The GCV technique is based on the sta- some nonlinear inverse solutions. If f (x) = Gx1 ,
tistical consideration that a good value of the reg- the cost function becomes the objective func-
ularization parameter should predict missing data tion using L1-norm methods in the framework of
values; therefore, no a priori knowledge about the Tikhonov regularization. Furthermore, from the
error norms is required.
Bayesian theory, it is known that a Gaussian a Then, the forward solution for VCD is the same
priori likelihoods, such as those implemented in as the forward solution for CCD with the only
linear inverse methods, usually result in smooth difference in the definition of source space. On
solutions, while an exponential a priori likeli- the other hand, the 3D source imaging approach
hoods, such as those in nonlinear L1-norm meth- faces greater technical challenges: by extending
ods, lead to sparse solutions. This explains the the solution space from 2D cortical surface to 3D
characteristics of inverse source reconstructions brain volume, the number of unknown sources
from both types of methods. Sparsity-enforcing increases dramatically. As a result, the source
regularizations can also be cast as convex opti- imaging problem is even more underdetermined,
mization problems and can be solved efficiently and the inverse solution is usually smeared due
with accurate numerical techniques [80, 81]. to regularization procedures. In addition, it be-
The major advantage using the Bayesian the- comes more important to retrieve depth infor-
ory in developing different EEG/MEG inverse mation of sources in 3D source imaging. While
solutions is that this framework provides the flex- the cortex can be modeled as a folded surface in
ibility to incorporate different a priori likelihoods cortical source imaging approach so that sources
through f (x). For a more mathematical treatment in sulci and gyri have different eccentricities,
of Bayesian methods in source imaging, refer to deeper sources probably exist below the corti-
Sekihara and Nagarajan’s book [82]. cal layer, such as in amygdala and hippocampal
formation.
13.4.4 Volume Current Density 13.4.4.2 Inverse Estimation Techniques

Source Imaging in Volume Current Density
Imaging
13.4.4.1 Challenges of the 3D Source The most popular 3D linear inverse solution is the
Imaging minimum-norm (MN) solution, which estimates
Tremendous progress has been made during the the 3D brain source distribution with the smallest
past decades for the 3D source imaging, in which L2-norm solution vector that would match the
the brain electric sources are distributed in the 3D measured data [39, 69]. It is equivalent to select
brain volume. Similar to the CCD source imaging G as an identity matrix in Eq. 13.15. Different
problem, the 3D source imaging approach is also regularization parameter selection techniques as
based on a distributed source model, i.e., volume detailed in linear inverse filters can be used here
current density (VCD) source model, and is im- to suppress the effects of noise.
plemented by solving the linear inverse problem However, the standard minimum-norm
as detailed in Sect. 13.4.3. The source space of solution has intrinsic bias that favors superficial
the VCD model usually consists of the entire sources because the weak sources close to the
human brain, including the deep structure such as sensors can produce scalp EEG/MEG with
hippocampus. Since the white matter is believed similar strength as strong sources at deep
of no generators for EEG/MEG, it can be removed locations. To compensate for the undesired
in some applications. A common approach in depth dependency of the original minimum-
numerical representation of the human brain is to norm solution, different weighting methods have
divide the brain volume into many small voxels. been introduced. The representative approaches
Each voxel is modeled by a current dipole similar include the normalized weighted minimum-norm
as in the CCD source model. However, the orien- (WMN) solution [76, 83] and the Laplacian
tation of the dipole at each voxel is not fixed as in weighted minimum-norm (LWMN) solution, also
CCD models. The dipole at each voxel is usually termed LORETA [41, 84].
decomposed into three orthogonal components The WMN compensates for the lower gains of
with each having fixed orientation. The selection deeper sources by using lead field normalization.
of orientations of these three components is usu- In the absence of noise, the inverse source esti-
ally dependent on the utilized coordinate system. mates can be given as:
398 B. He et al.
This approach combines the lead field nor-

φ = AW − W X (13.23)
malization with the spatial Laplacian operator,
thus giving the depth-compensated inverse solu-
The concomitant WMN inverse solution is
tions under the constraint of smoothly distributed
given by [76, 83]:
sources.
− Many variants of the minimum-norm solution
X WMN = W W T AT AW W T AT φ (13.24) were also proposed, by incorporating a priori in-
formation as constraint in a Bayesian formulation
where W is the weighting matrix acting on the or by estimating the source-current covariance
solution space. Most commonly, W is constructed matrix from the measured data in a Wiener for-
as a diagonal matrix [76, 83, 84]: mulation. All these efforts were made to improve
certain aspects of 3D source imaging techniques;
W = diag (a1 , a2 , · · · , an ) (13.25) however, they are not universally suitable for all
3D volume current density imaging applications.
where A = (a1 , a2 , · · · , an ). Thus, by using the In addition, both the MUSIC algorithm
norm of each column of the transfer matrix as the [61] and beamformer techniques [62], which
weighting factor for the corresponding position have been discussed in sections for dipole
in the solution space, the contributions of the source localization methods earlier, can be used
entries of the transfer matrix to a solution are to reconstruct 3D brain source distributions.
normalized. However, it should be noted that both MUSIC and
The LWMN approach defines a combined beamformer techniques are scanning techniques,
weighting operator LW, where L is a 3D discrete which are not based on distributed source models.
Laplacian operator acting on the 3D solution Beamformer techniques utilize the spatial filter
space and W is defined the same as in Eq. 13.24. designed for each scanned point in a 3D source
The corresponding LWMN inverse solution, or space, while the MUSIC algorithm computes
the LORETA solution, is then [41, 84]: the correlation between field vectors originated
− − −
by a dipole at the scanned position against the

X LWMN = W LT LW AT A W LT LW AT φ covariance structure of measurements.
(13.26) Figure 13.8 shows an example of 3D source
imaging of seizure activities by using a combined
Fig. 13.8 Seizure onset zones (SOZs) and the source is co-localized with surgically resected zones (shown in
time frequency representations estimated from a typical green) in patients 1 and 2. (From Yang et al. [85] with
seizure in two patients. The estimated SOZ (left and mid- permission)
dle panels, 60% threshold, yellow to orange color bar)
approach consisting of independent component computational effort is needed. Different nonlin-

analysis and LORETA [85]. Yellow color refers ear optimization approaches have been suggested,
to volume sources, and green color refers to surgi- including the iteratively reweighted least-squares
cally resected regions. The patients were seizure- method and the linear programming techniques
free after 1-year follow-up from the surgery. [81, 94, 95].
Solving the inverse problem for 3D source Imposing sparsity on the current density is
space using ECoG or sEEG measurements has the direct result of using L1-norm regularization
also been attempted [86–88]. Given the surge in terms or priors, which can lead to overly focused
using sEEG recordings to determine the epilep- solutions. On the other hand, such focal solu-
togenic zone in epilepsy patients, such studies tions do not seem to be physiologically viable;
indicate the value of using source imaging tech- thus, recent studies have imposed the sparsity
niques even with invasive recordings. Hosseini priors on other mathematical domains such as
and colleagues studied the potential advantages the wavelet transform [96, 97], spatial gradient
and disadvantages of this approach and proposed [98–100], and Laplacian of underlying current
to combine scalp and intracranial EEG measure- densities or multiple mathematical domains [80].
ments to eliminate the potential disadvantages These regularization priors encourage solutions
[88]. which are sparsely represented in those math-
ematical domains, which in turn determine the
13.4.4.3 Nonlinear Inverse Techniques solutions’ characteristics and features. For in-
Because the 3D EEG/MEG inverse problem is stance, a solution sparsely represented in the spa-
highly underdetermined, the linear solutions ob- tial gradient domain encourages piecewise ho-
tained by the minimum-norm inverse and its vari- mogeneous solutions [98]. These studies indicate
ants are usually associated with relatively low that by enforcing sparsity to transformations of
spatial resolution. To overcome this problem, sev- the current density, such as the spatial gradient,
eral nonlinear inverse approaches have been in- as opposed to the current density, the obtained
troduced to achieve more localized imaging resolutions are not overly focused and demonstrate
sults. more desirable and realistic features more in line
One recent popular method in reconstructing with our physiological intuitions.
focal sources is to solve the inverse problem using A question that might be raised is that how
the L1-norm instead of commonly used L2-norm are such improvements possible, given the limited
[89–93] on the penalty term of inverse solutions measurements at hand? The reason lies in the fact
in Eq. 13.15 or on the a priori likelihood function that sparse signals only contain a limited amount
in Eq. 13.22. The L1-norm methods prefer sparse of information, as such signals only contain a lim-
solutions since the L1-norm of a sparse solu- ited number of nonzero elements. Once a signal
tion vector is usually less than the L1-norm of a itself, or its representation in another domain, can
smooth solution vector on the condition that both be represented in a sparse fashion, this indicates
generate the similar scalp EEG/MEG signals. On that its redundancies are discovered and, conse-
the contrary, the L2-norm methods prefer smooth quently, fewer measurements are needed to recon-
solutions since the L2-norm of a smooth solution struct. Hence, with limited MEG or EEG mea-
vector is usually less than the L2-norm of a sparse surements, much better signal reconstructions can
solution vector on the condition that both gen- be achieved. Furthermore, combining MEG and
erate the similar scalp EEG/MEG. The L1-norm EEG signals improves the performance of sparse
methods, thus, provide much more focal solutions source imaging algorithms, as more measure-
and a more robust behavior against outliers in the ments are at dispense [67]. Figure 13.9 shows
measured data [94]. However, the use of the L1- widely distributed cortical sources from multiple
norm requires solving a nonlinear system of equa- time points for face perception and recognition
tions for the same number of unknowns as the obtained with the use of sparse source imaging
L2-norm inverse approach; therefore, much more on combined MEG and EEG data. The spatial
400 B. He et al.
Fig. 13.9 Dynamic patterns of sparse source reconstruc- maximal difference between faces and scrambled faces.
tions using combined EEG and MEG within P100/M100 (b) Cortical current density maps reconstructed within
and P170/M170 components from a face recognition task. P100/M100. (c) Cortical current density maps recon-
(a) An EEG waveform from one channel (red electrode structed within P170/M170. (From Ding and Yuan [67]
shown registered with the head) and an MEG waveform with permission)
from one channel (green sensor), both of which show the
distributions of these cortical sources and their inverse solution, such as the minimum-norm
temporal dynamics further revealed similarities inverse solution, and the iteration process that
and differences at different stages of neural pro- refines the initial estimate to the final focal
cesses for different conditions. Consistent spatial source solution. The iterations are based on
patterns in the visual cortex between actual faces weighted norm minimization of the dependent
and scrambled faces are observed during the time variable (similar as the weight process used in
window of P100/M100 for perception. During weighted minimum-norm inverse solutions) with
N170/M170 for face recognition, it is observed the weights being a function of the preceding
that bilateral fusiform (i.e., 150–160 ms) and iterative solutions. Similarly, a self-coherence
lateral ventral occipital regions (i.e., 160–175 ms) enhancement algorithm (SCEA) has also been
are more active to actual faces than scrambled proposed to enhance the spatial resolution of
faces, which has been similarly reported using the 3D inverse estimate [102]. This algorithm
fMRI data [101]. provides a noniterative self-coherence solution,
Through a different approach, a nonparametric which enhances the spatial resolution of an
algorithm for finding localized 3D inverse unbiased smooth estimate of the underdetermined
solutions, termed focal underdetermined system 3D inverse solution through a self-coherence
solution (FOCUSS), was proposed by Gorodnit- process.
sky et al. [83]. This algorithm has two integral Following these lines of investigation,
parts: a low-resolution initial estimate of the Sohrabpour et al. proposed a new inverse source
imaging technique that not only was capable 13.4.5 Multimodal Source Imaging
of imaging the location of underlying brain Integrating Electromagnetic
sources using scalp EEG/MEG measurements and Hemodynamic Imaging
but also was capable of estimating the underlying
sources extent, i.e., size [81]. Determining Until now, we only discussed the source
the size of underlying brain activity is of imaging problems and methods using single
particular importance in many applications modality data, such as EEG or MEG. Efforts
such as determining the seizure onset zone have been made to attempt to improve the
in epilepsy patients, as such information is performance of EEG/MEG source imaging by
necessary for optimizing treatments. One of integrating electromagnetic and hemodynamic
the features of this work was to use an iterative measurements [54, 105]. Neuronal activity
re-weighting approach to, ultimately, eliminate elevates electrical and magnetic field changes
the need for applying thresholds to solutions (the primary effects) as well as hemodynamic
to separate background activity from desirable and metabolic changes (the secondary effects).
signals. Sohrabpour et al. validated their proposed The observation of electrical and magnetic
technique by comparing it to clinical findings field changes is mainly made using EEG and
derived from invasive measures (in addition MEG, respectively, as what have been discussed.
to comprehensive Monte Carlo simulations). Furthermore, both EEG and MEG have high
This approach has inspired other researchers temporal resolution at sub-millisecond scale but
to introduce these ideas in Bayesian algorithms limited spatial resolution. On the other hand,
as well [103]. functional magnetic resonance imaging (fMRI)
In addition to applying L1-norms instead [106–108], based on the endogenous blood
of L2-norms, more elaborate mathematical oxygenation level-dependent (BOLD) contrast
constructs, such as the mixed-norm, have also [109], is another well-established technique
been proposed [104]. The mixed-norm operator in mapping human brain function (see Chap.
is basically the generalization of the Lp-norm 11 of this book). The benefit of fMRI is,
to multiple dimensions of a high-dimensional conversely, its high spatial resolution to the level
matrix, where each dimension can be measured of millimeters but of slow response time and
(or regularized) distinctly. One of the issues thus low temporal resolution. In combination,
associated with pure L1-norm estimates is that these two complementary noninvasive methods
the reconstructed time course of activity is would lead to an integrated neuroimaging
not smooth and random location of the cortex technology with high resolution in both space
gets activated for brief moments of time. In and time domains that cannot be achieved by any
order to alleviate this issue, mixed-norm was modality alone. Such superior joint spatial and
introduced into source imaging algorithms. The temporal resolution would be highly desirable
general intuition behind the mixed-norm operator to delineate complex neural networks related
is that each dimension of a high-dimensional to cognitive function, allowing answering the
matrix can be regularized uniquely to induce a question of “where” as well as the question of
specific structure in the solution; for instance, the “when.” It can also permit delineation about
spatial dimension might be regularized with an the hypotheses of top-down versus bottom-up
L1-norm type regularization to induce sparsity processing with the temporal resolution provided
in the spatial domain where only a limited by electrophysiology. The integration of EEG,
number of sites get activated but an L2-norm MEG, and fMRI is thus of significant interest
type regularization on the temporal dimension to to provide enhanced spatiotemporal imaging
induce smooth activity over time. performance.
402 B. He et al.
Fig. 13.10 Illustration of multimodal imaging ap- disparate temporal scales of the responses in the EEG and
proaches based on the spatial and temporal integrations. BOLD fMRI signals. Also, responses of both modalities
Waveforms of a typical EEG event-related potential and are widely distributed in the brain. (From He et al. [105]
a block-designed BOLD change are shown. Notice the with permission, © 2011, IEEE)
As illustrated in Fig. 13.10, integration of the vascular and electrophysiological responses

fMRI with EEG/MEG has been pursued in two may lead to fMRI displacement. Thus, the
directions, which either relies on (1) the spatial mismatch between a single static fMRI map
correspondence or (2) the temporal coupling of and consecutive snapshots of EEG/MEG during
fMRI and EEG/MEG signals. The first approach the same period can lead to biased estimates
of spatial integration typically utilizes the fMRI such as the fMRI extra sources (seen in fMRI
maps as a priori information to inform the but not EEG/MEG), the fMRI invisible sources
locations of the electromagnetic sources [52, (seen in EEG/MEG but not in fMRI), and the
65]. In these methods, fMRI analysis yields displacement sources (see detailed discussion
statistical parametric maps with several fMRI in [54]). New methods have been proposed
hotspots, which each constrains the location toward overcoming this limitation, by means
of an equivalent current dipole or collectively of a time-variant spatial constraint estimated
produces weighting factors to evenly distributed from a combination of quantified fMRI and EEG
current sources. With the spatial constrains, the responses [53] or estimating regionally fMRI-
ill-posedness of the EEG/MEG inverse problem informed models by allowing model parameters
is moderated, and continuous time course of jointly computed from electrophysiological
electromagnetic waveforms can be resolved from source estimates and fMRI data rather than
the fMRI hotspots, thus allowing inferences about exclusively dependent on fMRI [110]. Examples
the underlying neural processes [65]. of applying EEG/MEG-fMRI integration in the
A major limitation of the spatial integration investigation of visual processing function have
approach is that fMRI yields relatively static demonstrated how the subtle spatiotemporal
maps compared to dynamic evolution of dynamics revealed from electrophysiological
electromagnetic signals, owing to the highly imaging were able to delineate the hypotheses
different temporal scales in which the signals in with regard to the underlying neural processes
these two modalities are generated and collected [53]. Figure 13.11 [53] shows an example of time-
[54]. Additionally, the spatial difference between varying fMRI/EEG integration to mapping visual
Fig. 13.11 fMRI-EEG multimodal neuroimaging. (a) (1st row) or fMRI-EEG integration using our proposed
The pattern-reversal checkerboard visual stimulation, (b) adaptive wiener filter (2nd row) and the conventional
fMRI activation map with a corrected threshold p <0.01, 90% fMRI weighted algorithm (3rd row). Both the source
and (c) the global field power of VEP and the dynamic images and the fMRI activation map are visualized on an
cortical source distribution at three VEP latencies (76, 112, inflated representation of cortical surface. (From Liu and
212 ms after the visual onset) imaged from EEG alone He [53] with permission)
information processing pathways. In response to the reconstructed contralateral CCD distribution

the unilateral visual stimulation (Fig. 13.11a), using three imaging algorithms, respectively.
the activated cortical areas at the contralateral From the CCD images reconstructed by only
hemisphere were revealed in the fMRI activation using the VEP data, the dorsal pathway was
map (Fig. 13.11b). The fMRI activation map seen gradually extending from lower-tier visual
indicated a dorsal visual pathway covering V1, areas to high-tier visual areas. By using an
V2, dorsomedial areas (such as V3 and V7), adaptive Wiener filter to integrate the fMRI and
intraparietal sulcus (IPS), as well as medial EEG data, a consistent sequence of activities
temporal (MT) area (also known as V5). The was observed with a much enhanced spatial
top row of Fig. 13.11c shows the time course of resolution, showing the pathway starting from
global field power of VEP, which indicates three V1/V2 and then V3/V3a and finally V5/V7 and
VEP peak latencies (76, 112, and 212 ms). The IPS. The observed cortical visual pathway was
second through fourth rows of Fig. 13.11c show generally in agreement with the well-known
404 B. He et al.
hierarchical organization of the visual system. As mentioned in this chapter, EEG and MEG
In contrast, the imaging results obtained by using signals need to be analyzed and pre-processed
the conventional 90% fMRI-weighted approach before use. EEGLAB is one such toolbox particu-
also had an improved spatial resolution compared larly specialized in time series analysis and blind
to the EEG-alone source imaging. However, source separation techniques [112]. CARTOOL is
it shows a false positive source region in and a toolbox tailored for topographical data analysis
around V1/V2 at the latency of 212 ms, whereas and clustering, highly useful when performing
a more likely high-tier EEG source around V5, electrophysiological mapping studies (Sect. 13.2)
observable from the EEG data, is missed. [113].
On the other hand, the second approach of Subject-specific head models are necessary
temporal integration utilizes the EEG/MEG dy- for accurate reconstruction of sources, and these
namic signatures in the time or frequency domain models are usually built from subjects’ own MRI,
to inform the statistical mapping of fMRI. These when available. Some of the toolboxes designed
quantities obtained from electromagnetic record- for this purpose include FreeSurfer [114],
ings are typically convolved with a canonical BrainSuite [115], and BrainVISA anatomist
hemodynamic response function and then corre- [116]. OpenMEEG can also be used to generate
lated to BOLD signals on a voxel-by-voxel basis subject-specific BEM models [117].
to identify the statistical fMRI maps correspond- There are many toolboxes that specialize on
ing to the electromagnetic temporal signatures of source imaging. Among these toolboxes, some
interests. In this way, the integration method can are capable of analyzing functional connectiv-
recover the neural substrates by answering the ity such as eConnectome [118], FieldTrip [119],
question of “where” in joint with the question of MNE [120], and Nutmeg [121], while some fo-
“when.” An intriguing example is the study of cus more on source imaging such as Brainstorm
nonrepeatable effects in epileptic patients, i.e., the [122].
interictal activities. Correlates of the dynamics The availability of these computational tool-
of interictal discharges with the BOLD have led boxes means that students, researcher, and clini-
to insights into the problem of localizing the cians (and even interested members of the public)
epileptic foci from fMRI [111]. can have easy access to these programs for learn-
ing purposes or to follow their own line of inquiry.
13.5 Getting Started

with Electrophysiological 13.6 Discussions
Imaging and Data Processing
The ultimate goal of the electrophysiological
EEG and MEG data processing and source source imaging is to image brain electric
imaging algorithms have a far wider reach than activity with high resolution in both time and
the research community. There are, fortunately, space domains based on noninvasive EEG and
many publicly available and free-of-charge MEG recordings. Such noninvasive and high-
analysis toolboxes available on the internet. We resolution brain mapping technique would
intend to introduce a few of the more popular bring significant advancement in the fields of
ones, in this section, merely as a guide to the clinical neurosurgery, clinical neurology, neural
readers. This list is by no means a comprehensive pathophysiology, cognitive neuroscience, and
list of available EEG/MEG analysis toolboxes neurophysiology. For example, it will facilitate
available to date. All of the software and epilepsy presurgical planning, noninvasive
toolboxes we introduce here are developed by localization, and delineation of the epileptic
active research groups and include extensive zone in seizure patients; characterize the
online tutorials and/or online help communities brain dysfunction in schizophrenic, depression,
and can be achieved freely on the web. alcoholics, and Alzheimer’s patients; localize and
image cortical regions contributing to cognitive proposed [71, 73, 77], none of them has been
tasks; and even assist in neural decoding in brain- demonstrated to be universal, and different meth-
computer interfaces. ods should be considered depending on different
During the past decades, numerous techniques applications. On the other hand, different inverse
have been developed for brain electric source filters have been developed for specific applica-
imaging by solving the EEG and MEG forward tions based on various assumptions, such as the
and inverse problems. Dipole source localization presence or absence of noise, the availability of
is particularly useful for localizing descrete statistical information on signal and noise, and
focal brain electric sources, while the distributed so on. Not surprisingly, more robust and accurate
source imaging has the capability of imaging inverse imaging solutions can be obtained by
spatially distributed sources and multiple areas incorporating more a priori information as con-
of activities, such as the 2D cortical imaging and straints, for example, the anatomical constraint,
the 3D brain tomographic imaging. The choice of the temporal constraint, and the functional
using which inverse imaging approach depends constraint. The anatomical constraint can be
on the particular application, since each inverse easily implemented by the co-registration of
imaging algorithm has its own advantages and EEG and MEG inverse imaging solutions with the
limitations. structural brain images obtained from MR images
The major limitation of the dipole source [40]. The temporal constraint can be achieved by
localization is that it requires a priori knowledge selecting an epoch of EEG or MEG data as input
on the number of dipole sources. The distributed to the inverse imaging procedure with assumption
source imaging, on the other hand, makes no that the underlying bioelectric sources remain
assumption on the number of neural sources, relatively invariant. The functional constraint has
whereas it has to deal with a highly underde- shown great promise by combining the electro-
termined inverse problem. The cortical imaging magnetic and hemodynamic imaging modalities
technique has the potential to compensate for that were recorded using the same paradigm in
the head volume conduction effect and achieve the same subjects [52, 53]. The rationale for this
high-resolution mapping of cortical activities, multimodal integration is that neural activity
whereas the 3D neuroimaging approach has the generating EEG and MEG signals increases
capability of retrieving the depth information of glucose and oxygen demands [123]. The growing
the distributed brain electric sources. A recent body of evidence suggests that there is close
trend in the 3D distributed source imaging is spatial coupling between electrophysiological
to use the realistic geometry volume conductor signals and hemodynamic response [4]. However,
model constructed from the MR or CT images of many technical challenges still exist, and caution
individual subjects, through which the anatomical must be taken when interpreting multimodal
constraints become feasible and the obtained studies [124].
results can be interpreted more meaningfully In conclusion, the electrophysiological source
and in line with clinical intuition. Another major imaging, by means of reconstructing the under-
trend in the 3D neuroimaging is the development lying brain sources from the EEG and MEG,
of novel techniques that aim to overcome the has great potential for noninvasively mapping
smoothing effect of the inverse imaging solution, the brain activation and function, with high spa-
either by reducing the under-determination of tiotemporal resolution. Despite many challenges,
the inverse problem [43] or by some nonlinear with the integrated effort of algorithm develop-
inverse approaches [80, 81, 83, 93, 104]. ment, computer simulation, experimental explo-
The performance of the distributed source ration, clinical validation, and the availability of
imaging depends on the linear inverse filter and more powerful computing resources, it can be
regularization technique being selected. The confidently foreseen that the electrophysiological
regularization technique is critical to suppress source imaging will become an important neu-
noise and obtain stable inverse solution. Although roimaging tool for imaging neural abnormalities
many regularization techniques have been and understanding the human mind.
406 B. He et al.
Acknowledgments This work was supported in part (i) How could you distinguish between
by NIH EB021027, EB006433, NS096761, AT009263, condition A and B if you were only
MH114233, NSF CBET-0933067, NSF CAREER Award
given Vφ ?
ECCS-0955260, NSF NRI 1208639, and NSF EPSCoR
RII Track-2 1539068. (ii) Let us assume that Vφ under conditions
A and B has the same exact distribution
except that for condition A the distribu-
Homework tion is centered around the point (1,1)T
and for condition B around the point
1. What is the Nyquist frequency? How is it (−1,−1)T . Let us denote this probability
related to the sampling frequency of a band- distribution with p(x, y) and also let us
limited signal? assume symmetry with respect to origin,
2. If we believe that our signals of interest in the that is, p(x, y) = p(−x, −y), and indif-
EEG/MEG recordings are within the 1–50 Hz ference to input variables’ order, i.e.,
frequency bands, what would be the mini- p(x, y) = p(y, x). Now if we want to fit
mum sampling rate you propose that will al- the line y − αx − δ = 0, such that any
low the recovery of the full information con- point lying on one side of this line is
tent within this particular frequency band? designated as condition A and the other
3. Could you think of a way to define the min- side as condition B, how should we find
imum number of EEG/MEG sensors neces- (α, δ)?
sary to avoid aliasing the spatial frequency (iii) Based on your answer in (ii), find the
content of surface recordings? optimal set of (α, δ), if any.
4. We only record from two EEG electrodes, 5. If we assume that a dipole is placed at coordi-
say C3 and C4 , for two conditions A and nates (x, y, z), the distance between the dipole

B. These two conditions are elicited when source and field space, (x , y , z ), is defined
stimuli A and B are presented to our exper- as r = (x − x ) + (y − y )2 + (z − z )2 .
2
iment subject. Each stimuli is presented 100 (i) Calculate ∇ 1r , where ∇ is the
times, and the voltage recorded from C3 and gradient operator defined as ∇f =
T
C4 at 100 ms poststimulus is recorded in a ∂f ∂f ∂f
T ,
∂x ∂y ∂z
, .
vector, Vφ = φC3 (100), φC4 (100) where
1
(ii) Calculate ∇ r , where ∇ is the gradi-
φC3 (100) and φC4 (100) are the recorded sig-
ent operator with respect to (x , y , z ),
nals from C3 and C4 electrodes at 100 ms T
poststimulus, and is plotted below: i.e., ∇ f = ∂x ∂f ∂f ∂f
, ∂y , ∂z .
1 1

(iii) Show that ∇ r = −∇ r .
6. Assuming that a current dipole is placed
at the origin of an infinitely homogeneous
space pointing toward the z-direction, i.e.,
(x, y, z)T = (0, 0, 0)T and J i = (0, 0, 1)T ,
using Eq. 13.3:
(a) Can you calculate the potential field
generated by this dipole in any point
(x , y , z )?

Hint. ∇ 1r .J i (x, y, z) dv =
v
∇ 1r .J i , where J i is the current dipole

moment at the origin and ∇ 1r .J i is the
inner product of the dipole moment and
the gradient of the reciprocal of field
point distance to dipole. This equality is (ii) What would the MEG sensors record if
due to the fact that we assumed the dipole S = (1,1,1)T ?
source to be a point source at the origin. (iii) What if S = (2,−1,2)T ?
This basically is the impulse response of (iv) What if S = (3,0,3)T ?
the Poisson’s equations, more generally (v) What if S = (1,1,1)T + t(2,−1,2)T ,
referred to as the Green’s function. The ( t ∈ R)?
inner product between vectors A = (Ax , (vi) Can you calculate the null space of ma-
Ay , Az )T and B = (Bx , By , Bz )T is defined trix A, that is, all vectors x such that
as follows: A. B = Ax Bx + Ay By + Az Bz . Ax = 0?
(b) Assuming that the EEG sensor is located (vii) Can you briefly explain why the inverse
at (0, 0, 1)T , what number would it read problem is not unique? You can math-
as the potential (ideal conditions, noise is ematically prove this using the concept
nonexistent)? of null space of a matrix.
(c) What if the sensor is located at (1, 0, 0)T ? 10. Combining Eqs. 13.5 and 13.8:
(d) What if the sensor is located at (0, − 1, (i) Can you formulate the relationship be-
0)T ? tween estimated, X, and true source, S?
7. Repeat problem 6 to calculate the magnetic (ii) Based on the relation derived in (i), what
field an MEG magnetometer would should be the relationship between A
sense

at the same 1 locations. Use (B = and B, for the estimated source to be
μ
4π
J × ∇ r dv) and the Green’s
i exactly the same as the true source?
function hint given before. The cross product (iii) Can linear methods, as studied in this
between vectors A = (Ax , Ay , Az )T and problem, ever truly estimate the true
B = (Bx , By , Bz )T is defined as follows: source without any further priors or
A × B = (Ay Bz − Az By , Az Bx − Ax Bz , Ax By − assumptions?
Ay Bx ). 11. Can you derive Eq. 13.16 from Eq. 13.15
8. Based on problems 6 and 7, can you explain by differentiating Eq. 13.15 and setting it to
[and prove mathematically] why EEG signals zero?
are less sensitive to tangential sources and 12. Let us study the Bayesian approaches in more
MEG signals to radial sources? detail (Eqs. 13.20, 13.21, and 13.22). Let us
9. Let us simply assume that the lead field ma- assume that φ = Ax + n and that n is a white
trix A, of an MEG recording system with two Gaussian noise, n ∼ N 0, σn 2 :
recording channels and 3 possible sources, is (a) What is the probability distribution func-
given as follows: tion (pdf) of n?
(i) Assuming that the given lead field (b) What does p(φ| x) mean? Convince
matrix models the relationship between yourself that p (φ|x) = p(n) ∝
source current density and the magnetic − 1 φ−Ax2
e 2σn 2 .
field in z-direction, what is the relation- (c) If we assume x ∼ N(0, σ s 2 ), what is p(x)?
ship between the recorded magnetic (d) Using Bayes’ rule (Eq. 13.20), formulate
field (in z-direction) B at these sensors the posterior distribution p(x| φ).
and the current density S = (S1 ,S2 ,S3 )T ? (e) Define the likelihood of a distribution as
Assume ideal conditions where no L(x) = ln p(x). Derive the posterior
noise exists. likelihood calculated in (iv).
408 B. He et al.
(f) Formulate x̂ = argmaxL (x|φ) and tions to induce sparsity in the solution? Spar-
x
sity in case of a 2D signal means only 1
derive a similar formula to Eq. 13.15,
nonzero element!
showing that weighted minimum-norm
(WMN) solutions are a form of maximum
a posteriori (MAP) estimators.
References
13. TheL2 -norm of a 2D vector, (x, y), is defined
as x 2 + y 2 , and the L1 -norm is defined as 1. R. Caton, The electric currents of the brain. Br. Med.
x + y. The level sets of norm functions are J. 2, 278 (1875)
closed curves partitioning the space to inside 2. H. Berger, Über das Elektrenkephalogramm des
Menschen. Archiv f. Psychiatrie 87(1), 527–570
and outside. On the other hand, some func- (1929). https://doi.org/10.1007/BF01797193
tions, such as lines or hyperplanes, partition 3. D. Cohen, Magnetoencephalography: Detection of
the space to above and below. We will explore the brain’s electrical activity with a superconducting
the level sets of these functions in simple magnetometer. Science 175(4022), 664–666 (1972).
https://doi.org/10.1126/science.175.4022.664
cases and in a two-dimensional space. We 4. B. He, A. Sohrabpour, E. Brown, Z. Liu, Electro-
will examine how these simple functions can physiological source imaging: A noninvasive win-
be combined to form optimization problems, dow to brain dynamics. Annu. Rev. Biomed. Eng.
in later questions. 20(1), 171–196 (2018)
5. B. Pesaran, M. Vinck, G.T. Einevoll, A. Sirota, P.
(a) Can you plot x 2 + y 2 = 1 and Fries, M. Siegel, W. Truccolo, C.E. Schroder, R.
|x| + |y| = 1? Srinivasan, Investigating large-scale brain dynamics
(b) Can you plot and describe the set of lines using field potential recordings: analysis and inter-
described by y + 2x = K0 for K0 ∈ R? pretation. Nat. Neurosci. 21(7), 903 (2018). https://
doi.org/10.1038/s41593-018-0171-8
If K0 decreases, which direction will the 6. F.H. Lopes da Silva, EEG and MEG: Relevance
line move toward? What happens when to neuroscience. Neuron 80(5), 1112–1128 (2013).
K0 increases? https://doi.org/10.1016/j.neuron.2013.10.017
14. Assuming x, y ≥ 0, how would you describe 7. M.S. Hämäläinen, R. Hari, R.J. Ilmoniemi, J. Knuu-
tila, O.V. Lounasmaa, Magnetoencephalography—
the following optimization problem? theory, instrumentation, and applications to noninva-
(a) argmin (y + 2x) sive studies of the working human brain. Rev Mod.
x,y
Phys. 65(2), 413 (1993)
Subject to |x| + |y| = 1 x, y ≥ 0 8. J. Malmivuo, R. Plonsey, Bioelectromagnetism:
Hint. Basically, you want to minimize Principles and Applications of Bioelectric and Bio-
K0 (where y + 2x = K0 ) for nonnegative magnetic Fields (Oxford University Press, New
x, y with L1 -norm of 1. York, Oxford, 1995)
9. P.L. Nunez, R. Srinivasan, Electric Fields of the
(b) Can you graphically depict this optimiza- Brain: The Neurophysics of EEG (Oxford University
tion problem, by varying the values of Press, New York, 2006)
K0 ? 10. A. Sohrabpour, Y. Lu, P. Kankirawatana, J. Blount,
(c) Based on (b), can you propose a system- H. Kim, B. He, Effect of EEG electrode number
on epileptic source localization in pediatric patients.
atic way to solve this type of an opti- Clin. Neurophysiol. 126(3), 472–480 (2015)
mization problem? What are the optimal 11. M. Seeck, L. Koessler, T. Bast, F. Leijten, C. Michel,
values of x∗ , y∗ , and K0 ∗ in this problem? C. Baumgartner, B. He, S. Beniczky, The standard-
15. Repeat problem 6 for the following optimiza- ized EEG electrode array of the IFCN. Clin. Neuro-
physiol. 128, 2070 (2017)
tion problem: 12. G.A. Worrell, A.B. Gardner, S.M. Stead, S. Hu,
S. Goerss, G.J. Cascino, F.B. Meyer, R. Marsh, B.
argmin (y + 2x) Litt, High-frequency oscillations in human temporal
x,y lobe: Simultaneous microwire and clinical macro-
electrode recordings. Brain 131(4), 928–937 (2008)
13. G. Buzsáki, F. Lopes da Silva, High frequency
Subject to x 2 + y 2 = 1 x, y ≥ 0
oscillations in the intact brain. Prog. Neurobiol.
16. From problems 14 and 15, can you explain 98(3), 241–249 (2012). https://doi.org/10.1016/
why you would expect L1 -norm regulariza- j.pneurobio.2012.02.004
14. J.D. Jirsch, E. Urrestarazu, P. LeVan, A. Olivier, 26. S. Baillet, Magnetoencephalography for brain elec-
F. Dubeau, J. Gotman, High-frequency oscillations trophysiology and imaging. Nat. Neurosci. 20(3),
during human focal seizures. Brain 129(6), 1593– 327–339 (2017)
1608 (2006) 27. M. Seeber, L.-M. Cantonas, M. Hoevels, T. Sesia,
15. Y. Lu, G.A. Worrell, H.C. Zhang, L. Yang, B. V. Visser-Vandewalle, C.M. Michel, Subcortical
Brinkmann, C. Nelson, B. He, Noninvasive imaging electrophysiological activity is detectable with
of the high frequency brain activity in focal epilepsy high-density EEG source imaging. Nat. Commun.
patients. IEEE Trans. Biomed. Eng. 61(6), 1660– 10(1), 753 (2019). https://doi.org/10.1038/s41467-
1667 (2014) 019-08725-w
16. N. von Ellenrieder, L.P. Andrade-Valença, F. 28. F. Pizzo, N. Roehri, S.M. Villalon, A. Trebu-
Dubeau, J. Gotman, Automatic detection of fast chon, S. Chen, S. Lagarde, R. Carron, M. Gavaret,
oscillations (40–200Hz) in scalp EEG recordings. B. Giusiano, A. McGonigal, F. Bartolomei, J.M.
Clin. Neurophysiol. 123(4), 670–680 (2012). https:/ Badier, C.G. Benar, Deep brain activities can be
/doi.org/10.1016/j.clinph.2011.07.050 detected with magnetoencephalography. Nat. Com-
17. J. Jacobs, M. Zijlmans, R. Zelmann, C.E. Chatillon, mun. 10(1), 971 (2019). https://doi.org/10.1038/
J. Hall, A. Olivier, F. Dubeau, J. Gotman, High- s41467-019-08665-5
frequency electroencephalographic oscillations cor- 29. F. Perrin, O. Bertrand, J. Pernier, Scalp current
relate with outcome of epilepsy surgery. Ann. Neu- density mapping: Value and estimation from
rol. 67(2), 209–220 (2010) potential data. IEEE Trans. Biomed. Eng. BME-
18. S.V. Gliske, Z.T. Irwin, C. Chestek, G.L. Hegeman, 34(4), 283–288 (1987). https://doi.org/10.1109/
B. Brinkmann, O. Sagher, H.J.L. Garton, G.A. Wor- TBME.1987.326089
rell, W. Stacey, Variability in the location of high 30. B. He, R.J. Cohen, Body surface Laplacian ECG
frequency oscillations during prolonged intracranial mapping. IEEE Trans. Biomed. Eng. 39(11), 1179–
EEG recordings. Nat. Commun. 9(1), 2155 (2018). 1191 (1992). https://doi.org/10.1109/10.168684
https://doi.org/10.1038/s41467-018-04549-2 31. B. Hjorth, An on-line transformation of EEG
19. E. Boto, N. Holmes, J. Leggett, G. Roberts, V. Shah, scalp potentials into orthogonal source derivations.
S.S. Meyer, L.D. Munoz, K.J. Mullinger, T.M. Tier- Electroencephalogr. Clin. Neurophysiol. 39(5),
ney, S. Bestmann, G.R. Barnes, R. Bowtell, M.J. 526–530 (1975). https://doi.org/10.1016/0013-
Brookes, Moving magnetoencephalography towards 4694(75)90056-5
real-world applications with a wearable system. 32. F. Babiloni, C. Babiloni, F. Carducci, L. Fattorini,
Nature 555(7698), 657–661 (2018). https://doi.org/ P. Onorati, A. Urbano, Spline Laplacian estimate
10.1038/nature26147 of EEG potentials over a realistic magnetic
20. R. Hari et al., IFCN-endorsed practical guidelines resonance-constructed scalp surface model.
for clinical magnetoencephalography (MEG). Clin. Electroencephalogr. Clin. Neurophysiol. 98(4),
Neurophysiol. 129(8), 1720–1747 (2018). https:// 363–373 (1996). https://doi.org/10.1016/0013-
doi.org/10.1016/j.clinph.2018.03.042 4694(96)00284-2
21. A. Gevins, J. Le, N.K. Martin, P. Brickett, J. 33. W. Besio, T. Chen, Tripolar Laplacian electrocar-
Desmond, B. Reutter, High resolution EEG: 124- diogram and moment of activation isochronal map-
channel recording, spatial deblurring and MRI in- ping. Physiol. Meas. 28(5), 515–529 (2007). https:/
tegration methods. Electroencephalogr. Clin. Neu- /doi.org/10.1088/0967-3334/28/5/006
rophysiol. 90(5), 337–358 (1994). https://doi.org/ 34. J.V. Haxby, A.C. Connolly, J.S. Guntupalli, Decod-
10.1016/0013-4694(94)90050-7 ing neural representational spaces using multivariate
22. P. Zhang, K. Jamison, S. Engel, B. He, S. He, Binoc- pattern analysis. Annu. Rev. Neurosci. 37(1), 435–
ular rivalry requires visual attention. Neuron 71(2), 456 (2014). https://doi.org/10.1146/annurev-neuro-
362–369 (2011) 062012-170325
23. B.M. Savers, H.A. Beagley, W.R. Henshall, The 35. J. Linde-Domingo, M.S. Treder, C. Kerrén, M. Wim-
mechanism of auditory evoked EEG responses. Na- ber, Evidence that neural information flow is re-
ture 247(5441), 481 (1974). https://doi.org/10.1038/ versed between object perception and object recon-
247481a0 struction from memory. Nat. Commun. 10(1), 179
24. B. He, J. Lian, G. Li, High-resolution EEG: A new (2019). https://doi.org/10.1038/s41467-018-08080-
realistic geometry spline Laplacian estimation tech- 2
nique. Clin. Neurophysiol. 112(5), 845–852 (2001). 36. B. Gohel, S. Lim, M.-Y. Kim, H. Kwon, K. Kim,
https://doi.org/10.1016/S1388-2457(00)00546-0 Dynamic pattern decoding of source-reconstructed
25. A. Hillebrand, G.R. Barnes, A quantitative assess- MEG or EEG data: Perspective of multivariate pat-
ment of the sensitivity of whole-head MEG to ac- tern analysis and signal leakage. Comput. Biol. Med.
tivity in the adult human cortex. NeuroImage 16(3, 93, 106–116 (Feb. 2018). https://doi.org/10.1016/
Part A), 638–650 (2002). https://doi.org/10.1006/ j.compbiomed.2017.12.020
nimg.2002.1102 37. R. Plonsey, Bioelectric Phenomena (Wiley Online
Library, 1969). http://onlinelibrary.wiley.com/doi/
10.1002/047134608X.W1403/full
410 B. He et al.
38. B. He, T. Musha, Y. Okamoto, S. Homma, Y. Naka- 50. W.H. Lee, Z. Liu, B.A. Mueller, K. Lim, B. He,
jima, T. Sato, Electric dipole tracing in the brain Influence of white matter anisotropic conductivity
by means of the boundary element method and its on EEG source localization: Comparison to fMRI
accuracy. IEEE Trans. Biomed. Eng. 34(6), 406–414 in human primary visual cortex. Clin. Neurophysiol.
(1987) 120(12), 2071–2081 (2009)
39. M.S. Hämäläinen, Interpreting Measured Magnetic 51. R.M. Gulrajani, Bioelectricity and Biomagnetism
Fields of the Brain: Estimates of Current Distribu- (New York, Wiley, 1998)
tions (Helsinki University of Technology, Otaniemi, 52. A.M. Dale, A.K. Liu, B.R. Fischl, R.L. Buckner,
1984) J.W. Belliveau, J.D. Lewine, E. Halgren, Dynamic
40. A.M. Dale, M.I. Sereno, Improved localizadon of statistical parametric mapping: Combining fMRI
cortical activity by combining EEG and MEG and MEG for high-resolution imaging of cortical
with MRI cortical surface reconstruction: A lin- activity. Neuron 26(1), 55–67 (2000)
ear approach. J. Cogn. Neurosci. 5(2), 162–176 53. Z. Liu, B. He, fMRI–EEG integrated cortical source
(1993) imaging by use of time-variant spatial constraints.
41. R.D. Pascual-Marqui, C.M. Michel, D. Lehmann, NeuroImage 39(3), 1198–1214 (2008)
Low resolution electromagnetic tomography: A new 54. B. He, Z. Liu, Multimodal functional neuroimaging:
method for localizing electrical activity in the brain. Integrating functional MRI and EEG/MEG. IEEE
Int. J. Psychophysiol. 18(1), 49–65 (1994) Rev. Biomed. Eng. 1, 23–40 (2008). https://doi.org/
42. B. He, X. Zhang, J. Lian, H. Sasaki, D. Wu, V.L. 10.1109/RBME.2008.2008233
Towle, Boundary element method-based cortical po- 55. T. Musha, Y. Okamoto, Forward and inverse prob-
tential imaging of somatosensory evoked potentials lems of EEG dipole localization. Crit. Rev. Biomed.
using subjects’ magnetic resonance images. Neu- Eng. 27(3–5), 189–239 (1999)
roImage 16(3, Part A), 564–576 (2002). https:// 56. M. Scherg, D. Von Cramon, Two bilateral sources
doi.org/10.1006/nimg.2002.1127 of the late AEP as identified by a spatio-temporal
43. R.G. de Peralta Menendez, S.L.G. Andino, S. dipole model. Electroencephalogr. Clin. Neurophys-
Morand, C.M. Michel, T. Landis, Imaging the elec- iol./Evoked Potentials Sect. 62(1), 32–44 (1985)
trical activity of the brain: ELECTRA. Hum. Brain 57. H. Stefan, C. Hummel, G. Scheler, A. Genow, K. Dr-
Mapp. 9(1), 1–12 (2000) uschky, C. Tilz, M. Kaltenhauser, R. Hopfengartner,
44. S. Rush, D.A. Driscoll, EEG electrode sensitivity- M. Buchfelder, J. Romstock, Magnetic brain source
an application of reciprocity. IEEE Trans. Biomed. imaging of focal epileptic activity: A synopsis of
Eng. BME-16(1), 15–22 (1969). https://doi.org/ 455 cases. Brain 126(11), 2396–2405 (2003). https:/
10.1109/TBME.1969.4502598 /doi.org/10.1093/brain/awg239
45. M.S. Hämäläinen, J. Sarvas, Realistic conductivity 58. K. Kaiboriboon, S. Nagarajan, M. Mantle, H.E.
geometry model of the human head for interpretation Kirsch, Interictal MEG/MSI in intractable mesial
of neuromagnetic data. IEEE Trans. Biomed. Eng. temporal lobe epilepsy: Spike yield and characteri-
36(2), 165–171 (1989) zation. Clin. Neurophysiol. 121(3), 325–331 (2010).
46. J.C. Mosher, R.M. Leahy, P.S. Lewis, EEG and https://doi.org/10.1016/j.clinph.2009.12.001
MEG: Forward solutions for inverse methods. 59. D. Cohen, B.N. Cuffin, A method for combining
IEEE Trans. Biomed. Eng. 46(3), 245–259 MEG and EEG to determine the sources. Phys. Med.
(1999) Biol. 32(1), 85–89 (1987). https://doi.org/10.1088/
47. Y. Yan, P.L. Nunez, R.T. Hart, Finite-element model 0031-9155/32/1/013
of the human head: Scalp potentials due to dipole 60. M. Fuchs, M. Wagner, H.A. Wischmann, T. Kohler,
sources. Med. Biol. Eng. Comput. 29(5), 475–481 A. Theiben, R. Drenckhahn, H. Buchner, Improving
(1991) source reconstructions by combining bioelectric and
48. Y. Zhang, L. Ding, W. van Drongelen, K. Hecox, biomagnetic data. Electroencephalogr. Clin. Neu-
D.M. Frim, B. He, A cortical potential imaging rophysiol. 107(2), 93–111 (1998). https://doi.org/
study from simultaneous extra- and intracranial elec- 10.1016/S0013-4694(98)00046-7
trical recordings by means of the finite element 61. J.C. Mosher, P.S. Lewis, R.M. Leahy, Multiple
method. NeuroImage 31(4), 1513–1524 (2006). dipole modeling and localization from spatio-
https://doi.org/10.1016/j.neuroimage.2006.02.027 temporal MEG data. IEEE Trans. Biomed. Eng.
49. C.H. Wolters, A. Anwander, X. Tricoche, D. We- 39(6), 541–557 (1992)
instein, M.A. Koch, R.S. MacLeod, Influence of 62. B.D. Van Veen, W. Van Drongelen, M. Yuchtman,
tissue conductivity anisotropy on EEG/MEG field A. Suzuki, Localization of brain electrical activity
and return current computation in a realistic head via linearly constrained minimum variance spatial
model: A simulation and visualization study using filtering. IEEE Trans. Biomed. Eng. 44(9), 867–880
high-resolution finite element modeling. NeuroIm- (1997)
age 30(3), 813–826 (2006). https://doi.org/10.1016/ 63. F. Babiloni, C. Babiloni, F. Carducci, L. Fattorini, C.
j.neuroimage.2005.10.014 Anello, P. Onorati, A. Urbano, High resolution EEG:
A new model-dependent spatial deblurring method
using a realistically-shaped MR-constructed sub- 77. G.H. Golub, M. Heath, G. Wahba, Generalized
ject’s head model. Electroencephalogr. Clin. Neu- cross-validation as a method for choosing a
rophysiol. 102(2), 69–80 (1997). https://doi.org/ good ridge parameter. Technometrics 21(2),
10.1016/S0921-884X(96)96508-X 215–223 (1979). https://doi.org/10.1080/
64. Y. Lai, X. Zhang, W. van Drongelen, M. Kohrman, 00401706.1979.10489751
K. Hecox, Y. Ni, B. He, Noninvasive cortical imag- 78. S. Baillet, J.C. Mosher, R.M. Leahy, Electromag-
ing of epileptiform activities from interictal spikes netic brain mapping. IEEE Signal Process. Mag.
in pediatric patients. NeuroImage 54(1), 244–252 18(6), 14–30 (2001)
(2011) 79. J. Daunizeau, J. Mattout, D. Clonda, B. Goulard,
65. F. Babiloni, F. Cincotti, C. Babiloni, F. Carducci, H. Benali, J. Lina, Bayesian spatio-temporal ap-
D. Mattia, L. Astolfi, A. Basilisco, P.M. Rossini, proach for EEG source reconstruction: Concili-
L. Ding, Y. Ni, B. He, Estimation of the cortical ating ECD and distributed models. IEEE Trans.
functional connectivity with the multimodal inte- Biomed. Eng. 53(3), 503–516 (2006). https://
gration of high-resolution EEG and fMRI data by doi.org/10.1109/TBME.2005.869791
directed transfer function. NeuroImage 24(1), 118– 80. M. Zhu, W. Zhang, D.L. Dickens, L. Ding, Recon-
131 (2005) structing spatially extended brain sources via enforc-
66. C. Grova, J. Daunizeau, J.-M. Lina, C.G. Bénar, ing multiple transform sparseness. NeuroImage 86,
H. Benali, J. Gotman, Evaluation of EEG localiza- 280–293 (2014)
tion methods using realistic simulations of interictal 81. A. Sohrabpour, Y. Lu, G. Worrell, B. He, Imaging
spikes. NeuroImage 29(3), 734–753 (2006) brain source extent from EEG/MEG by means of
67. L. Ding, H. Yuan, Simultaneous EEG and MEG an iteratively reweighted edge sparsity minimization
source reconstruction in sparse electromagnetic (IRES) strategy. NeuroImage 142, 27–42 (2016)
source imaging. Hum. Brain Mapp. 34(4), 775–795 82. K. Sekihara, S.S. Nagarajan, Electromagnetic Brain
(2013). https://doi.org/10.1002/hbm.21473 Imaging: A Bayesian Perspective (Springer, Cham,
68. U. Mitzdorf, Current source-density method and 2015)
application in cat cerebral cortex: Investigation of 83. I.F. Gorodnitsky, J.S. George, B.D. Rao,
evoked potentials and EEG phenomena. Physiol. Neuromagnetic source imaging with FOCUSS:
Rev. 65(1), 37–100 (1985). https://doi.org/10.1152/ A recursive weighted minimum norm algorithm.
physrev.1985.65.1.37 Electroencephalogr. Clin. Neurophysiol. 95(4),
69. J.-Z. Wang, S.J. Williamson, L. Kaufman, Magnetic 231–251 (1995). https://doi.org/10.1016/0013-
source images determined by a lead-field analysis: 4694(95)00107-A
The unique minimum-norm least-squares estima- 84. R. Pascual-Marqui, LORETA (Low Resolution
tion. IEEE Trans. Biomed. Eng. 39(7), 665–675 Brain Electromagnetic Tomography): New authentic
(1992) 3D functional Images of the brain. ISBET Newslett.
70. E. Biglieri, K. Yao, Some properties of singular Issue 5, 4–8 (1994)
value decomposition and their applications to 85. L. Yang, C. Wilke, B. Brinkmann, G.A. Worrell, B.
digital signal processing. Signal Process. 18(3), He, Dynamic imaging of ictal oscillations using non-
277–289 (1989). https://doi.org/10.1016/0165- invasive high-resolution EEG. NeuroImage 56(4),
1684(89)90039-X 1908–1917 (2011)
71. Y. Shim, Z. Cho, SVD pseudoinversion image 86. Y. Zhang, W. van Drongelen, M. Kohrman, B. He,
reconstruction. IEEE Trans. Acoust. Speech Sig- Three-dimensional brain current source reconstruc-
nal Process. 29(4), 904–909 (1981). https://doi.org/ tion from intra-cranial ECoG recordings. NeuroIm-
10.1109/TASSP.1981.1163632 age 42(2), 683–695 (2008). https://doi.org/10.1016/
72. A.N. Tikhonov, V.Y. Arsenin, Solutions of Ill-Posed j.neuroimage.2008.04.263
Problems (Wiley, New York, 1977) 87. V. Caune, R. Ranta, S. Le Cam, J. Hofma-
73. V.A. Morozov, Methods for Solving Incorrectly nis, L. Maillard, L. Koessler, V. Louis-Dorr,
Posed Problems (Springer Science & Business Me- Evaluating dipolar source localization feasibility
dia, Berlin, 1984) from intracerebral SEEG recordings. NeuroIm-
74. P.C. Hansen, Analysis of discrete ill-posed problems age 98, 118–133 (2014). https://doi.org/10.1016/
by means of the L-curve. SIAM Rev. 34(4), 561–580 j.neuroimage.2014.04.058
(1992) 88. S.A.H. Hosseini, A. Sohrabpour, B. He, Electro-
75. P.C. Hansen, D.P. O’Leary, The use of the L-curve magnetic source imaging using simultaneous scalp
in the regularization of discrete ill-posed problems. EEG and intracranial EEG: An emerging tool for
SIAM J. Sci. Comput. 14(6), 1487–1503 (1993) interacting with pathological brain networks. Clin.
76. B. Jeffs, R. Leahy, M. Singh, An evaluation of meth- Neurophysiol. 129(1), 168–187 (2018)
ods for neuromagnetic image reconstruction. IEEE 89. K. Matsuura, Y. Okabe, A robust reconstruc-
Trans. Biomed. Eng. BME-34(9), 713–723 (1987). tion of sparse biomagnetic sources. IEEE Trans.
https://doi.org/10.1109/TBME.1987.325996 Biomed. Eng. 44(8), 720–726 (1997). https://
doi.org/10.1109/10.605428
412 B. He et al.
90. K. Matsuura, Y. Okabe, Selective minimum-norm 104. A. Gramfort, D. Strohmeier, J. Haueisen, M.S.
solution of the biomagnetic inverse problem. IEEE Hämäläinen, M. Kowalski, Time-frequency mixed-
Trans. Biomed. Eng. 42(6), 608–615 (1995) norm estimates: Sparse M/EEG imaging with non-
91. K. Uutela, M.S. Hämäläinen, E. Somersalo, Visual- stationary source activations. NeuroImage 70, 410–
ization of magnetoencephalographic data using min- 422 (2013)
imum current estimates. NeuroImage 10(2), 173– 105. B. He, L. Yang, C. Wilke, H. Yuan, Electrophysio-
180 (1999) logical imaging of brain activity and connectivity—
92. M.X. Huang, A.M. Dale, T. Song, E. Halgren, D.L. Challenges and opportunities. IEEE Trans. Biomed.
Harrington, I. Podogorny, J.M. Canive, S. Lewis, Eng. 58(7), 1918–1931 (2011)
R.R. Lee, Vector-based spatial–Temporal minimum 106. S. Ogawa, D.W. Tank, R. Menon, J.M. Ellermann,
L1-norm solution for MEG. NeuroImage 31(3), S.G. Kim, H. Merkle, K. Ugurbil, Intrinsic signal
1025–1037 (2006) changes accompanying sensory stimulation: Func-
93. L. Ding, B. He, Sparse source imaging in electroen- tional brain mapping with magnetic resonance imag-
cephalography with accurate field modeling. Hum. ing. PNAS 89(13), 5951–5955 (1992)
Brain Mapp. 29(9), 1053–1067 (2008) 107. K.K. Kwong, J.W. Belliveau, D.A. Chesler, I.E.
94. M. Fuchs, M. Wagner, T. Köhler, H.-A. Wischmann, Goldberg, R.M. Weisskoff, B.P. Poncelet, D.N.
Linear and nonlinear current density reconstruc- Kennedy, B.E. Hoppel, M.S. Cohen, R. Turner, Dy-
tions. J. Clin. Neurophysiol. 16(3), 267–295 (1999) namic magnetic resonance imaging of human brain
95. D. Wipf, S. Nagarajan, Iterative reweighted 1 and activity during primary sensory stimulation. PNAS
2 methods for finding sparse solutions. IEEE J. Sel. 89(12), 5675–5679 (1992). https://doi.org/10.1073/
Topics Signal Process. 4(2), 317–329 (2010) pnas.89.12.5675
96. K. Liao, M. Zhu, L. Ding, S. Valette, W. Zhang, 108. P.A. Bandettini, E.C. Wong, R.S. Hinks, R.S.
D. Dickens, Sparse imaging of cortical electrical Tikofsky, J.S. Hyde, Time course EPI of human
current densities via wavelet transforms. Phys. Med. brain function during task activation. Magn. Re-
Biol. 57(21), 6881 (2012). https://doi.org/10.1088/ son. Med. 25(2), 390–397 (1992). https://doi.org/
0031-9155/57/21/6881 10.1002/mrm.1910250220
97. K. Liao, M. Zhu, L. Ding, A new wavelet trans- 109. S. Ogawa, T.M. Lee, A.R. Kay, D.W. Tank, Brain
form to sparsely represent cortical current densities magnetic resonance imaging with contrast depen-
for EEG/MEG inverse problems. Comput. Meth- dent on blood oxygenation. PNAS 87(24), 9868–
ods Prog. Biomed. 111(2), 376–388 (2013). https:/ 9872 (1990)
/doi.org/10.1016/j.cmpb.2013.04.015 110. W. Ou, A. Nummenmaa, J. Ahveninen, J.W. Bel-
98. L. Ding, Reconstructing cortical current density by liveau, M.S. Hämäläinen, P. Golland, Multimodal
exploring sparseness in the transform domain. Phys. functional imaging using fMRI-informed regional
Med. Biol. 54(9), 2683 (2009) EEG/MEG source estimation. NeuroImage 52(1),
99. L. Ding, Y. Ni, J. Sweeney, B. He, Sparse cor- 97–108 (2010)
tical current density imaging in motor potentials 111. J. Gotman, C. Grova, A. Bagshaw, E. Kobayashi,
induced by finger movement. J. Neural Eng. 8(3), Y. Aghakhani, F. Dubeau, Generalized epileptic
036008 (2011). https://doi.org/10.1088/1741-2560/ discharges show thalamocortical activation and
8/3/036008 suspension of the default state of the brain. PNAS
100. M. Zhu, W. Zhang, D.L. Dickens, J.A. King, 102(42), 15236–15240 (2005). https://doi.org/
L. Ding, Sparse MEG source imaging for 10.1073/pnas.0504935102
reconstructing dynamic sources of interictal 112. A. Delorme, S. Makeig, EEGLAB: An open source
spikes in partial epilepsy. J. Clin. Neurophysiol. toolbox for analysis of single-trial EEG dynamics
30(4), 313–328 (2013). https://doi.org/10.1097/ including independent component analysis. J. Neu-
WNP.0b013e31829dda27 rosci. Methods 134(1), 9–21 (2004)
101. R.N. Henson, Y. Goshen-Gottstein, T. Ganel, L.J. 113. D. Brunet, M.M. Murray, C.M. Michel, Spatiotem-
Otten, A. Quayle, M.D. Rugg, Electrophysiological poral analysis of multichannel EEG: CARTOOL.
and haemodynamic correlates of face perception, Comput. Intell. Neurosci. 2011, 2 (2011)
recognition and priming. Cereb. Cortex 13(7), 793– 114. B. Fischl, FreeSurfer. NeuroImage 62(2), 774–781
805 (2003). https://doi.org/10.1093/cercor/13.7.793 (2012)
102. D. Yao, B. He, A self-coherence enhancement al- 115. D.W. Shattuck, R.M. Leahy, BrainSuite: An auto-
gorithm and its application to enhancing three- mated cortical surface identification tool. Med. Im-
dimensional source estimation from EEGs. Ann. age Anal. 6(2), 129–142 (2002)
Biomed. Eng. 29(11), 1019–1027 (2001) 116. D. Rivière, D. Geffroy, I. Denghien, N. Souedet,
103. K. Liu, Z.L. Yu, W. Wu, Z. Gu, Y. Li, S. Nagarajan, Y. Cointepas, BrainVISA: An extensible software
Variation sparse source imaging based on condi- environment for sharing multimodal neuroimaging
tional mean for electromagnetic extended sources. data and processing tools. NeuroImage 47, S163
Neurocomputing 313, 96 (2018). https://doi.org/ (2009)
10.1016/j.neucom.2018.06.004
117. A. Gramfort, T. Papadopoulo, E. Olivi, M. Clerc, 121. S.S. Dalal, J.M. Zumer, V. Agrawal, K.E. Hild, K.
OpenMEEG: opensource software for quasistatic Sekihara, S.S. Nagarajan, NUTMEG: A neuromag-
bioelectromagnetics. Biomed. Eng. Online 9(1), 45 netic source reconstruction toolbox. Neurol. Clin.
(2010) Neurophysiol. 2004, 52 (2004)
118. B. He, Y. Dai, L. Astolfi, F. Babiloni, H. Yuan, L. 122. F. Tadel, S. Baillet, J.C. Mosher, D. Pantazis,
Yang, eConnectome: A MATLAB toolbox for map- R.M. Leahy, Brainstorm: A user-friendly application
ping and imaging of brain functional connectivity. J. for MEG/EEG analysis. Comput. Intell. Neurosci.
Neurosci. Methods 195(2), 261–269 (2011) 2011, 8 (2011)
119. R. Oostenveld, P. Fries, E. Maris, J.-M. Schoffelen, 123. P.J. Magistretti, L. Pellerin, D.L. Rothman,
FieldTrip: Open source software for advanced analy- R.G. Shulman, Energy on demand. Science
sis of MEG, EEG, and invasive electrophysiological 283(5401), 496–497 (1999). https://doi.org/
data. Comput. Intell. Neurosci. 2011, 1 (2011) 10.1126/science.283.5401.496
120. A. Gramfort, M. Luessi, E. Larson, D.A. Engemann, 124. P.L. Nunez, R.B. Silberstein, On the relationship
D. Strohmeier, C. Brodbeck, L. Parkkonen, M.S. of synaptic activity to macroscopic measurements:
Hamalainen, MNE software for processing MEG Does co-registration of EEG with fMRI make sense?
and EEG data. NeuroImage 86, 446–460 (2014) Brain Topogr. 13(2), 79–96 (2000)
Exploring Functional and Causal
Connectivity in the Brain 14
Mingzhou Ding and Bin He
Abstract
14.1 Introduction
Functional and causal connectivity is widely
used in basic and clinical neuroscience. This Normal operations of the brain are achieved
chapter reviews the basic principles of func- through cooperative neural computation.
tional and causal connectivity analysis and il- Multielectrode recording and functional imaging
lustrates the concepts using numerical and ex- are the key technologies that afford us the op-
perimental examples. The theory, implemen- portunity to study neuronal mechanisms of brain
tation, and application of Granger causality functioning and its breakdown in disease from
are reviewed. Also covered is functional and a network perspective. Analytically, the main
causal connectivity imaging from electrophys- statistics for assessing functional connectivity
iological recordings such as electrocorticogra- between different neurons and between different
phy, electroencephalography, and magnetoen- brain areas are cross correlations and ordinary
cephalography. coherence spectra. More recent advances
have begun to emphasize effective (causal)
Keywords connectivity measures that yield information on
Stochastic process · Functional connectivity · the direction of neural signal transmission [1–
Causal connectivity · Granger causality · 7]. One of the most commonly applied causal
ECoG · EEG · MEG connectivity measures is Granger causality [8, 9].
It has been shown that the directional influence
derived from Granger causality can be used to
indicate the direction of neural transmission and
information flow between cortical areas [2, 3].
Because Granger causality is a method based on
stochastic processes, we begin in this chapter
of this chapter (https://doi.org/10.1007/978-3-030-43395- by reviewing the essential ideas of stochastic
6_14) contains supplementary material, which is available processes and then proceed to define methods
to authorized users. for functional and causal connectivity analysis.
M. Ding
We illustrate the various methods by applying
The J. Crayton Pruitt Family Department of Biomedical them to multimodal neuronal data recorded
Engineering, University of Florida, Gainesville, FL, USA from both animal preparations and humans
e-mail: [email protected] under normal and pathological conditions.
B. He () Finally, we introduce the readers to open-
Department of Biomedical Engineering, Carnegie Mellon source software packages for causal connectivity
analysis. These packages have been applied

416 M. Ding and B. He
to electrophysiological measurements such as A class of stochastic processes that have found

local field potentials (LFP), electrocorticogram wide applications in real-world problems is called
(ECoG), electroencephalogram (EEG), and stationary stochastic processes or simply station-
magnetoencephalogram (MEG), as well as ary processes. For a stationary process, the mean
functional imaging data. and variance are both constant, i.e., μ(t) = μ,
σ 2 (t) = σ 2 , and the autocovariance function has
the form Rxx (t1 , t2 ) = Rxx (τ ), where τ = t2 − t1
14.2 Basics of Functional is referred to as the time lag or lag. From these
and Causal Connectivity definitions, it is clear that σ 2 = Rxx (0). Because
Analysis the autocovariance function depends on the mag-
nitude of the time series, making the comparison
14.2.1 Stochastic Processes between different experimental conditions diffi-
and Their Characterization cult, we typically normalize the autocovariance
function by Rxx (0), to yield the autocorrelation
In many neurophysiological and neuroimaging function, ρ xx (τ ) = Rxx (τ )/Rxx (0). The autocor-
experiments, the data are collected in the form relation function measures the degree of linear
of time series. Some examples are shown in Fig. dependence between two variables separated by
14.1. These time series often have a random ap- the time lag τ . For stochastic processes encoun-
pearance and can be described mathematically as tered in applications, such as those in Fig. 14.1,
realizations of stochastic processes. A stochas- ρ xx (τ ) → 0 as τ → ∞, meaning that when the
tic process, denoted X(t), is a family of random two random variables are sufficiently separated in
variables indexed by time t. Mathematically, X(t) time, they are no longer correlated.
can be characterized by three quantities: mean, The autocorrelation function can provide
variance, and autocovariance function, defined as many insights into a stochastic process. For
μ(t) = E(X(t)), σ 2 (t) = E((X(t) − μ(t))2 ), and example, by measuring how fast ρ xx (τ ) decays to
Rxx (t1 , t2 ) = E((X(t1 ) − μ(t1 ))(X(t2 ) − μ(t2 ))), zero, we get the correlation time, which gauges
respectively. the memory effects in the times series. If ρ xx (τ )
Here E stands for mathematical expectation. approaches zero like a damped sinusoid, we
When analyzing actual data, E can be replaced by can infer that the system contains stochastic
sample averaging. oscillatory activity. As such, it is not surprising
Fig. 14.1 Example of neural time series. (a) Local field potential data from the brain of a macaque monkey. (b) EEG
data from the occipital cortex of a human subject
14 Exploring Functional and Causal Connectivity in the Brain 417
that this method has been used extensively to This is an essential function in quantifying the
study neuronal oscillation phenomena in local functional connectivity between two time series
field potential (LFP), ECoG, and EEG/MEG in the time domain. In particular, the zero-lag
recordings [10]. cross-correlation, ρ xy (0), is widely used to char-
A more systematic way to study the frequency acterize intrinsically connected brain networks in
content of a time series is through spectral analy- resting-state functional MRI data [14].
sis. The power spectrum of a stationary stochastic Functional connectivity analysis can also
process is defined as be done in the frequency domain. The key
quantity is the cross-spectrum Γxy (f ) =
∞

∞
Γxx (f ) = Rxx (0) + 2 Rxx (τ ) cos (2πf τ ) . Rxx (τ ) e−i2πf τ . Normalized cross-spectrum
τ =−∞
τ =1
is the spectral coherence function:
A power spectrum tells us how energy is dis-
Cxy (f ) =| Γxy (f ) | / Γxx (f )Γxy (f )
tributed in different frequency bands. This is very
useful in neuroscience because ensembles of neu-
which sometimes is also called the ordinary
rons are known to produce oscillatory activity
coherence function or simply coherence function.
in a variety of frequency bands, including alpha
There is a similar periodogram approach to
(8–12 Hz), beta (15–30 Hz), and gamma (30–
compute this function. Let X(f ) and Y(f ) be
80 Hz) bands [10]. Oscillations in the brain play
the Fourier transform of X(t) and Y(t). Let
an instrumental role in mediating many cognitive
Γˆxy (f ) = E (X(f )Y ∗ (f )) /N where ∗ is
and autonomic functions, and abnormal oscilla-
complex conjugate. Normalizing the cross-
tions are found to be associated with various brain
spectrum by the power spectra, we obtain
disorders like schizophrenia and epilepsy [11,
12].
A more intuitive way to estimate power spectra Ĉxy (f ) =| Γˆxy (f ) | / Γˆxx (f )Γˆyy (f ).
is the periodogram method. Let’s use X(t) to
denote a discrete-time stochastic process of finite According to the Wiener-Khinchin theorem,
duration where t = 1, 2, . . . ,N. Let X(f ) be the dis- lim Ĉxy (f ) = Cxy (f ). The coherence function
N →∞
crete Fourier transform of X(t). The periodogram
is an essential function in functional connectivity
is defined as Γˆxx (f ) = E |X(f )|2 /N. The analysis in the frequency domain. It has been
expectation operation E means averaging over widely used to study oscillatory neuronal net-
multiple realizations in actual data analysis. What works [2, 3, 15].
is the relation between Γˆxx (f ) and xx (f )? It can
be shown that lim Γˆxx (f ) = Γxx (f ). This result
N →∞
is sometimes referred to as the Wiener-Khinchin 14.2.2 Granger Causality
theorem [13].
For two neuronal time series X(t) and Y(t), the Interactions between different neurons and dif-
cross-covariance function is defined as ferent brain areas are mediated by the transmis-
sion of action potentials and are inherently di-

Rxy (τ ) = E (X (t1 ) − μx ) Y (t2 ) − μy rectional. Functional connectivity measures such
as cross-correlation function and ordinary co-
where τ = t2 − t1 . Normalizing the cross- herence may yield directional information only
covariance function gives us the cross-correlation under very ideal conditions. Recent work has
function, that is, shown that Granger causality is a robust method
to furnish the direction of neuronal interactions
ρxy (τ ) = Rxy (τ ) / Rxx (0)Ryy (0). [1–4, 16 –18, 46]. At the heart of this causal
connectivity method is the idea of time series ∞

prediction [9]. Y (t) = c2 (j )X (t − j )
Consider two simultaneously recorded station- j =1
(14.4)
ary time series. According to Wiener [19], if ∞

the prediction of one time series is improved by + d2 (j )Y (t − j ) + η2 (t)
incorporating the knowledge of a second one, j =1
then the second series is said to have a causal

influence on the first. Wiener’s proposal lacks the where ε2 (t) and η2 (t) are uncorrelated over time,
machinery for practical implementation. Granger and their contemporaneous covariance matrix is
[9] later formalized the prediction idea in the
Σ 2 γ2
context of linear regression models. Specifically, Σ= . (14.5)
if the variance of the autoregressive prediction γ2 Γ2
error of the first time series at the present time is
Here, 2 = var (ε2 (t)), 2 = var (η2 (t)),
reduced by inclusion of past measurements from
and ϒ 2 = cov (ε2 (t), η2 (t)). Intuitively, the linear
the second time series, then the second time series
combination on the right-hand side of (14.1),
is said to have a causal influence on the first one. ∞
Reversing the role of the two time series gives a1 (j )X (t − j ), can be thought of as a pre-
j =1
the causal influence in the opposite direction. The
diction of X(t) using past measurements of X,
interaction discovered this way could be either
namely, X(t − 1), X(t − 2), X(t − 3), . . . etc.
reciprocal or unidirectional.
The noise term is then the prediction error whose
The mathematical formulation of Granger
variance, var(ε1 (t)) = 1 , gauges the quality of
causality is based on the autoregressive
the linear predictor. Similarly, in (14.3), the term
representation of time series [5, 8]. Let two ∞
∞
stationary time series be denoted by X(t) and a2 (j )X (t − j ) + b2 (j )Y (t − j ) can be
j =1 j =1
Y(t). Individually, under general conditions, X(t) thought of as a prediction of X(t) by combin-
and Y(t) can each be written as the following ing the past measurements of both X and Y. In
autoregressive models: this case, the variance of the prediction error is
∞ 2 . According to the idea explained above, the

X(t) = a1 (j )X (t − j ) Granger causality from Y to X can be defined as
j =1 (14.1)
Σ1
+ ε1 (t), var (ε1 (t)) = Σ1 FY →X = ln . (14.6)
Σ2
If the prediction of X(t) is improved by in-

∞
cluding the past measurements of Y, we have
Y (t) = d1 (j )Y (t − j ) 2 < 1 , which means FY → X > 0, and there is
j =1 (14.2) causal influence from Y to X. On the other hand, if
+ η1 (t), var (η1 (t)) = Γ1 . there is no improvement in prediction, 2 = 1 ,
FY → X = 0, and there is no causal influence from
Jointly, they are represented as the following Y to X. Using the same reasoning, we can define
bivariate autoregressive model: the Granger causality from X to Y as
∞
Γ1
FX→Y = ln . (14.7)
X(t) = a2 (j )X (t − j ) Γ2
j =1
(14.3) The interaction between two neuronal
∞

+ b2 (j )Y (t − j ) + ε2 (t) ensembles can be unidirectional or can be
j =1 reciprocal. Both types of interaction patterns have
been found in experimental recordings. Examples Y Z Y Z

in the subsequent sections will illustrate both
possibilities.
The derivation of frequency domain Granger
causality is quite involved although the basic
X X
idea is straightforward to appreciate. Geweke
[8] showed that the spectral power at a given (a) (b)
frequency f can be written as the sum of two
terms: power(f ) = intrinsic power(f ) + causal Fig. 14.2 Two different connectivity patterns between
power(f ). The intrinsic power can be thought three recording channels. A bivariate Granger causality
of as the power generated locally near the analysis cannot distinguish the two patterns in (a) and (b),
but a conditional Granger causality analysis can
recording site, whereas the causal power term
is the power brought about by the input from
the other time series. The spectral Granger All these methods have been used in neural data
causality is then conceptually defined to be analysis.
W(f ) = ln(power(f )/intrinsic power(f )). From
this definition, it is clear that when the causal
power is zero, namely, there is no causal input 14.3 Numerical and Experimental
from the other time series, W(f ) = 0. With causal Examples
power greater than zero, W(f ) > 0, signaling
nonzero causal influence from the other time Below we present numerical and experimental
series at frequency f. A crucial result proven examples to illustrate the application of some of
by Geweke [8] is that the spectral measure the methods discussed above. Coupled autore-
defined this way can be related to its time-domain gressive models of varying network complexity
counterpart through the following integration: are used to generate the simulation data.

1/2
FY →X = −1/2 WY →X (f )df, Example 1 A two variable model is

1/2
FX→Y = −1/2 WX→Y (f )df.
x(t) = ε(t)
The conditions necessary for these equalities y(t) = 0.5y (t − 1) + x (t − 1) + η(t)
to hold are expected to be met in practical appli-
cations. where ε(t), η(t) are independent Gaussian white
Before we conclude this section, we make two noise processes with mean of 0 and variance of
remarks. First, for three or more time series, pat- 1 and 0.09, respectively. From the model, we
terns of connectivity can become more intricate. see that there is causal influence from X to Y
To illustrate, consider three time series X, Y, and but not from Y to X. Assume a sampling rate of
Z. A pairwise analysis will not be able to resolve 200 Hz. The coherence can be derived analyt-
the two patterns shown in Fig. 14.2. In this case, ically to be 0.92, which is independent of fre-
the concept of conditional Granger causality will quency. For Granger causality, one can show that
become necessary [20, 21]. Second, the Granger FY → X = 0 and FX → Y = 2.49 which corresponds
causality spectra defined above is but one ap- to the pattern of connectivity in the model. We
proach to the frequency representation of Granger simulated the model to generate a dataset of 500
causality. There are other spectral representations realizations each consisting of 100 time points.
of Granger causality, including directed trans- An AR model is fit to the data and coherence and
fer function [22, 46], partial Granger causality the Granger causality spectra are derived from
spectra [23], and partial directed coherence [29]. the model shown in Fig. 14.3. The agreement
(a) (b)
1 3
0.8 2.5
Granger causality
XoY
2
coherence
0.6 YoX
1.5
0.4 1
0.2 0.5
0 0
0 20 40 60 80 100 0 20 40 60 80 100
frequency (Hz) frequency (Hz)
Fig. 14.3 Simulation results of a two-variable model. (a) Coherence and (b) Granger causality
between the numerical results and the theoretical A dataset of 500 realizations each with 50 time
results is excellent. points was generated. The sampling rate is taken
to be 200 Hz. Assuming no knowledge of the
Example 2 Consider a 5-node oscillatory net- model equations, Eq. (14.8), we fitted a 5th order
work. The network configuration is shown in Fig. MVAR model to the simulated dataset and calcu-
14.4a. The mathematical equations are: lated power, coherence, and conditional Granger
causality spectra from it. The power spectra are
x1 (t) = 0.55x1 (t − 1) − 0.7x1 (t − 2) + ε1 (t) given in the panels along the diagonal direction
x2 (t) = 0.56x2 (t − 1) − 0.75x2 (t − 2) in Fig. 14.4b. All five oscillators have a spectral
+ 0.6x1 (t − 1) + ε2 (t) peak at around 40 Hz. The conditional Granger
x3 (t) = 0.57x3 (t − 1) − 0.8x3 (t − 2) causality spectra are shown in the off-diagonal
+ 0.4x1 (t − 2) + ε3 (t) , panels of Fig. 14.4b. Only the first column has
x4 (t) = 0.58x4 (t − 1) − 0.85x4 (t − 2) nonzero conditional Granger causality values, re-
+ 0.5x1 (t − 3) + ε4 (t) flecting the driving influence emanating from
x5 (t) = 0.59x5 (t − 1) − 0.9x5 (t − 2) node 1. The conditional Granger causality among
+ 0.8x1 (t − 4) + ε5 (t) other pairs of nodes is uniformly zero. This cor-
(14.8) responds precisely to the structural connectivity
pattern in Fig. 14.4a. One noteworthy feature
where ε1 (t), ε2 (t), ε3 (t), ε4 (t), ε5 (t) are in- about Fig. 14.4b is the consistency of spectral
dependent Gaussian white noise processes with features (i.e., peak frequency) across both power
zero means and variances σ12 = 1.0, σ22 = and Granger causality spectra. This is important
2.0, σ32 = 0.8, σ42 = 1.0, σ52 = 1.5, respectively. since it allows us to link local dynamics with that
The intrinsic dynamics of each node is chosen in of the global network.
such a way that it exhibits a prominent spectral
peak. From construction, the signal from the first Example 3 Laminar organization of cortical al-
node (the source) is propagated to the other four pha rhythm. Hans Berger is the first to coin the
nodes with differential time delays. A pairwise term alpha rhythm or alpha oscillations to de-
analysis will reveal nonzero Granger causality scribe the 8–12 Hz oscillations observed over
from the nodes that receive an early input from human occipital-parietal cortex [24]. More than
the source node to the nodes that receive a late 90 years since this initial discovery, the phys-
input (e.g., node 3 ➔ node 4). Clearly, this does iological mechanisms of alpha rhythm and its
not depict the true connectivity of this dynami- function remain a topic of intense investigations
cal network. A conditional Granger causality can [25]. Prior to the 1970s, the thalamus was thought
help to resolve this problem [20, 5]. to be the pacemaker of cortical alpha [26]. More
Fig. 14.4 Simulation (a)

results of a five-variable
mode. (a) Schematic 1
illustration of the network
topology. (b) Power
(diagonal) and conditional
Granger causality
(off-diagonal) results from
2 3 4 5
a multivariate parametric
analysis. Causal influence
is from the horizontal axis
onto the vertical axis (b)
0.2 1 1 1 1
0 0 0 0 0
1 2 1 1 1
0 0 0 0 0
1 1 1 1 1
0 0 0 0 0
1 1 1 4 1
0 0 0 0 0
1 1 1 1 20
0 0 0 0 0
0 100 0 100 0 100 0 100 0 100
recent studies using lesion techniques have tested ning all 6 cortical layers in visual area V4. The
the role of infragranular layer pyramidal cells in intercontact spacing was 200 μm. To examine
alpha pacemaking in cortical slice preparations the laminar organization of alpha oscillations, we
[27]. Here we demonstrate that Granger causality followed a two-step analysis protocol [2, 28].
can be used in lieu of the lesion technique to First, laminar generators of LFP oscillations at
identify the cortical pacemakers of alpha activity the alpha frequency are identified by calculating
in behaving monkeys [2, 3]. Because the lesion the current source density (CSD) using the phase
techniques commonly used in in vitro prepara- realigned averaging technique (PRAT). Second,
tions are not available in in vivo experiments, a the patterns of interaction between different lam-
computational method that can accomplish the inar alpha generators are identified using Granger
same goal will represent a major advance. causality. Figure 14.5a, b displays the schematic
A macaque monkey was trained to perform of the linear multielectrode and 200 ms unfiltered
an auditory discrimination task so that in the single-sweep LFPs. Oscillations around 10 Hz
visual cortex we can examine spontaneous neural are apparent. Current source density analysis re-
activity under verifiably alert conditions. Local veals alpha current generators in granular (G),
field potential (LFP) was sampled (2 kHz) with infragranular (IG), and supragranular (SG) layers.
a linear array electrode with 14 contacts span- Applying Granger causality to these alpha current
Fig. 14.5 Laminar organization of alpha oscillations. (a) Granger causality results. SG supragranular, G granular,
Schematic of a multicontact electrode capable of sampling IG infragranular. (Figure adapted from Ding et al. [28]
neuronal activity from all six layers. (b) Local field poten- with permission)
tial data demonstrating alpha oscillations (10 Hz). (c, d)
generators, we found that IG➔SG and IG➔G

causal influences in the alpha band are large, 14.4 Brain Causal Mapping
whereas the SG➔IG and G➔IG causal influences from Electrophysiological
are close to zero (Fig. 14.5c, d). This finding is Measurements in Humans
consistent with the in vitro result mentioned ear-
lier demonstrating that alpha frequency pacemak- 14.4.1 Analysis of Directed Cortical
ers are located in infragranular layers. In a sense Interactions
this study can be seen as providing a validation of
There are a variety of connectivity estimators
Granger causality as a method to infer direction
based upon the principle of Granger causality [9].
of synaptic transmission in neuronal circuits.
A shortcoming of bivariate causality, however, is
that the estimation of the connectivity is limited defined as the transfer matrix of the system. From
to pairwise systems and may incorrectly estimate the transfer matrix, the DTF measure, γ2 ij (f ),
the causal interactions in a multivariate setting. which describes the directional connectivity from
Several techniques have been developed to pro- ROI j to ROI i, is defined by the elements of the
vide estimates of connectivity in multivariate sys- transfer matrix in the spectrum domain [1, 46]:
tems. Many of these techniques, such as the di-
rected transfer function (DTF) [46] and partial di-
Hij (f )2
rected coherence (PDC) [29], are based upon the γij2 (f ) = N . (14.12)

spectral characteristics of the physiologic signals |Him (f )| 2
and are able to differentiate causal interactions m=1

within specific frequency bands of interest.
The DTF is a frequency-domain estimator
of causal interaction based on the multivariate 14.4.2 Connectivity Analysis
autoregressive (MVAR) modeling [46]. Let from Electrocorticogram
Y = [y1 (t), y2 (t), . . . , yN (t)]T be a set of
electrophysiological measurements or estimates The DTF has been applied to ECoG recordings
at N selected regions of interest (ROIs), where t from epilepsy patients to identify zones [17, 18,
refers to time. The following MVAR process will 30, 31, 47]. The connectivity analysis using DTF
be an adequate description of the dataset Y: has been shown useful in elucidating seizure on-
set zones in patients with temporal lobe epilepsy

p originating in either the mesial or lateral cortical
Λ(k)Y (t − k) = E(t), with Λ(0) = I structures as well as in patients with neocortical
k=0 onset extratemporal lobe epilepsy. In such ap-
(14.9) proach, the seizure foci were identified by calcu-
lating the DTF-calculated causal activity, which
where E(t) is a vector of a multivariate zero-mean originates at each ECoG electrode. The direc-
uncorrelated white noise process, (1), (2), tional functional connectivity was further inves-
. . . , (p) are the N × N matrices of model tigated using a within frequency framework as
coefficients, and the model order p can be chosen discussed above and using a cross-frequency di-
with the Akaike information criteria (AIC) [44] rectionality analysis [30], revealing the push-pull
for a MVAR process. In order to investigate the interplay between seizure-onset zone and the sur-
spectral properties of the examined process, the rounding regions.
above equation can be transformed to the fre- Treating each ECoG signals as a time series,
quency domain. the DTF connectivity values can be estimated
using the algorithms as discussed in Sect. 14.4.1.
Λ(f )Y(f ) = E(f ), After this procedure, the question arises as to

p
(14.10) whether the calculated values constitute a sig-
where Λ(f ) = Λk e−j 2πf Δtk nificant causal interaction. Since the DTF has a
k=0 highly nonlinear relationship to the time series
from which it is derived, the distribution of the
This equation can be rewritten as
estimator under the null hypothesis of no con-
nectivity is not well-established. Therefore, con-
Y (f ) = Λ−1 (f )E(f ) = H (f )E(f ) (14.11)
ventional parametric statistical analysis cannot be
used. To overcome this problem, a nonparametric
where H(f ) is the inverse of the frequency-
statistical test using surrogate data [4, 7] canbe
transformed coefficient matrix, #(f ), and is

performed to determine the significance of the θkj2
calculated DTF values. k∈n =j
Φj2 = . (14.14)
From the DTF calculation, the causal relation- n−1
ships among the ECoG channels in the selected
ictal frequency bands can be identified. Once Following calculation of the causal links, the
the causal interactions from the DTF calculation sum can be obtained of the DTF-calculated activ-
for the analyzed epoch are obtained, statistical ity which arises from each channel. The resulting
significance testing can be performed in order value can be interpreted as the degree to which
to remove the links, which may form spurious each electrode acts as a generator of the observed
interactions between ECoG channels. The sur- ictal activity. This value, which is termed the
rogate data method can be applied to each an- causal source activity, is usually normalized such
alyzed epoch in which the temporal correlation that the electrode(s) with the maximum activity
between the ECoG channels is destroyed [4, 7]. in each analyzed seizure had unit strength. A
The DTF method can be applied to the surrogate diagram outlining the methods for application
datasets, and a distribution of DTF values can be of DTF causal connectivity analysis to epilepsy
obtained which correspond to the null hypothesis ECoG data is shown in Fig. 14.6 [18].
of no causal interactions. From this distribution, From the thresholded DTF results, Fig. 14.7
a threshold is normally set (e.g., p = 0.01), and [18] shows an example of epilepsy source iden-
links in which the strength of the causal interac- tification, where two regions of source activity
tion does not exceed this threshold are discarded can be observed in a patient (Fig. 14.7). From
from further analysis. this figure, a good correlation can be observed
In situations where the frequency-derived in- between the spatial locations of the causal source
formation is confined to a relatively narrow band- activity and the SOZ identified by the epileptol-
width, the DTF values can be integrated over a ogist. A right temporal lobectomy and resection
specific frequency band of interest in order to of the frontal focus were performed. Following
provide a better visualization of the connectivity surgery, the patient experienced a roughly 70%
pattern [47]. This is denoted as the integrated reduction in seizure frequency.
DTF (IDTF) and is given by In a study of 11 selected patients [18], the
sources estimated from the DTF method were

f2 found to be in agreement with the seizure foci
γij2 (k) identified by the epileptologists. The use of such
k=f1
θij2 = , (14.13) causal analysis tools could provide greater insight
f2 − f1 into the sources of the epileptogenic networks
which give rise to the ictal activity and pave the
where f1 and f2 correspond to the lower and upper
way for better and more focused treatment of
indices, respectively, of the frequency band of
patients with intractable epilepsy.
interest.
If the total amount of information entering or
leaving a node is desired, the afferent and efferent
14.4.3 Connectivity Analysis
IDTF values can be summed accordingly. This
from E/MEG Source Imaging
technique has been previously demonstrated in
identifying frequency-dependent sources (and
Recently, a noninvasive electrophysiological con-
sinks) of cortical activity [1]. Here, the total
nectivity analysis approach has been developed
amount of information leaving a channel is
based on the unique feature of EEG (and MEG)
calculated by summing the IDTF values for each
source imaging in conjunction with causal anal-
jth input channel over all i output channels. This
ysis such as DTF method [1, 4, 6, 7, 32–35].
outflow value can also be normalized by dividing
Through the utilization of EEG/MEG-based non-
the sum by the number of output nodes.
invasive source imaging techniques, it is pos-
Fig. 14.6 A diagram outlining the causal connectivity amount of information leaving each electrode (strength of
analysis from ECoG. First, a time segment following the the outgoing arrows) is summed, and the electrode with
ictal onset is selected from the ECoG recordings. The DTF the maximum amount of source activity for each seizure
method is applied to the time series, and the connectivity is noted. This process is repeated for each seizure, and the
pattern between the ECoG electrodes is obtained. Signif- statistically significant source activity is summed to obtain
icance testing by means of a surrogate data method is the total DTF-calculated source activity for each patient.
performed to obtain the causal interactions. From here, the (Figure adapted from [18] with permission)
Fig. 14.7 (a) The seizure onset zones identified clinically maximum source activity in each of the analyzed seizures.
by the epileptologists. (b) The DTF-calculated source ac- (Figure adapted from [18] with permission)
tivity obtained by selecting the cortical regions having the
sible to reconstruct the cortical neural activity modeling [36], aim to explain the local network
with a high degree of fidelity (see Chap. 13 of dynamics at the neural level.
this book). The network connectivity can then Electrophysiological connectome (eConnec-
be directly estimated from source waveforms at tome) is a concept born from the combination
regions of interest (ROIs) within the brain. These of inverse imaging algorithms and causal
types of connectivity estimation approximate the connectivity measures such as Granger causality,
macroscopic causal interactions between func- in which the estimated time series of sources are
tionally distinct brain regions. Still other recently analyzed to delineate the underlying effective
developed techniques, such as dynamic causal connectivity of brain networks [1, 6, 37]. Similar
Fig. 14.8 The concept of electrophysiological connec- used to determine the directional connectivity among these
tome (eConnectome). Source imaging can estimate the nodes of activity. This approach, eConnectome, is suitable
location and time-course of underlying brain sources from for imaging dynamic brain networks. (Figure from [37]
noninvasive EEG and MEG measurements, which can be with permission)
to the ECoG-based connectivity analysis, one can 14.8. schematically illustrates the eConnectome
first estimate cortical current density distributions approach for mapping functional and causal
from noninvasively recorded scalp EEG [1, 4] or brain networks from scalp EEG/MEG signals
from MEG [33, 35] and then perform causal [37]. Figure 14.9 shows an example of causal
analysis on waveforms at the cortical ROI level. connectivity measures estimated from EEG
In this case, the estimation of causal interactions source imaging and DTF analysis during a motor
from the EEG/MEG data can be complicated task [1]. From the directionality information
by the volume conductor effect, whereas it is provided by the DTF, the degree to which each
less an issue for the near-field ECoG recordings. cortical region acts as either a source or sink of
The volume conduction effect can be reduced cortical activity during the task can be calculated.
by reconstructing the source signals in the brain The interpretation of cortical network activity
that underlie the sensor signals. The cortical obtained from functional and effective connectiv-
current density (CCD) source model [38] can ity estimates may not be entirely straightforward
be used to solve the inverse problem from the [6, 7]. While the relationship between anatomic
scalp EEG to cortical source distribution [1, 37, connectivity measurements, such as diffusion
39]. Alternatively, a volume source scanning tensor MRI [40], and the physiologic cortical
method can be used to estimate current source networks is easily discernable, the precise
distribution within the brain and then used to anatomic relationship between functionally
assess the causal relations among the activities coupled disparate brain regions is less obvious.
located in various brain regions [4]. Figure Granger causality and other similar connectivity-
Fig. 14.9 Causal

connectivity patterns
estimated from EEG source
imaging and DTF analysis
during a motor task in
human subject. Spheres
show in-flow or out-flow at
the cortical ROIs. Arrows
show causal connectivity
estimated from the source
waveforms averaged over
the cortical ROIs among
brain regions. (Figure from
[1] with permission)
based techniques do not provide information [41], the toolbox developed by Anil Seth [42],
regarding the underlying physical connections and eConnectome toolbox developed by He and
between functionally coupled ROIs. Currently, co-workers [6, 34]. In this section, we describe
as a result of this disconnect between structure a MATLAB-based toolbox, eConnectome
and function measures, the identification of (electrophysiological connectome) [33, 34],
the precise neural networks which denote how which has been developed for mapping and
functionally coupled brain regions interact is imaging functional and causal connectivity at
not trivial. Imaging modalities and connectivity both the scalp and cortical levels from EEG/MEG
techniques, which are able to incorporate the and ECoG. Graphical user interfaces were
structural neural network information into the designed for interactive and intuitive use of
causal interactions, are needed to improve the toolbox. Major functions of eConnectome
the accuracy and precision of the calculated include EEG/ECoG/MEG preprocessing, scalp
cortical network activity. An effort has been spatial mapping, cortical source estimation,
made to first estimate brain sources using connectivity analysis, and visualization. Granger
anatomically realistic head models, and then causality measures such as directed transfer
causal connectivity among selected ROIs is function and adaptive directed transfer function
assessed (Fig. 14.9) [1, 7]. Such approach [45] are implemented to estimate the directional
represents an initial albeit important direction interactions of brain functional networks, over
to integrate anatomic and functional information the scalp and cortical sensor spaces. Granger
to estimate the causal interactions within the brain causality can be further estimated over the
networks. cortical source domain from the inversely
reconstructed cortical source signals as derived
from the scalp EEG [34] or MEG [33]. The
14.5 Software Packages for toolbox package is open-source and freely
Functional and Causal available at https://www.nitrc.org/projects/
Connectivity Analysis econnectome/ under the GNU general public
license for noncommercial and academic uses.
A number of toolboxes for causal connectivity Figure 14.10 illustrates a source connectiv-
analysis have appeared in the past several years. ity analysis from an interictal MEG spike in a
These include the BSMART software package patient with focal epilepsy [33]. The waveform
Fig. 14.10 Source connectivity analysis from the inter- cant activity at the selected time points were considered as
ictal MEG spike. The waveform and global field power regions of interest (see b). Representative waveforms for
of the interictal spike were inspected for the selection of the cortical regions of interest were computed. Directional
significant time points around the peak of the spike (see connectivity among the cortical ROIs was estimated from
a). Cortical current distributions were reconstructed from the ROI waveforms using directed transfer function and
the interictal spike with an individual realistic three-layer visualized over the cortical surface (see c). L left view, R
boundary element model, and cortical regions with signifi- right view. (Figure from [33] with permission)
and global field power of the interictal spike for EEG/ECoG/MEG processing. While the
can be inspected for the selection of significant focus of the toolbox lies on the mapping
time points using eConnectome (see a). Corti- and imaging of causal connectivity, a set of
cal current distributions were reconstructed from preprocessing tools are easily available to handle
the interictal spike with an individual realistic the raw electrophysiological signals in the
three-layer boundary element model, and cortical time, frequency, and spatial domains. Three-
regions with significant activity at the selected dimensional visualization of the brain activity
time points were considered as regions of inter- images and connectivity patterns is implemented
est using the cortical source imaging function in at both the sensor and source levels based on the
eConnectome (see b). Representative waveforms standard Montreal Neurological Institute (MNI)
for the cortical regions of interest were computed. brain [43] or a user-defined anatomy.
Directional connectivity among the cortical ROIs The graphical user interfaces of the eConnec-
was estimated from the ROI waveforms using tome allow users to analyze EEG/MEG/ECoG
directed transfer function and visualized over the data interactively and intuitively without MAT-
cortical surface using eConnectome (see c). LAB programming experience. The MATLAB-
The eConnectome toolbox is developed in based interface also allows users to run modules
MATLAB (MathWorks, Inc.) with graphical in command line or write customized modules
user interfaces as an open-source package. It with available functions and interfaces. A uni-
is integrated by the modules of preprocessing, form structure “ECOM” was designed to store
source imaging, and connectivity analysis, EEG/MEG/ECoG data including acquisition in-
which can be called individually or coordinately formation (e.g., sampling rate), electrodes loca-
tions, time series, and event information (e.g., for the interested reader to begin the exploration
onset time). Intermediate data such as prepro- of functional and causal connectivity in brain
cessed EEG/MEG/ECoG data, estimated cortical networks.
sources, and connectivity measures can be ex-
ported for later analysis and review. Acknowledgments We thank Yonghong Chen for help
with numerical simulations. This work was supported
by NIH grants MH079388 and MH070498 (M.D.)
and by NIH grants EB007920, EB006433, EB021027,
14.6 Concluding Remarks MH114233, and NS096761 (B.H.).
With the advent of data acquisition technology,

multielectrode neural recordings and functional Homework
brain imaging are becoming commonplace.
Such technologies promise to offer unparalleled 1. Suppose that X(t) is a process with μ(t) = 0
insights into how different areas of the brain work and RXX (t1 , t2 ) = 2−|t2 −t1 | . Determine the
together to achieve thought and behavior and how mean, variance, and covariance of the ran-
such coordinated brain activity breaks down in dom variables X(3) and X(9).
disease. While the accumulation of data continues 2. Consider the process
at an astonishing rate, how to effectively analyze
these data to extract information about the X(t) = A cos (ωt) + B sin (ωt)
workings of the brain remains a key challenge.
Recent years have witnessed rapid growth where A and B are independent random vari-
in the applications of various functional and ables with zero mean and equal variance.
causal connectivity measures to multichannel Show that this process is stationary.
neural data. In this chapter, we give a brief For problems 3–4, the process {Z(t)} is
introduction to some of the commonly applied white and Gaussian with mean zero and vari-
methods, including cross-correlation, spectral ance σZ2 .
coherence, and Grange causality. In addition 3. Consider the following AR(2) process
to time-domain formulations, we also discuss
spectral domain formulations, which are useful X(t) = α1 X (t − 1) + α2 X (t − 2) + Z(t)
in analyzing oscillatory neuronal networks.
Numerical examples are provided, where the where α 1 = 1/3 and α 2 = 2/9. It can be shown
connectivity pattern is known a priori. An that the autocorrelation function of X(t) is
experimental example is also provided in which
local field potential data from monkeys were ρXX (τ ) = 16/21(2/3)|τ | + 5/21(−1/3)|τ |
analyzed to reveal the laminar organization
of cortical alpha oscillations. Uses of Granger where τ = 0, ± 1, ± 2, . . . Simulate this
causality and in particular the directed transfer model (assume σZ2 = 1) to generate a dataset
function analysis have been discussed in human of 1000 data points. Plot the time series.
subjects from both scalp EEG/MEG and ECoG Estimate the autocorrelation function up to
data. Our emphasis is placed on the insights lag 10 from the dataset and compare it with
generated by the directional information provided the theoretical autocorrelation function. Vary
by these methods. Readers can consult a recent the dataset size (2000 points, 5000 points,
tutorial [7] on functional connectivity analysis for 10000 points, etc.) and see what happens.
more advanced treatments of various connectivity 4. Consider the following MA process
estimation algorithms. Finally, we introduce
several open-source MATLAB Toolboxes, X(t) = Z(t) + 0.4Z (t − 1)
especially the eConnectome package in detail. Y (t) = Z(t) − 0.4Z (t − 1)
These toolboxes can serve as the starting point
Find the cross covariance function RXY (τ ).
5. Consider the following AR(2) model: where a1 = 2 cos (2π /T) exp (−1/τ ) and
a2 = − exp (−2/τ ). Let T =10, τ = 50, and
X(t) = 0.9X (t − 1) − 0.5X (t − 2) + (t) σU2 = 1. Simulate the process to generate a
Y (t) = 0.8Y (t − 1) data set of 100 realizations each containing
− 0.5Y (t − 2) + 0.16X (t − 1) 500 data points. Estimate the spectral density
− 0.2X (t − 2) + η(t) using the periodogram approach.
9. Consider the following MA process
where (t), η(t) are independent Gaussian
white noise processes with zero means and X(t) = U (t) + 0.4U (t − 1)
variances σ12 = 1, σ22 = 0.7, respectively. Y (t) = U (t − 1) − 0.4U (t − 2)
Simulate this model to generate a data set
of 500 points. Your task is to estimate the Find the cross-covariance function, the
AR model from the data according to the cross-spectrum, and spectral coherence.
following procedure. (a) Plot the time series. 10. Consider the following process
(b) Determine the model order. (c) Obtain
the model coefficients using the Yule-Walker X(t) = U (t)
procedure and compare the estimated model Y (t) = αY (t − 1) + X (t − 1) + V (t)
equations with the above theoretical equa-
tions. (d) Increase the length of the time Compute the theoretical Granger causality
series to 2000 points or some other number, between the two processes in the time do-
repeat (a) to (c), and compare the results. main.
6. Suppose that X(t) is a process with 11. Download eConnectome software from
RXX (t1 , t2 ) = 4e−0.2|t2 −t1 | . Find its power (https://www.nitrc.org/projects/econnectome),
spectral density function. and add it to your MATLAB path. You might
For problems 7–10, the processes {U(t)} have to use earlier MATLAB versions to
and {V(t)} are independent normal white pro- avoid compatibility issues.
cesses with mean zero and variance σU2 and 12. Read and follow eConnectome’s tutorial.
σV2 . 13. Can you simulate two sources randomly on
7. Consider the following AR(1) process the cortex, assign two time-courses of inter-
est to these sources, solve the forward prob-
X(t) = αX (t − 1) + U (t). lem, add noise to the simulated EEG, and
solve the inverse problem to recover the two
(a) Find the power spectral density func- sources and study the connectivity of the two
tion of {X(t)}. Here α < 1. (b) For σU2 = 1 sources?
and α = 0.5 plot the power spectral density
function. (c) Simulate the process to generate
a data set of 100 realizations each containing References
500 data points. Estimate the spectral density
1. F. Babiloni, F. Cincotti, C. Babiloni, F. Carducci,
with the periodogram approach. Compare the D. Mattia, L. Astolfi, A. Basilisco, P.M. Rossini, L.
result with the result from (b). (d) What hap- Ding, Y. Ni, J. Cheng, K. Christine, J. Sweeney, B.
pens if you have only 5 realizations? (e) What He, Estimation of the cortical functional connectivity
happens if you have 100 realizations but each with the multimodal integration of high-resolution
EEG and fMRI data by directed transfer function.
realization only has 10 data points? NeuroImage 24, 118–131 (2005)
8. Consider the following AR(2) process 2. A. Bollimunta, Y. Chen, C.E. Schroeder, M. Ding,
Neuronal mechanisms of cortical alpha oscillations
X(t) = α1 X (t − 1) + α2 X (t − 2) + U (t)
in awake-behaving macaques. J. Neurosci. 28, 9976– directed transfer function method. Epilepsia 51, 564–
9988 (2008) 572 (2010)
3. A. Bollimunta, J. Mo, C.E. Schroeder, M. Ding, 19. N. Wiener, The theory of prediction, in Modern Math-
Neuronal mechanisms and attentional modulation of ematics for Engineers, Chap 8, ed. by E. F. Becken-
corticothalamic alpha oscillations. J. Neurosci. 31, bach, (McGraw-Hill, New York, 1956)
4935–4943 (2011) 20. Y. Chen, S.L. Bressler, M. Ding, Frequency decom-
4. L. Ding, G.A. Worrell, T.D. Lagerlund, B. He, Ictal position of conditional Granger causality and appli-
source analysis: Localization and imaging of causal cation to multivariate neural field potential data. J.
interactions in humans. NeuroImage 34(2), 575–586 Neurosci. Methods 150, 228–237 (2006)
(2007) 21. J. Geweke, Measures of conditional linear-
5. M. Ding, Y. Chen, S.L. Bressler, Granger causal- dependence and feedback between time-series.
ity: Basic theory and application to neuroscience, in J. Am. Stat. Assoc. 79, 907–915 (1984)
Handbook of Time Series Analysis, ed. by B. Schelter, 22. M. Kaminski, M. Ding, W.A. Truccolo, S.L.
M. Winderhalder, J. Timmer, (Wiley-VCH, Berlin, Bressler, Evaluating causal relations in neural
2006), pp. 437–460 systems: Granger causality, directed transfer function
6. B. He, L. Yang, C. Wilke, H. Yuan, Electrophysio- and statistical assessment of significance. Biol.
logical imaging of brain activity and connectivity – Cybern. 85, 145–157 (2001)
Challenges and opportunities. IEEE Trans. Biomed. 23. S. Guo, A.K. Seth, K.M. Kendricke, C. Zhou, J.
Eng. 58(7), 1918–1931 (2011) Feng, Partial Granger causality—Eliminating exoge-
7. B. He, L. Astolfi, P.A. Valdés-Sosa, D. Marinazzo, nous inputs and latent variables. J. Neurosci. Methods
S.O. Palva, C.G. Bénar, C.M. Michel, T. Koenig, 172, 79–93 (2008)
Electrophysiological brain connectivity: Theory and 24. H. Berger, Über das Elektroencephalogramm des
implementation. IEEE Trans. Biomed. Eng. 66(7), menschen (on the electroencephalogram of man).
2115–2137 (2019) Arch. Psychiatr. Nervenkr. 87, 527–570 (1929)
8. J. Geweke, Measurement of linear-dependence and 25. J.C. Shaw, Brain’s Alpha Rhythm and the Mind (El-
feedback between multiple time-series. J. Am. Stat. sevier, Amsterdam, 2003)
Assoc. 77, 304–313 (1982) 26. P. Andersen, S.A. Andersson, Physiological Basis
9. C.W.J. Granger, Investigating causal relations by of the Alpha Rhythm (Appleton-Century-Crofts, New
econometric models and cross-spectral methods. York, 1968)
Econometrica 37(3), 424–438 (1969) 27. L.R. Silva, Y. Amitai, B.W. Connors, Intrinsic oscil-
10. G. Buzsaki, Rhythms of the Brain (Oxford University lations of neocortex generated by layer 5 pyramidal
Press, New York, 2006) neurons. Science 251, 432–435 (1991)
11. A. Schnitzler, J. Gross, Normal and pathological os- 28. M. Ding, J. Mo, C.E. Schroeder, X. Wen, Analyzing
cillatory communication in the brain. Nat. Rev. Neu- coherent brain networks with Granger causality. Pro-
rosci. 6(4), 285–296 (2005) ceedings of the 33rd Annual International Conference
12. P.J. Uhlhaas, W. Singer, Abnormal neural oscillations of the IEEE EMBS, (2011), pp. 5916–5918.
and synchrony in schizophrenia. Nat. Rev. Neurosci. 29. L.A. Baccala, K. Sameshima, Partial directed coher-
11, 100–113 (2010) ence: A new concept in neural structure determina-
13. C. Chatfield, The Analysis of Time Series: An Intro- tion. Biol Cyber 84, 463–474 (2001)
duction (Chapman and Hall, Boca Raton, 2004) 30. H. Jiang, Z. Cai, G. Worrell, B. He, Multiple oscilla-
14. M.D. Fox, M. Corbetta, A.Z. Snyder, J.L. Vincent, tory push-pull antagonisms constrain seizure propa-
M.E. Raichle, Spontaneous neuronal activity distin- gation. Ann. Neurol. 86(5), 683–694 (2019)
guishes human dorsal and ventral attention systems. 31. C. Wilke, G. Worrell, B. He, Graph analysis of epilep-
Proc. Natl. Acad. Sci. U. S. A. 103, 10046–10051 togenic networks. Epilepsia 52(1), 84–93 (2011)
(2006) 32. L. Astolfi, F. Cincotti, D. Mattia, M.G. Marciani,
15. Y. Zhang, X. Wang, S.L. Bressler, Y.H. Chen, M.Z. L. Baccala, F.F. de Vico, S. Salinari, M. Ursino, M.
Ding, Prestimulus cortical activity is correlated with Zavaglia, L. Ding, J.C. Edgar, G.A. Miller, B. He, F.
speed of visuomotor processing. J. Cognitive Neu- Babiloni, A comparison of different cortical connec-
rosci. 20, 1915–1925 (2008) tivity estimators for high resolution EEG recordings.
16. A. Brovelli, M. Ding, A. Ledberg, Y. Chen, R. Naka- Hum. Brain Mapp. 28(2), 143–157 (2007)
mura, S.L. Bressler, Beta oscillations in a large-scale 33. Y. Dai, W. Zhang, D.L. Dickens, B. He, Source
sensorimotor cortical network: Directional influences connectivity analysis from MEG and its application
revealed by Granger causality. Proc. Natl. Acad. Sci. to epilepsy source localization. Brain Topogr. 25(2),
101, 9849–9854 (2004) 157–166 (2012)
17. C. Wilke, W. van Drongelen, M. Kohrman, B. He, 34. B. He, Y. Dai, L. Astolfi, F. Babiloni, H. Yuan, L.
Identification of epileptogenic foci from causal analy- Yang, eConnectome: A MATLAB toolbox for map-
sis of ECoG interictal spike activity. Clin. Neurophys- ping and imaging of brain functional connectivity. J.
iol. 120(8), 1449–1456 (2009) Neurosci. Methods 195(2), 261–269 (2011)
18. C. Wilke, W. van Drongelen, M. Kohrman, B. He, 35. A. Sohrabpour, S. Ye, G. Worrell, W. Zhang, B.
Neocortical seizure foci localization by means of a He, Noninvasive electromagnetic source imaging and
granger causality analysis: An electrophysiological 42. A.K. Seth, A MATLAB toolbox for Granger causal
connectome (eConnectome) approach. IEEE Trans. connectivity analysis. J. Neurosci. Methods 186, 262–
Biomed. Eng. 63, 2474–2487 (2016) 273 (2011)
36. K.J. Friston, L. Harrison, W. Penny, Dynamic causal 43. D.L. Collins, P. Neelin, T.M. Peters, A.C. Evans, Au-
modeling. NeuroImage 19(4), 1273–1302 (2003) tomatic 3D intersubject registration of MR volumet-
37. B. He, A. Sohrabpour, E. Brown, Z. Liu, Electrophys- ric data in standardized Talairach space. J. Comput.
iological source imaging: A noninvasive window to Assist. Tomogr. 18, 192–205 (1994)
brain dynamics. Annu. Rev. Biomed. Eng. 20, 171– 44. H. Akaike, Autoregressive model fitting for control.
196 (2018) Ann. Inst. Stat. Math. 23(1), 163–180 (1971)
38. A.M. Dale, M.I. Sereno, Improved localization of cor- 45. C. Wilke, L. Ding, B. He, Estimation of time-varying
tical activity by combining EEG and MEG with MRI connectivity patterns through the use of an adaptive
cortical surface reconstruction: A linear approach. J. directed transfer function. IEEE Trans. Biomed. Eng.
Cogn. Neurosci. 5, 162–176 (1993) 55, 2557–2564 (2008)
39. Z.M. Liu, B. He, FMRI-EEG integrated corti- 46. M.J. Kaminski, K.J. Blinowska, A new method of
cal source imaging by use of time-variant spa- the description of the information flow in the brain
tial constraints. NeuroImage. 39(3), 1198–1214 structures. Biol. Cybern. 65(3), 203–210 (1991)
(2008) 47. P.J. Franaszczuk, G.K. Bergey, M.J. Kaminski, Anal-
40. P.J. Basser, J. Mattiello, D. LeBihan, MR diffusion ysis of mesial temporal seizure onset and propa-
tensor spectroscopy and imaging. Biophys. J. 66, gation using the directed transfer function method.
259–267 (1994) Electroencephalogr. Clin. Neurophysiol. 91, 413–427
41. J. Cui, L. Xu, S.L. Bressler, M. Ding, H. Liang, (1994)
BSMART: A Matlab/C toolbox for analysis of mul-
tichannel neural time series. Neural Netw. 21, 1094–
1104 (2008)
Deep Learning Models with
Applications to Brain Image Analysis 15
Dinggang Shen, Luping Zhou, and Mingxia Liu
Abstract rections of deep learning in brain image anal-

ysis are also pointed out.
Deep learning, rooted in artificial neural net-
works, has received increasing attention in the
Keywords
field of brain image analysis. In this chap-
ter, the pre-processing steps for brain images Deep learning · Brain image analysis ·
and the fundamental concepts of deep neural Convolutional neural network ·
networks are first introduced. After that, four Classification · Segmentation · Registration ·
typical types of deep neural networks used for Synthesis/augmentation
brain image analysis are elaborated, includ-
ing (i) convolutional neural networks (CNNs)
and the variants (i.e., fully convolutional net-
works and U-net), (ii) recurrent neural net-
15.1 Background
works (RNNs) and the variant (i.e., long short-
term memory model), (iii) auto-encoder, and
The significant advances of neuroimaging
(iv) generative adversarial networks (GANs)
techniques have deeply reshaped modern neu-
and the variants (i.e., Pix2Pix GAN and Cycle-
roscience in recent decades, offering researchers
GAN), as well as their applications in brain im-
unprecedented opportunities to noninvasively
age classification, segmentation, registration,
investigate the anatomy and functions of the
and image synthesis/augmentation. In addi-
brain. The imaging-based measurements are
tion, several challenges and future research di-
heralded more sensitive and consistent than
the traditional cognitive tests, thus critical for
the early diagnosis of brain disorders, e.g.,
Alzheimer’s disease (AD) and schizophrenia.
As a relatively new discipline within medicine,
Electronic Supplementary Material: The online version neuroscience, and psychology, neuroimaging
of this chapter (https://doi.org/10.1007/978-3-030-43395- falls into two broad categories: (1) structural
imaging, such as magnetic resonance imaging
(MRI), and (2) functional imaging, such as
D. Shen () · M. Liu functional magnetic resonance imaging (fMRI)
Department of Radiology and BRIC, University of North
and positron emission tomography (PET).
e-mail: [email protected] Neuroimaging-based studies not only focus
on investigating how the brain is organized
L. Zhou
School of Electrical and Information Engineering, The around regions (such as local morphometry
University of Sydney, Sydney, NSW, Australia and functions) but may also consider the

434 D. Shen et al.
connections between brain regions, in order to using prior knowledge or data-driven strate-
identify imaging-based biomarkers for automated gies [10]. However, it is usually challenging
diagnosis of brain diseases. Such tasks could be to define such ROIs in each 3D brain image.
very challenging, giving the highly non-linear (2) Missing data problem. The missing data
mapping between image-based observations problem is usually unavoidable in the field
and diagnosis outputs. Recently this research of brain image analysis (especially for multi-
field significantly benefits from the emerging modality applications) [15], because subjects
deep learning techniques that have demonstrated may lack some modalities (e.g., PET) due
excellent performance in learning non-linear to patient dropouts and/or poor data quality.
function mapping in various tasks. Conventional methods typically discard data-
As part of a broader family of machine learn- missing subjects, but this will significantly
ing methods, deep learning models are generally reduce the number of training subjects and
based on artificial neural networks. Even though affect the robustness of the learned model.
artificial neural networks were proposed in the (3) Spatiotemporal dynamics of the brain. Pre-
1950s, their development still suffered from some vious studies on brain functional connectiv-
limitations, such as lack of computing power, ity have shown that the human brain is in-
lack of sufficient training data, and difficulty in trinsically organized into spatiotemporal dy-
training deep networks. The rapid development namic interaction network [16, 17], demon-
of deep learning in recent years can be attributed strating remarkable spatiotemporal variabil-
to the enhanced computer capabilities of high- ity over time in its function and structure [18,
tech central processing units (CPUs) and graphics 19]. Hence, it’s essential to model the spa-
processing units (GPUs), the availability of big tiotemporal dynamics of brain images to im-
data, and the novel algorithms for training deep prove the performance of deep learning mod-
neural networks. Deep learning algorithms can els.
be supervised (i.e., learning with only labeled
data), semi-supervised (i.e., learning with both To address these issues, various deep learning
labeled and unlabeled data), or unsupervised (i.e., models have been proposed for analyzing brain
learning with only unlabeled data) [1–3]. In par- images, resulting in promising results in different
ticular, deep learning models such as convolu- applications. In the following, the pre-processing
tional neural network (CNN), recurrent neural steps for brain images and fundamental concepts
network (RNN), auto-encoder (AE), and gener- of neural networks will be first introduced in
ative adversarial network (GAN) have rapidly Sect. 15.2. Then, four typical deep neural net-
become a methodology of choice for analyzing works for brain image analysis will be presented
brain images in various applications [4,5], such as in Sects. 15.3, 15.4, 15.5, and 15.6. Section 15.7
brain image segmentation [6–8], brain image reg- presents the limitations of current deep learning
istration [9], brain disease diagnosis [10–12], and models and several possible future research
brain image synthesis [13–15]. However, there directions. Finally, Sect. 15.8 concludes this
are several challenges for deep learning models chapter.
in dealing with neuroimages, as summarized be-
low:
15.2 Image Processing and
(1) Small-sample-size problem. In the domain Concept of Deep Learning
of brain image analysis, deep learning models
generally suffer from the small-sample-size 15.2.1 Brain Image Pre-processing
problem, because there are millions of voxels
in each 3D volume and a very limited number Brain images usually need to be pre-processed
(e.g., tens or hundreds) of subjects/images for so that the acquisition artifacts and undesired
model training. A popular solution is to locate tissues could be removed to better serve subse-
regions of interest (ROIs) in brain images quent tasks. In the following, a typical pipeline
15 Deep Learning Models with Applications to Brain Image Analysis 435
to pre-process structural brain MR images is in- Spatial registration, which transforms images
troduced, since MRI is the most widely used into a common coordinate space, is often
imaging modality to explore human brains. The needed when integrating multiple imaging
pipeline includes the steps of skull stripping, bias modalities/MR sequences for analysis. The trans-
field correction, intensity normalization, and spa- formation could be linear (such as translation,
tial registration [20]. Based on the characteristic rotation, scaling and other affine transforms) or
of the analysis task, some steps could be optional. non-linear, which could locally warp images to
Skull stripping is used to remove non-brain tis- match them. Image registration by itself is a big
sues from MR images. The existence of non-brain category of research problems [24, 25].
tissues, such as skull and eyes, could negatively Many off-the-shelf toolkits provide basic
affect the algorithms for the subsequent segmen- tools for the pre-processing of brain images, for
tation or diagnosis. Therefore, skull stripping is example, FMRIB Software Library (FSL)1 and
used for pre-processing in many brain image anal- Statistical Parametric Mapping (SPM).2 Such
ysis models, including those deep learning-based pre-processing steps are often indispensable
ones. Either under- or over-segmentation of the in traditional non-deep learning-based models.
brain will lead to inaccurate estimation of brain For deep learning models, despite their strong
tissues. A commonly used skull stripping method capacity of learning highly non-linear function
is based on BET [21, 22]. mapping, they could still benefit from these pre-
A common artifact seen in an MR image is processing steps to reduce the complexity of the
the smooth variation of signal intensity across the learning task, especially when there are only a
image, called as bias field. This may be caused limited number of images for training.
by factors such as poor radio-frequency field uni-
formity and eddy currents driven by the switch-
15.2.2 Fundamentals About Neural
ing of field gradients, etc. This intensity non-
Network Models
uniformity is known as “bias field” which is the
low-frequency multiplicative noise in the images.
Deep learning is rooted in artificial neural
Many methods have been proposed for the bias
networks. An artificial neural network consists
field correction by estimating both the uncor-
of multiple layers of interconnected processing
rupted image intensities and the spatially smooth
units, known as neurons. If the connections
and multiplicative model of the bias field. Among
between neurons do not form a circle, the artificial
these methods, a typical representative is the non-
neural network is feed-forward. The most
parametric non-uniform intensity normalization
common class of feed-forward neural network is
(N3) algorithm [23].
known as multilayer perception (MLP). An MLP
MRI scans are acquired in arbitrary units, mak-
has at least three layers, (1) an input layer, (2) a
ing them not amiable for the comparison of the
hidden layer, and (3) an output layer, as shown in
same tissue across different studies of a same
Fig. 15.1. The network takes data from the input
subject or across different subjects. Such intensity
layer, non-linearly transforms the data via the
difference could make simple operations such as
hidden layer, and produces the prediction at the
thresholding difficult across images. Therefore,
output layer. In MLP, neurons in the neighboring
intensity normalization is sometimes involved for
layers are fully connected, while neurons within
pre-processing MR images. Methods for intensity
the same layer have no connection. Multiple
normalization usually manipulate the histograms
hidden layers can be stacked to increase the non-
of MR images so that they are aligned after the
linearity of the model.
normalization, i.e., the discrepancy between his-
Specifically, an MLP learns an embedding
tograms is minimized. However, pre-processing
function y = f (x; w), where x is the input, y
images with intensity normalization needs to be
done carefully to avoid the elimination of critical 1 https://fsl.fmrib.ox.ac.uk/fsl/fslwiki
or discriminative information carried in image 2 https://www.fil.ion.ucl.ac.uk/spm/
intensities.
436 D. Shen et al.
Fig. 15.1 A basic

multilayer perception
network (MLP)
Fig. 15.2 Common activation functions: sigmoid (left), hyperbolic tangent (middle), and rectified linear unit (right)
is the output, f = (fn ◦ fn−1 · · · ◦ f1 )(x) is a Output layer

set of non-linear transforms parameterized by w,
and n denotes the number of layers. Usually, a
non-linear transform fi at the i-th layer takes the Hidden layer
form of fi = σi (wi fi−1 ), consisting of a linear
transform on the output of the previous layer fi−1
and a following non-linear activation function Input layer
σi . The linear transform matrix wi is called
the parameters or weights of the model, which Fig. 15.3 An example for backpropagation
is automatically learned during training. The
commonly used activation function σi includes where α is the user-predefined learning rate and
sigmoid function, hyperbolic tangent function t is the index of iteration. Repeating in this way,
(tanh), and rectified linear unit (ReLu), as shown the value of the loss function will be gradually
in Fig. 15.2. reduced until a certain stopping criterion can be
Training and testing neural networks require met. In the test stage, the test set (unseen in
non-overlapped training and test datasets. The the training stage) is simply fed forward through
training set consists of N paired input and the learned neural network model, using w∗ for
expected output samples {xi , yi }N i=1 . An MLP prediction.
is trained to minimize the difference between
its prediction and the expected output by An example for backpropagation Figure 15.3
optimizing the loss function L. For example,
shows an MLP consisting of an input layer with
L = minw N i=1 N yi − f (xi ; w) . The
1 2
two scalar input variables x1 and x2 , a hidden
∗
optimal model parameters w can be effectively layer with the output h, and an output layer with
attained by a family of methods known as the output
“backpropagation.” The basic idea is to exploit o. The 2loss function of the MLP is
L = i (oi − yi ) , where yi indicates ground-
chain rule to first compute the gradient of the loss truth and
function with respect to each model parameter
wij in w and then update wij using the direction hi = f1 (x1,i , x2,i ; w1 , w2 ) = w1 x1,i + w2 x2,i ,
of gradient descent iteratively as
oi = f2 (hi ) = w3 hi .
∂ L(wtij )
wijt+1 = wtij − α , (15.1)
∂wij
Solution Applying the chain rule, we have the 15.3 Convolutional Neural
following solution Networks
∂L ∂ L ∂oi
= = 2(oi − yi )hi , MLPs have some drawbacks when they are used
∂w3 i
∂oi ∂w3 i for image processing. They use one neuron for
∂ L ∂oi ∂hi each input (e.g., a pixel in an image), and ev-
∂L
= = 2(oi − yi )w3 x2,i , ery neuron connects to all neurons in the next
∂w2 ∂oi ∂hi ∂w2
i i layer. This makes the parameters of the model
∂L ∂ L ∂oi ∂hi increase dramatically when the size of the image
= = 2(oi − yi )w3 x1,i . is relatively large. Also, flattening an image to
∂w1 i
∂oi ∂hi ∂w1 i
MLP causes the loss of spatial information in
Gradient descent computed in backpropa- the image. Instead, convolutional neural networks
gation can be trained in three ways: (1) batch, (CNNs) are the most commonly used deep learn-
(2) stochastic, and (3) mini-batch. In gradient ing models in medical image analysis. They are
descent, batch means the total number of samples biologically inspired variants of MLPs, utilizing
used to update the gradient in one iteration. A local receptive fields, weight sharing, and sparse
large batch (e.g., the entire training set) may connectivity to preserve spatial information and
even cause a single iteration to take a long reduce the number of network parameters.
time to compute. On the contrary, stochastic
gradient descent (SGD) uses a single training Receptive field Being 2D or 3D grids, images
sample to calculate the objective loss and form high-dimensional input to neural networks.
update the gradient for each iteration. The It is inefficient to fully connect a neuron with
increased frequency of model update may every pixel/voxel in this case. Instead, as known,
lead to faster learning in some problems, but pixels/voxels are mostly useful in the context of
also bring noisy gradient estimation. Mini- their neighbors. Therefore, in CNNs, a neuron
batch stochastic gradient descent (mini-batch is connected only to a local region of the input
SGD) balances the full-batch training and SGD. grid. This input region is known as the recep-
It splits the training set into small batches tive field of the neuron. It can be described by
and uses these small batches to calculate its centroid location and size. In CNNs, the re-
objective loss and update the model. Mini-batch ceptive field can be increased by stacking more
SGD improves the efficiency of the full-batch layers.
training and reduces the noise in SGD, which
is commonly used for training deep learning Weight sharing At every layer in CNNs, filters
models. are applied to detect the presence of specific fea-
The following terms are related to mini-batch tures or patterns. These filters act on the receptive
training. Mini-batch size refers to the number of field of the input image. The numbers within
training samples in one mini-batch used to update each filter are called weights. Weight sharing
the model. The number of epochs refers to the happens across the filters in a particular layer.
times that the entire training set is passed forward That is, when the filter moves through the image,
and backward through the neural network model. its weights do not change. The idea behind is that
The number of iterations refers to the number of if a detected pattern is important in a particular
passes using the samples of the mini-batch size, region of the image, it may be important in other
where each pass includes a forward pass and a regions of the image too.
backward pass. For example, if the training set
contains 1,000 samples, and the mini-batch size Sparse connectivity In CNNs, filters are con-
is 50, it then takes 20 iterations to complete one volved with the input image to calculate the ac-
epoch. tivation of neurons. When the size of the filter,
438 D. Shen et al.
or equally the receptive field, is smaller than Each filter moves across all input locations via
the input volume, each neuron in the activation a step called stride and uses the same weights
map is only connected to a local region of the for convolution. This produces a feature map.
input, leading to “sparse” (rather than “dense”) In other words, a feature map is formed by
connectivity across layers. units that share the same weights and bias in a
convolutional layer. The stack of filters produces
a tensor of feature maps. The feature maps
15.3.1 CNN Fundamentals are further sent through a non-linear activation
function, such as ReLu (See Fig. 15.2), to model
A CNN model consists of convolutional layers non-linear mapping and produce activation
interspersed with pooling layers, on top of which maps.
are often fully connected layers as in standard
MLPs. A typical example of CNN, i.e., LeNet, is Pooling layers After convolutional layer, there
shown in Fig. 15.4. is typically a pooling layer to down-sample the
feature maps produced in the convolutional layer.
Convolutional layers In convolutional layers, By pooling, each small region in a feature map
the input of the layer is convolved with stacks of is summarized into a single value. There are two
filters of predefined size. The weights of filters common methods for pooling: max-pooling and
are automatically learned by optimizing CNN average-pooling. In max-pooling, a small region
via backpropagation. Filters are also known as is represented by the maximum value inside it.
convolutional kernels. The output of convolution In average-pooling, a small region is represented
is the sum of element-wise multiplication by the average of all values inside it. Figure 15.6
between pixels in the receptive field and weights illustrates the ideas of max-pooling and average-
in the convolution filter, as shown in Fig. 15.5. pooling. The down-sampled feature maps could
Fig. 15.4 LeNet: an example of CNN
Fig. 15.5 Illustration of convolution operation in 2D. tween the weights in the filter and the pixel values in the
When a filter is convolved with a local region (receptive region is calculated, and these multiplications are summed
field) in the image, the element-wise multiplication be- up to produce a value in the feature map
Fig. 15.6 Illustration of

0 3 8 0
max-pooling and
average-pooling 0 5 5 3 5 8 2 4
3 2 0 0
3 3 2 1
1 2 3 1
feature map max-pooling average-pooling
be more robust to small changes in the positions 15.3.2 CNN Variants

of the features. Therefore, pooling could give the
network model a certain degree of “translation Based on CNNs, many novel network architec-
invariance.” Another way to down-sample feature tures have been proposed to enable the network
maps is to conduct convolution with the “stride” model to go deeper with less parameters and
larger than 1. better performance. For example, the residual
module [26] and inception module in [27] have
Fully connected layers When CNNs are been proved effective in both general and medi-
used for classification, following multiple cal image classification. Meanwhile, it is noticed
convolution-pooling blocks, there are usually that when CNN models are used for prediction
several fully connected (FC) layers that flatten at pixel level (e.g., segmentation and synthesis),
feature maps for prediction. FC layers are in fully convolutional networks (FCNs) [28] without
principle the same as the traditional MLP: every using fully connected layers on top of convo-
neuron in one layer is connected to every neuron lutional blocks demonstrate several advantages
in another layer. For example, in LeNet (see over the traditional CNNs, such as allowing input
Fig. 15.4), there are two FC layers on top of the images with different image sizes and being more
convolution-pooling blocks. The output of the efficient than patch-wise training for pixel-level
final FC layer is sent through a softmax function prediction. In the following, residual CNNs and
to classify the image with probabilistic values a typical FCN model called U-net [29] are intro-
between 0 and 1. duced, respectively, as well as different ways to
combine CNNs for analysis.
Dropout Deep models may contain a large
number of parameters to learn, and they have
a lot of freedom to fit complex datasets. This 15.3.3 Residual Learning Based on
may lead to an overfitting problem, i.e., the CNN
learned model fits well to the training data but
fails to generalize well to unseen test data. It is When neural networks go deep, they are expected
known that the ensembles of neural networks to become more powerful and better approxi-
with different model configurations can reduce mate complicated functions. However, it turns
overfitting. This can be achieved in a single out that they often encounter the gradient van-
model by “dropout” (i.e., randomly dropping ishing problem. That is, the gradient of the loss
out neurons during training). When a neuron function approaches zero, especially at the shal-
is dropped out, it is temporally removed from low layers (the layers closer to the input) during
the network with all its incoming and outgoing backpropagation. In this case, we may observe
connections. This leads to slightly different the performance saturates or even degrades when
network for each batch of training data, which more layers are added. To mitigate this problem,
effectively reduces overfitting. Dropout takes the residual module is proposed. As illustrated
place only in the training stage, but not in the test in Fig. 15.7, in the residual module, the inputs x
stage. from the previous layers are directly connected
440 D. Shen et al.
with the output f (x) of the new convolutional around every pixel as input and predicts the
layers in the module, which is known as residual label of the patch centroid. Such approaches
connection. They are then added to approximate have some potential disadvantages. First, fully
the target output y. In other words, in common connected layers consist of predetermined
CNNs, we directly learn the output y, while in the number of neurons, which constrains the size
residual setting we learn the difference (residual) of the input image. That is, all input images
between the output and the input: f (x) = y − have to be rescaled to the same size that is
x. In this way, the gradient of the input can be predefined. Second, by flattening the feature
better preserved, as x is transmitted by the identity map output from a convolutional layer into a
matrix without loss of information. The residual fully connected layer, some spatial information
modules can be stacked to form very deep neural is lost. Third, predicting pixel label in a patch-
networks, such as ResNet (Fig. 15.8) that is able wise manner could be very time-consuming as
to train up to hundreds or even thousands of layers the prediction needs to go through pixels one by
with compelling performance. one.
Recently, fully convolutional networks
(FCNs) [28] are proposed to better deal with
15.3.4 Fully Convolutional Networks per-pixel prediction tasks and demonstrate
and U-Net promising performance, especially in semantic
segmentation. Different from CNNs used in
Common CNN models with fully connected patch-wise manner that predicts the label of
layers can be used for pixel-level prediction, one pixel each time, FCNs can produce dense
such as segmentation and synthesis. This is outputs from the input images of arbitrary size.
usually conducted in a patch-wise manner, For example, FCNs can generate segmentation
i.e., the CNN takes image patch(es) extracted map that has the same size as the input image at
one shot, by concatenating convolutional layers
and deconvolutional layers. Deconvolution is
simply backward strided convolution. Similar
to convolutional layers that down-sample the
feature maps, deconvolutional layers upsample
the feature maps. Figure 15.9 illustrates the
deconvolution with 3 × 3 kernel using stride
one on the feature map with size 2 × 2.
A typical FCN widely used in medical image
analysis is U-net [29], as shown in Fig. 15.10. It
consists of the contracting and expanding paths.
The contracting path consists of convolutional
layers, while the expanding path consists of de-
convolutional layers. They share the same num-
Fig. 15.7 Illustration of the difference between the plain ber of layers. Between these two paths, multi-
layers (left) and the residual module (right) ple skip connections are used to link the corre-
Fig. 15.8 Network architecture of ResNet with 32 layers

sponding convolutional and deconvolutional lay- in the feature maps of shallow layers, forming a
ers. Similar to the residual module, using the hierarchy of visual clues.
skip connections, U-net can mitigate the common
gradient vanishing problem during the training 15.3.5 Combination of CNNs
of deep learning models, as the gradient of the
deeper layers can be directly backpropagated to CNN models could also be combined sequentially
the shallower layers via the skip connections. or in parallel for analysis. When combined
Moreover, this structure allows U-net to acquire sequentially, the output of the preceding network
multi-depth information of the input image. In becomes the input of the successive network.
this way, it can preserve the contextual informa- Such combination is often used to gradually
tion from the input as well as the spatial details refine the output results. When combined in
parallel, the networks are used to process the
same input, and their outputs are integrated for
decision. Such combination is often used to
extract complementary features from the input
image. For example, a cascade of networks is
proposed (see Fig. 15.11) in [30]. It sequentially
combines CNNs to segment the regions of interest
from coarse to fine, including (1) segmentation
of whole tumor, (2) segmentation of tumor
core segmentation, and (3) segmentation of
enhancing tumor core. Each network takes
a fully convolutional network using dilated
convolutions [31] and residual connections. In
contrast, the work in [32] proposed a CNN model
(Fig. 15.14) that consists of two parallel CNN
Fig. 15.9 Illustration of deconvolution (kernel size 3×3) pathways, each coping with a different receptive
using stride one. The original input is only the 2 × 2 green
field on the input image, to incorporate both the
region, while the white region is filled with zeros
Fig. 15.10 Illustration of U-net model. (Image courtesy to [29])

442 D. Shen et al.
Fig. 15.11 Illustration of Cascade Net. (Image courtesy to [30])
local and the contextual information for brain as Alzheimer’s disease (AD). Many methods
tumor segmentation. Also, a multi-instance CNN train CNN models to extract visual features
(MICNN) model with multiple parallel sub- based on predefined anatomical landmarks,
networks was designed in [33] for diagnosis such as hippocampus, for classification. These
and prognosis of brain diseases, and each sub- approaches demonstrated improved diagnostic
network was used to extract local patch-level rep- accuracy over the conventional approaches using
resentations of an input image. These patch-level handcrafted features. However, in these methods,
features were further concatenated and fed into the localization of atrophy and the diagnosis
several FC layers for brain disease identification. of diseases are treated separately. Different
In MICNN, each sub-network was corresponding from them, a hierarchically fully convolutional
to a disease-associated location (defined by network (H-FCN) was proposed to jointly learn
anatomical landmarks) in the input brain MR atrophy location and perform AD diagnosis [10].
image, and these sub-networks share the same ar- The architecture of H-FCN is shown in
chitecture but with different network parameters, Fig. 15.12. It consists of four components:
learn specific features from local patches. This location proposal sub-network, patch-level sub-
method was further extended to be a multi-task network, region-level sub-network, and subject-
learning model in [11] for joint classification and level sub-network, aiming to learn features in a
regression in brain MRI-based disease diagnosis. hierarchical way. Specifically, co-registered brain
images were sent to the location proposal sub-
network to generate image patches distributed
15.3.6 CNN Applications to Brain
over the whole brain. These image patches
Image Classification and
were fed into patch-level sub-networks to output
Segmentation
patch-level features and patch-level classification
scores. After that, spatially neighboring patches
CNNs have been applied to analyze brain im-
were then grouped into local regions, and their
ages in a variety of tasks, such as mental disease
patch-level outputs (features concatenated with
classification [10, 33], neuroanatomy segmenta-
classification scores) were processed by the
tion [34], lesion/tumor detection and segmenta-
region-level sub-networks to produce regional
tion [32, 35], brain image registration [36, 37],
features and regional classification scores. The
etc. In the following, some examples of CNNs
outputs of the region-level sub-networks were
used for brain image classification, segmentation,
eventually integrated by the subject-level sub-
and brain network analysis are introduced, re-
network to classify each subject. The proposed
spectively.
H-FCN demonstrated promising performance
when evaluated on two datasets, i.e., Alzheimer’s
15.3.7 Brain Image Classification Disease Neuroimaging Initiative 1 (ADNI-1)
and ADNI-2, that contain a large cohort of
With the capability of learning highly non- subjects. Visual examples of discriminative
linear and task-oriented features, CNNs have regions identified by H-FCN are provided in
been applied to diagnosing brain diseases, such Fig. 15.13.
PSN
Class_P
PSN Conv_R Class_R
PSN
32 64 64 128 128 64
PSN 64
Conv_S
4 × 4 × 4 Conv 3 × 3 × 3 Conv
64
…
1 × 1 × 1 Conv 2 × 2 × 2 max pooling
Class_S
PSN Classification (1 × 1 × 1 Conv)
Conv_R Class_R
Input: PSN
Output: Region-level Conv Subject-level Conv
sMRI Class label
PSN
64 Channel concatenation Spatial concatenation
PSN
Potentially pruned sub-networks
1) Location proposals 2) Patch-level sub-networks 3) Region-level 4) Subject-level
Skipped connection Conv: Convolution
(PSN) (shared weights) sub-networks sub-network
Fig. 15.12 Illustration of the hierarchically fully convolutional network (H-FCN) [10] for joint atrophy localization
and disease diagnosis. (Image courtesy to [10])
Fig. 15.13 Voxel-level AD heatmaps for discriminative patches automatically identified by H-FCN [10] in six different
subjects. Warmer color in each heatmap indicates higher discriminative capacity. (Image courtesy to [10])
15.3.8 Brain Image Segmentation contextual information. Features extracted from

the two parallel pathways were concatenated and
As mentioned, segmentation of tissues or lesions processed by two fully connected layers to predict
in brain images could be conducted either by the label of the patch centroid. Additionally, on
patch-wise classification (predicting the label of top of the CNNs’ soft segmentation maps, fully
the patch centroid) or by FCN-based models that connected conditional random field (CRF) model
directly generate dense output for segmentation was used for final post-processing. DeepMedic
labels. DeepMedic [32] is a representative patch- model achieved top rankings in two brain le-
wise classification method for brain tumor seg- sion segmentation challenges ISLES2015 [38]
mentation, with architecture shown in Fig. 15.14. and BRAT2015 [39]. Example segmentation re-
This method proposed a two-pathway 3D CNN sults are given in Fig. 15.15.
architecture to capture multi-scale features that Rather than classifying the centroid of each
incorporate both the local and the contextual in- patch, another type of CNN-based segmentation
formation to improve brain tumor segmentation. approaches directly produce dense outputs corre-
As shown in Fig. 15.14, the inputs of the two sponding to the segmentation labels. For exam-
pathways are 3D patches at the same image lo- ple, in [6], a 3D U-net architecture was proposed
cation but with different resolutions. The normal- for brain tumor segmentation (see Fig. 15.16).
resolution one focused on the local information, Just like U-net, this model consists of a context
while the low-resolution one was extracted from aggregation pathway to extract the abstract rep-
a down-sampled version of the image, providing resentation of the input large 3D blocks and a
444 D. Shen et al.
Fig. 15.14 Network architecture of DeepMedic. The inputs of the two parallel pathways are centered at the same image
location but with different resolutions. (Image courtesy to [32])
Fig. 15.15 Brain tumor segmentation examples (eval- dicates edema; orange indicates non-enhancing core; and
uated on the training set of BRATS2015 dataset) by red indicates enhancing core. (Image courtesy to [32])
DeepMedic [32]. Cyan indicates necrotic core; green in-
localization pathway that localizes the structures tion results at different layers in the localization
of interest based on combined features from shal- pathway. Meanwhile, the proposed model also
low layers. The context pathway is constructed benefits from the objective function of Dice loss
by residual blocks, while the localization path- as well as data augmentation. It achieves promis-
way is constructed by deconvolutional blocks for ing results on both BRATS2015 and BRATS2017
upsampling. Upon all convolution computation, datasets. Figure 15.17 shows an example segmen-
Leaky ReLu is used as the activation function tation result achieved by [6]. Moreover, based
for non-linearity. The final output of the network on the segmentation mask, the work in [6] could
is the element-wise summation of the segmenta- further extract imaging-based features (such as
Fig. 15.16 Network architecture of [6]. (Image courtesy to [6])
Fig. 15.17 A visual example of brain tumor segmentation. (Image courtesy to [6])
shape and first-order statistics) for survival pre- hold. For example, the prediction of a word in a
diction. Another state-of-the-art multi-scale FCN sentence usually depends on which words came
method was proposed in [8] for segmentation of before it. Recurrent neural networks (RNNs) ad-
perivascular spaces (PVSs) in brain images. dress this issue. RNNs are called recurrent be-
Furthermore, ensemble methods integrating cause they perform the same task for every el-
different segmentation networks (e.g., 3D ement of a sequence (with the output being de-
FCN [28], 3D U-net [40] and DeepMedic [32]) pended on the previous computations) and they
and trained with different loss functions and have a “memory” which captures information
normalization schemes could further improve about what has been calculated so far [42]. As
the segmentation performance over individual shown in Fig. 15.18 (with a fold and an unfold
models, as demonstrated in [41]. structure), the basic RNN model consists of a
sequence of non-linear units, and at least one
connection of units forms a directed cycle. This
15.4 Recurrent Neural Networks chain-like nature reveals that RNNs are intimately
related to sequences and lists. RNNs allow us to
15.4.1 Recurrent Neural Networks operate over sequences of vectors: sequences in
(RNNs): Basic Model the input, the output, or in the most general case
both.
Traditional neural networks assume that all in- A typical RNN consists of three types of lay-
puts (and outputs) are independent of each other. ers (see Fig. 15.18): (1) input layer, (2) recurrent
But for many tasks, this assumption may not hidden layers, and (3) output layer. The input
446 D. Shen et al.
Fig. 15.18 Architecture of the basic recurrent neural net- units in the hidden layer, and V is the weight matrix that
work (RNN). Each x is an input example, U is the weight connects the hidden and output layers. For simplicity, bias
matrix between the input and hidden layers, W is the terms are not shown in this figure
weight matrix between the previous and current hidden
layers is a sequence of vector through time step t, where σo and bo denote the activation function
i.e., {. . . , xt−1 , xt , xt+1 , . . . }, where xt is the input and bias term of the output layer. To learn net-
vector. The input units are fully connected with work parameters (W , U , V , bh , bo ), RNNs use
the units in the hidden layers via a weight matrix the backpropagation algorithm for network train-
U. ing.
The hidden layers are connected with each
other through time via recurrent connections, and
W is the weight matrix between the previous 15.4.2 Long Short-Term Memory
and current hidden units of the layer. With the (LSTM) Model
recurrent hidden layers, RNNs can obtain the
state space or “memory” as follows: Even though RNNs with recurrent connections
are capable of understanding sequential depen-
ht = σh (W xt + U ht−1 + bh ), (15.2) dencies, the backpropagation is usually time-
consuming and falls victim to exploding and
where ht and ht−1 denote the hidden state at time vanishing gradient during network training.
steps t and t − 1, respectively, bh is the bias Thus, in practice, RNNs don’t seem to be able
vector of the hidden units, and σh is the activation to learn “long-term dependencies” from data.
function used in the hidden layer. The weight Many methods have been designed to address
matrices W and U are filters that determine how this problem. Among these methods, long short-
much importance to accord to both the present term memory (LSTM) network, introduced by
input and the past hidden states. The error they Hocheriter et al. [43], is the most popular and
generate will return via backpropagation and be efficient method, capable of learning “long-term
used to adjust their weights until error cannot go dependencies.”
any lower. Because the feedback loop occurs at As shown in Fig. 15.19a, a typical LSTM
every time step in the series, each hidden state model consists of a memory cell Ct , an input gate
contains traces not only of the previous hidden it , an output gate ot , and a forget gate ft for the
state but also of all those that preceded ht−1 for time step t. The memory cell transfers relevant
as long as memory can persist. information all the way down the sequence chain,
The output units are connected with the hidden and these gates control the activation singles
units via a weight matrix V , and the output can be from various sources to decide what information
computed as follows: to add to or remove from the memory cell. The
input gate it , the output gate ot , and the forget
ot = σo (V ht + bo ), (15.3) gate ft of an LSTM at time step t are defined as
follows:
Fig. 15.19 Architectures of (a) long short-term memory the previous and current hidden units in the hidden layer,
(LSTM) network and (b) gated recurrent unit (GRU). Each and V is the weight matrix that connects the hidden and
x is an input example, U is the weight matrix between the output layers. For simplicity, bias terms are not shown in
input and hidden layers, W is the weight matrix between this figure
it = σ (W i xt + Ui ht−1 + bi ), It combines the forget and input gates into a

single “update gate.” It also merges the cell state
ot = σ (W o xt + Uo ht−1 + bo ), (15.4) and hidden state, making each recurrent unit to
ft = σ (W f xt + Uf ht−1 + bf ), adaptively capture dependencies of different time
scales. The resulting model is simpler than stan-
where W ∗ and U ∗ are weight matrices from one dard LSTM models, with an illustration shown in
state to the corresponding gate and b∗ is the bias Fig. 15.19b. The activation ht in GRU at time step
term. The memory cell Ct is updated by forgetting t is linearly modeled as
the existing memory and adding the new memory
content C̃t as follows: ht = (1 − zt )· ht−1 + zt · h̃t , (15.8)
Ct = ft Ct−1 + it C̃t , (15.5) where the update gate zt and the candidate activa-
tion h̃t are defined as
where the new memory content C̃t is defined as
zt = σ (W z xt + Uz ht−1 + bz ),
C̃t = tanh(W c xt + Uc ht−1 + bc ). (15.6) (15.9)
h̃t = tanh(W h xt + U h (ht−1 · rt ) + bn ),
As can be seen, the existing memory and the new
where the term rt = σ (W r xt + U r ht−1 + br )
memory are modulated by the forget data ft and
denotes the reset gate. The update gate decides
the input data it , respectively. The hidden state is
how much information to add and throw away,
finally computed as
and the reset gate decides how much previous
ht = ot tanh(Ct ). (15.7) information to forget. Detailed comparisons be-
tween LSTM and GRU can be found in [45, 46].
Unlike conventional RNN models, LSTM is
able to decide whether to preserve the existing
memory by the above-introduced gates. Theoret- 15.4.3 RNN Applications to Time
ically, if LSTM learns an important feature from Series Data Analysis
the input sequential data, it can keep this feature
over a long time, thus capturing potential long- RNNs have shown their advantage in exploiting
term dependencies. the temporal information in various tasks, such as
A popular LSTM variant, called gated recur- disease diagnosis and object detection [12,47,48].
rent unit (GRU), is introduced by Cho et al. [44]. In the following, we introduce an example of
448 D. Shen et al.
RNN models used for brain image classification The dense layer is followed by a softmax
based on time series data. layer to get the final classification result. The
Brain functional connectivity (FC) extracted proposed Full-BiLSTM method demonstrates
from resting-state fMRI (RS-fMRI) has become good performance when evaluated on a rigorously
a popular approach for disease diagnosis, where built large-scale multi-site database (i.e., with
discriminating subjects with mild cognitive 164 RS-fMRI scans from HCs and 330 scans
impairment (MCI) from normal controls (HC) from MCIs), achieving an accuracy of 73.6% in
is still one of the most challenging problems. the task of MCI vs. HC classification.
Dynamic functional connectivity (dFC) charac-
terizes “chronnectome” diagnostic information
for improving MCI classification, consisting of 15.5 Auto-encoder
time-varying spatiotemporal dynamics. In [12],
a fully connected bidirectional LSTM model The above-mentioned CNNs and RNNs can be
(called Full-BiLSTM) is designed to learn the treated as supervised deep learning models, since
periodic brain status changes using both past and they require labeled data for network training.
future information for each brief time segment However, the acquisition of these ground truth
(of blood oxygen-level-dependent signal of labels needs massive human efforts from experts
distributed brain regions) for MCI identification. and considerable time cost for manual annotation.
The architecture of Full-BiLSTM is shown in Therefore, many unsupervised deep feature learn-
Fig. 15.20. As can be seen from this figure, the ing models, such as stacked auto-encoder [7, 49],
outputs of every repeating cell are concatenated deep belief networks [50], and deep Boltzmann
into a dense layer (i.e., “concatenation layer”). machine [51], have been proposed to mitigate this
With this dense layer, one can abstract a common issue by using unlabeled training data. In addi-
and time-invariant dynamic transition pattern tion, these unsupervised models learn imaging
from all the BiLSTM cells which may represent features without knowing the exact analysis tasks
a constant “trait” information of each subject. in advance and directly capture the visual clues
Fig. 15.20 Overview of the fully connected bidirectional LSTM model (Full-BiLSTM) for mild cognitive impairment
(MCI) classification. (Image courtesy to [12])
that would be robust for different analysis tasks the medical cases where only scarce labeled im-
from brain images. This means that, feature rep- ages are available in clinic and research. Second,
resentations extracted by unsupervised learning the generated low-dimensional features largely
methods could have good capacity of generaliza- reduce the complexity of the learning task and
tion for the subsequent complex analysis. Among benefit the subsequent analysis.
them, the deep variants of auto-encoder (AE) [52] However, due to the simple and shallow struc-
have been widely applied to brain image analysis ture of the original AE models, they have limited
and achieved promising results. power to capture the complicated non-linear pat-
A typical auto-encoder (AE) model contains terns from the input data. To address this prob-
an encoder to first transform the input into its lem, deep stacked auto-encoders (SAEs) are con-
low-dimensional latent representation space and structed to improve the representational power.
a decoder to reconstruct the initial data from the Specifically, SAEs organize AEs on top of each
representation by closing the distance between other by using the hidden features from one AE
the input and the output. Once the models are as the input of the successive AE, as shown in
well trained, the latent representations can be Fig. 15.22. Similar to training the original AE
leveraged as the extracted features in the follow- models, SAEs could be trained by directly opti-
ing tasks. Originally, the AE model consists of mizing the parameters of all layers at the same
two layers for its encoder and decoder, respec- time. However, this training approach could eas-
tively [52]. The first layer maps the input data x to ily lead these parameters to be stuck in local opti-
its feature representation h by a specific function mum, reducing the stability of SAEs. Therefore,
h = σ (Wx x+bx ), where Wx and bx are trainable
parameters and σ is an activation function. The
second layer decodes h to the output y by the
mapping y = σ (Wh h + bh ) with the parameters
Wh and bh . The AE is trained to minimize the
reconstruction loss to optimize parameters Wx ,
bx , Wh , and bh , as follows:
{Wx , bx , Wh , bh }

= arg min Distance(x, y).
{Wx ,bx ,Wh ,bh }
Figure 15.21 illustrates the basic structure of

an AE model. With this structure, AE models
have at least two prominent advantages in feature
learning [53]. First, they can be applied as feature
extractors without any training labels, which fits Fig. 15.22 SAE model
Fig. 15.21 Original AE

model
450 D. Shen et al.
a greedy layer-wise strategy is employed to train disease (AD) and its prodromal stage, i.e., mild
SAEs [54]. It gradually optimizes the layers in cognitive impairment (MCI). However, training
SAEs one by one: when training the l-th layer, a high-quality deep model requires sufficient
the former pre-trained l − 1 layers only need labeled brain images which are not always
fine-tuning. In this way, SAEs can benefit from accessible. The work in [56] gives an SAE-based
their deep architecture and derive the high-quality approach to mitigate this issue. Figure 15.23
hierarchical hidden patterns from the input. illustrates a diagram of the proposed approach
In addition, to better apply SAEs to unsu- for brain image classification. Specifically, it first
pervised image feature extraction, the conven- extracts the traditional handcrafted features as the
tional MLP layers in SAEs are replaced by 2D low-level representations of input brain images.
or 3D convolutional layers as stacked convolu- Then, a deep SAE is trained to reconstruct
tional auto-encoders (SCAEs) [55]. By building a the low-level features in a greedy layer-wise
symmetrical architecture of CNNs, SCAEs could manner without using disease labels. After
learn the localized features of image structures. this unsupervised learning, SAE can efficiently
Furthermore, the pre-trained SCAEs can also be discover the hidden representation of these target-
used to better initialize a CNN model of the same unrelated samples and be applied to initialize
architecture before supervised learning. another deep model for supervised classification.
This deep model is then fine-tuned by the labeled
15.5.1 AE Applications to Feature samples to extract their deep features of brain
Learning in Brain Image images. The extracted deep features and the
Analysis low-level features are concatenated to select the
best features for disease diagnosis. Finally, the
15.5.2 Brain Image Classification selected features are passed to a support vector
machine (SVM) model for AD/MCI diagnosis.
As mentioned in Sect. 15.3.6, supervised Figure 15.24 compares the diagnosis results by
learning- based deep models are usually lever- separately using low-level features (LLF), SAE
aged to classify MRI and PET images of patients features (SAEF), and the fused LLF and SAEF.
for early diagnosis and prognosis of Alzheimer’s Since this work is among the first attempts to use
Fig. 15.23 An illustration

for AD/MCI diagnosis
in [56]. (Image courtesy
to [56])
Fig. 15.24 Comparison of different feature extraction methods for AD, MCI, MCI converter (MCI-C), MCI non-
converter (MCI-NC), and healthy normal control (HC) classification. (Image courtesy to [56])
deep learning for feature representation in brain SCAEs could be directly applied to different
disease diagnosis and prognosis in 2013, its SAE types of medical images, such as 1.5-T MR and
is simply constructed with MLP layers rather 7.0-T MR brain images. In [9], the SCAEs (for
than convolutional layers. Thus, the extracted extracting deep features) are cooperated with
SAEF performs worse than LLF in this case. the existing HAMMER registration framework,
These reported results still validate that the use of denoted as “H + DP”. Another two registra-
SAEF could improve the diagnosis performance tion approaches, i.e., image intensity-based
by only using traditional LLF. Demons [57] and handcrafted feature-based
HAMMER [58], are used to evaluate its effective-
ness. A visual example of their registration results
15.5.3 Brain Image Registration on 7.0-T MR brain images is shown in Fig. 15.25,
where the manually labeled hippocampus on
Deformable image registration, which aims to the template image and the deformed subject
register medical images to a target template hippocampi achieved by different registration
for anatomical alignment, is an important pre- methods are indicated through red and blue
process for various brain image analysis tasks. contours, respectively. As observed, H + DP
For more accurate registration, better features achieves most accurate registration results, while
should be extracted to reflect the intrinsic local Demons almost fails to register these 7.0-T MR
characteristics of both brain images and template. images. These results demonstrate the power of
To satisfy this target, [9] proposed a SCAEs feature representations learned by the SCAEs
model to capture low-dimension anatomical model for brain image registration.
latent representations of brain MR images via
unsupervised feature learning. After training the
SCAE model in a greedy layer-wise manner, the 15.6 Generative Adversarial
extracted deep imaging features from its encoder Networks
could be directly utilized as the input of existing
image registration frameworks. Therefore, the The recent occurrence of generative adversarial
hierarchical features can be learned without using networks (GANs) has well tackled many
manually annotated labels, and the constructed challenging problems in brain image analysis.
452 D. Shen et al.
Fig. 15.25 Typical registration results on 7.0-T MR brain different slices in the template, subject, and registered
images by Demons, HAMMER, and HAMMER with subjects. (a) Template. (b) Subject. (c) By Demons. (d)
SCAEs features (H + DP). Three rows represent three By HAMMER. (e) By H + DP. (Image courtesy to [9])
Fig. 15.26 Original GAN model
GANs have been used for a wide variety of one beating the other. Specifically, the generator
applications such as brain lesion detection and tries to produce fake samples that resemble the
segmentation, brain image registration, brain real ones to fool the discriminator, while the
image reconstruction and super-resolution, cross- discriminator struggles to tell the fake samples
modality brain image synthesis, etc. GANs were from the real ones. Through the competition,
originally proposed as generative models for both the generator and the discriminator could
unsupervised learning, which focus on learning improve their models for better performance.
the distribution of given data and therefore can
generate new samples from the learned distribu-
tion. That is, given the input data x, GANs focus 15.6.1 Principle of GAN
on learning the probability P (x). This is different
from the discriminative models, such as CNNs In 2014, the original GANs were first proposed
used for classification, which focus on classifying for the generic image synthesis tasks [59]. Differ-
the input data, i.e., learning P (y|x), where y ent from the common CNN-based deep learning
indicates class labels. The key idea of GANs is models, a GAN model consists of two agents, i.e.,
adversarial training. It refers to the simultaneous a generator G and a discriminator D, which are
training of two agents in a GAN model, i.e., a trained by adversarial learning, as illustrated in
generator and a discriminator, with the goal of Fig. 15.26. Given a training set of real samples
X with the distribution Pdata , the goal of the D is trained to learn an embedding function D(·)
generator is to learn an embedding function G(·) to maximize the probability of the correct label
that transforms the random inputs z (drawn from assignment to discriminate the real and the fake
the distribution Pnoise ) to the output synthetic samples. The generator and the discriminator play
images G(z) whose distribution matches the data a two-player minmax game with the following
distribution Pdata . Meanwhile, the discriminator objective function
arg min max V (G, D) = Ex∼pdata (x) [log (D(x)] + Ez∼pnoise (z) [log (1 − D(G(z))], (15.10)
G D
where the symbol E denotes mathematical expec-

tation. With the condition variable c, the objective of
In addition, prior information could also be in- cGANs becomes
corporated via conditional GANs (cGANs) [60].
arg min max V (G, D) =Ex∼pdata (x) [log (D(x|c)] (15.11)

G D
+ Ez∼pnoise (z) [log (1 − D(G(z|c))].
When the prior information is an input image real target image y. At the same time, the
x ∼ pdata (x), cGANs can be trained for paired discriminator D is trained to differentiate
image-to-image translation. That is, generating between the fake image pair (x, G(x)) and the
the corresponding image y ∼ pdata (y) with the real image pair (x, y). The training loss of the
specific control from x. For example, when x is generator G is as follows:
an input brain image and y is the corresponding
segmentation map of x, the cGAN model can be cGAN = Ex∼Pdata (x) [log (1 − D(x, G(x)))
LG
trained for segmentation tasks.
+ λl1 Ex,y∼Pdata (x,y) [y − G(x)1 ],
(15.12)
where the first term in Eq. 15.12 is the common
15.6.2 GAN Variants adversarial loss of a generator as in the orig-
inal GANs. In addition, Pix2Pix also enforces
15.6.3 Pix2Pix GANs the pixel-wise similarity between the generated
image and the real image, as described in the
Many GAN models used in medical image second term. Here · 1 indicates the l1 norm,
analysis [14, 61–63] follow the image translation i.e., the average absolute pixel-wise difference
framework Pix2Pix proposed in [64] and achieve between G(x) and y. The hyper-parameter λl1 is
promising results. As a cGAN model, given a user-defined to balance these two terms.
source image x, the generator G in Pix2Pix The loss function of the discriminator D is
produces an image G(x) that resembles the defined as
cGAN = − Ex,y∼Pdata (x,y) [log D(x, y)]

LD
(15.13)
− Ex∼Pdata (x) [log (1 − D(x, G(x)))].
454 D. Shen et al.
By minimizing Eq. 15.13, the discriminator D is That is, the training data consists of paired im-
trained to assign the correct labels (0 or 1) to the ages {x, y}, where x and y are the corresponding
fake or the real image pairs. samples in the two domains X and Y, respectively.
As image generation and image discrimination This requirement could be relaxed by cycle GAN
are trained together, the final loss function that (CycleGAN) [65] that only needs unpaired train-
integrates the objectives of G and D becomes ing data to learn the mapping between images in
two domains, such as computerized tomography
LcGAN = LG
cGAN + LcGAN .
D
(15.14) (CT) and MR images. The needs of paired images
are eliminated by learning two mappings X → Y
In Pix2Pix, CNN architectures are used for
and Y → X simultaneously and enforcing a cycle
both the generator and the discriminator to extract
consistency loss during training. CycleGAN con-
powerful deep features. Especially, the generator
sists of two generators – G to learn the mapping
uses a U-net-like architecture to utilize the hier-
X → Y and F to learn the mapping Y → X –
archy of contextual information for image gen-
as well as two discriminators, DX to differentiate
eration. The discriminator follows the common
x̂ = F (G(x)) from x and DY to differentiate ŷ =
CNNs used for classification.
G(F (y)) from y. The basic idea is illustrated in
Fig. 15.27. The objective function of CycleGAN
15.6.4 CycleGAN
is as follows:
The Pix2Pix model transforms images between
two domains with one-to-one correspondence.
L(G, F, DX , DY ) = LG,D
GAN
Y ,X,Y G,DX ,Y,X
+ LGAN + λLG,F
cycle , (15.15)
where the first and the second terms are the ad-
versarial loss of GANs and the third term is the
cycle consistency loss, which is defined as
LG,F
cycle = Ex∼Pdata (x) [F (G(x)) − x1 ] + Ey∼Pdata (y) [G(F (y)) − y1 ]. (15.16)
As can be seen, the forward cycle x → G(x) → example, a 3D cGAN model was proposed in [13]
F (G(x)) ≈ x should be able to bring x back. to synthesize full-dose brain positron emission to-
Similarly, the backward cycle y → F (y) → mography (PET) images from the low dose ones.
G(F (y)) ≈ y should be able to bring y back. PET imaging reveals the metabolism processes of
In this way, the samples x and y in two domains human and is widely exploited in clinics and re-
do not need to have one-to-one correspondence in search. During PET scanning, radioactive tracers
CycleGAN. are injected into the patient’s body for imaging.
Usually, a full dose of radioactive tracer is needed
15.6.5 GAN Applications to Brain to generate PET images of diagnostic quality. The
Image Analysis exposure to radiation inevitably brings healthy
concerns, especially for those patients who have
15.6.6 Brain Image Synthesis to undertake multiple scanning during their treat-
ment. On the other hand, lowering the dosage of
GANs have been widely used for brain image tracers could significantly reduce the quality of
synthesis either within the same imaging modal- PET images, as shown in Fig. 15.28. Therefore,
ity or across different imaging modalities. For the work in [13] proposed to fill the gap between
Fig. 15.27 Illustration of

CycleGAN for unpaired
image-to-image
translation. (Image
courtesy to [65])
GANs have also been intensively studied

for cross-modality brain image synthesis. For
example, when setting different scanning param-
eters, MRI can generate multi-modality images
(e.g., T1-weighted, T2-weighted, and FLAIR)
to reflect soft tissues with different contrast,
providing complementary information for disease
diagnosis [68] and treatment planning [69].
Cross-modality MR image synthesis is therefore
often needed to deal with the potential modality
loss in clinics so that the diagnosis could benefit
Fig. 15.28 Comparison between a low-dose PET (L- from the enriched information in the multiple
PET) image and its corresponding full-dose PET (F-PET).
(Image courtesy to [13])
imaging modalities [70, 71]. Such tasks could be
more challenging than the synthesis within the
same imaging modality. Meanwhile, generating
the low-dose PET images and the full-dose ones the images of new modality is often not the end:
by using a 3D conditional GAN (cGAN) model. these images are expected to well preserve the
The overview of the proposed method is given pathology that is critical for the subsequent
in Fig. 15.29. It follows the Pix2Pix framework, analysis. For example, when the brain images
where the generator is a U-net-like structure and contain tumors, the boundaries of tumors are
the discriminator is a CNN-based classifier to expected to be well depicted in the generated
differentiate the real and the fake image pairs. images. To develop pathology-centered synthesis,
Different from Pix2Pix and many other cGAN- different regularization terms have been proposed
based medical image synthesis models [61, 63, and embedded into the learning process of GANs
66, 67] that use 2D slices of a PET image as models. For example, the work in [14] proposed
the input, the cGAN model [13] is completely edge-aware conditional GAN models (Ea-GANs)
3D. It takes 3D patches of PET images as in- to enforce the preservation of edges for cross-
put and processes them with 3D up- and down- modality MR image synthesis to assist brain
convolutions. In this way, it mitigates the problem tumor segmentation. In addition to enforcing the
of discontinuous estimation across slices, which pixel-wise intensity similarity as in Pix2Pix, Ea-
is however often observed in 2D-based synthe- GANs also require that the edge maps extracted
sis models. A visual comparison of the results from the synthesized images should resemble
using 2D and 3D cGAN models, respectively, those from the real images. These edge maps
is given in Fig. 15.30. As shown, the full-dose are computed by the commonly used Sobel
PET images synthesized by 3D cGAN show high filters. As shown in Fig. 15.31, the work in [14]
image quality in all three views. However, three proposed two frameworks to incorporate the edge
2D cGAN models only produce good results in maps, i.e., a generator-induced Ea-GAN (gEa-
their corresponding trained views as indicated GAN) and a discriminator-induced Ea-GAN
in the red circles, but not along with the other (dEa-GAN). In gEa-GAN, the edge maps are
two directions since they lose the 3D structural incorporated into the generator side only, while
information during the synthesis. in dEa-GAN, the edge maps are also introduced to
456 D. Shen et al.
Fig. 15.29 Framework of training a 3D conditional GAN (cGAN) to estimate the full-dose PET image from low-dose
counterpart. (Image courtesy to [62])
Fig. 15.30 Visual comparison between the results esti- corresponding axial, coronal, and sagittal views. (Image
mated by 2D cGAN and the 3D cGAN in [62]. These 2D courtesy to [62])
cGANs are separately trained with the 2D slices from the
the discriminator side, so that they participate the GANs using the edge information outperformed
adversarial training to help improve the synthesis the 3D cGAN in both the whole image and the
quality. As shown in Table 15.1, in a synthesis tumor areas. This is consistent with the visual
task from T1-weighted MRI to FLAIR MRI, Ea- comparison in Fig. 15.32.
Fig. 15.31 Frameworks of Ea-GANs proposed in [14]. train the discriminator to classify the triplets comprising
(Image courtesy to [14]). In gEa-GAN, the generator en- of the source-modality image, the real/generated target-
forces the similarity between the real and the generated modality image, and the edge map of the corresponding
images, as well as the similarity of their corresponding real/generated target-modality image
edge maps. In dEa-GAN, the edge maps are also used to
Table 15.1 Method comparison: synthesizing FLAIR-like images from T1-weighted images on the BRATS2015
dataset (mean)
Whole image Tumor part
Methods
PSNR NMSE SSIM PSNR NMSE SSIM
3D cGAN [72] 29.26 0.119 0.958 15.95 0.098 0.681
gEa-GAN [14] 29.55 0.115 0.960 16.37 0.090 0.697
dEa-GAN [14] 30.11 0.105 0.963 16.90 0.084 0.705
15.6.7 Brain Image Augmentation Neuroimaging Initiative (ADNI) database, which

limits the number of subjects available for the
Brain disease diagnosis benefits from multi- research. A common practice to deal with the
modality imaging data that provides comple- missing-modality problem is to impute the
mentary information. For example, the structural images of the missing modality. For example,
imaging MRI and the functional imaging PET in [15], a 3D CycleGAN model was proposed
have been widely used for the diagnosis of to impute the missing PET images by learning
Alzheimer’s disease (AD). However, the missing- the bidirectional mapping between MRI and
modality problem often occurs in clinic, for PET. Based on complete (after imputation) MRI-
example, patients taking MRI scanning may PET pairs, a multi-modal multi-instance learning
reject to also take PET scanning due to the method was further proposed for AD diagnosis.
concerns about the cost. Such a problem also The architecture of the 3D CycleGAN is shown
exists in the widely used Alzheimer’s Disease in Fig. 15.33. It consists of two generators to learn
458 D. Shen et al.
Fig. 15.32 FLAIR image synthesis from T1-weighted approaches. (a) Axial slices, (b) zoomed parts of axial
MR images: visual comparison between the results esti- slices, (c) coronal slices, (d) zoomed parts of coronal
mated by methods proposed in [14] and several competing slices, (e) sagittal slices, and (f) zoomed parts of sagittal
slices
Synthetic PET Real MRI

1/0 1/0
RNB
RNB
RNB
RNB
RNB
RNB
1 128 64 32 16 2 1 16 32 1 32 16 8 16 32 64 128 1 1
7×7×7 Conv
ResNet Block (RNB)
4×4×4 Conv
3×3×3 Conv
RNB
RNB
RNB
RNB
RNB
RNB
3×3×3 Deconv
32 32
Addition 8 16 32 32 16 1
Real PET Synthetic MRI
Fig. 15.33 The architecture of the 3D CycleGAN proposed in [15]. (Image courtesy to [15])
Fig. 15.34 Visual comparison on two subjects: synthetic PET images generated by 3D CycleGAN proposed in [15]
(top) vs. the corresponding real PET images (bottom). (Image courtesy to [15])
the mappings from MRI to PET and the mapping network blocks to transfer the features from the
from PET to MRI, respectively. Each generator source domain to the target domain (e.g., PET).
consists of three parts: encoding, transferring, The decoding part contains two deconvolutional
and decoding components. The encoding part layers and one convolutional layer to generate
consists of three convolutional layers to extract images in the target domain. Examples of the
features in the source domain (e.g., MRI). The synthesized PET images are shown in Fig. 15.34.
transferring part is constructed by six residual By integrating image synthesis and disease
diagnosis into a unified framework, this work brain image analysis, which could accelerate the
was further extended to a more advanced model acceptance of deep learning applications among
to generate disease-specific PET/MRI scans [73], clinicians and patients.
providing an exciting research direction for
synthesizing task-oriented neuroimages through
15.8 Conclusion
GANs.
In this chapter, deep learning models and their ap-
15.7 Discussion plications to brain image analysis are introduced.
Specifically, four typical deep learning models
Even though deep learning has achieved record- (i.e., CNN, RNN, AE, and GAN) and their appli-
breaking performance in brain image analysis, cations (i.e., brain image segmentation, brain im-
there are still several potential limitations to con- age registration, neuroimaging-based brain dis-
sider. ease diagnosis, and brain image synthesis) are
First, deep learning models usually have a introduced. Limitations of current deep learning
very high computational cost for network training models and possible future research directions are
because of the high dimensionality of input brain also discussed. It is expected that deep learning
images as well as the huge number of to-be- will have a great impact on brain image analysis.
optimized network parameters. Using GPUs with
higher computation power and designing models Homework
in a parallel way can partly address this issue. It
is also interesting to perform dimension reduction 1. In practice, we usually have only limited
for input brain images, by defining regions of number of brain images to train deep models
interests in the brain (empirically or in a data- for analysis. Please explain the problem and
driven manner) to reduce the negative influence list at least two strategies to deal with this
of uninformative regions [8]. situation.
Second, they generally require a large number 2. Consider the following MLP model with two
of training images for generating reliable models. hidden layers and the loss function shown in
The latest success of GAN models in the syn- ∂h1,i ∂J
Table 15.2. Please calculate ∂w 1,2
and ∂w 1,2
.
thesis of neuroimages has brought new solutions 3. What are the benefits to use CNN to analyze
to the augmentation of training samples. Transfer brain images, compared with MLP?
learning [33, 74], which can enable knowledge 4. What are the key difference and advantages
sharing between related tasks/domains, is also of the fully convolutional networks (FCNs)
an interesting solution that reduces the need for over convolutional neural networks (CNNs)
a large number of training samples required for in brain image segmentation (pixel-level pre-
network training. diction)?
In addition, deep learning models have often 5. Please describe the network structure of U-
been described as “black boxes,” without explic- net. What are the benefits to use skip connec-
itly articulating themselves in a certain way. In tions in U-net?
many neuroimaging-based applications, it is of-
ten not enough to have a good prediction sys-
tem. To understand what intermediate layers of Table 15.2 An MLP model with two hidden layers
convolutional networks are responding to, several Input x1,i , x2,i , where i = 1, · · · N and N is
the number of samples
strategies have been proposed, such as deconvo-
Layer 1 h1,i = max(w1,1 x1,i + w1,2 x2,i , 0),
lution networks [75], deep Taylor composition
h2,i = max(w2,1 x1,i + w2,2 x2,i , 0)
backpropagation [76], and Bayesian deep net- Layer 2 pi = w5 h1,i + w6 h2,i
works [77]. It is desired to develop new strategies
Loss J (p, w) = i (pi − yi )2
to further understand deep learning methods in
460 D. Shen et al.
6. What’s the difference between RNNs and work by unsupervised deep feature representations
CNNs? What is the advantage of LSTM over learning. IEEE Trans. Biomed. Eng. 63(7), 1505–
1516 (2015)
RNN?
10. C. Lian, M. Liu, J. Zhang, D. Shen, Hierarchical fully
7. The auto-encoder (AE) model can extract convolutional network for joint atrophy localization
imaging features in an unsupervised manner. and Alzheimer’s disease diagnosis using structural
What is the principle of AE? MRI. IEEE Trans. Pattern Anal. Mach. Intell. 42(4),
880–893 (2020)
8. What’s the purpose of using the generator and
11. M. Liu, J. Zhang, E. Adeli, D. Shen, Joint classi-
discriminator in GANs? fication and regression via deep multi-task multi-
9. What’s the advantage of CycleGAN over channel learning for Alzheimer’s disease diagno-
GAN? sis. IEEE Trans. Biomed. Eng. 66(5), 1195–1206
(2018)
10. Please list possible deep learning models that
12. W. Yan, H. Zhang, J. Sui, D. Shen, Deep chron-
could be used for the following brain image nectome learning via full bidirectional long short-
analysis tasks: (a) brain disease diagnosis, term memory networks for MCI diagnosis. In: In-
(b) brain lesion segmentation, (c) brain net- ternational conference on medical image computing
and computer-assisted intervention (Springer, 2018),
work analysis based on fMRI images, (d)
pp. 249–257
brain image transferring across modality, (e) 13. Y. Wang, L. Zhou, B. Yu, L. Wang, C. Zu, D.S.
brain disease diagnosis without labeled train- Lalush, W. Lin, X. Wu, J. Zhou, D. Shen, 3D auto-
ing samples, (f) brain image generation with- context-based locality adaptive multi-modality GANs
for PET synthesis. IEEE Trans. Med. Imaging 38(6),
out paired training samples.
1328 (2019)
14. B. Yu, L. Zhou, L. Wang, Y. Shi, J. Fripp, P. Bourgeat,
Ea-GANs: edge-aware generative adversarial net-
References works for cross-modality MR image synthesis. IEEE
Trans. Med. Imaging 38(7), 1750–1762 (2019)
15. Y. Pan, M. Liu, C. Lian, T. Zhou, Y. Xia,
1. Y. Bengio, A. Courville, P. Vincent, Representation
D. Shen, Synthesizing missing PET from MRI
learning: a review and new perspectives. IEEE Trans.
with cycle-consistent generative adversarial net-
Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
works for Alzheimer’s disease diagnosis. In: In-
2. Y. LeCun, Y. Bengio, G. Hinton, Deep learning. Na-
ternational conference on medical image comput-
ture 521(7553), 436 (2015)
ing and computer assisted intervention, pp. 455–463
3. J. Schmidhuber, Deep learning in neural networks: an
(2018)
overview. Neural Netw. 61, 85–117 (2015)
16. S. Sadaghiani, G. Hesselmann, K.J. Friston, A. Klein-
4. D. Shen, G. Wu, H.I. Suk, Deep learning in medical
schmidt, The relation of ongoing brain activity,
image analysis. Ann. Rev. Biomed. Eng. 19, 221–248
evoked neural responses, and cognition. Front. Syst.
(2017)
Neurosci. 4, 20 (2020)
5. J.G. Lee, S. Jun, Y.W. Cho, H. Lee, G.B. Kim, J.B.
17. V. Kiviniemi, T. Vire, J. Remes, A.A. Elseoud, T.
Seo, N. Kim, Deep learning in medical imaging:
Starck, O. Tervonen, J. Nikkinen, A sliding time-
general overview. Korean J. Radiol. 18(4), 570–584
window ICA reveals spatial variability of the default
(2017)
mode network in time. Brain Connect. 1(4), 339–347
6. F. Isensee, P. Kickingereder, W. Wick, M. Bend-
(2011)
szus, K.H. Maier-Hein, Brain tumor segmentation
18. M. Kudela, J. Harezlak, M.A. Lindquist, Assessing
and radiomics survival prediction: contribution to
uncertainty in dynamic functional connectivity. Neu-
the BRATS 2017 challenge. In: Brainlesion: glioma,
roImage 149, 165–177 (2017)
multiple sclerosis, stroke and traumatic brain injuries.
19. S. Sadaghiani, A. Kleinschmidt, Functional interac-
BrainLes 2017 (2018)
tions between intrinsic brain activity and behavior.
7. Y. Zhu, L. Wang, M. Liu, C. Qian, A. Yousuf, A. Oto,
NeuroImage 80, 379–386 (2013)
D. Shen, MRI-based prostate cancer detection with
20. J. Bernal, K. Kushibar, D.S. Asfaw, S. Valverde, A.
high-level representation and hierarchical classifica-
Oliver, R. Marti, X. Llado, Deep convolutional neural
tion. Med. Phys. 44(3), 1028–1039 (2017)
networks for brain image analysis on magnetic reso-
8. C. Lian, J. Zhang, M. Liu, X. Zong, S.C. Hung, W.
nance imaging: a review. Artif. Intell. Med. 95, 64–81
Lin, D. Shen, Multi-channel multi-scale fully convo-
(2019)
lutional network for 3D perivascular spaces segmen-
21. S.M. Smith, Fast robust automated brain extraction.
tation in 7T MR images. Med. Image Anal. 46, 106–
Hum. Brain Mapp. 17(3), 143–155 (2002)
117 (2018)
22. M. Jenkinson, M. Pechaud, S. Smith, BET2: MR-
9. G. Wu, M. Kim, Q. Wang, B.C. Munsell, D. Shen,
based estimation of brain, skull and scalp surfaces.
Scalable high-performance image registration frame-
In: Eleventh annual meeting of the organization for 37. J. Fan, X. Cao, P.T. Yap, D. Shen, BIRNet: brain
human brain mapping (2005) image registration using dual-supervised fully con-
23. J.G. Sled, A.P. Zijdenbos, A.C. Evans, A non- volutional networks. Med. Image Anal. 54, 193–206
parametric method for automatic correction of inten- (2019)
sity nonuniformity in MRI data. IEEE Trans. Med. 38. O. Maier, B. Menze, J. von der Gablentz, L. Häni, M.
Imaging 17(1), 87–97 (1998) Heinrich, M. Liebrand, S. Winzeck et al., ISLES 2015
24. D. Shen, C. Davatzikos, Hammer: hierarchical at- – a public evaluation benchmark for ischemic stroke
tribute matching mechanism for elastic registra- lesion segmentation from multi-spectral MRI. Med.
tion. IEEE Trans. Med. Imaging 21(11), 1421–1439 Image Anal. 35, 250–269 (2015)
(2002) 39. B. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer,
25. A. Sotiras, C. Davatzikos, N. Paragios, Deformable K. Farahani, J. Kirby, Y. Burren et al., The multi-
medical image registration: a survey. IEEE Trans. modal brain tumor image segmentation benchmark
Med. Imaging 32(7), 1153 (2013) (BRATS). IEEE Trans. Med. Imaging 36(10), 1993–
26. K. He, X. Zhang, S. Ren, J. Sun, Deep residual 2024 (2015)
learning for image recognition. In: Proceedings of 40. Ö. Çiçek, A. Abdulkadir, S.S. Lienkamp, T. Brox,
the IEEE conference on computer vision and pattern O. Ronneberger, 3D U-Net: learning dense volu-
recognition, pp. 770–778 (2016) metric segmentation from sparse annotation. In: In-
27. C. Szegedy, S. Ioffe, V. Vanhoucke, A.A. Alemi, ternational conference on medical image computing
Inception-v4, inception-resnet and the impact of and computer-assisted intervention (Springer, 2016),
residual connections on learning. In: Thirty-first pp. 424–432
AAAI conference on artificial intelligence (2017) 41. K. Kamnitsas, W. Bai, E. Ferrante, S. McDonagh,
28. J. Long, E. Shelhamer, T. Darrell, Fully convolutional M. Sinclair, N. Pawlowski, M. Rajchl, M. Lee, B.
networks for semantic segmentation. In: Proceedings Kainz, D. Rueckert, B. Glocker, Ensembles of multi-
of the IEEE conference on computer vision and pat- ple models and architectures for robust brain tumour
tern recognition, pp. 3431–3440 (2015) segmentation. In: International MICCAI brainlesion
29. O. Ronneberger, P. Fischer, T. Brox, U-net: convolu- workshop, 2017, pp. 450–462
tional networks for biomedical image segmentation. 42. J.L. Elman, Finding structure in time. Cogn. Sci.
In: International conference on medical image com- 14(2), 179–211 (1990)
puting and computer-assisted intervention (Springer, 43. S. Hochreiter, J. Schmidhuber, Long short-term mem-
2015), pp. 234–241 ory. Neural Comput. 9(8), 1735–1780 (1997)
30. G. Wang, W. Li, S. Ourselin, T. Vercauteren, Au- 44. K. Cho, B. Van Merriënboer, D. Bahdanau, Y. Ben-
tomatic brain tumor segmentation using cascaded gio, On the properties of neural machine translation:
anisotropic convolutional neural networks. In: In- encoder-decoder approaches (2014). arXiv preprint
ternational MICCAI brainlesion workshop, 2017, arXiv:1409.1259
pp. 178–190 45. J. Chung, C. Gulcehre, K. Cho, Y. Bengio, Em-
31. F. Yu, V. Koltun, T. Funkhouser, Dilated residual pirical evaluation of gated recurrent neural net-
networks. In: Proceedings of the IEEE conference works on sequence modeling (2014). arXiv preprint
on computer vision and pattern recognition, 2017, arXiv:1412.3555
pp. 472–480 46. K. Greff, R.K. Srivastava, J. Koutník, B.R. Ste-
32. K. Kamnitsas, C. Ledig, V.F. Newcombe, J.P. Simp- unebrink, J. Schmidhuber, LSTM: a search space
son, A.D. Kane, D.K. Menon, D. Rueckert, B. odyssey. IEEE Trans. Neural Netw. Learn. Syst.
Glocker, Efficient multi-scale 3D CNN with fully 28(10), 2222–2232 (2017)
connected CRF for accurate brain lesion segmenta- 47. H. Chen, Q. Dou, D. Ni, J.Z. Cheng, J. Qin, S. Li,
tion. Med. Image Anal. 36, 61–78 (2017) P.A. Heng, Automatic fetal ultrasound standard plane
33. M. Liu, J. Zhang, E. Adeli, D. Shen, Landmark- detection using knowledge transferred recurrent neu-
based deep multi-instance learning for brain dis- ral networks. In: International conference on medical
ease diagnosis. Med. Image Anal. 43, 157–168 image computing and computer-assisted intervention
(2018) (Springer, 2015), pp. 507–514
34. A.G. Roy, S. Conjeti, N. Navab, C. Wachinger, Quick- 48. B. Kong, Y. Zhan, M. Shin, T. Denny, S. Zhang,
NAT: a fully convolutional network for quick and Recognizing end-diastole and end-systole frames
accurate segmentation of neuroanatomy. NeuroImage via deep temporal regression network. In: Inter-
186, 713–727 (2019) national conference on medical image computing
35. M. Havaei, A. Davy, D. Warde-Farley, A. Biard, and computer-assisted intervention (Springer, 2016),
A. Courville, Y. Bengio, C. Pal, P.M. Jodoin, H. pp. 264–272
Larochelle, Brain tumor segmentation with deep neu- 49. H.C. Shin, M.R. Orton, D.J. Collins, S.J. Doran,
ral networks. Med. Image Anal. 35, 18–31 (2017) M.O. Leach, Stacked autoencoders for unsupervised
36. X. Yang, R. Kwitt, M. Styner, M. Niethammer, Quick- feature learning and multiple organ detection in a
silver: fast predictive image registration–a deep learn- pilot study using 4D patient data. IEEE Trans. Pattern
ing approach. NeuroImage 158, 378–396 (2017) Anal. Mach. Intell. 35(8), 1930–1943 (2012)
462 D. Shen et al.
50. Y. Bengio et al., Learning deep architectures for AI. 66. X. Yi, P. Babyn, Sharpness-aware low-dose CT de-
Found. Trends Mach. Learn. 2(1), 1–127 (2009) noising using conditional generative adversarial net-
51. R. Salakhutdinov, G. Hinton, Deep Boltzmann ma- work. J. Digit. Imaging 31(5), 655–669 (2018)
chines. In: Artificial intelligence and statistics, 2009, 67. L. Bi, J. Kim, A. Kumar, D. Feng, M. Fulham, Syn-
pp. 448–455 thesis of positron emission tomography (PET) images
52. H. Bourlard, Y. Kamp, Auto-association by multi- via multi-channel generative adversarial networks
layer perceptrons and singular value decomposition. (GANs). In: Molecular imaging, reconstruction and
Biol. Cybern. 59(4–5), 291–294 (1988) analysis of moving body organs, and stroke imaging
53. J. Ker, L. Wang, J. Rao, T. Lim, Deep learning ap- and treatment (Springer, 2017), pp. 43–51
plications in medical image analysis. IEEE Access 6, 68. M. Dadar, T.A. Pascoal, S. Manitsirikul, K. Misquitta,
9375–9389 (2017) V.S. Fonov, M.C. Tartaglia, J. Breitner, P. Rosa-Neto,
54. Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, O.T. Carmichael, C. Decarli et al., Validation of a
Greedy layer-wise training of deep networks. In: regression technique for segmentation of white matter
Advances in neural information processing systems, hyperintensities in Alzheimer’s disease. IEEE Trans.
2007, pp. 153–160 Med. Imaging 99, 1–1 (2017)
55. J. Masci, U. Meier, D. Cireşan, J. Schmidhuber, 69. M. Lê, H. Delingette, J. Kalpathy-Cramer, E.R. Gerst-
Stacked convolutional auto-encoders for hierarchical ner, T. Batchelor, J. Unkelbach, N. Ayache, Personal-
feature extraction. In: International conference on ized radiotherapy planning based on a computational
artificial neural networks (Springer, 2011), pp. 52–59 tumor growth model. IEEE Trans. Med. Imaging
56. H.I. Suk, S.W. Lee, D. Shen, Latent feature represen- 36(3), 815–825 (2017)
tation with stacked auto-encoder for AD/MCI diag- 70. D.H. Ye, D. Zikic, B. Glocker, A. Criminisi, E.
nosis. Brain Struct. Funct. 220(2), 841–859 (2015) Konukoglu, Modality propagation: coherent synthe-
57. T. Vercauteren, X. Pennec, A. Perchant, N. Ayache, sis of subject-specific scans with data-driven regu-
Diffeomorphic demons: efficient non-parametric im- larization. In: International conference on medical
age registration. NeuroImage 45(1), S61–S72 (2009) image computing and computer-assisted intervention
58. G. Wu, P.T. Yap, M. Kim, D. Shen, TPS-HAMMER: (Springer, 2013), pp. 606–613
improving HAMMER registration algorithm by soft 71. A. Jog, A. Carass, S. Roy, D.L. Pham, J.L. Prince,
correspondence matching and thin-plate splines based Random forest regression for magnetic resonance
deformation interpolation. NeuroImage 49(3), 2225– image synthesis. Med. Image Anal. 35, 475–488
2233 (2010) (2017)
59. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, 72. B. Yu, L. Zhou, L. Wang, J. Fripp, P. Bourgeat, 3D
D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, cGAN based cross-modality MR image synthesis for
Generative adversarial nets. In: Advances in neural in- brain tumor segmentation. In: 2018 IEEE 15th in-
formation processing systems, 2014, pp. 2672–2680 ternational symposium on biomedical imaging (ISBI
60. M. Mirza, S. Osindero, Conditional generative adver- 2018) (IEEE, 2018), pp. 626–630
sarial nets (2014). arXiv preprint arXiv:1411.1784 73. Y. Pan, M. Liu, C. Lian, Y. Xia, D. Shen, Disease-
61. P. Costa, A. Galdran, M.I. Meyer, M. Niemeijer, M. image specific generative adversarial network for
Abràmoff, A.M. Mendonça, A. Campilho, End-to- brain disease diagnosis with incomplete multi-modal
end adversarial retinal image synthesis. IEEE Trans. neuroimages. In: International conference on medical
Med. Imaging 37(3), 781–791 (2018) image computing and computer assisted intervention,
62. Y. Wang, B. Yu, L. Wang, C. Zu, D.S. Lalush, W. Lin, 2019, pp. 1–9
X. Wu, J. Zhou, D. Shen, L. Zhou, 3D conditional 74. B. Cheng, M. Liu, D. Zhang, B.C. Munsell, D. Shen,
generative adversarial networks for high-quality PET Domain transfer learning for MCI conversion predic-
image estimation at low dose. NeuroImage 174, 550– tion. IEEE Trans. Biomed. Eng. 62(7), 1805–1817
562 (2018) (2015)
63. S.U.H. Dar, M. Yurt, L. Karacan, A. Erdem, E. 75. M.D. Zeiler, R. Fergus, Visualizing and understand-
Erdem, T. Çukur, Image synthesis in multi-contrast ing convolutional networks. In: European conference
MRI with conditional generative adversarial networks on computer vision (Springer, 2014), pp. 818–833
(2018). arXiv preprint arXiv:1802.01221 76. G. Montavon, S. Lapuschkin, A. Binder, W. Samek,
64. P. Isola, J.Y. Zhu, T. Zhou, A.A. Efros, Image-to- K.R. Müller, Explaining nonlinear classification de-
image translation with conditional adversarial net- cisions with deep Taylor decomposition. Pattern
works (2016). arXiv preprint arXiv:1611.07004 Recogn. 65, 211–222 (2017)
65. J.Y. Zhu, T. Park, P. Isola, A. Efros, Unpaired image- 77. A. Kendall, Y. Gal, What uncertainties do we need
to-image translation using cycle-consistent adversar- in Bayesian deep learning for computer vision? In:
ial networks. In: IEEE international conference on Advances in neural information processing systems,
computer vision, 2017, pp. 2242–2251 2017, pp. 5574–5584
Neural Modeling
16
Michael N. Economo, Jad Noueihed, Joan J. Martinez,
and John A. White
Abstract processing, and synaptic communication. This

chapter also covers mathematical models that
The brain is extraordinarily complex, contain-
replicate basic neural behaviors through more
ing 1011 neurons linked with 1014 connec-
abstract mechanisms. We briefly explore ef-
tions. We can improve our understanding of
forts to extend single-neuron models to the
individual neurons and neuronal networks by
network level and provide several examples of
describing their behavior in mathematical and
insights gained through this process. Finally,
computational models. This chapter provides
we list common resources, including modeling
an introduction to neural modeling, laying the
environments and repositories, that provide
foundation for several basic models and sur-
the guidance and parameter sets necessary to
veying key topics. After some discussion on
begin building neural models.
the motivations of modelers and the uses of
neural models, we explore the properties of
Keywords
electrically excitable membranes. We describe
in some detail the Hodgkin-Huxley model, the Neural modeling · Nernst/reversal potential ·
first neural model to describe biophysically Membrane potential · Conductance ·
the behavior of biological membranes. We ex- Hodgkin-Huxley · Integrate-and-fire models ·
plore how this model can be extended to de- Propagation · Excitability · Synapse model ·
scribe a variety of excitable membrane behav- Plasticity model
iors, including axonal propagation, dendritic
16.1 Why Build Neural Models?

6_16) contains supplementary material, which is available Given the immense complexity of the brain and
to authorized users. nervous system, it seems reasonable to attempt to
understand its behavior by building and studying
Michael N. Economo and Jad Noueihed contributed computational models. The approaches used fall
equally with all other contributors.
along a continuum between two extremes. At one
M. N. Economo · J. Noueihed · J. A. White () extreme, practitioners attempt to include every
Department of Biomedical Engineering, Boston relevant detail, in essence trying to build an in
University, Boston, MA, USA
silico representation of the full neural system that
e-mail: [email protected]; [email protected]; [email protected]
can then be monitored, dismantled, and altered
J. J. Martinez
Technology Ventures, Columbia University, New York
at will. Although this approach is philosophically
City, NY, USA attractive, it is impossible to build a truly accu-
e-mail: [email protected] rate and complete model of even a single neuron

464 M. N. Economo et al.
within the foreseeable future, because so many of 2. Fitting experimental data. A model with a
the critical parameters for such a complete model tractable number of parameters can be used ex-
are very difficult, or even impossible, to measure. tremely effectively to fit experimental data and
Such factors include the dendritic densities of thus to make those data useful more broadly.
channels, the precise neuromodulatory states of If a given class of model can’t fit trustworthy
the channels, and the spatiotemporal distributions data, then one is in a good position to reject
of a vast number of second messengers and cy- that class of model and posit a new class.
toskeletal elements. 3. Understanding the implications of collected
At the other extreme lie modelers who pur- data. We as experimentalists often collect
posefully and unapologetically construct over- our data using deliberately simplified stimuli.
simplified models. This approach allows one to Computational models can represent an im-
build a model with a fairly limited number of free portant step in understanding the implications
parameters and thus a model that can potentially of a given biophysical mechanism for neural
be understood fully and in detail. However, by data processing under more general conditions
construction, such simple models leave out a host than in the original experiment.
of presumably important details. Modelers taking 4. Guiding novel experiments. For many labo-
this approach are criticized for picking the impor- ratories, neural models represent a rigorously
tant details, rather than letting the model tell them stated hypothesis, hopefully with clear im-
what is important. In the worst cases, such models plications that can be tested experimentally.
can often legitimately be criticized as having been In this way, the refinement of computational
“rigged” to give a particular answer, without pro- models via feedback from experiments repre-
viding meaningful insight into the mechanisms. sents a particularly rigorous form of the scien-
As in most ideological arguments, day-to-day tific method. We are well behind the physics
neural modelers lie at some point on this contin- community in the degree of productive inter-
uum. Most who take more detailed approaches action between experiments and models, but
are aware that they, too, make many unavoid- we are making progress.
able choices in building the model. Most of the 5. Testing one’s understanding of a proposed
detail-focused work is at the cellular level, but the mechanistic explanation. It is one thing to have
best practitioners of detailed network models are a basic understanding of the Carnot cycle; it
meticulous in exploring the relevant parameter is quite another to build an efficient internal-
spaces and in trying to draw general conclusions combustion engine. Building a computational
from complex simulations. The best of the work model that can reproduce data or perform a
with more reduced models avoids the trap of task is often a strong test of our understanding
“rigging” the model to give an inescapable result of a neural circuit. Often, we find that our
and instead produces a result that the community less-than-rigorous hypothesis is inadequate
would not have guessed a priori. No matter what in some fundamental way. An even more
the style of the model, the most useful modeling rigorous test comes in building a physical,
work makes non-trivial predictions and is experi- rather than simulated, version of the model.
mentally falsifiable.
Regardless of the style and choices of the mod- In building a model, one’s methods should
eler, what are the purposes of neural modeling? meet one’s goals. Conductance-based models
Below is a partial list: of the Hodgkin-Huxley form (Sects. 16.3, 16.4,
16.5, and 16.6) are superb for understanding the
1. Keeping track of complex nonlinear interac- effects of a novel set of ion channels on neuronal
tions. Even many of the simplest neural mod- integration, but these models are known to be
els are highly nonlinear, making it difficult wrong in some of the fine details and are not
to reason through multiple interactions. Com- adequate to describe the gating (turning off and
putational models allow us to examine such on) of single ion channels. Integrate-and-fire and
interactions, quantitatively and qualitatively. similar models (Sect. 16.7) have the immense
16 Neural Modeling 465
advantage of mathematical tractability, but they tion 16.3 describes the manner in which distinct
are too abstract for many studies of membrane ion fluxes are controlled by excitable cells in
mechanisms. Commonly, the most detailed order to integrate, process, and transmit electrical
portions of a model represent the most detailed signals.
and trustworthy data and/or the mechanism of
particular interest of the study.
16.2.2 Equivalent Circuit
Representation
16.2 Basic Properties of Excitable
Membranes We first consider a small isopotential cell with
a lipid bilayer membrane. In this case, there is
16.2.1 Membrane Properties no spatial variation in electrical potential within
the cell or in the extracellular space. The passive
Excitable cells, including neurons in the nervous properties of this cell can be easily represented
system, cardiac myocytes in the heart, and beta using elements common in electrical circuits.
cells of the pancreas, carry information in the The circuit analogs of cellular components and
voltage difference across their membranes. The their contribution to the flow of ionic currents are
lipid membrane of a cell separates the charged described.
ions within the cell from charged ions in the
extracellular medium. The collection of ions near 16.2.2.1 Membrane Capacitance
both sides of the membrane creates an electric The lipid bilayer of a neuron acts as a thin, insulat-
potential difference across the cell membrane. ing membrane, separating positive and negative
Static in non-excitable cells, this electrical charges and endowing it with an intrinsic capac-
potential changes dynamically in their excitable itance. The capacitance of a small isopotential
counterparts. The membrane potential of a cell is cell, Cm , is defined as the amount of charge that
defined as must accumulate across the membrane to achieve
a membrane potential Vm . In any cell, Cm = Q/Vm ,
Vm = Vin − Vout (16.1) where charge Q is a measure of the ions near
the membrane, and Vm is the resulting membrane
where Vin and Vout are the internal and external potential. Commonly, units of millivolts (mV) for
potentials, respectively. Vm and nanofarads (nF) for Cm are used when
Cell membranes are primarily composed of describing these quantities.
two elements: a lipid bilayer that forms the bulk Following a change in voltage across the mem-
of the surface area and membrane-bound pro- brane, ions redistribute on each side of the mem-
teins interspersed throughout the lipid bilayer. brane. This produces a capacitive current, which
Although the lipid bilayer is largely impermeable is defined as
to charged ions, transmembrane ion channel pro-
dVm
teins are able to selectively control the flow of Ic = Cm , (16.2)
distinct ion species into and out of the membrane. dt
The inward flow of a positively charged ion (com- where Ic is in nanoamps (nA) when units for the
monly Na+ or Ca2+ ) results in an inward current other quantities are as defined above and time is
and increases the membrane potential. Similarly, in milliseconds (ms).
the outward flow of positive ions (commonly
K+ ) produces an outward current and moves the 16.2.2.2 Membrane Conductance
membrane potential to more negative values. If A current may also flow across the cell membrane
a negatively charged ion, such as Cl− , moves in response to a static potential difference. In this
across the membrane, the effects are reversed case, the magnitude of current flow is governed
with respect to the direction of ion flow. Sec-
by the resistance of this structure, Rm , which is surface area of many mammalian neurons is much
itself a function of the composition of the mem- less than one centimeter squared.
brane and the presence of ion channel proteins.
The membrane conductance, Gm , is the inverse of 16.2.2.4 Passive Membrane
the membrane resistance, Representation
To account for the resistive and capacitive qual-
1 ities of the cell membrane, we may construct a
Gm = (16.3)
Rm circuit model of a patch of passive membrane
by considering a resistance in parallel with a
and is measured in nanosiemens (nS) when re- capacitance (Fig. 16.1a). These elements separate
sistance has units of gigaohms (G ). When a the interior of the cell, with voltage, Vin , from
voltage difference exists between the interior and the extracellular space, with voltage, Vout . If an
exterior of a small isopotential cell, a resistive extrinsic current, Iinj , is applied to the interior
current flows through the membrane according to of the cell, as may be introduced experimentally
Ohm’s law: with a recording pipette, then we may determine
the behavior of the membrane using Kirchhoff’s
IR = Gm · Vm . (16.4) current law. Under this condition, all currents
flowing into each node must sum to zero. This
Analysis of current flux becomes more com-
condition can be represented as
plex when the flows of distinct ionic species
are considered. In general, Eq. 16.4 will include dVm
fluxes of multiple ions, each with variable con- 0 = Iinj − Ic − IR = Iinj − Cm − G m Vm .
dt
ductances and behavior that accounts for differ- (16.5)
ent internal and external concentrations of each
ion. These additional factors will be discussed in Rearranging Eq. 16.5 produces an expression
detail in Sect. 16.3. describing the behavior of the membrane poten-
tial in response to internal current injection. This
16.2.2.3 Normalized Units for is the passive membrane equation:
the Passive Membrane
In cellular neuroscience, membrane properties dVm
τ = −Vm + Iinj /Gm , (16.6)
are often normalized by the surface area of the dt
membrane, so as to describe the intrinsic char-
where τ is the time constant of the membrane,
acteristics of a patch of membrane independent
defined as τ = Cm /Gm , in units of milliseconds.
of the size or morphology of a given cell. In
The time constant is a measure of how fast the
normalized units, we have the specific membrane
membrane potential changes. Specifically, it mea-
capacitance, cm , in units of microfarads per square
sures the time it takes for the membrane potential
centimeter (μF/cm2 ); the specific membrane re-
to reach e−1 (approximately 37%) of its steady-
sistance, rm , in units of kilohms per square cen-
state value following a step change in voltage.
timeter (k /cm2 ); specific membrane conduc-
Neuronal membranes typically have a time con-
tance, gm , in millisiemens per square centimeter
stant between 1 and 100 ms. The behavior of a
(mS/cm2 ); and current densities, ic and iR , in
passive membrane to step changes in injected cur-
nanoamps per square centimeter (nA/cm2 ). When
rent is depicted in Fig. 16.1b. For both charging
considering normalized quantities, one may cal-
and discharging, Vm moves exponentially from its
culate the characteristics of an entire cell by mul-
initial value (either zero or Vss in this example) to
tiplying each quantity by the surface area of the
its final value (either Vss or zero here) with time
cell of interest. Quantities normalized by surface
constant τ .
area generally have larger units than in the non-
normalized case (e.g., μA/cm2 vs. nA), as the
a b Vm =Vss (1 - e -t/τ)
I inj Vin Vss
Vm
Cm Gm Vm Vm =Vss e -t/τ
I inj
V out
Fig. 16.1 (a) Simple electrical-circuit representation of a passive membrane. (b) Response of passive membrane to
external injected current pulses
is impermeable (and therefore do not contribute

16.3 Excitability any flux). Because of these differences in con-
centration, a diffusive flux exists for each ion as
16.3.1 Electric Potentials it diffuses down its concentration gradient. This
density of diffusive flux is described by Fick’s
Although the passive membrane is a useful tool
first law in one dimension, here expressed in
for the investigation of basic membrane proper-
steady-state (non-time-varying) form:
ties, one must consider the behavior of individ-
ual ions in order to understand the dynamics of d [i]
voltage in excitable cells. Here, we review the jdiff,i = −Di , (16.7)
dx
highlights of this material, which is covered su-
perbly and in detail elsewhere [1, 2]. The central where the coordinate x represents distance across
point in considering ionic fluxes is that they are the membrane, jdiff,i is the diffusive flux (in mol
driven by gradients in both electrical potential cm−2 s−1 ) of an ion, i, [i] is the concentration of
and chemical potential. This chemical potential the ion (in moles) as a function of distance, and Di
arises when the concentration of an ion differs in is the diffusion coefficient of the ion, a measure
the cytoplasm of a cell and in the extracellular of how fast the ion can diffuse across the mem-
milieu. The principal charge carriers responsible brane (in cm−1 s−1 ). This concentration gradient
for excitable behavior are sodium, potassium, and represents a chemical potential (analogous to a
chloride ions (Na+ , K+ , and Cl− ), and so we will voltage gradient or electrical potential) that drives
describe the chemical potentials acting on these a flux of the ion to the right in Fig. 16.2a. The
species. The concentration of potassium ions is subsequent flow of ion i down its concentration
much higher inside the cell than outside, while the gradient and across the cell membrane results in
concentrations of sodium and chloride are much an accumulation of excess charge on one side of
lower inside the cell relative to the exterior (Table the membrane and produces a shift in the electric
16.1). Although the concentrations of these ions potential across the membrane. An ionic flux in
are different inside and outside of the cell, the the opposite direction is initiated resulting from
sum total of all charges on both sides of the mem- the electric potential. The net flux density of the
brane must be zero to maintain electroneutrality. ion is
In the extracellular space, the charge introduced
d [i] Di zi F dV
by sodium ions is largely canceled by negatively jnet,i = −Di − [i] , (16.8)
charged chloride ions, and inside of the cell, the dx RT dx
positive charges introduced by potassium ions are where zi is the valence of the ion, R is the ideal
canceled by the presence of a large population gas constant (8.314 J K−1 mol−1 ), F is Faraday’s
of organic anions, to which the cell membrane constant (96485.4 C mol−1 ), and T is absolute
temperature in degrees Kelvin [2]. Multiplying Table 16.1 Internal and external concentrations of
by Faraday’s constant and the ion’s valence pro- common ions in mammalian neurons [3]
duces the Nernst-Planck equation for steady-state [Internal] [External] Nernst potential
conditions: Ion (mM) (mM) (mV)
Na+ 15 145 +61
Ii ≡ zi Fjnet,i K+ 140 5 −90
Cl− 4 110 −89
d [i] Di zi F dV
= −zi F Di + [i] Ca2+ 0.1 2.5 +136
dx RT dx
(16.9)
16.3.2 Resting Potential
where Ii is the flow of ion i in amperes per
centimeter squared (A cm−2 ) and the sign “≡”
Cellular membranes contain a variety of ion chan-
means “equivalent by definition.” The Nernst-
nels and are permeable to several different ions.
Planck equation describes the current density of
The Nernst equation may be generalized to de-
a particular ionic species through the membrane.
termine the equilibrium potential when multiple
At equilibrium, the magnitude of flux of ion i
ionic species are present with unique permeabil-
due to its chemical potential equals its flux due
ities and concentration gradients. In this case,
to the electrical potential but flows in the opposite
a dynamic equilibrium occurs, in which the net
direction (Fig. 16.2b). Solving for the equilibrium
ionic current of each species is nonzero, but the
condition of zero net current for an individual ion
total current summed over all ions equals zero. If
and integrating yields the Nernst equation,
one assumes that the change in voltage across the
RT [i]out membrane is linear (the so-called constant field
eq
Vi = ln , (16.10) assumption), the resting potential for a membrane
zi F [i]in
permeable to Na+ , K+ , and Cl− can be calculated
where the concentrations, [i], denote the concen- using the Goldman-Hodgkin-Katz (GHK) equa-
tration of the ion on each side of the membrane. tion,
eq
Vi is called the reversal potential, or Nernst
RT PK K+ out + PNa Na+ out + PCl Cl− in
potential, of ion i and is commonly written as Vi Vm = ln + − ,
F PK K+ in + PNa Na in + PCl Cl out
for simplicity. The term “reversal potential” is a
reference to the fact that the net flux of an ion (16.11)
reverses direction when the membrane potential where Pi denotes the permeability of the mem-
crosses the Nernst potential for that ion. When brane to ion i. In the GHK derivation, Pi is defined
the membrane voltage exactly equals the Nernst as the diffusion coefficient Di divided by mem-
potential, no net movement of charge across the brane thickness. Note that the Cl− concentrations
membrane occurs for that particular ion. For ex- appear on opposite sides of the fraction compared
ample, K+ has a valence of +1, a typical mam- to the K+ and Na+ terms; this is due to the
malian cytoplasmic concentration of 140 mM, negative valence of Cl− .
and an extracellular concentration of 5 mM. The A second method of deriving the resting poten-
Nernst potential for K+ can be calculated at 37 ◦ C tial arises when one assumes that resting fluxes
using Eq. 16.10 as −90 mV. At this potential, of each ion are linearly related to the difference
there is no net influx or efflux of K+ ions. If the between membrane potential and the Nernst po-
cell membrane was permeable only to K+ ions, tential for that particular ion. In this case, the
K+ ion flux would tend to bring the membrane resting potential is given for our three-ion sys-
potential toward its reversal potential, and its tem by a simple combination of conductances
resting membrane potential would be −90 mV at (GNa for sodium, GK for potassium, and GCl for
steady state. Approximate Nernst potentials for chloride) and Nernst potentials (VNa , VK , and VCl ,
common ions are given in Table 16.1 for mam- respectively):
malian neurons under physiological conditions.
Fig. 16.2 (a) A difference

in concentration [i] of a a
cation i across a
semi-permeable
membrane, with no voltage
gradient, results in
diffusion of the ion down
its concentration gradient.
(b) If the system is allowed
to equilibrate, the cation
redistributes, reducing the
chemical potential that
drives diffusion, while
simultaneously increasing
the electrical potential.
When the electrical
potential is equal and
opposite to the chemical
potential, the system is in
equilibrium. At
equilibrium, the electrical
potential is equal to the
Nernst potential for ion i b
GNa VNa + GK VK + GCl VCl −70 mV, as the cell membrane is more permeable
Vm = (16.12)
GNa + GK + GCl to K+ and Cl− ions than Na+ ions. A membrane
potential shift away from this “polarized”
Both methods of calculating the resting resting potential and toward zero is called a
potential yield close approximations of the depolarization. Alternatively, a shift to potentials
correct value, although the stated assumptions of more negative than the resting potential is termed
the GHK equation generally yield smaller errors. a hyperpolarization. If the membrane potential
In the absence of activity, most neurons have shifts from the resting potential to zero and
a resting membrane potential of approximately then to increasingly positive potentials, it is still
referred to as a depolarization, contrary to a literal Eccles “for their discoveries concerning the ionic
interpretation of the term. mechanisms involved in excitation and inhibition
in the peripheral and central portions of the nerve
cell membrane.” For reasons discussed in Sect.
16.3.3 Voltage-Gated Conductances 16.3.6, the Hodgkin-Huxley formulation remains
extremely useful after more than a half century.
Thus far, we have described membrane perme-
ability as a static quantity. However, a host of 16.3.4.1 Voltage Clamp and Space
transmembrane channels whose permeabilities Clamp
change dynamically are present in the membranes In the late 1940s, it was understood that mem-
of excitable cells. Of particular importance are brane currents both contributed to and were de-
voltage-gated ion channels, whose pores switch pendent on the membrane potential. This cou-
between open and closed states probabilistically pling between membrane potential and current
depending on the voltage across the membrane presented a significant experimental hurdle, as it
at the position of the channel. Ubiquitous among was difficult to study either without controlling
all excitable cells are voltage-gated Na+ and K+ for one. Additionally, the capacitive current com-
channels, which display preferential specificity plicated the study of these ionic currents, as it
for a single ionic species in the open state. The was not possible, given electrophysiological tech-
presence of these, and other, channel populations niques at the time, to separate individual compo-
in the cell membrane results in highly nonlinear nents of membrane current. Finally, experimental
behaviors of membrane potential in response constraints led many researchers to focus on the
to perturbations. Chief among these behaviors study of very large neurons, which cannot be
are the generation of the action potential, an approximated as isopotential. Instead, large axial
all-or-nothing transient depolarizing spike in currents, those flowing between sections of the
membrane potential responsible for representing cell that do not have the same transmembrane
and transmitting information in the nervous potential, were present and difficult to distinguish
system of all animals. from transmembrane currents.
Hodgkin and Huxley exploited two techniques
to address these problems: the voltage clamp and
16.3.4 The Hodgkin-Huxley Model: space clamp. To allow for easier experimental
Action Potentials in the Squid access, they studied the giant axon of the squid
Giant Axon Sepia loligo, taking advantage of its large diam-
eter. They introduced electrodes to measure po-
In 1952, Alan Lloyd Hodgkin and Andrew Hux- tential and inject current in both the extracellular
ley published a series of papers, culminating in medium and within the cell. The voltage clamp
a quantitative biophysical model of membrane technique (Fig. 16.3a) used a feedback amplifier
currents and the mechanism by which they pro- to hold the voltage across the membrane constant.
duce action potentials [4]. This work established This technique uncoupled the ionic currents from
a phenomenological model of action potential the membrane potential, and custom circuitry was
generation in terms of experimentally measurable employed to eliminate the capacitive current. The
ionic conductances, membrane potential, and cur- space clamp technique utilized a long wire, care-
rent flow. The approach of Hodgkin and Hux- fully threaded into the squid axon intracellularly,
ley has offered an experimental and theoretical to short-circuit the voltage of the axon, resulting
framework for examining a broad class of neu- in the same value of voltage at all positions along
ronal models and, upon its introduction, became a its length. As such, this made the entire intracellu-
foundation for understanding neural excitability. lar space of the cell isopotential, eliminating any
Hodgkin and Huxley shared the Nobel Prize in axial currents. The combination of these technical
physiology or medicine in 1963 with John Carew advances allowed Hodgkin and Huxley to analyze
a b
Fig. 16.3 (a) Schematic of a two-electrode voltage-clamp experiment. (b) Circuit diagram of the space-clamped
Hodgkin-Huxley model of neuronal excitation
the dynamics of the conductances mediated by each population (the total conductance of a chan-
voltage-gated Na+ and K+ channels. In order to nel population when all constituent channels are
separate voltage-clamp responses into their ionic open). In addition to this information, they also
components, they used the painstaking method determined the reversal potential of each channel
of ionic substitution within their salt solutions population, equivalent to the Nernst potentials for
to manipulate the Nernst potentials VNa and VK Na+ and K+ , in order to determine the appropriate
independently. (Thankfully, we now have access current flow mediated by each resulting from a
to channel blockers, making the process of sep- given level of activation. The membrane model
arating ionic currents much easier than it was a proposed by Hodgkin and Huxley is shown in
half-century ago.) Fig. 16.3b, and the components are described in
the following sections.
16.3.4.2 Ionic Conductances
In order to reveal the mechanisms responsible for 16.3.4.3 Model of the Potassium
action potential generation, Hodgkin and Hux- and Sodium Conductance
ley quantitatively described three principal ionic Hodgkin and Huxley represented the voltage
conductances present in the squid giant axon: dependence of the Na+ and K+ conductances
the Na+ , K+ , and leak conductances. Although as static maximal conductances, GNa and GK ,
voltage-gated Cl− channels do not play a major multiplied by voltage-dependent gating variables.
role in the squid giant axon, these ions do con- Upon stepping the voltage of the squid axon from
tribute to the leak conductance, a “catch-all” term near the resting potential to more depolarized
used to denote all static conductances in the mem- values, under conditions that isolated the K+ -
brane. All currents that result from static con- mediated current, this current was found to
ductances linearly and instantaneously depend on activate and remain activated for the duration
voltage according to Ohm’s law. As a result, they of the step. A single voltage-dependent gating
may be combined into a single conductance math- variable, n, representing the fraction of open
ematically, the leak conductance, the reversal po- channels, was used to describe the voltage
tential of which does not necessarily correspond dependence of this channel. The isolated Na+
to the Nernst potential of any single ion. current was found to quickly activate and then,
To fully describe the dynamics of each con- after a short period of time, inactivate. To describe
ductance, Hodgkin and Huxley determined the this behavior, two gating variables were ascribed
amount of activation and time scale of activa- to this channel. The variable m describes the
tion of each channel population as a function of activation process of the Na+ current, and another
voltage, as well as the maximal conductance of variable, h, describes the inactivation process. All
a b
Fig. 16.4 Steady-state amplitudes (a) and time constants (b) of the Hodgkin-Huxley gating variables vs. membrane
potential
three gating variables were assumed by Hodgkin 1

τn (Vm ) = . (16.17)
and Huxley to possess first-order kinetics. The n αn (Vm ) + βn (Vm )
gating variable, for example, was described by
We leave the derivation of these expressions
αn (Vm )
C ↔ O (16.13) as a useful exercise for the reader. The time con-
βn (Vm ) stant (Eq. 16.17) and steady-state activation (Eq.
16.16) of the gating variable can be measured
where C is the closed state and O is the open experimentally in voltage-clamp experiments.
state of the channel. α n (Vm ) and β n (Vm ) represent The m and h variables governing the Na+
the rates of channel opening and closure, respec- conductance are described in the same manner.
tively, both voltage-dependent quantities. If n is The steady-state value and time constant for each
the fraction of open channels, then 1-n represents gating variable are depicted in Fig. 16.4. Notice
the fraction of closed channels, and one may write that m and n, representing activation of the Na+
an expression for the change in the fraction of and K+ currents, increase with increasing volt-
open channels: age, while h decreases with increasing voltage. In
addition, the activation of the Na+ channel takes
dn
= αn (Vm ) · (1 − n) − βn (Vm ) · n. (16.14) place on a substantially shorter time scale than the
dt activation of the K+ channel or the inactivation of
Alternatively, we can write the sodium channel.
dn n∞ (Vm ) − n
= (16.15) 16.3.4.4 Potassium and Sodium
dt τn (Vm ) Currents
where n∞ (Vm ) is the steady-state fraction of open With descriptions of the rate constants describing
channels at a voltage, Vm , calculated as opening and closure of the m, n, and h, gates,
Hodgkin and Huxley were able to calculate the
αn (Vm ) current produced by each channel. When fitting
n∞ (Vm ) = , (16.16) this model to their data, the K+ current, Ik , in
αn (Vm ) + βn (Vm )
nanoamps (nA), was found to depend roughly on
and τ n (Vm ) is the time constant of this process, the fourth power of the gating variable n, in the
defined as squid axon,
IK = Gk n4 (Vm − VK ) , (16.18) Hodgkin and Huxley model:
dVm
where Gk is the maximal conductance, in Cm = Iinj − Iion (16.21)
nanosiemens (nS), of the potassium channel dt
population and VK is the Nernst potential of where Iion is the sum of the Na+ , K+ , and leak
K+ . The reversal potential for potassium, VK , is currents:
approximately −90 mV, slightly more negative
than the resting potential. Therefore, when the Iion = GK n4 (Vm − VK ) + GNa m3 h (Vm − VNa )
membrane voltage is more positive than VK ,
activation of this conductance leads to increased + Gleak (Vm − Vleak ) .
K+ permeability and hyperpolarization of the (16.22)
membrane, as the membrane potential moves
closer to the Nernst potential for K+ . The Hodgkin-Huxley model is comprised of
In an analogous fashion, the Na+ current INa four ordinary differential equations describing the
was found to depend on approximately the third time evolution of the three gating variables n,
power of m, the gating variable controlling activa- m, and h and the voltage of the membrane, Vm .
tion of the Na+ conductance, and the first power Notice that its form is similar to the passive
of h, which controls the inactivation process of membrane equation with the addition of two non-
this channel. The Na+ current, INa , therefore, may linear, voltage-gated conductances and reversal
be represented as potentials appropriate for the ions to which each
conductance is permeable. The Hodgkin-Huxley
INa = GNa m3 h (Vm − VNa ) (16.19) equations (Eqs. 16.18–16.22) describe exactly the
dynamics of the circuit depicted in Fig. 16.3b.
When endowed with physiologically accurate
where GNa is the maximal conductance of the
parameters, this model reproduces the action po-
population of Na+ channels and VNa is the Nernst
tential waveform observed in the giant axon of
potential for sodium. The Nernst potential for
the squid as well as many other aspects of the
sodium is approximately +61 mV, and so activa-
voltage dynamics observed in this system. Fig-
tion of Na+ channels leads to depolarization of
ure 16.5 illustrates the voltage waveform of the
the membrane.
action potential and the time course of the gating
Finally, the leak current, Ileak , is modeled as a
variables, conductances, and currents during the
linear function of voltage, with reversal potential,
action potential.
Vleak . Because this conductance is static, it is
not controlled by a gating variable, but instead
16.3.4.6 Normalized Units in the
modeled simply as
Hodgkin-Huxley Model
This description of the Hodgkin-Huxley model
Ileak = Gleak (Vm − Vleak ) (16.20)
has been provided in terms of the macroscopic
quantities, current, conductance, and capacitance.
where Gleak represents the constant leak conduc-
Normalizing by surface area, as in Sect. 16.2.2.3,
tance. The leak current has a reversal potential
provides an analogous description of the model in
near the resting membrane potential, approxi-
terms of current density, conductance density, and
mately −60 mV.
specific capacitance. The membrane equation for
the Hodgkin-Huxley model in normalized units is
16.3.4.5 Complete Hodgkin-Huxley
Model dVm
Combining these currents with the current due to cm = iinj − iion (16.23)
dt
membrane capacitance and an extrinsic injected
current produces the membrane equation for the with
Fig. 16.5 An action

potential (top panel). The
associated values of the
Hodgkin-Huxley gating
variables (second panel),
sodium and potassium
conductances (third panel),
and ionic currents (bottom
panel)
iion =g K n4 (Vm − VK ) + g Na m3 h (Vm − VNa ) potential. If the magnitude of this current is large
enough, it will trigger activation of the Na+ con-
+ gleak (Vm − Vleak ) . ductance (i.e., an increase in the variable, m,
(16.24) which possesses fast kinetics; see Fig. 16.4), re-
sulting in additional depolarization and an in-
In Eqs. 16.23–16.24, each gi is a conductance creased inward current. This produces a positive
density, in mS/cm2 ; cm is the specific capacitance feedback loop in which the membrane poten-
of the membrane, in μF/cm2 ; and the current tial and the Na+ current continue to increase
terms are replaced by current densities, iinj and at increasingly higher rates. This process is re-
iion , in μA/cm2 . The gating variables m, n, and sponsible for the upstroke of the action poten-
h are unitless fractions and therefore remain the tial. This depolarization is halted as the slower
same. h variable begins to decrease (representing in-
activation of the Na+ conductance; dotted line
in second panel of Fig. 16.5) and the n vari-
16.3.5 Behavior of theHodgkin- able begins to increase (activation of the hyper-
Huxley Model polarizing K+ conductance; dashed line of sec-
ond panel in Fig. 16.5). The hyperpolarization
16.3.5.1 Action Potentials brought about by the combination of decreased
and Threshold Na+ conductance and enhanced K+ conductance
Introducing a positive current pulse, Iinj , into the results in the downstroke of the action potential.
axon results in a depolarization of the membrane Because the n and h variables are relatively slow
to respond to changes in voltage, a small un- 16.3.6 Assumptions of the Model

dershoot of the resting membrane potential oc-
curs before these variables return to their equi- The ability of the Hodgkin-Huxley model to re-
librium states. This brief hyperpolarization fol- produce the biophysical phenomena observed in
lowing the action potential is termed the after- the squid axon remains one of the most impor-
hyperpolarization and is discussed in more detail tant achievements in the modeling of excitable
in Sect. 16.5.3. cells. However, it is important to keep in mind
The action potential is an all-or-none event the assumptions that were made in its construc-
triggered when the state variables of the tion. First, Hodgkin and Huxley took the various
model cross a threshold hyperplane in its ionic currents to be independent, meaning that
four-dimensional (m, n, h, Vm ) state space. the effect on the membrane potential of each
However, this threshold is often approximated current could be described as a simple sum of the
as a simple scalar voltage threshold. Small individual components. Second, ionic currents
depolarizing currents that do not move the were assumed to be ohmic. Thus, they could be
membrane voltage across the voltage threshold calculated as the product of the ionic conduc-
produce insufficient change in the activation tance and the difference between the membrane
of the Na+ conductance to recruit the positive potential and the reversal potential of each ionic
feedback necessary for an action potential. species. Finally, the model assumes that the ionic
Depolarizing stimuli that are too small to elicit conductances can be described simply as a frac-
an action potential are termed subthreshold, in tion of the maximal conductances, according to
contrast with larger, suprathreshold stimuli that their respective gating variables, which for the
produce a full spike in membrane potential. Na+ conductance are assumed to be indepen-
After a subthreshold stimulus, the membrane dent [4]. Subsequent research has shown that
relaxes back to its resting potential, and because many of these assumptions do not strictly hold.
the voltage-gated channels are minimally However, when more accurate models have been
recruited, this subthreshold response is very constructed, they behave mostly in a qualitatively
similar to the response to current input of the similar manner. The immense and continued in-
passive membrane considered in Sect. 16.2 fluence of the Hodgkin-Huxley approach, despite
(Fig. 16.1). its known inaccuracies, lies in its direct connec-
tion to voltage-clamp data, its relative simplic-
16.3.5.2 Refractory Period ity, and its demonstrated ability to account for a
Following an action potential, the squid giant diverse body of experimental results, including
axon displays a refractory period, during which those described in Sects. 16.4, 16.5, and 16.6.
the membrane is less excitable than at rest. The
refractory period follows from two factors: the
residual activation of the K+ current, as indicated 16.4 Propagating Activity
by the elevated value of the n gate in Fig. 16.5,
and inactivation of the Na+ current, as indicated The Hodgkin-Huxley model was derived from
by the depressed value of the h gate in Fig. 16.5. data collected under conditions of “space clamp”
This feature is captured by the Hodgkin-Huxley (i.e., with a wire inserted down the length of
model. During this period of reduced excitability, the axon to short-circuit the interior; Fig. 16.3a).
a larger depolarization may be required to cause However, the crowning achievement of their work
an action potential (the relative refractory period), was that they were able to model propagation
or it may not be possible at all (the absolute of the action potential in the non-space-clamped
refractory period). During the refractory period, cylindrical squid giant axon and to show that
the assumption of a fixed voltage threshold does the propagating solution is stable only for veloc-
not apply. ities near those measured experimentally. They
a Another relationship for Ia (x, t) can be derived

by applying Kirchhoff’s current law:
Ia (x, t) = Δxι̂m (x + Δx, t) + Ia (x + Δx, t)

(16.27)
where, for mathematical convenience, we have
defined membrane current per unit length ι̂m =
Im /Δx. Rearrangement and taking the infinitesi-
b
mal limit of this equation yields
∂Ia (x, t)
ι̂m (x, t) = − . (16.28)
∂x
We can combine Eqs. 16.26 and 16.28 as fol-

lows:
Fig. 16.6 (a) Electrical-circuit representation of an un-
myelinated axon. (b) A propagating action potential in a
∂I a (x, t) ∂ 2 Vm (x, t)
model of an unmyelinated axon −ri = = ri ι̂m (x, t) .
∂x ∂x 2
(16.29)
accomplished this goal by assuming a travel-
ing wave solution and thus simplifying the prob- To write a more specific version of this equa-
lem considerably [4]. With the benefit of modern tion, we must specify the relationship between
computer technology, it is fairly straightforward membrane potential Vm and membrane current
to dispense with the traveling wave assumption per length, ι̂m . For the Hodgkin-Huxley model,
and instead to simulate directly electrical behav-
∂Vm (x, t)
ior in spatially extended processes with arbitrary im (x, t) = cm + iion (x, t) (16.30)
geometry and channel properties. ∂t
Figure 16.6a shows a circuit diagram of a long where iion (x,t) is the sum of the conductance-
process with membrane potential that varies with based current fluxes,
distance along the membrane. From Ohm’s law,
iion (x, t) =g K n4 (Vm (x, t) − VK )
ri ΔxI a (x, t) = Vin (x, t) − Vin (x + Δx, t)
(16.25) + g Na m3 h (Vm (x, t) − VNa )
+ gleak (Vm (x, t) − Vleak ) .
where ri is the intracellular resistance per unit (16.31)
length, x is the length of the “compartment” of
membrane that can be considered isopotential, To bridge between im , with units of current
and Ia is the magnitude of the axial current per unit area, and ι̂m , with units of current per
between neighboring compartments. Because unit length, we must consider the geometry of
the exterior environment is assumed to be short- the axon or other neuronal process. If we assume
circuited at all locations, we can assume that that our neural compartments are cylindrical with
Vout = 0 and, thus, that Vm (x,t) = Vin (x,t). Making length x and radius a, then ι̂m = 2π aim . Thus,
this substitution, dividing both sides by x, and for a cable with Hodgkin-Huxley conductances,
taking the limit as Δx → 0 yields we have the equation
∂V m (x, t)
ri Ia (x, t) = − . (16.26) ∂ 2 Vm (x, t) ∂Vm (x, t)
∂x = 2π ar i cm + iion (x, t) .
∂x 2 ∂t
(16.32)
In practice, this nonlinear partial differential aged to solve this problem with 1950s technology
equation is difficult or impossible to solve ana- by using an ingenious traveling wave assump-
lytically under most conditions. The more use- tion. However, solving for the voltage profile
ful approach is to build a compartmentally dis- in a branching or nonhomogeneous structure re-
cretized version of the equations and to solve a quires numerical simulations of equations like
set of nonlinear ordinary differential equations those above. Although good software packages
computationally. One way to derive this equation exist to solve conductance-based equations for
is to use a spatially discrete approximation of the spatially extended neural models, it is a useful
second spatial derivative above. In this example, exercise to solve a set of equations from scratch,
one would solve the ordinary differential equation using a general numerical analysis program like
below for each compartment, MATLAB or XPP. One of the major issues that
∂Vm (x, t) 1 arises in such models is that of equation stiffness:
= a very small value of x can be required for accu-
∂t cm
racy, but small values of x can force the numeri-
1 Vm (x + Δx, t) − 2Vm (x, t) + Vm (x − Δx, t)
2π ari (Δx)2 cal algorithm to take exceedingly small time steps
and in fact can make the problem numerically
−iion (x, t) . unstable in some cases. Good software packages
(16.33) like NEURON or GENESIS have been written
especially to solve these problems efficiently and
This intimidating-looking equation is easy to accurately and are described in Sect. 16.14.
derive [5, 6] and is conceptually simple. The Many of the large axons in mammals are
second spatial derivative gives rise to currents myelinated. The myelin sheath is electrically
that are proportional to the voltage differences passive, as is the axonal membrane underneath
between the current compartment and its imme- the sheath. This portion of the axonal membrane
diate neighbors, according to Ohm’s law. The can be modeled as passive, with very high
term, iion , comes from the membrane within the membrane resistivity and low membrane
compartment. One must also find a sensible way capacitance. Between myelin sheaths are hot
to determine ri , the intracellular resistance per spots, called nodes of Ranvier, with very
unit length. For intracellular fluid with volume high densities of voltage-gated sodium and
resistivity ρ i , in units of -cm, ri = πρai 2 , giv- potassium channels. Action potential propagation
ing a version of the compartmental equation that in myelinated axons is saltatory, jumping from
depends only on the geometry of the cylinder one node of Ranvier to another. The effect of
and fundamental properties of the intracellular myelination is to speed up propagation in large
medium and membrane: axons substantially and to make propagation
velocity proportional to axonal radius a [7].
dVm (i) 1
=
dt cm

a Vm (i + 1) − 2Vm (i) + Vm (i − 1)
− i ion (i) . 16.5 Diversity in Channels
2ρi (Δx)2
(16.34) and Electrical Activity
In this equation, we have also indexed the Although the squid giant axon provided an
compartments using the compartment number i ideal preparation for early studies of neuronal
rather than the position of the compartment, x. excitability, it represents only one of the
For physiological parameters, an action poten- many patterns of electrical excitability that
tial initiated in a long, unbranching process such have been observed in neurons. Since the
as the squid giant axon propagates away from pioneering work of Hodgkin and Huxley,
the site of initiation without decrement in am- neurons have been described that exhibit a
plitude (Fig. 16.6b). Hodgkin and Huxley man- host of identifiable behaviors. These include
various forms of burst spiking, rebound spiking, ing is not preceded by hyperpolarization, as the
after-hyperpolarization trajectories, spike-rate T-type calcium current remains inactivated (Fig.
adaptation, and subthreshold membrane potential 16.7a top). The bursting and regular firing be-
oscillations. Electrophysiological characteristics haviors of TC neurons can be reproduced in a
such as these are mediated by voltage-gated ion model incorporating a representation of the T-
channels, including the sodium and potassium type calcium channel with the correct voltage
channels described in the squid giant axon, dependence and kinetics.
as well as ligand-gated channels sensitive In general, bursting requires the presence of a
to neurotransmitters and intracellular species channel population or other process that operates
such as calcium [8]. Here we briefly describe on a time scale that is slower than the transient
these forms of electrical activity and the ionic sodium and delayed rectifier potassium channels
mechanisms that have been used to construct responsible for the upstroke and subsequent re-
models of them. polarization of the action potential. In addition
to the T-type calcium channel, such mechanisms
can include calcium-activated potassium chan-
16.5.1 Bursting nels and slow, hyperpolarization-activated cation
channels.
Bursting is a pattern of spiking in which groups
of spikes occur closely spaced in time (the burst),
flanked by relatively long periods of inactivity 16.5.2 Subthreshold Oscillations
(the inter-burst intervals). Bursting can be gener-
ated intrinsically, as a result of the combination of Several types of neurons exhibit small-amplitude
ionic currents present in a neuron, or as a result oscillations of membrane voltage below spike
of extrinsic synaptic input originating from other threshold. These subthreshold oscillations
cells in a local network. Thalamocortical (TC) re- (STOs) vary in their frequency and amplitude and
lay neurons represent a neuron type that exhibits in the channels responsible for producing them.
action potential firing in both regular and bursting Mesencephalic V neurons of the brainstem and
patterns intrinsically, with mechanisms that are stellate neurons of the medial entorhinal cortex
well understood [9, 10]. In TC neurons, high- illustrate two distinct forms of STOs. In both
frequency bursts of action potentials are gener- cases, these oscillations emerge and increase in
ated following the activation of T-type calcium amplitude, reaching 5 mV or more, as the voltage
channels. T-type channels are activated at mem- of the neuron increases toward spike threshold.
brane potentials more negative than the resting The STOs in each cell type emerge, however,
membrane potential and have slow kinetics. As a at frequencies and by mechanisms which differ
result of these two properties, hyperpolarization substantially. Mesencephalic V neuron STOs are
induced by transient synaptic inhibition, or hy- often observed at frequencies between 50 and
perpolarizing current injected through a recording 100 Hz [12]. In contrast, the STOs of stellate
pipette, activates T-type calcium channels. The neurons are typically recorded in the 2–8 Hz
activation of this population of channels, which frequency band [13] (Fig. 16.7b), a difference of
is highly expressed in the proximal dendrites of more than an order of magnitude.
TC neurons, leads to a strong depolarization of Experimental and modeling studies have es-
the neuron and the generation of a high-frequency tablished that subthreshold oscillations in both
(300 Hz) burst of action potentials (Fig. 16.7a bot- cell types are generated by the interplay of a
tom). This burst is terminated when the calcium channel population possessing slow kinetics with
channels inactivate (among other factors) and the a regenerative current possessing fast kinetics. In
neuron’s voltage returns to rest. In contrast, TC both cases, the regenerative current is believed to
neurons exhibit a regular firing pattern when spik- be the persistent sodium current, which is similar
Fig. 16.7 (a) Tonic spiking and bursting in a thalamo- sponse to the same level of DC current; the current pulses
cortical relay neuron. (b) Subthreshold oscillations in an in the bottom trace can move the cell from one regime to
entorhinal stellate neuron. (c) After-hyperpolarization, in- the other. (f) Rebound spiking, in which a stellate neuron
dicated by the arrow, of a neocortical fast-spiking interneu- of the entorhinal cortex fires an action potential after being
ron. (d) Spike-frequency adaptation in response to DC released from hyperpolarization. (Panel (a) adapted from
current in a neocortical pyramidal neuron. (e) Bistability, [10] and panel (e) adapted from [11]. Other panels are
in which the neuron can fire at high rates or be silent in re- from our laboratory’s unpublished data)
to the sodium current described in the squid giant as an inductive element in the membrane, and the
axon, but does not inactivate following activa- membrane itself, which is inherently capacitive,
tion of the channel. This channel activates in results in a resonant membrane in which fluctu-
response to small depolarizations, thus depolar- ations in a certain frequency band are preferen-
izing the membrane further. The interplay be- tially amplified [14]. In mesencephalic V neu-
tween the slowly activating current, which acts rons, the slowly activating current is believed
to be a non-inactivating low-threshold potassium 16.5.4 Spike-Frequency Adaptation

current with an activation time constant of 10 ms
[12]. In stellate neurons, the hyperpolarization- In response to a constant injected current, many
activated cation current, Ih , having an activation neurons fire action potentials repetitively at a
time constant of 100 ms, fulfills this role [15]. frequency that depends on the amplitude of
The difference in activation kinetics controls the the injected current. Commonly, however, the
inductive strength of these currents and explains interval between successive action potentials
the frequency of observed STOs. becomes longer and longer as more and more
spikes are fired, until a steady-state firing
frequency is reached (Fig. 16.7d, unpublished
16.5.3 After-Hyperpolarizations data). This decreasing spike frequency in
and After-Depolarizations response to a stimulus is termed spike-frequency
adaptation (or accommodation). Spike-frequency
After-hyperpolarizations (AHPs) and after- adaptation occurs as a result of activating
depolarizations (ADPs) refer to the voltage outward currents or inactivating inward currents.
trajectory following an action potential. If the The change in activation of these currents
voltage follows a trajectory more negative than may be due to the sustained depolarization on
rest, then it is referred to as an AHP, as in the top of which spikes ride, or through a spike-
squid giant axon, while voltage trajectories more dependent mechanism in which each spike leads
depolarized than rest are termed ADPs. Neurons to an incremental change in some activation
can express one or more of these characteristics variable. Common ionic mechanisms underlying
sequentially as a result of the sequential activation spike-frequency adaptation include the M-type
of multiple membrane mechanisms. Figure 16.7c potassium current IM [16], calcium-activated
illustrates the voltage trajectory following a spike potassium currents [17], and cumulative partial
in a fast-spiking interneuron of the somatosensory inactivation of the fast sodium current responsible
cortex. This neuron exhibits a strong, fast AHP for the upstroke of the action potential [18].
under control conditions. In these neurons,
the AHP produces complete de-inactivation of
sodium channels immediately after the spike, 16.5.5 Bistability
as a result of the strong hyperpolarization,
allowing these neurons to fire at high rates. Some neurons exhibit two stable states for the
The presence of AHPs and/or ADPs has been same value of injected current. In one state, the
shown to impact the excitability of a neuron cell resides below spike threshold and remains
following an action potential in many ways. quiescent, while in the other state, it generates
AHPs contribute to post-spike refractoriness and action potentials repetitively. A brief stimulus
can determine the frequencies at which a neuron may transition the neuron from the resting state
may spike preferentially. ADPs typically provide to the spiking state, where it remains indefinitely,
a short time window of enhanced excitability until another, correctly timed stimulus transitions
following each action potential. This may result it back to the resting state again. Bistability is
in a propensity to fire pairs or bursts of closely exhibited by Purkinje neurons of the cerebel-
spaced spikes. AHPs and ADPs are shaped by the lum [19]. Figure 16.7e illustrates that a positive
particular combination of ion channels present pulse of current can switch the state of these
and by passive current fluxes determined by the neurons from resting to spiking and a second
morphology of the cell. A host of voltage- and pulse can subsequently facilitate a switch back to
calcium-gated ion channels have been shown to the resting state. Many neurons exhibit bistability
contribute to AHPs and ADPs, and they are often over a small range of injected current ampli-
generated by the combined activity of several tudes, including many neocortical fast-spiking in-
such populations. terneurons, which transition between fast spiking
and quiescence as a result of noisy membrane a b

fluctuations. In fact, under certain experimental
equations, even the squid giant axon has been
shown to exhibit bistability [20]. Bistability is not
generally associated with any specific membrane
mechanism, but rather is a product of the dynam-
ics produced by the sum total of all passive and
active elements in the cell membrane. For an in-
depth discussion of membrane behaviors from a
dynamical systems perspective, see the treatment
by Izhikevich [21].
16.5.6 Post-Inhibitory Rebound Fig. 16.8 (a) Filled pyramidal cell, showing locations
Spiking of simultaneous patch-clamp recordings. (b) In response
to depolarization via the dendritic electrode, the dendrite
can fire an action potential, which in this case does not
Post-inhibitory rebound spiking occurs when
propagate to the soma. (Adapted from [24])
spikes are induced following a hyperpolarization
of membrane voltage. This phenomenon may be
observed in response to inhibitory synaptic input a high signal-to-noise ratio (Fig. 16.8a). The in-
or the injection of hyperpolarizing current with troduction of differential interference contrast mi-
a recording pipette. As illustrated previously, croscopy, in particular, provided unprecedented
the T-type calcium current mediates rebound optical contrast in thick tissue specimens, per-
bursts in thalamocortical relay neurons, but mitting visualization of dendritic processes in
rebound spiking may be generated by any current live tissue. In studies taking advantage of these
that contributes to an increase in excitability in technical advances, it was observed that sodium-
response to hyperpolarization of the membrane. based action potentials generated in the somata
Channel populations contributing to rebound of neocortical pyramidal neurons could propa-
spiking must also have kinetics that are slower gate into dendrites as well as along the axon
than the kinetics of the cell membrane, so that the [25]. Although dendritic spikes are smaller in
increase in excitability effected at hyperpolarized amplitude and broader than somatically recorded
potentials does not subside before spike threshold spikes (Fig. 16.8b), this seminal work established
is reached. Post-inhibitory rebound spiking that dendrites may behave in a strongly nonlinear
may also be generated in cells expressing the manner.
hyperpolarization-activated cation current, Ih
(Fig. 16.7f).
16.6.1 Dendritic Channel Expression
16.6 Nonlinear Dendritic In addition to sodium channels, a plethora

Processing of other voltage-gated channels have been
described in dendrites. In pyramidal neurons
Initially treated as passive cables [5, 6, 22], den- residing in region CA1 of the hippocampus,
drites have since been shown to contain a host of these include a host of voltage-gated calcium and
nonlinear active conductances, serving to shape potassium channels, as well as Ih , the slow inward
synaptic inputs originating at dendritic locations current activated at hyperpolarized potentials
[23]. This realization was made possible by tech- [26]. The position-dependent expression of
nological advancements that allowed for access these ion channels indicates that different
to membrane voltage at dendritic locations with dendrites, and different positions along the
same dendrite, may integrate synaptic inputs represents one of the first, and simplest, models
with variable efficacy and at different preferred describing the neuron. The ideal integrate-and-
temporal frequencies. Although the data are fire model is described by
limited to a handful of cell types with large-
diameter primary dendrites, voltage-gated ion dVm
Cm = Iinj (t). (16.35)
channels have been documented in the dendrites dt
of all (or nearly all) the examples studied
This equation describes the charging of a ca-
[26], suggesting that nonlinear processing in
pacitive circuit (the membrane) with capacitance,
dendrites may be the rule rather than the
Cm , and represents the behavior of a neuron in the
exception.
subthreshold regime in response to a time-varying
injected current, Iinj (t). The left side of Eq. 16.35
is identical to the capacitive current described
16.6.2 Dendritic Excitability
in Sect. 16.2.2.1. Instead of explicitly modeling
the sodium and potassium currents responsible
Although sodium-based action potentials gener-
for spike generation and repolarization, a hard
ated at the soma of a neuron may propagate into
threshold is applied. Upon a threshold crossing
the dendrites, this type of spike is generally not
of voltage, a spike is considered to have occurred,
initiated in response to dendritic synaptic input.
and the following reset condition is applied:
However, voltage spikes mediated by voltage-
gated calcium channels and synaptic NMDA re-
If Vm > Vthresh , then Vm = Vreset . (16.36)
ceptors do occur in this manner in some cells.
In neocortical pyramidal neurons, activation of
This model is a perfect integrator, in that the
sufficient glutamatergic synapses in the distal api-
effect of any current input, no matter how small
cal tuft induces an all-or-nothing spike that re-
or brief, will affect the voltage of the model for
lies upon voltage-gated calcium channels [24].
all time. In real neurons, however, the effect on
These calcium spikes do not propagate actively
voltage of an input decays with time. For this
to the soma but do contribute to nonlinear in-
reason, the resistance, Rm , of a cell membrane is
tegration of distal inputs (Fig. 16.8b). In con-
commonly incorporated into this model in addi-
trast, convergent input onto smaller, basal den-
tion to the contribution of membrane capacitance.
drites of these neurons results in an all-or-none
In this case, it is termed the leaky integrate-and-
event that is heavily dependent upon the activa-
fire (or LIF) and is described by
tion of NMDA channels [27]. These results illus-
trate that dendrites are heterogeneous, nonlinear dVm Vm
structures whose function has only begun to be Cm · = Iinj (t) − . (16.37)
dt Rm
understood.
Here, the membrane leak current, RVmm , reverses
direction when the voltage changes sign. The
16.7 Simple Neural Models circuit equivalent of the leaky integrate-and-fire
neuron in the subthreshold regime is depicted in
16.7.1 Integrate-and-Fire Model Fig. 16.9a and is equivalent to the passive mem-
brane considered in Sect. 16.2. The more realistic
In addition to the Hodgkin-Huxley formalism, scenario, in which the membrane leak current
which describes model neurons in terms of reverses at a negative voltage, near resting po-
conductances representative of populations of tential, may be obtained by a simple substitution
ion channels with voltage-dependent gating of variables, where Vm = Vm − Vleak , although
variables, neuron models take on numerous the two forms are equivalent mathematically. The
other forms. The integrate-and-fire model, first LIF neuron model, while highly idealized, cap-
studied more than a century ago by Lapicque [28], tures the most basic aspects of neuronal respon-
Fig. 16.9 (a) Schematic representations of the leaky added artificially for clarity. (c) Responses to pulsatile
integrate-and-fire (LIF) model and the resonate-and-fire inputs at different rates in the two models. (Panel (c)
model (RIF). (b) Repetitive firing in the LIF model. Spikes adapted from [29])
siveness and is analytically tractable for many Vthresh

Irheo = . (16.39)
problems. Because its behavior can be described Rm
in a mathematically precise fashion, the LIF has
been an invaluable tool for neural modeling. If a subthreshold injected current is applied,
such that Iinj < Irheo , then the voltage of the LIF
will approach Vss exponentially. If the initial volt-
16.7.2 Behavior of the Leaky age of the LIF is V0 before the constant current is
Integrate-and-Fire Model applied, then its voltage trajectory becomes
Given a constant current input, the leaky Vm (t) = VSS + (V0 − VSS ) · e−t/τ , (16.40)
integrate-and-fire model approaches a steady-
state voltage: where τ = Rm Cm is the membrane time constant.
Hence, the steady-state voltage of the model is
VSS = Iinj · Rm . (16.38) determined only by the injected current and the
resistance of the membrane, while the speed with
If Vss is more depolarized than spike threshold, which the membrane voltage approaches a new
then repetitive spiking results (Fig. 16.9b). This steady state is determined by a combination of Rm
condition occurs when the injected current, Iinj , and Cm .
is greater than the rheobase current, Irheo , the If, on the other hand, Iinj > Irheo , then the neu-
minimum current that produces action potentials. ron will repeatedly cross spike threshold and be
Irheo can be solved for algebraically: subsequently reset to a subthreshold voltage. The
frequency of repetitive spiking can be calculated represents the resonant current. Like the LIF, the
directly [30] to give RIF has an artificial threshold, and upon threshold
crossing, the following condition applies:

Vthresh − Iinj Rm −1 ,
F Iinj = −τln . Vm = Vreset
Vreset − Iinj Rm If Vm > Vthresh , then . (16.44)
(16.41) U = Ureset
For more complicated input current wave- 16.7.3.2 Quadratic Integrate-and-Fire

forms, the firing frequency may be calculated Models
analytically as well, making the LIF a useful tool The quadratic integrate-and-fire (QIF) model [21]
for obtaining mathematically precise descriptions is constructed by adding a nonlinear term, a de-
of neural activity. pendency on the square of voltage, to the LIF.
Unlike the LIF model, the QIF model generates
spikes intrinsically, without the imposition of an
16.7.3 Modified Integrate-and-Fire artificial threshold. It does, however, still rely on a
Models reset for repolarization. The QIF model is written
generally as
The leaky integrate-and-fire model has been ex-
dVm
tended in several ways in order to obtain more re- = Iinj + a (Vm − Vrest ) (Vm − Vthresh )
alistic dynamics at the expense of reduced math- dt
(16.45)
ematical tractability. The following are common
extensions of the LIF.
with the same reset condition as the LIF. Here, the
variable a is a constant controlling the excitability
16.7.3.1 Resonate-and-Fire Models
of the model, and Vthresh is the threshold voltage
The resonate-and-fire (RIF) model [29] is ob-
when Iinj = 0.
tained when a second dynamic variable, U, is
added to the LIF model. The variable, U, repre-
16.7.3.3 Complexity in Simple Models
sents the current mediated by voltage-gated ion
Izhikevich [31] developed and characterized a
channels or other processes acting as inductive
model combining the built-in threshold of QIF
elements in the membrane, as in Fig. 16.9a. In
with the resonance of RIF. This model is capable
contrast to the one-variable LIF, the addition of
of reproducing many of the behaviors observed
a second variable allows for models that act as
in a diverse set of biological neurons [31]. The
resonators, in that they display intrinsic oscilla-
dynamics of the Izhikevich model are governed
tory behavior and/or respond preferentially to os-
by the equations
cillatory input at certain frequencies (Fig. 16.9c).
The resonate-and-fire model is described by two dVm
coupled ordinary differential equations: = Iinj + 0.04Vm2 + 5Vm + 140 − U
dt
(16.46)
dVm Vm
Cm · = Iinj (t) − −U (16.42)
dt Rm
dU
= a (bVm − U ) (16.47)
dt
dU Vm U
= − . (16.43)
dt a b and the reset condition.
,
The parameters a and b are the inductance and Vm = c
If Vm > 30 mV, then (16.48)
the time constant of the resonant process, and U U =U +d
Fig. 16.10 The Izhikevich model, consisting of only two differential equations, can exhibit a wide variety of firing
patterns. (Electronic version of the figure and reproduction permissions are freely available at www.izhikevich.com)
with the parameters a, b, c, and d controlling nel activity, currents, conductances, and mem-
the behavior of the model. Figure 16.10 depicts brane potential. Although they can model the
many of the firing patterns that may be observed dynamics of real neurons and explain variations
by varying parameters of the Izhikevich model. in the membrane potential, they are challeng-
This example illustrates that a wealth of behaving to study analytically and can become over-
iors may be obtained in models with as few as whelming when applied to neural coding prob-
two dependent variables, although the biophysi- lems and large-network activity. In contrast, the
cal interpretation of parameters in simple models simple models discussed in Sect. 16.7 are math-
may not always be straightforward. It is a useful ematically tractable and can be theoretically ana-
exercise for students to code their own version of lyzed. With such models, capturing the dynamics
this model and to explore its range of behaviors. of real neurons depends on the dimensionality and
complexity of the model. The two-dimensional
Izhikevich model [31] provides a richer repertoire
16.8 Generalized Linear Model of more complex dynamics than the simpler one-
dimensional LIF. The drawback with these simple
Hodgkin-Huxley and other biophysically based mathematical models is that parameters generally
models can provide detailed accounts of ion chan- do not directly map to measurable biological mea-
surements. By construction, biophysical, LIF, and y(t) is a discrete spike train with a spike rate λ(t).
Izhikevich models are deterministic, but they can The spike count y(t) is conditioned on the rate λ(t)
be altered by adding either channel- or current- and has a Poisson distribution in an interval Δt:
based noise sources [32].
Δt λ(t) −Δt
In this section, we highlight a more abstract Pr (y(t)|λ(t)) = e λ(t)
. (16.49)
type of model, the generalized linear model y(t)!
(GLM) [33]. GLMs are compatible with a wide
variety of neurophysiological data, ranging
from current-clamp intracellular recordings to 16.8.1 Linear-Nonlinear Poisson
in vivo, single-unit recordings in response to Model
a physiological stimulus. GLMs are inherently
stochastic and thus naturally attempt to describe The linear-nonlinear-Poisson (LNP) model
the variability of neuronal spiking activity seen shown in Fig. 16.11a is a reduced form of the
in vivo. In these models, regression is used GLM that does not incorporate feedback. The
to fit the model and find a relation between spike rate in the LNP is given by
the covariates and experimental measures of
neuronal spiking activity. GLMs have been used λ(t) = f (K · x(t)) . (16.50)
successfully to model spiking activity in the early
stages of sensory [34] and motor [35] processing The linear filter K, estimated from physiologic
pathways. recordings, represents the receptive field of the
The basic framework of the GLM model is a neuron and is used to integrate the input stimulus
series of three processing stages (Fig. 16.11a), vector x(t). The nonlinear function f is a thresh-
with the addition of a possible feedback loop olding function that provides a nonnegative spike-
(Fig. 16.11b), that take an input stimulus x(t) and rate. An exponential function is the most common
produce an output spike train y(t). The sequence nonlinearity, but a soft-rectification function can
is summarized by an input linear stimulus filter, work as well.
followed by a nonlinear thresholding stage, and
finally an output stochastic process. Because a
Poisson point process is most commonly used as
16.8.2 Generalized Linear Model
the stochastic output process, we will focus on
with Spike History Dynamics
that design choice in this section.
The LNP can be extended to a GLM by adding
The Poisson process is fully characterized by
the effect of spiking history via a feedback in the
its instantaneous rate, λ(t). In the GLM, the output
Fig. 16.11 (a) Processing

stages of the a
linear-nonlinear-Poisson
model showing the linear
x(t) K ƒ λ(t)
Poisson
y(t)
input stimulus filter K, the
nonlinearity f, and Poisson
output stochastic process.
(b) Generalized linear
b m
model with spike history
feedback via post-spike
filter H x(t) K S ƒ λ(t)
Poisson
y(t)
H
model as depicted in Fig. 16.11b. With feedback, measurement of parameters in a given neuronal
the rate of the Poisson process becomes type. Similar results have been seen in other stud-
ies [37, 38]. Overall, this body of work empha-
λ(t) = f K · x(t) + H · y hist (t) + μ(t) . sizes the point that particular values of param-
(16.51) eters do not seem to be preserved by neurons.
Instead, it is believed that individual neurons of
The post-spike filter H is used to capture the a given type appear to have a mechanism by
effects of spiking history of the neuron yhist (t) has which they co-vary channel densities and perhaps
on itself. The parameter μ(t) is used to set the other parameters in order to “tune” resulting out-
baseline spiking activity. These allow the GLM put behavior [37]. This fascinating set of results
to account for adaptation and refractoriness, as gives credence to more mathematically abstract,
well as other spiking-dependent dynamics that are dynamical systems-based approaches of under-
not possible in the simpler LNP. By changing the standing the physiological behaviors of neurons.
stimulus and post-spike filters, the GLM has been As described in detail elsewhere [21], dynamical
shown [33] to capture the same spiking dynamics systems models depend not on the specific ion
produced by the Izhikevich model [31], which channels and their densities but rather on the
is illustrated in Fig. 16.10. By linking neurons underlying mathematical forms that give rise to
using coupling filters, the GLM incorporates the particular behaviors.
covariates that affect a neuron’s activity and its
input, history, and network activity [35].
16.9 Similar Phenotypes Arising

from Disparate Mechanisms
Trying to build a model neuron by matching the

full complement of channel types and densities
is a laudable approach. However, this method
is fraught with difficulty, because there can be
a great deal of variability in channel densities
from cell to cell. Golowasch and colleagues [36]
studied this problem in a population of cells from
the crab that act as highly stereotyped bursting
neurons in vivo. They had three major findings.
First, measured densities of sodium and potas-
sium channels were highly variable among dif-
ferent cells of the same class. This result is quite
surprising, given the apparently stereotyped be-
havior of the neural outputs from these cells.
Second, models with randomly chosen channel
densities span a large range of electrophysio-
logical behaviors, but many disparate choices of
channel densities replicate the correct stereotyped
pattern (Fig. 16.12a). Third, an “average” model,
with mean conductance-density values from their Fig. 16.12 (a) Similar firing patterns can arise from sub-
measurements, did not replicate the known in stantially different conductance densities. (b) Somewhat
vivo responses. The implication of this finding for similar conductance densities can give rise to much differ-
ent firing patterns. In both cases, insets show the details in
modeling work is that it may be literally impos-
finer time resolution. (Adapted from [36])
sible to build an accurate model from painstaking
16.10 Synapse Models t−t t−t

− spike − spike
Isyn (t) = H(tspike ) a e τfall − e τrise ,
Constructing models containing more than
a single neuron requires an accurate and (16.53)
computationally efficient representation of the
that captures both the rise and fall kinetics of the
synapses connecting them. The most common
waveform. The alpha function,
type of synaptic transmission incorporated into
computational models is ionotropic transmission.
Ionotropic synaptic transmission is mediated Isyn(t) = H tspike α 2 te−α·(t−tspike ) , (16.54)
by the activation of ionotropic channels on
a postsynaptic neuron by neurotransmitters which requires fewer computations than a dou-
released from a presynaptic neuron. Ionotropic ble exponential waveform, also has non-zero, but
channels are those that act as both the receptor coupled, rise and decay times. In the above equa-
and the ion channel; one or more neurotransmitter tions, H(t) denotes the Heaviside unit-step func-
molecules bind to the protein, inducing a tion,
conformational change that opens a pore in the ,
0, t < 0
membrane through which ions may flow. H (t) = . (16.55)
1, t ≥ 0
The excitatory AMPA (2-amino-3-(5-methyl-
3-oxo-1,2- oxazol-4-yl)propanoic acid) and Although in many circumstances synaptic ac-
NMDA (N-Methyl-D-aspartic acid) receptors tivation is well-approximated by the addition of
and the inhibitory GABAA (γ-aminobutyric a current source to a model neuron, the postsy-
acid A-type) receptor are responsible for the naptic effect is more accurately represented by
majority of fast chemical synaptic transmission a change in synaptic conductance. This disparity
in the nervous system. Models of ionotropic has two important implications. First, represent-
synapses take many forms. The simplest and ing synaptic activation as a conductance change
most amenable to mathematical analysis is the introduces a voltage dependency in the synap-
representation of synaptic transmission by a tic current due to the reversal potential of each
scaled delta function of current. In this model, channel. Second, an increase in open channels
the voltage of a postsynaptic neuron is simply following synaptic transmission leads to a change
incremented (decremented) following an action in the effective membrane resistance, resulting in
potential in a synaptically connected excitatory the “shunting” of other membrane currents. This
(inhibitory) neuron. indirect effect on excitability has particular im-
To more accurately capture the postsynaptic portance when considering inhibitory synapses.
effect of a presynaptic spike, synapses may be Synaptic conductance inputs may be described
represented by an input current waveform qualita- like other membrane conductances,
tively similar to those recorded in voltage-clamp
experiments from intact neurons. This synaptic Isyn (t) = Gsyn s(t) Vm − Vsyn (16.56)
current waveform may be described by an expo-
nentially decaying current where Gsyn represents the maximal conductance
of the synapse; s(t) is the so-called activation
t−t
− spike variable, a time-varying quantity that may be
Isyn (t) = H tspike a· e τfall (16.52)
described as a single exponential, double expo-
nential, or alpha function (Eqs. 16.52–16.55); and
that captures the decay time course of the synaptic Vsyn is the reversal potential of the channel.
current but reaches its maximum instantaneously An even more realistic representation of
or a double exponential waveform, synaptic activation can be obtained by modeling
ate parameters, NMDA receptors have an addi-

tional dependency on postsynaptic voltage. When
the membrane voltage at the postsynapse is low,
the NMDA pore is effectively blocked by free
Mg2+ ions. Subsequent depolarization removes
this “Mg2+ block,” allowing for the flow of cur-
rent when activated by neurotransmitter binding.
The added voltage dependency may be incorpo-
rated into the synapse model by modifying Eq.
16.56 as follows [30]:

Isyn (t) = GMg2+ · s(t)· Gsyn Vm − Vsyn
(16.59)
where
−1
Mg 2+ − Vm
GMg2+ = 1+ e 16.13 (16.60)
3.57 mV
Fig. 16.13 Synaptic currents (top panel) and resulting
changes in membrane potential (bottom panel) generated and [Mg2+ ] is the concentration of magnesium
by a variety of models of chemical synaptic transmission ions in the extracellular solution. The dependency
of this receptor on presynaptic transmitter release
transmitter release and the open probabilities as well as postsynaptic depolarization endows
of a receptor population. For example, if the NMDA receptors with a unique ability to sense
concentration of neurotransmitter in the synaptic coincident activity in both neurons. Markov mod-
cleft, [T], is assumed to be a square pulse with els, which capture channel kinetics with the high-
unitary amplitude and short duration, then the est precision, are also employed for accurate rep-
activation function, s(t), may evolve according to resentation of synaptic activation but are beyond
the scope of this chapter.
ds(t)
= αs (1 − s(t)) − βs · s(t) (16.57) Electrical synapses, which connect the cyto-
dt
plasm of neighboring cells, are made up of pore-
with the forward rate constant α s equal to its forming gap junction proteins and provide an-
maximal rate, α s , when [T] is high and zero when other mechanism of cell-to-cell communication.
[T] is low: Electrical synapses are often modeled as simple
resistors, with the current through the gap junc-
,
αs , [T ] = 1 tion, Igap , defined as
αs ([T ]) = . (16.58)
0, [T ] = 0
Vm,pre − Vm,post
Igap = (16.61)
Figure 16.13 illustrates the shape of current Rgap
and voltage waveforms resulting from con-
ductance inputs modeled using the waveforms where Vm,pre is the voltage of the presynaptic
described above. With the exception of the neuron, Vm,post is the voltage of the postsynaptic
delta function representation, each of the neuron, and Rgap is the resistance of the gap
aforementioned models approximates synaptic junction. Some gap junctions display rectification
input closely. (i.e., pass current more readily in one direction
While these formulations can be used to model than the other), and in some models, this charac-
AMPA and GABAA receptors, given appropri- teristic is included as well.
Metabotropic receptors, whose postsynaptic depression. This model expands the models of
actions are mediated by G-protein-coupled path- synaptic transmission described in the previous
ways, operate on a much slower time scale and section to include an additional term, the
impact postsynaptic excitability in ways too nu- probability of release, p(t), that multiplies the
merous to describe here. previous expression for the synaptic conductance:

Isyn = p(t)Gsyn s (t) Vsyn − Vm . (16.62)
16.11 Short-Term Synaptic
Plasticity Here, p(t) relaxes exponentially to its steady-
state value, p0 , during periods of quiescence,
The efficacy of synaptic transmission has been
shown to undergo dynamic changes depending dp(t)
= − (p − p0 ) /τp (16.63)
on the recent history of synaptic activation. dt
Short-term synaptic plasticity encompasses the
family of processes controlling usage-dependent and undergoes activity-dependent modification
changes in synaptic strength on the time scale of whenever an action potential occurs. The up-
tens of milliseconds to tens of seconds. Various date rule for the probability of release takes the
forms of synaptic enhancement and depression, form
leading to increases and decreases in synaptic
strength, respectively, are categorized by the p = p + fF (1 − p) (16.64)
time scale and mechanism of their action. We
focus on facilitation, fast enhancement caused for facilitating synapses and
by an increase in the probability of release
of presynaptic neurotransmitter-containing p = fD p (16.65)
vesicles, and depression, a fast decrease in
synaptic strength resulting from depletion of for depressing synapses. The factors fF and fD
the immediately releasable pool vesicles in the are related to the strength of the facilitation and
presynaptic terminal. Examples of synaptic depression processes, and both parameters are
facilitation and depression are depicted in fractional values less than one. The combination
Fig. 16.14. The biochemistry of these processes of the decay time constant τ p and the factors fF
is complex, and so we will focus on a simple and fD control the kinetics of the onset of synaptic
phenomenological model [30] that captures modification, for a given presynaptic firing rate,
the basic attributes of synaptic facilitation and as well as the time scale with which it decays in
a Facilitation b Depression
post post
pre pre
Fig. 16.14 (a) In synaptic facilitation, repeated presy- (b) In synaptic depression, postsynaptic potentials become
naptic trains of action potentials (bottom trace) lead to smaller with repeated inputs
successively larger postsynaptic potentials (top trace).
the absence of activity. Solving for the steady- is represented but also the pattern of connectiv-
state probability of release at a given firing rate ity and the rules that govern each connection.
is straightforward and is left to the reader as an This follows from the abundance of neuronal and
exercise. synaptic dynamics described above as well as
This model assumes a large number of release the limitless topologies that may be envisioned.
sites per synaptic connection, so that the effective Here, we highlight several examples that have
postsynaptic current may be represented by the been particularly influential.
deterministic Eq. 16.62. In the case of a small
number of release sites, the effective postsynaptic
conductance must be determined stochastically. 16.12.1 Feed-Forward Networks
Networks that are purely feed-forward are those

16.12 Beyond Single Neurons comprised of multiple populations in which one
population, or layer, projects to another, and this
Although the modeling of single neurons is a one to a third, and so forth in a unidirectional
rich and diverse topic that has added much to chain (Fig. 16.15a). Cortical processing of sen-
our knowledge of how neurons operate, these sory input can be abstracted as occurring in a
models are relatively constrained compared to largely feed-forward manner. One particular in-
what might be represented computationally at carnation of a feed-forward network is the syn-
a network level. Network models allow one to fire chain, first described by Griffith [41] and
choose not only the manner in which each neuron later explored in detail by Abeles [42]. A synfire
Fig. 16.15 (a) Schematic representation of a feedfor- or maintained (c). (d) Sustained activity recorded exper-
ward network in which “packets” (correlated firing in imentally in a working memory task and in a model (e).
groups of modeled neurons) can either be dispersed (b) (Panel (d) adapted from [39] and panel (e) adapted from
[40])
chain is a strictly feed-forward network contain- 16.12.2 Persistent Activity

ing many layers in which the first layer receives
a stimulating input that excites a subset of neu- Persistent activity, the autonomous sustained fir-
rons in this layer. Input to the synfire chain may ing of a population or subpopulation of neurons
be a synchronous “packet,” or an asynchronous, in the absence of external input, has been ob-
continuous input. Connections between layers are served in a variety of vertebrates during specific
described as all-to-all if every neuron in the first behavioral states in vivo. Of particular interest has
layer projects to every neuron in the second layer been the persistent firing described in a variety
and so forth. A specific pattern or statistical rule of cortical areas of the monkey during work-
may be used to govern the connections between ing memory tasks. While performing a task that
layers as well. requires the subject to remember a cue for a
When the connections between layers are all short delay period, a subset of neurons display
excitatory and simple neuron models are used to an increased firing rate, which remains stable
represent constituent cells, propagation of activity over time, until the precise moment when the
through the chain may occur stably only when delay period ends (Fig. 16.15d). The mechanisms
synchronized volleys of activity are transmitted underlying persistent activity remain unknown,
from layer to layer (Fig. 16.15c). If the input although modeling studies have helped tremen-
to the first layer in the chain is incapable of dously in determining when and how persistent
exciting a sufficient number of cells, then activity activity may arise.
will eventually die out as it propagates through Most models of persistent activity during
successive layers (Fig. 16.15b). If the first layer working memory tasks assume that this form of
is initially excited in an asynchronous fashion, its activity originates at the network level, mediated
degree of synchrony will increase in successive in single neurons by synaptic input. Initial
layers until the activity among all of the neurons theoretical work implicated the involvement
in each layer becomes synchronous. This result of local recurrent excitation in this process.
has been examined in synfire chains that con- Recurrent excitation is capable of providing
tain more realistic neuron models, that contain long-lasting activation after the termination of
inhibitory connections, and whose connectivity is external input, as the activated subset of neurons
sparse (e.g., <10% probability of connection beat any instant provides additional excitation
tween neurons in successive layers). While chal- to other cells in the local network. Recurrent
lenging to ascertain generally what patterns may activation alone, however, does not lead to stable,
be stably propagated in these, more complex, constant activation at the firing rates observed
chains, it is believed that asynchronous activity in vivo. Much like the scenario described in
may only be transmitted under these conditions synfire chains, initial inputs activating only
when constituent neurons generate action poten- a small subpopulation decay with time and
tials periodically. Although the synfire chain, as input magnitudes above a certain threshold rise
described in isolation, represents a highly ideal- exponentially until unrealistic rates are obtained.
ized abstraction, the study of these networks has Local feedback inhibition from a population of
provided understanding of how activity patterns interneurons has been proposed as a means to
may propagate through networks of neurons. Ad- control excessive excitation. Networks containing
ditionally, further research on synfire chains has a balance of excitatory and inhibitory inputs were
indicated that the scenario in which many in- found capable of producing persistent activity,
terwoven chains are contained within a single but only when parameters governing this balance
neuronal population may have greater biological were tuned with a precision unlikely to occur in
relevance. a biological system.
Various solutions have been proposed that tions that determine the extracellularly evoked
alleviate the requirement for fine parameter electric field are well understood. Third, there ex-
tuning: saturating synaptic conductances or firing ist accurate models of axonal extracellular stimu-
rates, involvement of slow NMDA receptors, lation.
and the recruitment of multiple interneuron As an example, we highlight here a par-
populations. Using these modifications, modeling ticularly influential computational study of
studies have been able to closely approximate deep brain stimulation (DBS) by McIntyre and
the qualitative behavior recorded experimentally colleagues [44]. By coupling a finite-element
(Fig. 16.15e). However, several aspects of model of the electrode-induced field with a
persistent activity have eluded explanation in conductance-based model of neurons in the thala-
modeling studies. Spike-frequency adaptation mus, their simulation results suggested that DBS
and periods of enhanced neural synchrony stimuli can stimulate activity in appropriately
seem to disrupt persistent firing in models. oriented axons while simultaneously suppressing
When persistent firing is obtained in a model, activity in the cell bodies. This computational
termination of this behavior typically leads to result was important in helping researchers to
a brief enhancement or depression of firing understand paradoxical results in recordings from
rates relative to rest that are not observed in DBS patients. More recently, similar models
experimental recordings. Further observations have been used to design customized methods
that the irregularity of firing increases during that attempt to optimize DBS effectiveness
the persistently enhanced state relative to while minimizing side effects in individual
spontaneous firing in the rest state are not patients [45].
captured by many models. Although these
inconsistencies remain to be resolved, modeling
studies have helped to clarify which cellular 16.14 Modeling Resources
mechanisms may be plausible for the generation
of persistent activity and the conditions under A bevy of powerful tools and resources are avail-
which it cannot occur. able to aid in the construction and simulation of
neural models. NEURON (www.neuron.yale.edu/
neuron) specializes in the simulation of
16.13 Neural Modeling in Medicine conductance-based neurons and allows for
the construction of models with realistic
Given the number of uncertainties involved in morphologies. This software package has
constructing a detailed neural model, it is an undergone much development, resulting in
immense challenge to construct a model that is extensive functionality for tasks as diverse as
accurate enough for medical application. The suc- importing anatomical data and interfacing with
cessful examples, like the work of Traub and real-time dynamic clamp software. GENESIS
colleagues on computational models of epilepsy (genesis-sim.org) was developed for purposes
[43], involve immense amounts of work and ex- similar to NEURON and attempts to provide a
traordinary attention to detail. One medical do- platform for the investigation of neural models
main in which neural modeling has proven quite across spatial scales ranging from the subcellular
useful is that of modeling field effects related to to the systems level. BRIAN, a neural simulator
electrical stimulation of neural structures. Rela- developed using the Python programming
tive to many other neural modeling problems, this language (www.briansimulator.org), attempts to
application has distinct advantages. First, it de- simplify the process of constructing complicated
pends strongly on anatomy that can be measured networks. The NEST initiative (www.nest-
accurately in three dimensions using noninva- initiative.org) has focused on the development
sive methods like traditional and diffusion-tensor of algorithms and visualization methods as well
magnetic resonance imaging. Second, the equa- as in simulation software.
In addition to these simulators, repositories 2. Draw an equivalent circuit diagram of the cell
of computational models are another useful in Exercise 1 assuming it has a membrane
resource. Databases such as ModelDB, cellML, capacitance Cm . Find the resting potential
and Visiome have been introduced to solve two of the cell. How does doubling the external
inherent challenges in computational modeling. potassium concentration change the resting
The first goal of these databases is to make potential?
published models freely and easily available to 3. Use the Hodgkin-Huxley gating variables to
any individual who wishes to investigate them. describe how an action potential is generated.
This goal is largely accomplished through the What happens if the time constant τ m of the
cooperation of individuals who submit their sodium conductance activation is increased?
models to the databases voluntarily. ModelDB 4. Find the rheobase current of a leaky
(senselab.med.yale.edu/modeldb) alone currently integrate-and-fire neuron with the pa-
has well over a thousand submissions. A second rameters Rm = 50 M , Cm = 2000 pF,
goal of databases like the ones listed here is to Vthresh = −50 mV, and Vreset
standardize the framework of neural models. = −70 mV. What is the injected current
CellML (www.cellml.org), for example, in required to have the neuron fire repeatedly at
addition to serving as a central repository, has 10 Hz?
introduced a markup language proposed as a 5. Write a program to simulate the LIF neuron
common language for constructing and sharing in Exercise 4 using time steps of 0.5 ms. Use
mathematical models of biological systems. More an artificially added spike with amplitude
specific databases have also been introduced, 20 mV (for a single time step) to indicate
such as Visiome (visiome.neuroinf.jp), which the occurrence of an action potential. Plot the
specializes in tools and resources restricted to response to the following currents:
vision research. These efforts are important (1) 640 pA current step with 1 s duration.
in fostering collaboration and standardiza- (2) Current linearly increasing from 0 nA to
tion among those using neural modeling to 1 nA over a duration of 5 s.
better understand how the nervous system 6. Find expressions for the time of the peak tpeak
functions. of postsynaptic potentials for the difference
of exponentials synapse model and for the al-
pha function model. The expressions should
Homework be in terms of the model parameters: τ rise ,
τ fall , α, and tspike .
1. A cell with ion-selective channels has dif- 7. Derive Eq. 16.41 for the firing frequency of
ferent concentrations [Na+ ], [K+ ], and [Cl− ] an LIF neuron with a constant current injec-
inside and outside the cell as given in Table tion.
16.2. Calculate the reversal potential Veq for 8. Write a program that simulates a neuron
each of the ions at 310 K. using the Izhikevich model given by Eqs.
16.46–16.48. Find the parameters a, b, c, and
Table 16.2 Internal and external concentrations of d that produce tonic spiking, phasic spiking,
ions for cell tonic bursting, and phasic bursting.
[Internal] [External] Conductance 9. Write a program that simulates an exponen-
Ion (mM) (mM) (S/cm2 ) tially decaying synapse with τ fall = 20 ms,
Na+ 12 145 1 × 10−5 peak conductance of 25 nS, and excitatory
K+ 150 4 1 × 10−4 synaptic reversal potential of 0 mV. Use
Cl− 5 100 2 × 10−5 an LIF neuron with with Rm = 50 M ,
Cm = 2000 pF, Vthresh = −50 mV, and
Vreset = −70 mV for the model. Plot the
response of the neuron to 5 presynaptic action 14. B. Hutcheon, Y. Yarom, Resonance, oscillation and
potentials occurring at a rate of 40 Hz. Repeat the intrinsic frequency preferences of neurons. Trends
Neurosci. 23(5), 216–222 (2000)
the simulation with synaptic facilitation
15. C.T. Dickson, J. Magistretti, M.H. Shalinsky, E. Fran-
having p0 = 0.25, τ p = 100 ms, and fF = 0.3. sén, M.E. Hasselmo, A. Alonso, Properties and role
To compare the effect of facilitation, use of I(h) in the pacing of subthreshold oscillations in
the same plasticity factor p0 = 0.25 in both entorhinal cortex layer II neurons. J. Neurophysiol.
83(5), 2562–2579 (2000)
simulations.
16. S.P. Aiken, B.J. Lampe, P.A. Murphy, B.S. Brown,
10. For an unmyelinated fiber, describe quanti- Reduction of spike frequency adaptation and block-
tatively how the attenuation of a signal is ade of M-current in rat CA1 pyramidal neurones by
affected by the length and radius of the fiber. linopirdine (DuP 996), a neurotransmitter release en-
hancer. Br. J. Pharmacol. 115(7), 1163–1168 (1995)
17. D.V. Madison, R.A. Nicoll, Control of the repetitive
discharge of rat CA 1 pyramidal neurones in vitro. J.
References Physiol. 354, 319–331 (1984)
18. I.A. Fleidervish, A. Friedman, M.J. Gutnick, Slow
1. T.F. Weiss, Cellular Biophysics, Vol. 2: Electrical inactivation of Na+ current and slow cumulative
Properties (MIT Press, Cambridge, MA, 1996) spike adaptation in mouse and guinea-pig neocortical
2. T.F. Weiss, Cellular Biophysics, Vol. 1: Transport neurones in slices. J. Physiol. 493(Pt 1), 83–97 (1996)
(MIT Press, Cambridge, MA, 1996) 19. S.R. Williams, S.R. Christensen, G.J. Stuart, M.
3. D. Johnston, S.M.-S. Wu, Foundations of Cellular Häusser, Membrane potential bistability is controlled
Neurophysiology, 1st edn. (MIT Press, Cambridge, by the hyperpolarization-activated current I(H) in
MA, 1994) rat cerebellar Purkinje neurons in vitro. J. Physiol.
4. A.L. Hodgkin, A.F. Huxley, A quantitative descrip- 539(2), 469–483 (2002)
tion of membrane current and its application to con- 20. R. Guttman, S. Lewis, J. Rinzel, Control of repetitive
duction and excitation in nerve. J. Physiol. 117(4), firing in squid axon membrane as a model for a
500–544 (1952) neuroneoscillator. J. Physiol. 305, 377–395 (1980)
5. W. Rall, Theoretical significance of dendritic trees for 21. E.M. Izhikevich, Dynamical Systems in Neuro-
neuronal input-output relations, in Neural Theory and science: The Geometry of Excitability and Bursting
Modeling, ed. by R. F. Reiss, (Stanford University (MIT Press, Cambridge, MA, 2007)
Press, Stanford, 1964) 22. W. Rall, Membrane time constant of motoneurons.
6. W. Rall, The Theoretical Foundation of Dendritic Science (New York, N.Y.) 126(3271), 454 (1957)
Function: Selected Papers of Wilfrid Rall with Com- 23. G. Stuart, N. Spruston, M. Häusser, Dendrites, 2nd
mentaries (MIT Press, Cambridge, MA, 1995) edn. (Oxford University Press, Oxford, 2007)
7. L. Goldman, J.S. Albus, Computation of impulse 24. J. Schiller, Y. Schiller, G. Stuart, B. Sakmann, Cal-
conduction in myelinated fibers; theoretical basis of cium action potentials restricted to distal apical den-
the velocity-diameter relation. Biophys. J. 8(5), 596– drites of rat neocortical pyramidal neurons. J. Physiol.
607 (1968) 505(Pt 3), 605–616 (1997)
8. B. Hille, Ion Channels of Excitable Membranes, 3rd 25. G.J. Stuart, B. Sakmann, Active propagation of so-
edn. (Sinauer, Sunderland, 2001) matic action potentials into neocortical pyramidal cell
9. D.A. McCormick, H.C. Pape, Properties of a dendrites. Nature 367(6458), 69–72 (1994)
hyperpolarization-activated cation current and its role 26. D. Johnston, R. Narayanan, Active dendrites: Color-
in rhythmic oscillation in thalamic relay neurones. J. ful wings of the mysterious butterflies. Trends Neu-
Physiol. 431, 291–318 (1990) rosci. 31(6), 309–316 (2008)
10. S.M. Sherman, Tonic and burst firing: Dual modes of 27. J. Schiller, G. Major, H.J. Koester, Y. Schiller, NMDA
thalamocortical relay. Trends Neurosci. 24(2), 122– spikes in basal dendrites of cortical pyramidal neu-
126 (2001) rons. Nature 404(6775), 285–289 (2000)
11. F.R. Fernandez, J.D.T. Engbers, R.W. Turner, Firing 28. L. Lapicque, Recherches quantitatives sur l’excitation
dynamics of cerebellar purkinje cells. J. Neurophys- électrique des nerfs traitée comme une polarisation. J.
iol. 98(1), 278–294 (2007) Physiol. Pathol. Gen. 9, 620–635 (1907)
12. C.A. Del Negro, S.H. Chandler, Physiological and 29. E.M. Izhikevich, Resonate-and-fire neurons. Neural
theoretical analysis of K+ currents controlling dis- Netw. 14(6–7), 883–894 (2001)
charge in neonatal rat mesencephalic trigeminal neu- 30. P. Dayan, L.F. Abbott, Theoretical Neuroscience:
rons. J. Neurophysiol. 77(2), 537–553 (1997) Computational and Mathematical Modeling of Neu-
13. A. Alonso, R.R. Llinás, Subthreshold ral Systems (MIT Press, Cambridge, MA, 2001)
Na+−dependent theta-like rhythmicity in stellate 31. E.M. Izhikevich, Which model to use for cortical
cells of entorhinal cortex layer II. Nature 342(6246), spiking neurons? IEEE Trans. Neural Netw. 15(5),
175–177 (1989) 1063–1070 (2004)
32. J.A. White, J.T. Rubinstein, A.R. Kay, Channel noise 39. S. Funahashi, C.J. Bruce, P.S. Goldman-Rakic,
in neurons. Trends Neurosci. 23(3),break 131–137 Mnemonic coding of visual space in the monkey’s
(2000) dorsolateral prefrontal cortex. J. Neurophysiol. 61(2),
33. A.I. Weber, J.W. Pillow, Capturing the dynamical 331–349 (1989)
repertoire of single neurons with generalized linear 40. A. Compte, N. Brunel, P.S. Goldman-Rakic, X.J.
models. Neural Comput. 29(12), 3260–3289 (2017) Wang, Synaptic mechanisms and network dynamics
34. J.W. Pillow et al., Spatio-temporal correlations and underlying spatial working memory in a cortical net-
visual signalling in a complete neuronal population. work model. Cerebral Cortex (New York, N.Y.: 1991)
Nature 454(7207), 995–999 (2008) 10(9), 910–923 (2000)
35. W. Truccolo, U.T. Eden, M.R. Fellows, J.P. 41. J.S. Griffith, On the stability of brain-like structures.
Donoghue, E.N. Brown, A point process framework Biophys. J. 3, 299–308 (1963)
for relating neural spiking activity to spiking history, 42. M. Abeles, Local Cortical Circuits: An Electrophysi-
neural ensemble, and extrinsic covariate effects. J. ological Study (Springer, Berlin, 1982)
Neurophysiol. 93(2), 1074–1089 (2005) 43. R. Traub, R. Wong, Cellular mechanism of neuronal
36. J. Golowasch, M.S. Goldman, L.F. Abbott, E. synchronization in epilepsy. Science 216(4547), 745–
Marder, Failure of averaging in the construction of 747 (1982)
a conductance-based neuron model. J. Neurophysiol. 44. C.C. McIntyre, W.M. Grill, D.L. Sherman, N.V.
87(2), 1129–1131 (2002) Thakor, Cellular effects of deep brain stimulation:
37. E. Marder, J.-M. Goaillard, Variability, compensation Model-based analysis of activation and inhibition. J.
and homeostasis in neuron and network function. Nat. Neurophysiol. 91(4), 1457–1469 (2004)
Rev. Neurosci. 7(7), 563–574 (2006) 45. C.R. Butson, S.E. Cooper, J.M. Henderson, C.C.
38. A.M. Swensen, B.P. Bean, Robustness of burst firing McIntyre, Patient-specific analysis of the volume of
in dissociated purkinje neurons with acute or long- tissue activated during deep brain stimulation. Neu-
term reductions in sodium conductance. J. Neurosci. roImage 34(2), 661–670 (2007)
Off. J. Soc. Neurosci. 25(14), 3509–3520 (2005)
Linear Dynamics and Control of Brain
Networks 17
Jason Z. Kim and Danielle S. Bassett
Abstract as a useful tool both for capturing biophysi-

cally relevant parameters of neural activity and
The brain is an intricately structured organ
connectivity and for analytical and numerical
responsible for the rich emergent dynamics
study. We begin with a brief overview of state-
that support the complex cognitive functions
space representations and linearization of neu-
we enjoy as humans. With around 1011 neu-
ral models for non-linear dynamical systems.
rons and 1015 synapses, understanding how
We then derive core concepts in the theory of
the human brain works has proven to be a
linear systems such as the impulse and con-
daunting endeavor, requiring concerted collab-
trolled responses to external stimuli, achieving
oration across traditional disciplinary bound-
desired state transitions, controllability, and
aries. In some cases, that collaboration has
minimum energy control. Afterward, we dis-
occurred between experimentalists and techni-
cuss recent advances in the application of lin-
cians, who offer new physical tools to measure
ear systems theory to structural and functional
and manipulate neural function. In other con-
brain data across multiple spatial and temporal
texts, that collaboration has occurred between
scales, along with methodological considera-
experimentalists and theorists, who offer new
tions and limitations. We close with a brief
conceptual tools to explain existing data and
discussion of open frontiers and our vision for
inform new directions for empirical research.
the future.
In this chapter, we offer an example of the
latter. Specifically, we focus on the simple but Keywords
powerful framework of linear systems theory
Linear systems theory · Control theory ·
Dynamical brain networks · Controllability ·
Simple models
17.1 Emergence in the Structure
J. Z. Kim and Function of Complex
Department of Bioengineering, University of
Pennsylvania, Philadelphia, PA, USA Systems
D. S. Bassett ()
Departments of Bioengineering, Electrical and Systems In the observable world, some of the most beau-
Engineering, Physics and Astronomy, Neurology, and tiful and most puzzling phenomena arise in phys-
Psychiatry, University of Pennsylvania, Philadelphia, PA, ical and biological systems characterized by het-
USA erogeneous interactions between constituent ele-
Santa Fe Institute, Santa Fe, NM, USA ments. For example, in materials physics, hetero-

498 J. Z. Kim and D. S. Bassett
geneous interactions between particles in granu- Some efforts have sought to develop detailed
lar matter (such as a sand pile) constrain whether multiscale computational models [14]. Yet such
the matter acts as a liquid (flowing with grav- efforts are faced with the ever-present quandary
ity) or a solid (supporting load-bearing) [1, 2]. that, in point of fact, “the best material model
In sociology, heterogeneous interactions between of a cat is another, or preferably the same, cat”
humans in a society are thought to be responsi- [15]. Detailed models are difficult to construct
ble for surges in online activity, peaks in book and intractable to analytic approaches, require ex-
sales, traffic jams, and correlated spikes in de- tensive time to simulate, contain parameters that
mand for emergency services [3]. In biology, het- are frequently underconstrained by experimental
erogeneous interactions between computational data, and in the end produce dynamics that are
units in the brain are thought to support a di- themselves difficult to understand or to explain
vergence of the correlation length, an anomalous from any specific choices in the model. In con-
scaling of correlation fluctuations, and the man- trast, approaches from physics consider natural
ifestation of mesoscale structure in patterns of phenomena as if dynamics at macroscopic length
functional coupling between units, all features scales were almost independent of the underlying,
that allow for a diversity of dynamics underlying a shorter length scale details [16]. A hallmark of ef-
diversity of cognitive functions [4,5]. The feature fective physical theories is a marked compression
of these systems that often drives our fascination of the full parameter space into a few governing
is the capacity for heterogeneous interactions to variables that are sufficient to describe the observ-
produce suprising dynamics, in the form of dras- ables of interest at the scale of interest. Interest-
tic state transitions, spikes of collective activity, ingly, recent theoretical work demonstrates that
and multiple accessible dynamical regimes. such simple models are the natural culmination
Because element-element interactions are of processes maximizing the information learned
heterogeneous in such systems, traditional from finite data [17].
approaches from statistical mechanics – such Here we embrace simplicity by considering
as continuum models and mean-field approxima- the utility of linear systems theory for the under-
tions – fail to offer satisfying explanations for standing and control of neural systems comprised
system function. There exists a critical need to of computational units coupled by heterogeneous
develop alternative approaches to understand how interactions. We begin by placing our remarks
interactions map to emergent behavior. The need within the context of quantitative dynamical mod-
is particularly salient in the context of neural els of neurons and their interactions, as well as
systems, where such an understanding could the spatial and temporal considerations inherent
directly inform models of neurological disease in choosing such models. We will then turn to a
and psychiatric disorders [6, 7]. Moreover, gain- discussion of approximations to those dynamical
ing such an understanding is a prerequisite for models, the incorporation of exogeneous control
the well-reasoned development of interventions input, and model linearization. Our treatment then
[8], whether in the form of brain stimulation naturally brings us to a discussion of the theory of
[9, 10], pharmacological agents [11, 12], or other linear systems, as well as their response to pertur-
therapies [13]. Technically, such interventions bative impulses, and to explicit control strategies.
in systems characterized by heterogeneous We lay out the formalism for probing state tran-
interactions can be parsimoniously considered sitions, controllabilty, and the minumum control
as forms of network control, thus motivating energy needed for a given state transition. After
extensive recent interest in the utility of network completing our formal treatment, we discuss the
control theory for neural systems [8]. application of linear systems theory to neural
Despite the generic importance of understand- systems, and efforts to map network architecture
ing how interactions map to emergent properties, to control properties. We close with a description
and the specific importance of understanding that of several particularly pertinent methodological
mapping in the human brain, progress toward that considerations and limitations, before outlining
understanding has remained surprisingly slow. emerging frontiers.
17 Linear Dynamics and Control of Brain Networks 499
17.2 Quantitative Dynamical (Fig. 17.1, left) was developed by Alan Hodgkin
Models of Neural Systems and Andrew Huxley in 1952 (see [18] for details).
and Interactions The model is now known as the Hodgkin-Huxley
model. It treats a segment of a neuron as an elec-
Historically, many neural behaviors and mecha- trical circuit, where the membrane (capacitor) and
nisms have been successfully modeled quantita- voltage-gated ion channels (resistors) are parallel
tively. Here we briefly describe several illustra- circuit elements. The time evolution of membrane
tive examples of such models. The classic fun- voltage, Vm , between the inside and the outside of
damental biophysical model of a single neuron the neuron is given by
Cm V̇m (t) = ḡK n4 (t)(VK − Vm ) + ḡN a m3 (t)h(t)(VN a − Vm ) + ḡl (Vl − Vm ) + I (t),
where Cm is the membrane capacitance; ḡK , ḡN a ,

and ḡl are maximum ion conductances for potas- interrogation. An alternative approach is to
sium, sodium, and passive leaking ions; and I is capture the simplest aspects of neural interactions
an external stimulus current, all per unit area. In that are crucial for the phenomenon of interest.
addition, VK , VN a , and Vl represent the reversal Such was the approach taken by Warren
potential of these ions. The variables n, m, and h McCulloch and Walter Pitts [23], who developed
vary between 0 and 1 and model the ion channel what would later become a canonical model of
gate kinetics to determine the fraction of open an artificial neuron. In this model, each neuron i
sodium (m, h) and potassium (n) channels: at any point in time t exists in one of two states:
firing xi (t) = 1 or not firing xi (t) = 0. The
ṅ(t) = αn (Vm (t))(1 − n(t)) − βn (Vm (t))n(t) state of the neuron is determined by a weighted
sum of inputs from connected neurons j at the
ṁ(t) = αm (Vm (t))(1 − m(t)) − βm (Vm (t))m(t)
previous time step. Then, neuron i in a system of
ḣ(t) = αh (Vm (t))(1 − h(t)) − βh (Vm (t))h(t), N neurons evolves in time as
⎛ ⎞
where the functions αi (Vm ) and βi (Vm ) are em- N
pirically determined. These segments are then xi (t + 1) = fi ⎝ wij xj (t)⎠ ,
spatially connected together, such that the prop- j =1
agation of an action potential across a neuron is

modeled by a set of partial differential equations. where wij is the strength of excitation (wij >
Due to the biophysical realism of variables and 0) or inhibition (wij < 0) from neuron j to
parameters, this model can make powerful and neuron i and function fi is typically a threshold-
accurate predictions of neuron activity in different ing function (Fig. 17.1, center). Instantiations and
environments and stimulation regimes [19–21]. extensions of this model are used to study associa-
Simplified versions of this model, such as the tive memory (Hopfield [24]), machine learning
FitzHugh-Nagumo model [22], can also produce (perceptron [25]), and cellular automata [26].
many of the same neuronal dynamics. In many cases, the sheer number of neurons
However, many complex behaviors of neural and interactions renders even these simple models
systems arise from interactions between multiple difficult to study. A typical solution is to instead
neurons. With four variables (membrane voltage, model the average activity of a population of
gates) and even more parameters to model the neurons. This is the approach taken by Hugh Wil-
behavior of a single neuron, the space of models son and Jack Cowan [27] in the Wilson-Cowan
to explore interacting neurons quickly becomes model. Here, a group of neurons is separated into
intractable to both analytical and numerical excitatory and inhibitory populations, where the
Fig. 17.1 Schematic of neural models and controlling neurons (center). The Wilson-Cowan model describes the
perturbations at different scales. Here, the Hodgkin- activity of large neural populations in a region (right)
Huxley model describes the biophysical behavior of single or in a cortical column by modeling the excitatory and
neurons (left) that may be excitatory (blue) or inhibitory inhibitory connections of each population. In each case,
(gray). The artificial neuron models describe the sim- a controlling perturbation (yellow) can affect the neural
plified weighted connections and binary states of many system at different scales
fraction of cells firing at time t in each population 17.2.1 Spatial and Temporal
is E(t) and I (t), respectively, that evolve in time Considerations
as
When modeling neural systems, an immediately
τe Ė(t) = −E(t) + (ke − re E(t))Se (c1 E(t) salient consideration is the vast range of spatial
and temporal scales at which nontrivial – and
−c2 I (t) + P (t))
thus quite interesting – dynamics occur. It stands
τi I˙(t) = −I (t) + (ki − ri I (t))Si (c3 E(t) to reason that the most relevant type of model
for understanding a given phenomenon depends
−c4 I (t) + Q(t)) . on the spatiotemporal scale at which that phe-
nomenon is observed. For example, consider the
Here, c1 , c2 > 0 represent connection strength fact that while it is generally known that cer-
into the excitatory population, and c3 , c4 > 0 tain sensory regions such as the visual cortex
represent connection strength into the inhibitory are both anatomically linked to and functionally
population, re , ri are the refractory periods, and responsible for sensory inputs, it is more diffi-
Se , Si are sigmoid functions from the distribution cult to assign a set of neurons that are neces-
of neuron input thresholds for firing. Such models sary for distributed cognitive processes such as
produce oscillations such as those observed in attention and cognitive control. Thus, biophysical
noninvasive measurements of large-scale brain models at the level of single neurons may be
activity (Fig. 17.1, right) in patients with epilepsy viable for simulating receptive fields in visual
[28]. processing, but may be less useful for studies of
In these and many other models, a common task-switching or gating. Similarly, consider the
theme is the tradeoff between realism and fact that a single neuron may fire every few mil-
tractability. We desire sufficient realism to study liseconds, while human reaction times are on the
crucial features of neural systems such as the order of hundreds of milliseconds, and brain-wide
activity of each unit, the interaction strength fluctuations in activity on the order of seconds.
between units, the connection topology, and the Thus, the form of the model considered should
effect of external stimulation. We also desire match the temporal scales of the behavior to be
sufficient tractability (either to analytical or studied.
numerical interrogation) to make consistent From a modeling perspective, balancing these
and meaningful predictions about our neural considerations of spatial and temporal scales with
system by understanding relations between the model realism impacts the category of model that
model parameters and the model behavior. In this has the greatest utility. If one wishes to con-
chapter, we will discuss one such model from the sider small spatial scales, then a rather simplistic
theory of linear dynamical systems.
neuron-level model such as the McCulloch-Pitts tems as well. We begin our formulation with a set
may be particularly useful, where each neural of N neural units, where each unit has an associ-
unit has discrete states such that each neuron i ated level of activity xi (t) that is a real number at
is either firing xi (t) = 1 or not xi (t) = 0. In some time t ≥ 0 that is also a real number. Then
contrast, if one wishes to consider larger spatial the collection of activity for all units into column
scales characteristic of distributed cognitive pro- vector x(t) = [x1 (t); x2 (t); · · · ; xN (t)] is called
cesses, it may be more appropriate to consider the state of our system at time t. For example, in
models in which each neural unit reflects the the Hodgkin-Huxley equations, our state vector
average population activity of a brain region as is x = [V ; n; m; h]. In many models including
a continuous state, where xi (t) is a real number. Hodgkin-Huxley, the time evolution of the sys-
Similar considerations are relevant and important tem states can be written as a vector differential
in the time domain. For models that assume fairly equation:
uniform delays in neuronal interactions such as ⎡ ⎤ ⎡ ⎤
the McCulloch-Pitts, a discrete time model where ẋ1 (t) f1 (x(t))
⎢ ẋ2 (t) ⎥ ⎢ f2 (x(t)) ⎥
time evolves in integer increments may be appro- ⎢ ⎥ ⎢ ⎥
⎢ .. ⎥ = ⎢ .. ⎥,
priate. In contrast, if the timing of interactions ⎣ . ⎦ ⎣ . ⎦
between neural units such as myelinated versus ẋN (t) fN (x(t))
unmyelinated axons is heterogeneous, a continu- 0 12 3 0 12 3
ẋ(t) f (x(t))
ous time model may be more suitable, where time
t is a real number. where f , the vector of functions fi , determines
In addition to affecting the definition of neural how the system states change, ẋ, at every partic-
activity and the nature of its propagation, these ular state x. We can think of these equations as
considerations also affect the meaning of inter- generating a vector field, where at each point x,
actions between units. In a neuron-level model we draw an arrow with magnitude and direction
whose units reflect neurons, the unit-to-unit in- equal to f (x). As an example, consider the fol-
teractions may represent structural synapses be- lowing two neuron system x1 , x2 that evolves in
tween neurons. In contrast, in a population model time as:
whose units reflect average neural activity of a
brain region, unit-to-unit interactions may repre- ẋ1 (t) = 2x2 (t) − sin(x1 (t))
sent a summary measure of the collective strength
ẋ2 (t) = x12 (t) − x2 (t),
or extent of structural connections between re-
gions. Both types of connections can be empir- where the vector field and example trajectory
ically measured using either invasive (staining, from initial state x(0) = [−0.3; −0.4] are shown
flourescence imaging, tract tracing [29]) or non- (Fig. 17.2, top). Note how at every point x1 , x2
invasive (tractography [30]) methods. The spe- the above equation determines a vector of motion
cific type of interaction studied constrains the ẋ that the system traces from the initial point.
sorts of inferences that one can draw from the This quantitative modeling of neural dynamics
subsequent model, as well as the types of model- allows us to study and predict the response of our
generated hypotheses that one can test in new neural system to changes in interaction strength
experiments. or external stimulation.
17.2.2 Dynamical Model 17.2.3 Incorporating Exogenous

Approximations Control
Both here and in the following sections, we will While modeling intrinsic system behavior is al-
consider systems with both continuous state and ready a broad topic of current research, there is
time. However, we note that the theory of linear an increasing need for the principled study of
systems extends naturally to discrete time sys- therapeutic interventions to correct dysfunctional
neural activity. These interventions may take the As an example in our two-unit system, we can
form of targeted invasive (deep bran stimulation) apply an input to the first unit
or noninvasive (transcranial magnetic stimula-
tion) inputs, or more diffusive drug treatments. ẋ1 (t) = 2x2 (t) − sin(x1 (t)) + u(t)
Hence, in our modeling efforts, we also often ẋ2 (t) = x12 (t) − x2 (t),
desire to incorporate the effect of some external
stimuli u1 (t), · · · , uk (t). We collect these stimuli thereby changing our system of equations. We
into a vector u(t) = [u1 (t); u2 (t); · · · ; uk (t)] plot the vector field and trajectory of our system
and include their effect on the rates of change of under some constant input u(t) = 0.5 (Fig. 17.2,
system states in our function: bottom). Notice how the control input changes
⎡ ⎤ ⎡ ⎤ the trajectory and final state of our system by
ẋ1 (t) f1 (x(t), u(t))
⎢ ẋ2 (t) ⎥ ⎢ f2 (x(t), u(t)) ⎥ modifying the vector field. Also notice that our
⎢ ⎥ ⎢ ⎥ input only shifts the x1 component of our vectors
⎢ .. ⎥ = ⎢ .. ⎥.
⎣ . ⎦ ⎣ . ⎦ because we only stimulate x1 . These abilities to
ẋN (t) fN (x(t), u(t)) map neural interactions f to the full trajectory of
0 12 3 0 12 3 activity x(t) and to find control inputs u(t) that
ẋ(t) f (x(t),u(t))
0.5 0.5
-0.5 -0.5
-0.5 0.5 0 2
0.5 0.5
-0.5 -0.5
-0.5 0.5 0 2
Fig. 17.2 Vector fields and trajectories, with and with- corresponding plot of each state over time (top right) and
out control inputs. Example simple vector field of two the corresponding vector field and trajectory with control
states with a particular trajectory from initial condition input u(t) = 0.5 (bottom left) with corresponding states
x(0) = [−0.3; −0.4] (top left) in state space, with the over time (bottom right)
drive our neural system to a desired final state In systems of only a few neural units, there exist
x(T ) are among the core contributions of linear several powerful numerical and analytic tools.
systems theory. However, the study and control of large neural
systems is made difficult by our inability to know
how a stimulus will affect our system without first
17.2.4 Model Linearization simulating the full trajectory. Further, for multiple
stimuli, the number of possible stimulus patterns
While we have a quantitative framework for the grows exponentially.
evolution of a controlled neural system, there A special class of simplified systems called
are no general principles for determining the full linear systems circumvents this issue. In our state
trajectory x(t) or control input u(t) to reach a representation, a linear system is described by
desired final state for a general nonlinear system.
⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤⎡ ⎤
ẋ1 (t) a11 a12 · · · a1N x1 (t) b11 b12 ··· b1k u1 (t)
⎢ ẋ2 (t) ⎥ ⎢ a21 a22 · · · a2N ⎥ ⎢ x2 (t) ⎥ ⎢ b21 b22 ··· b2k ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢u2 (t)⎥
⎢ .. ⎥ = ⎢ .. .. . . .. ⎥ ⎢ .. ⎥ + ⎢ .. .. .. .. ⎥ ⎢ . ⎥, (17.1)
⎣ . ⎦ ⎣ . . . . ⎦ ⎣ . ⎦ ⎣ . . . . ⎦ ⎣ .. ⎦
ẋN (t) aN 1 aN 2 · · · aN N xN (t) bN 1 bN 2 · · · bN k uk (t)
0 12 3 0 12 3 0 12 3 0 12 3 0 12 3
ẋ(t) A x(t) B u(t)
that is characterized by the time evolution of any To transform the nonlinear system ẋ =
state ẋi (t) being a weighted sum of current states
N k f (x, u), into a linear system ẋ = Ax + Bu,
j =1 aij xj (t) and external inputs j =1 bij uj (t). we can create an approximate model of our
Here, aij is a real number that determines how vector field about a particular constant operating
activity in state xj influences the rate of change state x ∗ and input u∗ . We first evaluate the
of state xi and bij is a real number that deter- dynamics at this operating point, f (x ∗ , u∗ ).
mines how external input uj influences the rate of Then we approximate the vector field along
change of state xi . We see that our example two- small deviations from this point by computing the
unit system is not linear, because the first state derivative of f (x, u) with respect to the states to
ẋ1 (t) depends on sin(x1 (t)), and the second state get matrix A and with respect to control inputs to
ẋ2 (t) depends on x12 (t), and is therefore a non- get matrix B:
linear system.
⎡ ∂f1 ∂f1
⎤ ⎡ ∂f1 ∂f1
⎤
· · · ∂x
∂f1
∂x1 ∂x2 · · · ∂u
∂f1
∂u1 ∂u2
⎢ ∂f2 ∂f2 ∂f2 ⎥
· · · ∂x
N
⎢ ∂f2 ∂f2 ∂f2 ⎥
· · · ∂u
k
⎢ ∂x1 ∂x2 ⎥ ⎢ ∂u1 ∂u2 ⎥

⎢
A= ⎢ . . . N ⎥ ⎢
B=⎢ . . . k ⎥
.
. ⎥ . ⎥
⎣ .. .. . . .. ⎦ ⎣ .. .. . . .. ⎦
∂fN

∂fN ∂fN
· · · ∂x ∂fN ∂fN
· · · ∂f N
∂x1 ∂x2 N x=x ∗ ,u=u∗ ∂u1 ∂u2 ∂uk x=x ∗ ,u=u∗
Then, for states near x ∗ and inputs near u∗ , the A typical operating point for the input is u∗ =
vector field is approximately 0 corresponding to no input, because neural stim-
ulation is viewed as a perturbation to the natural
ẋ(t) = f (x, u) (17.2) and unstimulated dynamics. A typical operat-
ing point for the state x ∗ is a fixed point where
≈f (x ∗ , u∗ )+A(x(t)−x ∗ )+B(u(t)−u∗ ).
f (x ∗ , u∗ ) = 0, because then the evolution of our
(17.3)
system Eq. 17.2 only depends on deviations from

the point, and not on its actual value. Finally, we ẋ1 (t) −1 2 x1 (t) 1
≈ + u(t).
can write the linearized equation explicitly as a ẋ2 (t) 0 −1 x2 (t) 0
function of these deviations through a change of 0 12 3 0 12 3 0 12 3 0123
ẋ(t) A x(t) B
variables y(t) = x(t) − x ∗ :
We show the vector fields and trajectories for
ẏ(t) = ẋ(t) ≈ Ay(t) + Bu(t). both the nonlinear and linear equations without
control where u(t) = 0 (Fig. 17.3, top) and with
We will continue to use variable x instead of y control where u(t) = 0.5 (Fig. 17.3, bottom)
with the understanding that it represents devia- from the same initial condition, and we notice
tions from the fixed point. For example, in our that in the neighborhood of x1∗ = 0, x2∗ = 0,
two-unit system, we can linearize about x1∗ = the field and trajectories are similar. Hence, by
0, x2∗ = 0, and u∗ = 0 to yield linearizing our neural dynamics about x ∗ , u∗ , we
0.5 0.5
-0.5 -0.5
-0.5 0.5 -0.5 0.5
0.5 0.5
-0.5 -0.5
-0.5 0.5 -0.5 0.5
Fig. 17.3 Vector fields and trajectories for a nonlinear tem (top left), the uncontrolled linear system (top right),
system and its linearized form. Example vector field of the controlled nonlinear system (bottom left), and the
two states with a particular trajectory from initial condition controlled linear system (bottom right)
x(0) = [−0.3; −0.4] for the uncontrolled nonlinear sys-
can preserve the behavior of our neural system at system’s response to control through mathemati-
state x(t) and inputs u(t) near this point while cal relations as opposed to simulations. Then we
enabling the use of powerful tools developed in will use these principles to design stimuli that
the next section. optimally guide our system from some initial state
x(0) to some final state x(T ).
17.3 Theory of Linear Systems

17.3.1 Impulse Response
A useful model for therapeutic intervention in a
neural system should capture both how the activ- First, we find the natural evolution of system
ity over time depends on the connections between states from some initial neural state x(0) without
neural units and how to change the activity in a any external input. This task amounts to finding
desired way through stimulation. Now that we the state trajectory x(t) that solves our dynamic
have a model that captures features of neural equation ẋ(t) = Ax(t). For scalar systems where
activity and connectivity in a linearized form, we x(t) is not a vector, we are reminded of the
will develop equations that yield precisely these solution to ẋ = ax:
features. Specifically, we will first determine the
dx
= ax differential equation,
dt
1
dx = adt divide by x,
x

1
dx = adt + c integrate both sides,
x
ln |x| = at + c
x(t) = Ceat solution to differential equation,
∞ (at)k
where the constant is the initial condition C = tial function eat = k=0 k! . Taking the time
x(0). We can prove that this solution satisfies derivative of x(t) = eat , we see ẋ = ax:
ẋ = ax by using a Taylor series of the exponen-

d at d at a2t 2 a3t 3 ak t k
e = 1+ + + + ··· + + ··· Taylor series of eat ,
dt dt 1! 2! 3! k!
a a2t a3t 3 a k t k−1
=0+ +2 +3 + ··· + k + ··· differentiate each term,
1! 2! 3! k!

at a2t 2 ak t k
=a 1+ + + ··· + + ··· factor out scalar a,
1! 2! k!
= aeat substitute Taylor series.
A matrix exponential is defined exactly the same show that the time derivative satisfies the vector
∞ (At)k
as above with eAt = k=0 k! , and we again
relation ẋ(t) = Ax(t):

d At d At A2 t 2 A3 t 3 Ak t k
e = I+ + + + ··· + + ··· Taylor series of eAt ,
dt dt 1! 2! 3! k!
A A2 t A3 t 3 Ak t k−1
=0+ +2 +3 + ··· + k + ··· differentiate each term,
1! 2! 3! k!

At A2 t 2 Ak t k
=A I+ + + ··· + + ··· factor out matrix A,
1! 2! k!
= AeAt substitute Taylor series.
Hence, we see that the following solution any time T without solving for intermediate states
0 < t < T.
x(t) = eAt x(0) (17.4) As an example in our linearized two-unit
model, to find the state of our system at T = 2
satisfies our dynamic equation. Here, the matrix given an initial start at x(0) = [−0.3; −0.4], we
exponential eAt is called the state transition ma- can use a software to numerically compute the
trix, and Eq. 17.4 is called the impulse response matrix exponential at time t = 2 and multiply by
of our system. Hence, we can find the state at our initial state Eq. 17.4

0.1353 0.5413 −0.3 −0.2571
x(2) = e x(0) =
2A
= ,
0 0.1353 −0.4 −0.0541
which agrees with the simulation results

(Fig. 17.3). d −At
(e x(t)) = e−At Bu(t),
dt
and integrate both sides from t = 0 to t = T to
17.3.2 Control Response yield
Next, we derive the system response from an T

−AT
initial state x(0) to some controlling input u(t) e x(T ) − x(0) = e−At Bu(t)dt.
0
through some algebraic manipulation and calcu-
lus. We begin with our system equations ẋ(t) − We note the matrix exponential at t = 0 becomes
Ax(t) = Bu(t) and multiply both sides by a e−A·0 = I from the Taylor series. Next, we move
matrix exponential the initial state x(0) to the right-hand side and
multiply by eAT :
e−At ẋ(t) − e−At Ax(t) = e−At Bu(t).
T
Next, we see that the left-hand side is the re- eAT e−AT x(T ) = eAT x(0) + eAT e−At Bu(t)dt.
sult of a product rule where dtd (e−At x(t)) = 0
e−At ẋ(t) − Ae−At x(t), recalling that functions of

Finally we use the fact that eAT and e−AT are
matrices can switch orders of multiplication, such
inverses of each other where eAT e−AT = I ,
that Ae−At = e−At A. Hence, we can write our
and we bring eAT into the integral to derive the
equation as
system’s response to control input:
T L(u2 (t)) = c2 , then a weighted sum of inputs

x(T ) = eAT x(0) + eA(T −t) Bu(t)dt .
0 12 3 yields the same weighted sum of outputs, such
natural 00 12 3 that
controlled
(17.5)
L(au1 (t) + bu2 (t)) = ac1 + bc2 . (17.6)
Intuitively, we see that the first part of the re-
sponse, eAT x(0), is just the natural evolution of This linearity allows us to treat solutions to
our system from an initial state and that the sec- our control function problem the same as
ond part of the response is a convolution of our solutions to our linear system of equations.
mapped inputs, Bu(t), with the impulse response. Specifically, suppose the control input u∗ (t) is
We will next take advantage of the convolution’s a particular solution to our control problem such
property of linearity to draw powerful relations that L(u∗ (t)) = x T − eAT x 0 . Further, suppose
between the state evolution, control input, and that inputs u1 (t), u2 (t), · · · are homogeneous
system structure. solutions such that L(ui (t)) = 0. If we construct
a control input that is the particular solution added
to a weighted sum of homogeneous solutions
17.3.3 Linear Relation Between the
Convolution and Control Input u(t) = u∗ (t) + ai ui (t) ,
0 12 3 0 12 3
particular i homogeneous
Previously, we focused on the evolution of a neu-
ral system in response to a known control input then the convolution of this combined input yields
u(t) in Eq. 17.5. However, our goal is to design the desired output:
a control input that drives our neural system to
some desired final state that may stabilize an
∗
L(u(t)) = L u (t) + ai ui (t)
epileptic seizure [31], or aid in memory recall
i
[32]. In this scenario, we fix the initial state
x(0) = x 0 and the final state x(T ) = x T as con- = L(u∗ (t)) + L(ai ui (t))
stants and rewrite Eq. 17.5 to move the variables i
u(t) to the left-hand side and the constants to the
= x T − eAT x 0 + ai 0
right-hand side: i
T = x T − eAT x 0 .
eA(T −t) B u(t) dt = x(T ) − eAT x(0) .
0 0123 0 12 3
variable constant Hence, if we have a particular control input u∗ (t)
that drives our system to a desired final state,
This formulation is a linear equation with a struc- then the homogeneous control inputs ui (t) give us
ture that is similar to a typical system of linear the flexibility to design less costly, more energy-
equations used in regression, Mv = b, where v is efficient inputs.
the variable, b is a constant vector, and matrix M
is the linear function acting on v. Here, the control
input u(t) is the variable, x(T ) − eAT x(0) is the 17.3.4 Controllability
constant vector, and the convolution
T For any system, we would first like to know if a
A(T −t)
L(u(t)) = e Bu(t)dt particular solution exists to the control problem
0 described above. A system is controllable if there
is a control input that brings our system from any
is the linear function acting on our control inputs.
initial state to any final state in finite time. For
By linear function, we mean that for two control
nonlinear systems, if we know that the input u∗ (t)
inputs u1 (t) and u2 (t), if L(u1 (t)) = c1 , and
brings our system from the initial state 0 to some Further, any N-dimensional vector can be written
final state x T , there is in general no way to know as a weighted sum of N linearly independent vec-
what input will take our system to a scaled final tors v 1 , v 2 , · · · , v N . Here, linear independence
state ax T . means that no vector v i in the set can be written as
In contrast, due to the linearity of our convolu- a weighted sum of the remaining vectors v j =i . For
tion operator, we know that a scaled input au∗ (t) example, a column vector a = [a1 ; a2 ; · · · ; aN ]
will produce a scaled output L(au∗ (t)) = ax T . can be written as the weighted sum
⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤
a1 1 0 0
⎢ a2 ⎥ ⎢0⎥ ⎢1⎥ ⎢0⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ .. ⎥ = a1 ⎢ .. ⎥ +a2 ⎢ .. ⎥ + · · · + aN ⎢ .. ⎥,
⎣ . ⎦ ⎣.⎦ ⎣.⎦ ⎣.⎦
aN 0 0 1
0 12 3 0123 0123 0123
a v1 v2 vN
where none of the vectors v i can be written as a independent vectors L(u1 (t)), · · · , L(uN (t)),
weighted sum of remaining vectors v j =i . Hence, because then we can always reach any final state
our system is controllable if we can find input from any initial state through the weighted sum
functions u1 (t), · · · , uN (t) that reach N linearly
x T − eAT x 0 = a1 L(u1 (t)) +a2 L(u2 (t)) + · · · + aN L(uN (t)),

0 12 3 0 12 3 0 12 3 0 12 3
a v1 v2 vN
through the control input u(t) = a1 u1 (t) + As an example in our linearized two-unit sys-
a2 u2 (t) + · · · + aN uN (t). This information of tem, A, B, and C are written as
reachable states is encoded in the controllability
matrix −1 2 1
A= , B=
,
0 −1 0

C = B, AB, A2 B, · · · , AN −1 B , (17.7)
1 −1
C = B, AB = ,
0 0
where the rank of this matrix (given by the num-
ber of linearly independent columns of C) tells which is not controllable, because the rank of C
us how many of these N independent vectors can is 1. To consider the controllable subspace, notice
be reached using control input. If this rank = N, that the columns of C only have non-zero entry
then the system is controllable and can reach all in the first row. Hence, the controllable subspace
states. However, even if the rank < N, there still contains any desired value of x1 (T ), but excludes
exists a control input that drives the system from all values of x2 (T ). Intuitively, this loss of con-
x 0 to x T if the vector x T − eAT x 0 can be written trollability arises because x2 does not receive an
as a weighted sum of the columns of C. This set input, nor is it affected by x1 . Hence, there is no
of vectors spanned by the columns of C is called way to influence the activity of x2 in a desired
the controllable subspace and the remaining set way.
of vectors the uncontrollable subspace.
17.3.5 Minimum Energy Control scales quadratically with the input, such as with
resistive heating. This quadratic measure of size
Once we know a system is controllable, we would is mathematically and intuitively defined using
like to determine the control input function u(t) the inner product. For N-dimensional column
that transitions our system from initial x 0 to final vectors of numbers, a, the inner product is the
x T states. However, there are often limitations on well known dot product
the input magnitude such as electrical and thermal
damage of neural tissue or battery life of chronic < a, a >= a12 + a22 + · · · + aN2 = a a,
implanted stimulators. Due to the system’s linear-
ity, we can find not only an input function but an where a is the transpose that turns column vec-
optimal one u∗ (t) that minimizes input cost. tor a into a row vector. We see that doubling a will
First, we must define a measure of the size of quadruple the inner product. For k-dimensional
our control input functions u(t). In many applica- column vectors of functions a(t) from time t = 0
tions of electrical stimulation, the cost of control to t = T , the inner product is similarly defined as
T T
< a(t), a(t) >= a12 (t) + a22 (t) + ··· + aN2 (t)dt = a(t) a(t)dt
0 0
that has the same quadratic relation. Hence, we cost < u∗ (t), u∗ (t) >. However, while matrix
define the control energy as M inputs a vector of numbers v and outputs a
vector of numbers b, our linear function L inputs
E =< u(t), u(t) > . (17.8) a vector of functions and outputs a vector of
numbers. Hence, we need to carefully define the
Now that we have a measure of how large an adjoint of L; because L is not a finite matrix,
input is, we wish to find a minimal input u∗ (t) we cannot use L to denote the adjoint. Instead,
that minimizes the control energy. This task is we will use L∗ to denote the adjoint of L. In the
analogous to a typical linear system of equations, case of matrix M, the adjoint preserves the inner
Mv = b, where we want to find v ∗ that solves product between inputs and outputs such that
the equation with the smallest cost < v ∗ , v ∗ >.
Here, if M has full row rank where the rows of < Mv, b > =< v, M b >
M are linearly independent, then the minimum
(Mv) b = v (M b).
solution is given by the equation for least squares
v ∗ = M (MM )−1 b. Here, M is the transpose,
Identically, for state transition x = eAT x 0 −
or adjoint of M.
x T , the adjoint of L preserves the inner product
This same principle holds for our linear system
between the vectors of input functions u(t) and
L(u(t)) = x T − eAT x 0 , where we want to find
output numbers x as
u∗ (t) that solves the equation with the smallest
< L(u(t)), x > =< u(t), L∗ (x) >

T T

eA(T −t) Bu(t)dt x= u (t)(B eA (T −t) x)dt.
0 0
Notice that the inner product on the left is over 17.4 Mapping Network
vectors of numbers, while the inner product on Architecture to Control
the right is over vectors of functions. Then, we Properties
see that our adjoint is

By formulating our neural system in a linear way,
L∗ (x) = B eA (T −t)
x we can solve difficult problems such as predicting
the system’s response to control, finding the set
and takes as input a vector of numbers and outputs of states that the system can reach, and designing
a vector of functions. Then, just as our system efficient input stimuli, without the need to try
Mv = b, the minimum input u∗ (t) is given by every control input and simulate every trajectory.
Further, by directly mapping control properties
u∗ (t) = L∗ (LL∗ )−1 (x T − eAT x 0 ). (17.9) to neural activity and network architecture in
an algebraic way, we can study how features of
Finally, through substitution into Eq. 17.8, we can interaction patterns impact our ability to control
write the minimum control energy as neural activity [8]. As an active area of research,
the variety of questions being asked and systems
Emin = (x T − eAT x 0 ) (LL∗ )−1 (x T − eAT x 0 ). being studied is very large, and require simul-
(17.10) taneous innovations in experiment, computation,
and theory. In this section, we will describe a few
In conclusion, we point out the crucially im- recent applications and advances.
portant term of the minimum energy, LL , as the
controllability Gramian written as
T
17.4.1 Neuronal Control in Model
Wc (T ) = LL = ∗
e A(T −t)
BB e A (T −t)
dt. Organisms
0
(17.11) While most neural systems are too large to em-
pirically measure activity and connectivity or to
First, we notice that this Gramian is only a func- analyze numerically, there do exist a few suffi-
tion of the underlying neural relationships, A; the ciently simple model organisms. Among these is
matrix determining where the inputs are placed, the worm Caenorhabditis elegans [33] with sev-
B; and time T . Next, we notice that Wc (T ) is eral hundred neurons that can be recorded from si-
actually an N × N matrix and can therefore be multaneously [34]. Even for such a small system,
numerically evaluated and analytically studied. it is difficult to map the functional form of how
Finally, we see that if our system begins at an activity in neuron i affects the activity in neuron
initial state of x 0 = 0, then the minimum energy j . However, the presence or absence of connec-
can be written as tions between neurons in this organism, and by
consequence the presence or absence of elements
Emin = x −1
T Wc (T )x T , in the connectivity matrix A, is well known.
Advances in the study of structural control-
where the role of neural interactions and stim- lability [35] allow us to ask questions about our
ulation parameters on our ability to control the ability to control a system given only the bi-
system is fully encapsulated in the Gramian. This nary presence or absence of edges. Colloquially,
ability to decouple the states x T from the neural this framework focuses on connectivity matrices
interactions and stimulation parameters A, B, T A where non-zero entries can only exist in the
is a powerful tool for studying and designing presence of binary edges, and can be used to
control properties of neural systems. determine whether the system is controllable
for most values where an edge is present. Using during various cognitively demanding tasks [44].
this framework, recent work has sought to deter- Here, brain activity can be empirically measured
mine whether the removal of certain neurons in through methods such as magnetic resonance
C. elegans will reduce structural controllability imaging (blood oxygen level dependent) or elec-
[36]. Specifically, the modeling involves input trophysiology (aggregate electrical activity). Of
to the sensory receptor neurons as the control particular interest are large-scale functional brain
input that is mapped to the system through a networks that display stereotyped changes in
matrix B and the connectivity between neurons activity patterns during tasks that demand certain
and muscle cells through a matrix A. Further, in- cognitive or sensorimotor processes [45]. Here, it
stead of recording the activity of each neuron, the is thought that the brain uses underlying structural
motion of muscles was recorded. This framework connections to support circuit-level coordination,
involves the appended control framework as well as to guide itself to specific patterns of
activity using cognitive control [46, 47].
ẋ(t) = Ax(t) + Bu(t) Recent work has begun formulating cognitive
control as a linear systems problem [46, 48–51],
y(t) = Cx(t),
where matrix A is the network of white matter
connections between brain regions, B represents
where y(t) represents the states (muscles) that
the regions that were chosen to be responsible
are measured and C is the map from neurons and
for control, and x(t) represents the activity of
muscles x(t) to the measured output [37]. Here,
each region over time. Specifically in [48, 50],
the authors find that the ablation of a neuron not
the authors quantify cognitive states as vectors
previously implicated in motion, PDB, decreased
corresponding to activity in the brain regions
structural controllability, significantly reducing
during cognitive tasks and compute the mini-
ventral bias in deep body bends in C. elegans.
mum control energy Eq. 17.9 to transition be-
tween cognitive states for various sets of control
17.4.2 State Transitions in the regions. Colloquially, if a set of regions requires
Human Brain less input energy to transition between cognitive
states, then those regions may easily transition the
While neuron-level structural synapses map whole brain between these states along an optimal
most directly to functional relationships between trajectory given they are responsible for cogni-
neurons, there are also well-characterized tive control. Moreover, individual differences in
structural connections between larger-scale brain the minimal control energy are correlated with
regions. These connections contain thick bundles individual differences in performance on cogni-
of myelinated axonal fibers that run throughout tive control tasks [52]. In complementary studies,
the brain and are thought to play a crucial role in individual differences in controllability statistics
coupling the activity of distant brain regions [38]. calculated for distinct regions of the brain are
These fibers are resolved by measuring water correlated with individual differences in measures
diffusion throughout the brain using magnetic of cognitive control assessed with common neu-
resonance [39] and tracing fibers along this ropsychological test batteries [49, 51].
diffusion field using computational algorithms
[30]. The whole brain is typically divided into
hundreds to thousands of discrete brain regions 17.5 Methodological
using a variety of parcellation schemes [40, 41], Considerations and
and the strength of fibers between these regions Limitations
comprises the connectivity matrix A [42].
Such region-level study of brain dynamics has While the theory of linear systems is a powerful
led to the discovery of macroscopic functional quantitative framework for studying and control-
organization in the human brain at rest [43] and ling dynamical neural systems, there are several
important caveats. Here we mention three: dimen- While we typically use the structural connections
sionality and numerical stability, model valida- in synapses between neurons, or bundles of axons
tion and experimental data, and the assumption between brain regions as a proxy for A, it is very
of linearity. difficult to measure the true functional effect that
activity in unit i has on activity in unit j , partic-
ularly for large systems. This problem is exacer-
17.5.1 Dimensionality and Numerical bated by further methodological limitations such
Stability as the inability to resolve directionality of connec-
tions in diffusion tractography. Along these lines,
The benefit of studying linear systems is that many statistical and autoregressive methods have
we take difficult and largely intractable ques- been developed to infer functional relationships
tions of controllability and control input design from recordings of neural activity [55–59] and
and greatly simplify them into algebraic prob- to use that inferred activity to better understand
lems of computing objects like the controllability control [60]. However, the degree of causality in
matrix Eq. 17.7 and the controllability Gramian these methods as measured by true response to
Eq. 17.11. However, these matrices scale quadrat- external stimuli remains controversial.
ically with the number of neural units, and numer- Another such fundamental limitation is our in-
ical calculations and manipulations using these ability to fully measure every state of the system.
matrices quickly face computational issues. The state-space representation of our model re-
Most viable approaches to dealing with these quires that every state is observed. However, it is
issues involve numerically representing the ele- impossible to simultaneously record the activity
ments of our matrices and performing algebraic of every neuron in almost all biological systems,
operations. However, these representations are although this recording has been achieved in suf-
imperfect, as it is impossible to completely rep- ficiently simple organisms [34]. As a result of
resent irrational numbers such as π . Hence, the only being able to observe a small subset of the
matrices are truncated to numerical precision, full state-space, these models of interactions may
and this truncation error propagates with each become largely descriptive and phenomenologi-
computation. Further, the propagation of error cal in nature. In response, there is a continuing
tends to scale faster than the number of dimen- effort to improve the spatial and temporal resolu-
sions. This issue is prevalent in the computation tion of neuroimaging methods [61].
of the state-transition matrix [53], as well as in
the calculation of the controllability Gramian and
its inverse. With the application of this theory 17.5.3 Assumption of Linearity
to high-dimensional neural systems, the study of
useful controllability metrics is an active area of An inherent limitation is the lack of generality
research [54]. in our linear approximation of the full nonlinear
neural dynamics. In response, there is a sizable
quantity of research studying the control prop-
17.5.2 Model Validation and erties of nonlinear dynamical systems [62]. An
Experimental Data interesting bridge between these two disciplines
exists in the theory of the Koopman or composi-
A fundamental limitation for modeling any neural tion operator [63]. The underlying benefit of this
system is the ability to empirically and accu- theory is that, while our system of equations may
rately measure model parameters and variables. A evolve nonlinearly in time given the current set
crucial parameter is the network of connectivity of N states, there may exist a higher-dimensional
encoded by our adjacency matrix A, where the set of M > N state variables in which the dy-
element in the i-th column and j -th row models namical system does evolve linearly [64]. While
the effect of unit i on the rate of change of unit j . the extension of linear systems theory to actually
controlling this higher-dimensional system may [67]. Moreover, it would be fruitful in the future
be limited, it remains a promising future area of to further develop a broader set of controllablity
research. statistics, extending beyond node controllability
[54], and edge controllability [68], to the control
of motifs [69]. Finally, throughout such investi-
17.6 Open Frontiers gations, it will be useful to understand which fea-
tures of control are shared across networks with
Many exciting and open frontiers exist in the various topologies, versus those features which
study of brain network dynamics using linear are specific to networks with a particular topology
systems theory. Here we constrain our remarks to [70–72].
three main topic areas, but freely admit that this
discussion is far from comprehensive. First, we
describe opportunities in the further development 17.6.2 Context, Computations, and
of useful controllability statistics as well as in Information Processing
the development of foundational theory linking
control profiles to the system’s underlying net- Despite the emerging appreciation that linear sys-
work architecture. Second, we underscore the tems theory has considerable utility in the study of
need for a better understanding of how control is cognitive function, we still know very little about
implemented in the brain, how control strategies exactly how control is implemented in the brain,
might depend on context, and how control pro- across spatial scales, and capitalizing on the unit-
cesses could facilitate the effective manipulation to-unit interaction patterns at each of those scales.
of information. Third, we describe the relevance Some initial evidence suggests that features of
of the modeling efforts we discussed here for synaptic connectivity – and particularly autaptic
our understanding of neurological disease and connections – can serve to tune the excitability
psychiatric disorders as well as the development of the neural circuit, altering its controllability
of personalized and targeted therapeautic inter- profile and propensity to display synchronous
ventions for alterations in mental health. bursts of activity [73]. Complementary evidence
also at the cellular scale demonstrates how in-
trinsic network structure and exogeneous stim-
17.6.1 Theory and Statistics ulus patterns together determine the manner in
which a stimulus propagates through the network,
Linear systems theory has its basis in a rich liter- with important implications for cognitive facul-
ature stemming from now well-developed areas ties that require persistent activation of neuronal
of mathematics, physics, and engineering [65]. patterns such as working memory and attention
Yet, much is still unknown about exactly how [74]. There are interesting similarities between
the network topology of a given unit-to-unit in- these observations and evidence at larger spatial
teraction pattern impacts the capacity for control, scales, which suggests that the architecture of
the trajectories accessible to the systems, and white matter tracts connecting brain areas can be
the minimum control energy. Some preliminary used to infer the probability with which the brain
efforts have begun to make headway by using persists in certain states [75]. Such conceptual
linear network control theory to derive accurate similarities motivate concerted efforts to better
closed-form expressions that relate the connec- understand how the architecture of brain net-
tivity of a subset of structural connections (those works across spatial scales supports information
linking driver nodes to non-driver nodes) to the processing and cognitive computations and how
minimum energy required to control networked those processes and computations might depend
systems [66]. Further work is needed to gain an on the context in which the brain is placed. For-
intuition for the role of higher-order structures mally, it would be interesting to consider context
(e.g., cycles) in the control of the networked sys- as a form of exogeneous input to the system, in a
tem and any dependence on edge directionality manner reminiscent of how we currently consider
brain stimulation [8]. We speculate that such a and demonstrate that this point is a fixed
formulation of the problem could help to explain point where ẋ1 = ẋ2 = ẋ3 = 0.
a range of observations, such as the ability of that the matrix exponential of A =
2. Prove
cognitive effort to suppress epileptic activity [76]. a0
is
0b

17.6.3 Disease and Intervention ea 0
e =
A
,
0 eb
The fact that controllability can depend on net-
work topology [66, 70] and can be altered by using the Taylor series of the scalar and ma-
edge pruning [77] suggests that it might also be trix exponentials.
a useful biomarker in some neurological diseases 3. Prove that the system response to control
and psychiatric disorders, many of which are as- t
sociated with changes in the structural topology x(t) = eAt x 0 + eA(t−τ ) Bu(τ )dτ
of neural circuitry at various spatial scales [6, 7]. 0
Indeed, recent studies have reported differences
in controllability statistics estimated in brain net- satisfies the dynamical equation ẋ(t) =
works of patients with bipolar disorder [78], tem- Ax(t) + Bu(t) by substitution.
poral lobe epilepsy [79], and mild traumatic brain 4. Prove that the convolution operator
injury [50]. In a complementary line of work, T
studies are beginning to ask whether the altered L(u(t)) = eA(T −τ ) Bu(τ )dτ
controllability profiles of brain networks in these 0
patients could help to inform the development of
is linear according to Eq. 17.6; that is, if
more targeted interventions for their illness, in
L(u1 (t)) = c1 , and L(u2 (t)) = c2 , then
the form of brain stimulation [31, 80], pharma-
demonstrate that L(au1 (t)+bu2 (t)) = ac1 +
cological agents, or cognitive behavioral therapy.
bc2 .
Other efforts have begun to consider symptoms
5. Determine if the following system is control-
of a given disease as a network and to identify
lable
symptoms predicted to have high impulse re-
sponse in the patient’s daily life [81]. It would be ⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤
ẋ1 (t) 010 x1 (t) 1
interesting in the future to determine whether the ⎣ẋ2 (t)⎦ = ⎣0 0 1⎦ ⎣x2 (t)⎦ + ⎣0⎦ u(t),
linear systems approach could be useful in more ẋ3 (t) 100 x3 (t) 0
carefully formalizing that problem as a network
control problem, which in turn could be used by constructing the controllability matrix.
to determine which symptom to treat in order 6. Determine for what value of a the system is
to move the entire symptom network toward a not controllable
healthier state [82].
⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤
ẋ1 (t) 000 x1 (t) 1
⎣ẋ2 (t)⎦ = ⎣1 1 0⎦ ⎣x2 (t)⎦ + ⎣0⎦ u(t),
Homework ẋ3 (t) 10a x3 (t) 0
1. Linearize the following system about point by constructing the controllability matrix.
x1∗ = 1, x2∗ = −1, x3∗ = 0, 7. Derive the minimum energy equation
⎡ ⎤ ⎡ 2 ⎤ Eq. 17.10
ẋ1 (t) −x1 (t) − 2x2 (t) + x3 (t) − 1
⎣ẋ2 (t)⎦ = ⎣ 2x1 (t) − 2x22 (t) + 2x3 (t) ⎦ . Emin = (x T −eAT x 0 ) (LL∗ )−1 (x T −eAT x 0 ),
ẋ3 (t) x1 (t)x2 (t) − x3 (t) + 1
by substituting the minimum input u∗ (t) into W911NF-10-2-0022, the Army Research Office through
the control energy Eq. 17.8 contract numbers W911NF-14-1-0679 and W911NF-
16-1-0474, the National Institute of Health (2-R01-DC-
009209-11, 1R01HD086888-01, R01-MH107235, R01-
E =< u(t), u(t) > . MH107703, R01MH109520, 1R01NS099348 and R21-M
MH-106799), the Office of Naval Research, and the
8. Show that the controllability Gramian can be National Science Foundation (BCS-1441502, CAREER
PHY-1554488, BCS-1631550, and CNS-1626008).The
written as
content is solely the responsibility of the authors and does
T not necessarily represent the official views of any of the

WC (T ) = eA(T −t) BB eA (T −t)
dt funding agencies.
0
T

= eAτ BB eA τ dτ, References
0
1. M. Maier, A. Zippelius, M. Fuchs, Emergence of

using the substitution τ = T − t. long-ranged stress correlations at the liquid to glass
9. Show that the controllability Gramian for transition. Phys. Rev. Lett. 119(26), 265701 (2017).
system https://doi.org/10.1103/PhysRevLett.119.265701
2. S. Kivelson, S.A. Kivelson, Defining emergence in
physics. Quantum Mater. 1, 16024 (2016). https://doi.
a0 10
A= , B= org/10.1038/npjquantmats.2016.24
0b 01 3. C.W. Lynn, L. Papadopoulos, D. Lee, D.S. Bassett,
Surges of collective human activity emerge from sim-
is ple pairwise correlations. Phys. Rev. X 9, 011022-1–
011022-19 (2018, in Press)
4. D.S. Bassett, M.S. Gazzaniga, Understanding com-
1
e2aT − 1 0
WC (T ) = 2a plexity in the human brain. Trends Cogn. Sci. 15(5),
0 1
2b
e2bT − 1 200–209 (2011). https://doi.org/10.1016/j.tics.2011.
03.006
10. Compute the minimum energy required for 5. A. Haimovici, E. Tagliazucchi, P. Balenzuela,
the system D.R. Chialvo, Brain organization into resting state
networks emerges at criticality on a model of the
1 human connectome. Phys. Rev. Lett. 110(17), 178101
0 10
A= 2 , B= , (2013). https://doi.org/10.1103/PhysRevLett.110.
02 01 178101
6. U. Braun, A. Schaefer, R.F. Betzel, H. Tost, A.
Meyer-Lindenberg, D.S. Bassett, From maps to multi-
0
to transition from initial state x(0) = to dimensional network mechanisms of mental disor-
0 ders. Neuron 97(1), 14–31 (2018). https://doi.org/10.

1 1016/j.neuron.2017.11.007
final state x(T ) = in time T = 1. 7. C.J. Stam, Modern network science of neurologi-
2
cal disorders. Nat. Rev. Neurosci. 15(10), 683–695
(2014). https://doi.org/10.1038/nrn3801
Acknowledgements We gratefully acknowledge 8. E. Tang, D.S. Bassett, Control of dynamics in brain
comments and feedback from Arian Ashourvan, Ann networks. Rev. Mod. Phys. 90, 031003 (2018). https://
E. Sizemore, Melody X. Lim, Jennifer A. Stiso, Erin doi.org/10.1103/RevModPhys.90.031003
G. Teich, Teresa Karrer, Zhixin Lu, Harang Ju, and 9. J. Downar, J. Geraci, T.V. Salomons, K. Dunlop, S.
Eli J. Cornblath. We also thank Ann E. Sizemore for Wheeler, M.P. McAndrews, N. Bakker, D.M. Blum-
generous assistance with and input on schematic figure berger, Z.J. Daskalakis, S.H. Kennedy, A.J. Flint,
construction. JZK acknowledges support from the NIH P. Giacobbe, Anhedonia and reward-circuit connec-
T32-EB020087, PD: Felix W. Wehrli, and the National tivity distinguish nonresponders from responders to
Science Foundation Graduate Research Fellowship No. dorsomedial prefrontal repetitive transcranial mag-
DGE-1321851. DSB acknowledges support from the John netic stimulation in major depression. Biol. Psychi-
D. and Catherine T. MacArthur Foundation, the Alfred atry 76(3), 176–85 (2014). https://doi.org/10.1016/j.
P. Sloan Foundation, the Paul G. Allen Foundation, the biopsych.2013.10.026
Army Research Laboratory through contract number
10. J.D. Medaglia, D.Y. Harvey, N. White, A. Kelkar, J. 16. B.B. Machta, R. Chachra, M.K. Transtrum, J.P.
Zimmerman, D.S. Bassett, R.H. Hamilton, Network Sethna, Parameter space compression underlies
controllability in the inferior frontal gyrus relates to emergent theories and predictive models. Science
controlled language variability and susceptibility to 342(6158), 604–607 (2013) https://doi.org/10.1126/
TMS. J. Neurosci. 38(28), 6399–6410 (2018). https:// science.1238723
doi.org/10.1523/JNEUROSCI.0092-17.2018 17. H.H. Mattingly, M.K. Transtrum, M.C. Abbott, B.B.
11. N. Gass, R. Becker, M. Sack, A.J. Schwarz, J. Rein- Machta, Maximizing the information learned from
wald, A. Cosa-Linan, L. Zheng, C.C. von Hohenberg, finite data selects a simple model. Proc. Natl. Acad.
D. Inta, A. Meyer-Lindenberg, W. Weber-Fahr, P. Sci. U. S. A. 115(8), 1760–1765 (2018)
Gass, A. Sartorius, Antagonism at the NR2B subunit 18. A.L. Hodgkin, A.F. Huxley, A quantitative descrip-
of NMDA receptors induces increased connectivity of tion of membrane current and its application to con-
the prefrontal and subcortical regions regulating reduction and excitation in nerve. J. Physiol. (1952).
ward behavior. Psychopharmacology (Berl). 235(4), https://doi.org/10.1113/jphysiol.1952.sp004764
1055–1068 (2018). https://doi.org/10.1007/s00213- 19. G. Cano, R. Dilao, Intermittency in the Hodgkin-
017-4823-2 Huxley model. J. Comput. Neurosci. 43(2), 115–125
12. U. Braun, A. Schafer, D.S. Bassett, F. Rausch, J.I. (2017). https://doi.org/10.1007/s10827-017-0653-9
Schweiger, E. Bilek, S. Erk, N. Romanczuk-Seiferth, 20. J.H. Goldwyn, E. Shea-Brown, The what and where
O. Grimm, L.S. Geiger, L. Haddad, K. Otto, S. of adding channel noise to the Hodgkin-Huxley equa-
Mohnke, A. Heinz, M. Zink, H. Walter, E. Schwarz, tions. PLoS Comput. Biol. 7(11), e1002247 (2011).
A. Meyer-Lindenberg, H. Tost, Dynamic brain net- https://doi.org/10.1371/journal.pcbi.1002247
work reconfiguration as a potential schizophrenia ge- 21. W. Teka, D. Stockton, F. Santamaria, Power-law dy-
netic risk mechanism modulated by NMDA receptor namics of membrane conductances increase spiking
function. Proc. Natl. Acad. Sci. U. S. A. 113(44), diversity in a Hodgkin-Huxley model. PLoS Com-
12568–12573 (2016). https://doi.org/10.1073/pnas. put. Biol. 12(3), e1004776 (2016). https://doi.org/10.
1608819113 1371/journal.pcbi.1004776
13. Z. Yang, S. Gu, N. Honnorat, K.A. Linn, R.T. Shi- 22. R. FitzHugh, Impulse and physiological states
nohara, I. Aselcioglu, S. Bruce, D.J. Oathes, C. Da- in theoretical models of nerve membrane.
vatzikos, T.D. Satterthwaite, D.S. Bassett, Y.I. She- Biophys. J. (1961). https://doi.org/10.1016/S0006-
line, Network changes associated with transdiagnos- 3495(61)86902-6
tic depressive symptom improvement following cog- 23. W.S. McCulloch, W. Pitts, A logical calculus of the
nitive behavioral therapy in MDD and PTSD. Mol. ideas immanent in nervous activity. Bull. Math. Sci.
Psychiatry 23(12), 2314–2323 (2018). https://doi.org/ (1943). https://doi.org/10.1007/BF02478259
10.1038/s41380-018-0201-7 24. J.J. Hopfield, Neural networks and physical sys-
14. H. Markram, E. Muller, S. Ramaswamy, M.W. tems with emergent collective computational abili-
Reimann, M. Abdellah, C.A. Sanchez, A. Ailamaki, ties. Proc. Natl. Acad. Sci. (1982). https://doi.org/10.
L. Alonso-Nanclares, N. Antille, S. Arsever, G.A. 1073/pnas.79.8.2554
Kahou, T.K. Berger, A. Bilgili, N. Buncic, A. Chal- 25. F. Rosenblatt, The perceptron: a probabilistic model
imourda, G. Chindemi, J.D. Courcol, F. Delalondre, for information storage and organization in the
V. Delattre, S. Druckmann, R. Dumusc, J. Dynes, brain. Psychol. Rev. (1958). https://doi.org/10.1037/
S. Eilemann, E. Gal, M.E. Gevaert, J.P. Ghobril, A. h0042519
Gidon, J.W. Graham, A. Gupta, V. Haenel, E. Hay, 26. G.A. Hedlund, Math. Syst. Theory 3, 320 (1969).
T. Heinis, J.B. Hernando, M. Hines, L. Kanari, D. https://doi.org/10.1007/BF01691062
Keller, J. Kenyon, G. Khazen, Y. Kim, J.G. King, 27. H.R. Wilson, J.D. Cowan, Excitatory and inhibitory
Z. Kisvarday, P. Kumbhar, S. Lasserre, J.V. Le Be, interactions in localized populations of model neu-
B.R. Magalhaes, A. Merchan-Perez, J. Meystre, B.R. rons. Biophys. J. (1972). https://doi.org/10.1016/
Morrice, J. Muller, A. Munoz-Cespedes, S. Muralid- S0006-3495(72)86068-5
har, K. Muthurasa, D. Nachbaur, T.H. Newton, M. 28. V. Shusterman, W.C. Troy, From baseline to epilep-
Nolte, A. Ovcharenko, J. Palacios, L. Pastor, R. Perin, tiform activity: a path to synchronized rhythmicity
R. Ranjan, I. Riachi, J.R. Rodriguez, J.L. Riquelme, in large-scale neural networks. Phys. Rev. E. Stat.
C. Rossert, K. Sfyrakis, Y. Shi, J.C. Shillcock, G. Sil- Nonlinear Soft. Matter. Phys. (2008). https://doi.org/
berberg, R. Silva, F. Tauheed, M. Telefont, M. Toledo- 10.1103/PhysRevE.77.061911
Rodriguez, T. Trankler, W. Van Geit, J.V. Diaz, R. 29. S.W. Oh et al., A mesoscale connectome of the
Walker, Y. Wang, S.M. Zaninetta, J. DeFelipe, S.L. mouse brain. Nature (2014). https://doi.org/10.1038/
Hill, I. Segev, F. Schurmann, Reconstruction and nature13186
simulation of neocortical microcircuitry. Cell. 163(2), 30. P.J. Basser, S. Pajevic, C. Pierpaoli, J. Duda, A.
456–492 (2015). https://doi.org/10.1016/j.cell.2015. Aldroubi, In vivo fiber tractography using DT-
09.029 MRI data. Magn. Reson. Med. (2000). https://doi.
15. A. Rosenblueth, N. Wiener, The role of models in org/10.1002/1522-2594(200010)44:4<625::AID-
science. Philos. Sci. 12(4), 316–321 (1945) MRM17>3.0.CO;2-O
31. P.N. Taylor, J. Thomas, N. Sinha, J. Dauwels, M. structural brain networks. Nat. Commun. 6, 8414
Kaiser, T. Thesen, J. Ruths, Optimal control based (2015). https://doi.org/10.1038/ncomms9414
seizure abatement using patient derived connectiv- 47. J.D. Medaglia, W. Huang, E.A. Karuza, A. Kelkar,
ity. Front Neurosci. 9, 202 (2015). https://doi.org/10. S.L. Thompson-Schill, A. Ribeiro, D.S. Bassett,
3389/fnins.2015.00202 Functional alignment with anatomical networks is
32. Y. Ezzyat et al., Direct brain stimulation modulates associated with cognitive flexibility. Nat. Hum. Be-
encoding states and memory performance in hu- hav. 2(2), 156–164 (2018). https://doi.org/10.1038/
mans. Curr. Biol. (2017). https://doi.org/10.1016/j. s41562-017-0260-9
cub.2017.03.028 48. R.F. Betzel et al., Optimally controlling the human
33. J.G. White, E. Southgate, J.N. Thomson, S. Brenner, connectome: the role of network topology. Sci. Rep.
The structure of the nervous system of the nema- (2016). https://doi.org/10.1038/srep30770
tode Caenorhabditis elegans. Philos. Trans. R. Soc. 49. E. Tang, C. Giusti, G.L. Baum, S. Gu, E. Pollock,
Lond. B. Biol. Sci. (1986). https://doi.org/10.1098/ A.E. Kahn, D.R. Roalf, T.M. Moore, K. Ruparel,
rstb.1986.0056 R.C. Gur, R.E. Gur, T.D. Satterthwaite, D.S. Bassett,
34. J.P. Nguyen et al., Whole-brain calcium imaging with Developmental increases in white matter network
cellular resolution in freely behaving Caenorhabditis controllability support a growing diversity of brain
elegans. Proc. Natl. Acad. Sci. (2016). https://doi.org/ dynamics. Nat. Commun. 8(1), 1252 (2017)
10.1073/pnas.1507110112 50. S. Gu, R.F. Betzel, M.G. Mattar, M. Cieslak, P.R.
35. C.T. Lin, Structural controllability. IEEE Trans. Au- Delio, S.T. Grafton, F. Pasqualetti, D.S. Bassett, Op-
tom. Control (1974). https://doi.org/10.1109/TAC. timal trajectories of brain state transitions. Neuroim-
1974.1100557 age. 148, 305–317 (2017). https://doi.org/10.1016/j.
36. G. Yan et al., Network control principles pre- neuroimage.2017.01.003
dict neuron function in the Caenorhabditis elegans 51. E.J. Cornblath, E. Tang, G.L. Baum, T.M. Moore,
connectome. Nature (2017). https://doi.org/10.1038/ A. Adebimpe, D.R. Roalf, R.C. Gur, R.E. Gur, F.
nature24056 Pasqualetti, T.D. Satterthwaite, D.S. Bassett, Sex
37. E.K. Towlson et al., Caenorhabditis elegans and differences in network controllability as a pre-
the network control framework–FAQs. Philos. Trans. dictor of executive function in youth. Neuroim-
R. Soc. B. (2018). https://doi.org/10.1098/rstb.2017. age 188, 122–134 (2018). https://doi.org/10.1016/j.
0372 neuroimage.2018.11.048
38. A. Avena-Koenigsberger, B. Misic, O. Sporns, Com- 52. Z. Cui, J. Stiso, G.L. Baum, J.Z. Kim, D.R. Roalf,
munication dynamics in complex brain networks. R.F. Betzel, S. Gu, Z. Lu, C.H. Xia, R. Ciric, T.M.
Nat. Rev. Neurosci. 19(1), 17–33 (2017). https://doi. Moore, R.T. Shinohara, K. Ruparel, C. Davatzikos,
org/10.1038/nrn.2017.149 F. Pasqualetti, R.E. Gur, R.C. Gur, D.S. Bassett, T.D.
39. D.G. Taylor, M.C. Bushell, The spatial mapping of Satterthwaite, Optimization of energy state transi-
translational diffusion coefficients by the NMR imag- tion trajectory supports the development of executive
ing technique. Phys. Med. Biol. (1985). https://doi. function during youth. bioRxiv 424929; https://doi.
org/10.1088/0031-9155/30/4/009 org/10.1101/424929
40. P. Hagmann et al., Mapping the structural core of 53. C. Moler, C.V. Loan, Nineteen dubious ways to
human cerebral cortex. PLoS Biol. (2008). https://doi. compute the exponential of a matrix, twenty-five
org/10.1371/journal.pbio.0060159 years later. SIAM Rev. (2003). https://doi.org/10.
41. J.D. Power et al., Functional network organization of 1137/S00361445024180
the human brain. Neuron (2011). https://doi.org/10. 54. F. Pasqualetti, S. Zampiere, F. Bullo, Controllability
1016/j.neuron.2011.09.006 metrics, limitations and algorithms for complex net-
42. D.S. Bassett, P. Zurn, J.I. Gold, On the nature and works, in 2014 American Control Conference (2014).
use of models in network neuroscience. Nat. Rev. https://doi.org/10.1109/ACC.2014.6858621
Neurosci. 19(9), 566–578 (2018). https://doi.org/10. 55. C.W.J. Granger, Investigating causal relations by
1038/s41583-018-0038-8 econometric models and cross-spectral methods.
43. M.E. Raichle et al., A default mode of brain function. Econometrica (1969). https://doi.org/10.2307/
PNAS (2001). https://doi.org/10.1073/pnas.98.2.676 1912791
44. O. Sporns, R.F. Betzel, Modular brain networks. 56. A.K. Seth, A.B. Barrett, L. Barnett, Granger causality
Annu. Rev. Psychol. 67, 613–640 (2016). https://doi. analysis in neuroscience and neuroimaging. J. Neu-
org/10.1146/annurev-psych-122414-033634 rosci. 35(8), 3293–3297 (2015). https://doi.org/10.
45. S.L. Bressler, V. Menon, Large-scale brain net- 1523/JNEUROSCI.4399-14.2015
works in cognition: emerging methods and principles. 57. L. Barnett, A.B. Barrett, A.K. Seth, Misunderstand-
Trends. Cogn. Sci. (2010). https://doi.org/10.1016/j. ings regarding the application of Granger causality in
tics.2010.04.004 neuroscience. Proc. Natl. Acad. Sci. U. S. A. 115(29),
46. S. Gu, F. Pasqualetti, M. Cieslak, Q.K. Telesford, E6676–E6677 (2018). https://doi.org/10.1073/pnas.
A.B. Yu, A.E. Kahn, J.D. Medaglia, J.M. Vettel, M.B. 1714497115
Miller, S.T. Grafton, D.S. Bassett, Controllability of
58. K.J. Friston, Functional and effective connectivity: a 72. T. Menara, D.S. Bassett, F. Pasqualetti, Structural
review. Brain Connect. 1(1), 13–36 (2011). https:// controllability of symmetric networks. IEEE Trans.
doi.org/10.1089/brain.2011.0008 Autom. Control 64(9), 3740–3747 (2019). https://
59. A.R. McIntosh, Tracing the route to path analysis in ieeexplore.ieee.org/document/8533416
neuroimaging. Neuroimage. 62(2), 887–890 (2012). 73. L. Wiles, S. Gu, F. Pasqualetti, B. Parvesse, D.
https://doi.org/10.1016/j.neuroimage.2011.09.068 Gabrieli, D.S. Bassett, D.F. Meaney, Autaptic connec-
60. C.O. Becker, D.S. Bassett, V.M. Preciado, Large- tions shift network excitability and bursting. Sci. Rep.
scale dynamic modeling of task-fMRI signals via 7, 44006 (2017). https://doi.org/10.1038/srep44006
subspace system identification. J. Neural Eng. 15(6), 74. H. Ju, J.Z. Kim, D.S. Bassett, Network topology
066016 (2018). https://doi.org/10.1088/1741-2552/ of neural systems supporting avalanche dynamics
aad8c7 predicts stimulus propagation and recovery (2018).
61. C. Stosiek, O. Garaschuk, K. Holthoff, A. Konnerth, arXiv:1812.09361
In vivo two-photon calcium imaging of neuronal net- 75. E.J. Cornblath, A. Ashourvan, J.Z. Kim, R.F. Betzel,
works. Proc. Natl. Acad. Sci. (2003). https://doi.org/ R. Ciric, G.L. Baum, X. He, K. Ruparel, T.M. Moore,
10.1073/pnas.1232232100 R.C. Gur, R.E. Gur, R.T. Shinohara, D.R. Roalf, T.D.
62. A.E. Motter, Networkcontrology. Chaos 25(9), Satterthwaite, D.S. Bassett, Temporal sequences of
097621 (2015). https://doi.org/10.1063/1.4931570 brain activity at rest are constrained by white matter
63. B.O. Koopman, Hamiltonian systems and transforma- structure and modulated by cognitive demands. Com-
tions in Hilbert space. Proc. Natl. Acad. Sci. (1931). mun. Biol. (2020, In Press).
https://doi.org/10.1073/pnas.17.5.315 76. S.F. Muldoon, J. Costantini, W.R.S. Webber, R.
64. S.L. Brunton, B.W. Brunton, J.L. Proctor, J.N. Kutz, Lesser, D.S. Bassett, Locally stable brain states pre-
Koopman invariant subspaces and finite linear rep- dict suppression of epileptic activity by enhanced
resentations of nonlinear dynamical systems for cognitive effort. Neuroimage Clin. 18, 599–607
control. PLoS One (2016). https://doi.org/10.1371/ (2018). https://doi.org/10.1016/j.nicl.2018.02.027
journal.pone.0150171 77. S.A. Mengiste, A. Aertsen, A. Kumar, Effect of edge
65. T. Kailath, Linear Systems (Prentice-Hall, Englewood pruning on structural controllability and observability
Cliffs, 1980) of complex networks. Sci. Rep. 5, 18145 (2015)
66. J.Z. Kim, J.M. Soffer, A.E. Kahn, J.M. Vettel, F. 78. J. Jeganathan, A. Perry, D.S. Bassett, G. Roberts, P.B.
Pasqualetti, D.S. Bassett, Role of graph architecture Mitchell, M. Breakspear, Fronto-limbic dysconnec-
in controlling dynamical networks with applications tivity leads to impaired brain network controllability
to neural systems. Nat. Phys. 14, 91–98 (2018). in young people with bipolar disorder and those at
https://doi.org/10.1038/nphys4268 high genetic risk. Neuroimage Clin. 19, 71–81 (2018).
67. Y. Xiao, S. Lao, L. Hou, M. Small, L. Bai, Effects https://doi.org/10.1016/j.nicl.2018.03.032
of edge directions on the structural controllability of 79. B.C. Bernhardt, M. Liu, R. Vos de Wael, J.
complex networks. PLoS One 10(8), e0135282 Smallwood, E. Jefferies, S. Gu, D.S. Bassett, A.
(2015). https://doi.org/10.1371/journal.pone. Bernasconi, N. Bernasconi, Hippocampal pathology
0135282 modulates white matter connectome topology and
68. S.P. Pang, W.X. Wang, F. Hao, Y.C. Lai, Universal controllability in temporal lobe epilepsy. Neurology.
framework for edge controllability of complex net- 92(19), e2209-e2220 (2019).
works. Sci. Rep. 7(1), 4224 (2017). https://doi.org/10. 80. S.F. Muldoon, F. Pasqualetti, S. Gu, M. Cieslak,
1038/s41598-017-04463-5 S.T. Grafton, J.M. Vettel, D.S. Bassett, Stimulation-
69. A.J. Whalen, S.N. Brennan, T.D. Sauer, S.J. Schiff, based control of dynamic brain networks. PLoS Com-
observability and controllability of nonlinear net- put. Biol. 12(9), e1005076 (2016). https://doi.org/10.
works: the role of symmetry. Phys. Rev. X 5, 011005 1371/journal.pcbi.1005076
(2015). 81. X. Yang, N. Ram, S.D. Gest, D.M. Lydon-Staley, D.E.
70. E. Wu-Yan, R.F. Betzel, E. Tang, S. Gu, F. Pasqualetti, Conroy, A.L. Pincus, P.C.M. Molenaar, Socioemo-
D.S. Bassett, Benchmarking measures of network tional dynamics of emotion regulation and depres-
controllability on canonical graph models. J. Nonlin- sive symptoms: a person-specific network approach.
ear Sci. 1–39 (2018). https://doi.org/10.1007/s00332- Complexity 2018, pii: 5094179 (2018) https://doi.
018-9448-z org/10.1155/2018/5094179
71. C. Tu, R.P. Rocha, M. Corbetta, S. Zampieri, M. 82. D.M. Lydon-Staley, I. Barnett, T.D. Satterthwaite,
Zorzi, S. Suweis, Warnings and caveats in brain con- D.S. Bassett, Digital phenotyping for psychiatry: Ac-
trollability. Neuroimage 176, 83–91 (2018). https:// commodating data and theory with network science
doi.org/10.1016/j.neuroimage.2018.04.010 methodologies. Curr. Opin. Biomed. Eng. 9, 8–13
(2019). https://doi.org/10.1016/j.cobme.2018.12.003
Deciphering the Neuronal Population
Code 18
Sanjeev B. Khanna and Matthew A. Smith
Abstract pairs to more recent approaches leveraging the

newest technology to measure tens to hundreds
Neuroscience has long asked questions about
of neurons simultaneously.
how neurons represent both external sensory
information arriving from the outside world
Keywords
and motor and cognitive signals that are in-
ternal to an organism. These questions were Correlated variability · Noise correlation ·
first asked at the level of spiking activity in Signal correlation · Synchrony · Decoding ·
single neurons, but in the latter years of the Dimensionality reduction · Factor analysis ·
twentieth century, technological advances en- Generalized linear model · Choice
abled recording from small groups of neu- probability · Brain computer interface
rons. Along with the technology, computa-
tional frameworks have been developed to an-
alyze neuronal populations, starting with pairs
and moving into larger groups. This body of
18.1 Introduction
work has opened up avenues of inquiry that
range from primarily theoretical (how do neu-
In the latter portion of the nineteenth century,
rons represent information?) to highly practi-
developments in anatomical staining by Camillo
cal (how can we design a robust brain com-
Golgi [1] and later investigation of nervous sys-
puter interface?). This chapter reviews the his-
tem tissue by Santiago Ramon y Cajal led to
tory of analytic approaches and neuroscience
the neuron doctrine, the notion that the nervous
research aimed at deciphering the population
system is comprised of individual cells, along
code, from early work with single neurons and
with vivid pictures of the intricate connectivity
among these cells [2]. Work in the first half of
Electronic Supplementary Material: The online version the twentieth century led to understanding of fun-
of this chapter (https://doi.org/10.1007/978-3-030-43395- damental principles of the neural code – neurons
6_18) contains supplementary material, which is available signal by emitting “spikes” (action potentials),
and spikes convey information to their targets
S. B. Khanna through their rate or timing. Since this time, the
Department of Bioengineering, University of Pittsburgh, understanding of the neuronal population code
Pittsburgh, PA, USA
has been advanced greatly by experimental work
M. A. Smith () in which the activity of many tens to hundreds of
Department of Biomedical Engineering and
neurons is recorded simultaneously and by theo-
Neuroscience Institute; Carnegie Mellon University,
Pittsburgh, PA, USA retical work that has attempted to derive princi-
e-mail: [email protected] ples of information encoding and transfer. In this

520 S. B. Khanna and M. A. Smith
chapter, we review some of this ongoing literature 45◦ steps. This neuron emitted more spikes for
that delves into the computational principles of leftward target locations (135◦ ,180◦ ,225◦ ) com-
the brain. pared to rightward locations (0◦ ,45◦ ,315◦ ) as thus
can be considered “tuned” for leftward targets (it
was recorded in the right hemisphere, and the
18.2 Extracting Information preferred targets were, as is typical in this brain
from Single Neurons area, the contralateral visual field).
Like many signals in biological systems how-
Within the sensory domain, the information neu- ever, neuronal spiking is variable. If a target is
rons convey spans many different modalities in- presented repeatedly at the same location, re-
cluding vision, hearing, taste and smell, balance sponses will vary from trial to trial. For the exam-
and spatial orienting of the body, and somatosen- ple neuron shown, spiking responses to a down-
sory experiences such as touch and pain. Single ward target (orange dot, 270◦ ) were greater than
neurons can respond selectively to different stim- 20 spikes/second for some trials and as low as
ulus attributes such as the spatial location of a dot 0 spikes/second for others (orange histogram in
displayed on a computer screen, the constituent Fig. 18.1b). At the single-neuron level, variability
frequencies of a sound, or whether the index can be measured by computing the variance of the
finger or thumb is touched. This phenomenon is spiking response for each condition, across trials
not limited to sensory systems, but also occurs of that particular condition (Fig. 18.1c). The spik-
for motor systems, such as those that control the ing response and variance scale with each other,
six muscles that coordinate to produce a saccadic such that conditions with a high spiking response
eye movement, and the more complex muscle and also have high variance, while conditions with
joint kinematics required to kick a ball. low spiking responses have low variance. In a
A neuron is said to be tuned if across a given Poisson process, often used as a mathematical ap-
set of attributes (such as spatial location of a proximation of the statistics of neuronal spiking
visual stimulus), the neuron varies its activity in behavior, the ratio of the variance to the mean is
a systematic way. The optimal or preferred stim- one.
ulus is then the unique stimulus which elicits the By calculating the tuning curves of individual
largest response from the neuron. In many cases, neurons, researchers can gain some insight into
neuronal responses vary smoothly as a stimulus is how stimuli are represented in the brain. In Fig.
changed, and by varying a given stimulus attribute 18.1, spike responses for two conditions (blue and
across a wide range of values, a tuning curve orange histograms, 180◦ and 270◦ ) are plotted
can be generated. For example, in a region of for many repeats of these two conditions. If an
the prefrontal cortex known as the frontal eye observer was using the spiking response of this
fields (FEF), the spikes that a neuron emits tend neuron to determine the location of the stimulus
to be related to both the spatial location of a (left or down), for some responses it would be
visual stimulus and the direction and magnitude clear where the stimulus was being presented.
of eye movements. To determine the tuning of a For instance, only the leftward stimulus (blue his-
single neuron, experimenters typically monitor its togram) showed responses greater than 40 sp./s,
electrical activity while an experimental subject while the weaker responses (in the range of 15–
(in this example, a macaque monkey) performs a 20 sp./s) could arise from a stimulus in either
simple task in which it makes eye movements to the blue or orange condition. To quantify the
flashed stimuli. degree to which a neuron can accurately represent
To illustrate this with an example from real a stimulus, researchers have drawn from signal
neuronal data, we show (Fig. 18.1a) the spiking detection theory methods, including d-prime (d’).
response of a frontal eye field neuron that was d’ is calculated as the difference of the means of
recorded as the spatial location of a visual stim- two distributions normalized by the standard de-
ulus was varied across the visual hemifield in viation of the two distributions. A neuron’s d’ can
18 Deciphering the Neuronal Population Code 521
A 25
B 60
C
140
50 120
Frequency (number of trials)

20
Spiking response (sp/s)
Spiking variance (sp2)

100
40
15
80
30
10 60
20
40
5
10
20
0 0 0
0 90 180 270 0 20 40 60 0 90 180 270
Target location (deg) Spiking response (sp/s) Target location (deg)
Fig. 18.1 Tuning curve and variability. (a) In this simple hemisphere (blue dot), but much more weakly to stimuli
tuning curve, the firing rate (in spikes per second) of a flashed downward (270 degrees, orange dot). (b) These
single FEF neuron (recorded in the right hemisphere) is histograms show the distribution of spike counts in many
plotted as a mean value (across many trials) with the stan- repeated trials of the two stimulus conditions in panel A.
dard error of the mean (SEM) indicated by the error bars The means of the distributions are indicated by the vertical
(± 1 SEM). This single neuron responds briskly to visual lines. (c) Variance in the spike count (computed across
stimuli flashed contralateral (180 degrees) to the recorded trials) is shown here as a function of the stimulus condition
be used to calculate a “neurometric” function [3], Now, to extend this example to neuroscience,
which can be related to behavioral performance let’s consider a set of 8 sensory neurons, each
(the psychometric function). We will discuss this of which encode the presence or absence of a
issue further in the section Relating neurons to be- single sensory input (one of the coin flips) with
havior. Our next section, however, will deal with a spike. If we were recording from 8 neurons, the
generalizing the concept of neuronal responses population could represent the 8 coin flips (the
beyond the single neuron. variable x). That is, if each neuron fired a spike
(1) or did not (0), we could encode the results
of the coin flips in the spiking of the neurons (1
18.3 Correlation in Pairs spike for heads, 0 spike for tails). If those eight
of Neurons neurons were able to accurately encode the coin
flips and never made a mistake (i.e., fired a spike
To give some intuition on how variability can for tails, or failed to fire a spike for heads), then no
affect stimulus encoding, we start with a simple information would have been lost, and our coin-
coin flipping example. Imagine you flip a coin flip-detecting sensory system would have perfect
eight times and observe the number of times it fidelity in its encoding.
lands on each side of the coin (“heads” or “tails”). There are then two simple ways that this sen-
We can define a variable x that represents whether sory system could be imperfect. First, there could
for a given flip the result was heads or tails (x = 0 be noise in the encoding of the coin flips. That is,
for tails, x = 1 for heads). We can thus represent some of our sensory neurons could make mistakes
the outcome of the 8 trials as an 8 bit binary on occasion. Second, some of the neurons could
number (11111111 corresponds to 8 flips landing be correlated. If two of the neurons were perfectly
all on heads, 00000000 to 8 flips landing all on correlated, only 27 states would be possible in
tails, 10101010 to 50% heads and 50% tails) this set of neurons (instead of 28 ). Thus, in a
that would have 28 possible values (ignoring the very simple sense, correlation can have an im-
ordering of the responses). pact on the amount of information that can be
represented in a population. This seems unde- neurons could be averaged out, if each neuron’s
sirable in a simple sense – uncorrelated sensory variability was independent. However, variability
neurons would seem to provide the best possible in neurons is not in fact independent, but shared,
sensory encoding. However, the brain contains meaning it could not simply be averaged away by
neurons that are massively interconnected, and pooling across neurons [4].
correlation may be a necessary side effect of that Noise correlation (also known as rnoise , rsc , or
interconnection. spike count correlation) describes the tendency of
As another analogy to reinforce the effect of a pair of neurons to co-fluctuate their activity for
correlation, consider the same population of eight an identical stimulus presented repeatedly and has
neurons who now each fire spikes related to the been used as a measure of the shared variability
outcome of a single coin flip. If the coin landed in pairs of neurons. It is termed “noise” correla-
on heads, then they would all fire one spike (and tion because it involves fluctuations in neuronal
zero spikes for tails). As we mentioned above, response that are independent of the stimulus
noise in the encoding of the coin flip by individual (and are typically measured in repeated presen-
neurons could decrease the fidelity of their repre- tations of an identical stimulus). This tendency
sentation of the coin flip outcome. One way to ac- of neurons to fluctuate their responses together
count for noisiness in the responses of individual has been used as an index of their functional con-
neurons would be to average the spike responses nectivity, reflecting the direct and indirect con-
of the eight neurons and set a threshold such that if nections present in a network of neurons. Such
the total spike response across the eight neurons a common source of variability could arise from
is greater than 4 (0.5 spikes/neuron), a stimulus noise present in a common input to both neurons.
was present. This would be an effective way of By studying how groups of neurons vary their
dealing with noise in the encoding of the coin activity together, researchers can gain insights on
flip in individual neurons. If up to three of the the functional architecture of neuronal networks,
eight neurons made an error, the spike response a difficult task to achieve by only recording single
would still exceed our threshold, and we would neurons. Furthermore, it is possible that by learn-
know the outcome of the coin flip. However, if ing how pairs of neurons interact, and how those
there is correlation among some pairs within the interactions are linked to behavior, we can glean
group of eight neurons, when one neuron fired an principles that impact the operations of much
“accidental” spike when the coin turned up tails larger groups of neurons.
(and it was supposed to not fire), the other neurons Noise correlation (referred to hereafter as rsc )
positively correlated with that neuron also would is typically calculated across repeats of a given
exhibit increases in the likelihood or magnitude condition by computing the Pearson correlation
of their spiking response. Thus, the presence of coefficient of a pair of neuronal spike responses.
correlation in pairs of neurons can produce errors It is also possible to combine data across condi-
in the population’s encoding of a stimulus. tions, typically by z-scoring the spiking responses
An early motivation for studying correlations within each condition before computing the Pear-
came from the observation that in certain circum- son product-moment correlation coefficient. Crit-
stances, the behavioral performance of a subject ically, to calculate rsc , simultaneous recordings
in a task was not substantially better than the must be made of multiple neurons (at least 2)
single most sensitive neuron recorded during that across multiple trial repeats (at least 20 or 30 are
task [3]. Although it has already been highlighted needed to estimate rsc reliably).
that single-neuron responses can vary from trial When considering the effect of rsc on stimulus
to trial even in identical task conditions (Fig. encoding, the tuning of each neuron in the pair
18.1b), as described above pooling across a small must be considered. Signal correlation, or rsignal ,
subpopulation of these sensitive neurons could describes the similarity of tuning in two neurons
result in an accurate estimate of the stimulus. By by calculating the Pearson correlation coefficient
pooling across neurons, small variations in single of their tuning curves. An rsignal value of 1 cor-
A B
35 rsignal = .90 rsignal = -.42
30
neuron 1
25 neuron 2
Firing rate (sp/s)
20
15
10
neuron 1
5 neuron 2
0
0 90 180 270 0 90 180 270
Target location (deg) Target location (deg)
high rsc high rsc

Firing rate neuron 2 (sp/s)
condition 1 condition 1
condition 2
condition 2
low rsc low rsc

Firing rate neuron 2 (sp/s)
condition 1 condition 1
condition 2
condition 2
Firing rate neuron 1 (sp/s) Firing rate neuron 1 (sp/s)
Fig. 18.2 (a) An example pair of neurons with similar ability of an observer to determine the stimulus identity
tuning curves (top panel). When rsc is large and positive (bottom panel). (b) For an example pair of neurons with
(middle panel), discrimination between a pair of stimuli dissimilar tuning (top panel), the effect of rsc would be
can be difficult (ellipses cross the dashed decision bound- quite different, better with large values (middle panel) than
ary). A reduction of rsc in this case would enhance the with weakly correlated neurons (bottom panel)
responds to two neurons whose tuning curves A simple example of how correlation between
are identical (or scaled copies), while a rsignal of neurons can impact coding is shown in Fig. 18.2.
−1 would correspond to tuning curves that are Consider first two neurons in Fig. 18.2a (top
opposite (Fig. 18.2). panel). They have similar tuning curves, mean-
ing the stimuli that elicit large (and small) re- The results from Zohary et al. [4] and related
sponses are shared between the two neurons. With work brought about many theoretical studies de-
this pair of neurons, we can consider the goal voted to the impact of correlations on information
of discriminating the trial condition (1 and 2, encoding in populations of neurons. A common
which can be thought of as the blue and or- measure of information encoding is Fisher infor-
ange conditions from Fig. 18.1). In a joint fir- mation, which in a neural encoding context is
ing rate space (Fig. 18.2a, illustration in mid- meant to capture the amount of information about
dle and bottom panels), the neural responses are the stimulus carried by the spiking responses of a
plotted, with the filled circles representing the group of neurons. Some of the earliest studies to
mean firing rates for each condition and the el- examine Fisher information and neuronal corre-
lipses representing the variability (2 standard de- lations disagreed on the impact of correlations on
viations from the mean). Two cases are shown information in neuronal populations [6–8], with
for this pair, one in which the rsc is low (bottom a key point of disagreement coming from the
panel) and high (middle panel). These are visu- question of whether correlation could lead to a
ally distinguishable because of the shape of the saturation in the gains in information as popula-
joint response ellipses. If the two neurons were tion size grows. In other words, in the presence of
uncorrelated (rsc of zero), those ellipses would correlation, is the brain limited in how much it can
be completely circular. In the case of higher rsc do with a large group of neurons? Assumptions
(middle panel), the ellipses are more elongated, in the structure of neuronal responses and their
indicating that the noise in the response of one correlations can have an important impact on
neuron around its mean was highly predictive of this calculation. For populations of neurons with
the noise in another neuron’s response around its heterogenous tuning (tuning curves have different
mean. amplitudes and widths), Fisher information does
The noise shared among a pair of neurons not saturate [9, 10]. Similar assumptions in the
can impact the ability of an observer to determine correlation structure are critical, such as whether
which stimulus (red or blue) was shown. A simple correlations are uniform across the population or
way to distinguish the stimulus identity would decrease as a function of neuronal response char-
be to determine an optimal decision boundary, acteristics such as tuning similarity or distance.
considering the noise in each neuron’s response. Ultimately, great care should be taken when infer-
Such an optimal boundary could be sufficient ring the impact of correlations in pairs of neurons
when the rsc of the pair of neurons is low (dashed or simulated populations [11].
line, bottom left panel), and few mistakes would
occur. However, for the same pair of neurons
with the same mean responses to the two stimuli, 18.4 Synchrony in Pairs
a higher rsc (middle left panel) would lead to of Neurons
mistakes in identifying the stimulus – instances
where the response falls on the “wrong” side So far in this chapter, we have described cor-
of the decision boundary. When we consider relation in pairs of neurons in a fashion that is
a pair of neurons with dissimilar tuning (Fig. independent of time. However, co-fluctuations in
18.2b, rsignal < 0), however, the intuition changes. the responses of pairs of neurons can be measured
In this case, discrimination between the two across a wide range of time-courses, from mil-
conditions would be better when rsc was high liseconds to spanning the entire trial (hundreds
(middle right panel) when compared to lower of milliseconds). The precise synchronization of
values of rsc (bottom right panel). This simple spiking responses on a millisecond by millisec-
example highlights how the structure of both ond time scale is often referred to as synchrony.
signal and noise correlation can greatly impact To measure the synchrony of a pair of neurons, a
the ability of downstream neurons to decode common tool is the spike train cross-correlogram
neuronal population activity [5]. (CCG). For a given pair of neurons, the CCG mea-
A No synchrony
B No synchrony
C Synchrony
10
Trial #
100 Neuron 1
Firing rate
Neuron 2
(sp/s)
50
0
0 1 0 Time from stimulus onset (s) 1 0 1
Raw CCG
P (coinc.)
Trial-shuffled CCG
0.1 Shuffle-subtracted CCG
0
-1 0 1 -1 0 1 -1 0 1
Time lag (s)
Fig. 18.3 (a) The responses of a pair of independent neu- firing rates (shuffle-subtracted CCG in gray). (b) If the two
rons (red and blue) are shown as both a raster (top panel) neurons have no synchrony but a similar stimulus response
and smoothed temporal response function (middle panel) profile, an elevated synchrony level can be detected in
for a 1 second stimulus. In the bottom panel, the cross- the CCG but will be removed by trial shuffling. (c) A
correlogram (CCG) is shown between the two neurons. synchronous pair of neurons (with a time scale of less
The raw CCG (black) is very similar to the trial-shuffled than 10 ms) will exhibit a peak in the CCG which is not
CCG (green), indicating that these two neurons were not removed by trial shuffling
correlated above chance levels expected based on their
sures the number of coincident spikes between the behavior under study and then subtract it from
two neurons’ spike trains at different time lags the raw CCG. Whatever remains after that sub-
(Fig. 18.3). If a peak is observed at 0 ms, for traction is the “true” synchrony. For example, a
example, the pair of neurons tended to spike at pair of neurons that spike randomly will exhibit
the same time, while a peak at 3 ms would mean some level of synchrony (Fig. 18.3a). If one con-
one neuron tended to spike 3 ms after the other. structs a surrogate data set in which the trials are
Typically, to quantify synchrony, the area under rearranged such that the trials of the two neurons
the CCG is calculated for time lags in a small were not recorded simultaneously, synchrony that
window. In the calculation of synchrony (as with was due merely to the spike rate would be pre-
correlation overall), an important consideration served. If that “shuffled” CCG is subtracted from
is the expectation due to chance. That is, for a the raw CCG, a flat (albeit noisy) trace remains,
pair of neurons with moderate firing rates, even if indicating a lack of synchrony beyond the chance
their spiking activity is independent, some spikes expectation. This shuffled control can also correct
may still occur within a few milliseconds due to for synchrony that is observed merely due to
chance (Fig. 18.3a). Furthermore, if those neu- the response timing of the neurons (Fig. 18.3b).
rons share a particular feature of their response, However, if the neurons are truly synchronous
for instance, a transient increase in spiking when (Fig. 18.3b, simulated with synchronous spiking
the stimulus turns on, it may masquerade as a on a time scale of a few milliseconds), synchrony
precise relationship between the neurons when in will be preserved even after the shuffled control.
fact it only indicates their common relationship It is also possible to use other forms of shuf-
to the stimulus. How do we determine if the syn- fled correction, which attempt to isolate the spe-
chronously timed spikes we observe are simply cific time scale of synchrony [12–14]. In this
due to chance, or instead indicate some precise case, jittering the spike times within a particu-
temporal relationship between the neurons? lar designated window (e.g., 50 ms) can destroy
A widely used means to control for stimulus- the potential for synchrony on a fast time scale,
locked responses and those due to random coin- but preserve the overall PSTH and the potential
cidences in spiking involves using surrogate data for slower time scale interactions. Designing a
to compute a CCG. The goal of this procedure method to create surrogate spike trains requires
is to generate surrogate spike trains that are like careful thought about the response properties of
the real data in all ways except the synchronous the neurons and the time scale under study. Im-
portantly, the CCG approach to studying faster in machine learning. One such approach is
time scale interactions and the Pearson’s corre- dimensionality reduction, which reduces high-
lation metric of rsc are linked. The integrated area dimensional (tens to hundreds of neurons)
under the CCG, when normalized by the temporal neural activity into a more tractable subspace
pattern of spiking of each neuron in the pair, is (a smaller number of latent dimensions) that
mathematically equivalent to rsc when computed maintains many of the important features of the
at the same time scale [15]. high-dimensional neural activity. One common
dimensionality reduction method is principal
component analysis (PCA) which identifies the
18.5 Beyond Pairwise Correlation dimensions in the neural data that contain the
most variance (Fig. 18.4). PCA is particularly
Pairwise correlations and single-neuron metrics useful in identifying the separation of neural
provide a relatively simple measure of how neural activity due to stimulus tuning. For instance,
activity is evolving, that when averaged across in a population of FEF neurons with tuning to
many pairs or neurons can give some index into the location of a visual target, the activity on
the underlying circuitry. However, changes in cor- single trials is separated into distinct clouds in
relations can be difficult to interpret when relating a PCA subspace generated from the condition-
these measures to higher cognitive processes such averaged responses (Fig. 18.4). Importantly, a
as attention. Attention modulates the spiking of PCA subspace is defined to describe all of the
sensory neurons, typically increasing their firing variability present in the responses, whether
rate when compared to an unattended stimulus it derives from the stimulus, single neuron
[16–18]. Correlations have also been shown to variability, or neuronal correlation.
decrease with attention in pairs of neurons [19, Factor analysis (FA) expands upon PCA by
20], indicating that the changes in the activity of explicitly defining a noise term private to each
populations of neurons due to attention cannot neuron, allowing neural variability to be parti-
be described simply as the sum of single-neuron tioned into components shared across the pop-
changes. That is, a change in firing rate alone ulation and those independent to each neuron
does not capture the changes in the population [21, 22]. This conceptually links FA and pairwise
due to attention. However, it is difficult to de- correlation measures such as rsc , which also incor-
velop simple intuition for how the activity of porates covariance and variance. Similarly to how
single neurons and pairs of neurons influences synchrony and correlation are related but attempt
the population-level structure, particularly since to capture different time scales, population-level
previous modeling studies have shown correla- analyses like PCA and FA have been extended
tions can be both beneficial and detrimental to to incorporate temporal information. Two such
information capacity. Furthermore, correlations techniques are Gaussian process factor analysis
are measured across trials, but for directly link- (GPFA) [23] and latent factor analysis via dy-
ing population activity to behavior, it is desir- namical systems (LFADS) [24]. By assuming that
able to have population metrics that can be com- population activity is both low-dimensional and
puted on a trial-by-trial basis. As an example, smoothly varying over time, these methods can
one might want to identify simple descriptions identify the temporal structure of the population
of what changes in a population between con- activity, which could vary across different task
ditions or states (for instance, paying attention epochs of an experiment. An important additional
or not). The following paragraph will describe method to describe population activity involves
common measures of population activity, while the explicit identification of the relevant variables
later paragraphs will describe their applications within a regression framework, known as a gen-
in experimental paradigms. eralized linear model (GLM). In this case, rele-
Many approaches to measuring population vant variables like interaction strengths between
activity are based on statistical techniques rooted neurons, spiking history, stimulus input, and more
Define low dimensional space Project each individual trial into

based on condition average low dimensional space
Condition # (target direction) Principal component
1 (0˚) 3 (90˚) 5 (180˚) 7 (270˚) 1 2
40
1 μ1 , 1 μ1 , 3 μ1 , 5 μ1 , 7
30
20
Principal component 2
Neuron #
10
Trial #
0
-10
-20
21 -30
-30 -20 -10 0 10 20 30 40
Principal component 1
Fig. 18.4 Data from a population of 21 FEF neurons in shown). This method allows the projection of the individ-
which 8 different conditions were shown. In this example, ual trials (only four conditions are shown for simplicity)
a matrix containing the mean responses of those 21 neu- into that space. The condition means (large black-outlined
rons to 8 different stimulus conditions forms the starting circles) are well separated, but the individual trials have
point. PCA can identify a low-dimensional subspace that some overlap, indicating that imperfect decoding would
captures most of the variance (here, two dimensions are be obtained from this population of neurons
can be incorporated into a generative model that 18.5.1 Relating Neurons to Behavior
describes spiking activity. Such a framework can
be used to capture neuronal responses while tak- While descriptive statistics of neural activity,
ing into account the population [25]. The ability whether at the single neuron, pairwise, or
to explicitly identify and add parameters, and population level, are useful for establishing the
observe the change in prediction, makes it an response properties of neurons in a particular
important alternative to the dimensionality reduc- region, in order to truly understand how the brain
tion approach. works, these statistics must be related to high-
Advances in recording technologies have al- order processes. Identifying the neural correlates
lowed researchers to go from monitoring single of behavior has a long history, beginning with
neurons or small populations (tens of neurons) single-neuron experiments, extending to pairs
to recording from hundreds of neurons simulta- and populations, and even includes causal
neously [26]. To understand how all these neural manipulations of neural activity.
signals are combined in the brain to produce One of the first studies linking single-neuron
a behavior, statistical approaches from machine activity to behavior employed a binocular rivalry
learning have been applied to these data sets to paradigm. In this paradigm, two distinct stim-
identify the relevant signals in a more tractable uli (moving gratings) were presented separately
way. In particular, dimensionality reduction tech- to the two eyes, either moving in the same di-
niques allow researchers to extract population rection (i.e., upward, the nonrivalry condition)
signals from groups of simultaneously recorded or in different directions (i.e., one upward, one
neurons and relate these signals to behavior on downward, rivalry condition). During the rivalry
a trial-by-trial and moment-to-moment level. As condition, a perceptual instability occurs, as one
we will see in the next section, one of the ultimate eye would signal the motion to be downward,
goals of population analyses is to understand and while the other would signal it upward. The term
predict behavior. “rivalry” describes the phenomenon that subjects
report only one of the two motions (but not both) be used to define an “attention axis,” which goes
at a time, and this report can change across tri- beyond examining stimuli in unattended/attended
als or even within a trial. Logothetis and Schall conditions, but aims to quantify the degree of
[27] recorded from neurons in parietal cortex attention allocated to a stimulus on a trial-by-trial
and found that some neuronal responses reflected basis [37–39]. In the motor domain, shared vari-
the monkeys’ reported perception in the rivalry ability in populations decreases after the onset of
condition, indicating that the perceptual report of a stimulus [21] and is lower for faster eye move-
the subject could be read out from these neurons. ment reaction times [40]. Dimensionality reduc-
The typical way to quantify the relationship tion analyses have shown that population activ-
between the activity of individual neurons and ity occupies different low-dimensional subspaces
behavioral choices is with “choice probability,” when preparing to make an arm reach (denoted
which quantifies the ability of an ideal observer to the “null” space since the arm has not moved
determine which choice a subject will make based yet) and executing the arm movement (denoted
on the firing rate of a single neuron [28, 29]. Com- the “potent” space) [41]. Additionally, these low-
puted by taking the area under the receiver operat- dimensional trajectories across time can represent
ing characteristic (ROC) curve, choice probabil- vacillation between two reach targets, giving an
ity captures the tendency of a neuron’s variability indication of when the subject is “changing their
in response to an ambiguous stimulus to predict mind” [42].
an animal’s behavioral choices about that stimu- Here, we consider the same example of FEF
lus. In many cortical areas such as the middle tem- neurons in the context of an eye movement task
poral area (MT), lateral intraparietal area (LIP), used above (Fig. 18.1). In a group of FEF neurons,
somatosensory cortex (S1 and S2), premotor and FA can be used (similar to PCA in Fig. 18.4) to
motor cortex, and subcortical areas involved in identify a low-dimensional subspace that captures
eye movements (for review, see Crapse and Basso the shared variability among the neurons. In con-
[30]), neurons possess choice probability levels trast to the PCA procedure shown above, FA is
that easily exceed chance, meaning single neu- performed in this case on a matrix of neurons by
rons can accurately predict which decision the individual trials (Fig. 18.5). Because the behavior
subject will make. In general, choice probabili- of the animal is variable even though the target is
ties increase along the visual hierarchy, meaning in the same location (Fig. 18.5a, reaction times
higher-order “decision” areas have larger choice for one target location), this offers an opportunity
probabilities compared to earlier sensory areas. to predict behavior from neural activity. The 1st
Importantly, the presence of choice probabili- FA dimension captures the largest proportion of
ties in individual neurons by necessity will go shared variability that can be described from FA.
along with statistical relationships between neu- Although FA was not provided with information
rons, measured by pairwise correlation [29, 31, about the saccadic reaction time in the trials,
32]. Thus, an understanding of the relationship nonetheless individual trials fell in a different
between neurons and decisions must include con- portion of the 1st FA dimension axis (Fig. 18.5b)
sideration of both individual neurons and popula- when the eventual reaction time was particularly
tions. fast (green) or slow (purple). In a multidimen-
As previously mentioned, changes in pairwise sional subspace (Fig. 18.5c, 3 FA dimensions),
correlations have been linked to attention. While this was also true, where fast and slow trials were
numerous studies have found that correlations de- separable.
crease with attention [19, 20, 33, 34], this descrip- This FA result is one example of the link be-
tion does not capture the full picture. Correlation tween neuronal populations and behavior. How-
changes with attention depend on the tuning sim- ever, in the above example, FA was applied across
ilarity of neurons [35] and whether the pair is a large temporal window of several hundred mil-
within or between cortical areas [36]. At the pop- liseconds. Applying GPFA [23] to the same data
ulation level, activity from groups of neurons can yields a rich picture of the temporal extent to
A B C
50 6
fastest trials
slowest trials
5
40
FA Dimension 3
Number of trials
Number of trials
30
20
2
10 FA
1
Di
m
en 1
sio ion
0 0
n
2 imens
-4 -2 0 2 4 -150 -100 -50 0 50 FAD
Z-scored reaction time Projection onto 1st FA Dimension
Fig. 18.5 Activity of a population of FEF neurons around The activity of the population sits in a different position in
the time of an eye movement. (a) In an eye movement task, the FA subspace when fast (green) and slow (purple) eye
a subject tends to exhibit variable reaction times (reaction movements occur. (c) Using a larger subspace (here, 3 FA
times are z-scored in this example). (b) In a population dimensions), fast and slow reaction times can be predicted
of neurons, FA can identify an axis (1st FA dimension) in from the population-level activity of FEF neurons
which the neurons express substantial shared variability.
which population activity separates during eye timulation has been widely used to identify func-
movement preparation. In GPFA, each dimen- tional properties of different cortical and sub-
sion has its own time scale that best matches the cortical regions, identifying which areas, when
data. Once again, as with FA, individual trials stimulated, elicit a behavior such as a saccade
(grouped by reaction time) are well separated in [47–49], complex arm reach [50], and tactile sen-
the subspace (Fig. 18.6a). Considering the full sations [51]. In sensory areas, microstimulation
set of eight eye movement conditions, and the can influence the direction random motion dots
fastest and slowest trials in each (green and purple are perceived [52], while subthreshold microstim-
lines), there is a strong separation in the subspace ulation (not eliciting a movement) can improve
identified by GPFA (Fig. 18.6b). performance on visual discrimination task [53,
These data relating low-dimensional activity 54], delay arm reaches [55], and increase the
in FEF to reaction time [40] form but one of firing rates of neurons in visual cortex in a manner
many studies that have linked low-dimensionality similar to attention [56]. Causal interventions, of
motor activity to behavior. Importantly, they offer which electrical microstimulation is one of many,
the potential for large-scale population recordings are a powerful means to understand the relation-
to predict and account for behavior on a scale that ship between neuronal activity and behavior.
has not been possible with measurements from
individual neurons or pairs of neurons. Some fur-
ther examples of this type of analysis include re- 18.6 Developing Hypotheses
lating low-dimensional projections to reaching/- About the Structure
grasping reaction time [43, 44], task epoch from and Function of Neuronal
motor preparation to the onset of movement [41, Population Activity
45], and the motor preparatory state to one of two
targets [46]. Intracortical brain computer interfaces (BCIs) op-
Finally, one direct method of linking neural erating on spiking neural activity represent one
activity to behavior involves altering neural re- situation in which it is particularly important to
sponses through direct activation of neurons in link neural activity accurately to behavior. BCIs
the proximity of an electrode. Electrical micros- often involve a user controlling some kind of
A Factor loading (a.u.)

Reaction time decile
GPFA Dimension: fast 1 10 slow
1 2 3
0 1 0 1 0 1
Eye movement direction
fastest trials
slowest trials
Dimension 3
GPFA
G
Dim PFA GPFA 1
ens sion
ion
2 Dimen
Fig. 18.6 GPFA applied to FEF neurons during eye movement was made, and the values on the x-axis are
movement preparation. (a) For three GPFA dimensions, seconds, with 0 indicating the onset of fixation. (b) For
reaction times are separated into decile bins, and then all three GPFA dimensions, with all eight eye movement
trials are grouped and plotted in the appropriate color for directions, fast and slow trials are well separated in a
that bin. The vertical blue line represents the time an eye GPFA subspace
output, generally a computer cursor or robotic Another instance of using population activity
arm, by generating activity patterns across a pop- metrics to test hypotheses about the neural code
ulation of neurons. Because the mapping between is in the study of working memory. The nature
the activity and the control signal is defined by of the neural code for working memory has been
the experimenter, BCIs are useful tools to study hotly debated [60–62], in particular focusing
learning, when the mapping can be redefined or on whether persistent activity in single neurons
perturbed in some systematic way to identify how might form an effective code. One means to test
(and whether) the user can learn the new mapping that hypothesis involves investigating whether
[57]. In work of this nature, rich descriptions of the neural code (based on population activity)
population activity are essential to capture the can generalize across the memory period of a
nature of the population code in use [58] and to working memory task. That is, is the same code
recognize changes in activity patterns when they used at all times while a memory is stored? While
emerge [59]. some studies have shown evidence of neural
codes that do not generalize (in support of a examining single-neuron response properties can
more dynamic working memory framework (for aid in the interpretation of population signals,
review see Meyers [63]), an alternative avenue while understanding population signals can pro-
forward lies in the stable subspace approach that vide a holistic measure of how single-neuron ac-
permits a fixed population readout regardless of tivity is combined. Ultimately, the level (or levels)
the time in the memory period [64]. Others have at which researchers examine neural activity is
used decoding frameworks to compare cortical dependent on the research question posed.
regions, highlighting the presence of a stable
or dynamic code could depend on the role of a
particular area in the sensorimotor hierarchy [65, Homework
66]. Although this debate remains unresolved,
it is clear that population-level analyses provide 1. Neurons in the brain convey information
additional insights that cannot be gained from the about:
study of only single neurons. (a) Vision
(b) Hearing
(c) Touch
18.7 Conclusion (d) All the above
2. If neuron A and neuron B are perfectly cor-
Although spikes from single neurons can be con- related, no additional information is gained
sidered the fundamental unit of communication when taking into account neurons A and B as
in the brain, those single neurons do not act in- opposed to only neuron A
dependently. Vast connections exist between neu- (a) True
rons within and between regions. To understand (b) False
interactions between neurons, analyses were first 3. Signal correlation (rsignal ) and noise correla-
developed to account for how a pair of neurons tion (rsc ) differ in that:
might covary their activity, across trials and from (a) rsignal examines evoked activity; rsc exam-
millisecond to millisecond within a trial. With ines spontaneous activity.
the advent of technologies that allowed hundreds (b) There is no difference between rsignal and
of neurons to be recorded simultaneously, re- rsc ; they are only different names for the
searchers turned to statistical methods that bet- same thing.
ter describe population-level signals. With these (c) rsc is measured for pairs of neurons,
methods, researchers can uncover subtle signals whereas rsignal is measured for single
that are distributed across a population of neurons neurons.
that might be hidden from examination of single (d) rsc measures trial-to-trial fluctuations for
neurons individually. Relating these population- a repeated stimulus; rsignal is a measure
level signals to behavior is an extremely active of the similarity of two neuron’s tuning
area of current research in neuroscience. curves.
In this chapter, we aimed to cover a vari- 4. The impact of rsc on stimulus encoding is:
ety of techniques used for extracting information (a) When rsc decreases, stimulus decoding
from neurons, beginning with single neurons, ex- always increases.
tending to pairwise interactions, and finally con- (b) When rsc increases, stimulus decoding
cluding with population-level analyses. We high- always decreases.
lighted research findings at each of the levels and (c) rsc does not impact stimulus decoding.
demonstrated their relation to behavior, a com- (d) None of the above
ponent we believe will be critical in advancing 5. Single neurons in the brain cannot predict
our understanding of the brain. Examining neural behavior
activity at the single-neuron, pairwise, and pop- (a) True
ulation level provides complementary insights: (b) False
6. Attention’s effect on correlation is primarily 10. M. Shamir, H. Sompolinsky, Implications of neu-

to increase its value ronal diversity on population coding. Neural Comput.
18(8), 1951–1986 (2006). https://doi.org/10.1162/
(a) True
neco.2006.18.8.1951
(b) False 11. A. Kohn, R. Coen-Cagli, I. Kanitscheider, A. Pouget,
7. Describe the difference between correlation Correlations and neuronal population information.
and synchrony. Annu. Rev. Neurosci. 39, 237–256 (2016). https://
doi.org/10.1146/annurev-neuro-070815-013851
8. Explain the concept of surrogate data and
12. A. Amarasingham, M.T. Harrison, N.G. Hatsopoulos,
how it is used in correcting synchrony met- S. Geman, Conditional modeling and the jitter method
rics. of spike resampling. J. Neurophysiol. 107(2), 517–
9. What is the difference between PCA and FA 531 (2012). https://doi.org/10.1152/jn.00633.2011
13. M.T. Harrison, S. Geman, A rate and history-
as applied to neural spiking data?
preserving resampling algorithm for neural spike
10. What advantage do GPFA and LFADS have trains. Neural Comput. 21(5), 1244–1258 (2009).
over PCA and FA? https://doi.org/10.1162/neco.2008.03-08-730
14. M.A. Smith, A. Kohn, Spatial and temporal scales of
neuronal correlation in primary visual cortex. J. Neu-
rosci. 28(48), 12591–12603 (2008). https://doi.org/
References 10.1523/JNEUROSCI.2929-08.2008
15. W. Bair, E. Zohary, W.T. Newsome, Correlated fir-
1. G. Grant, How the 1906 Nobel Prize in Physiol- ing in macaque visual area MT: time scales and re-
ogy or Medicine was shared between Golgi and Ca- lationship to behavior. J Neurosci21(5), 1676–1697
jal. Brain Res. Rev. 55(2), 490–498 (2007). https:// (2001). Retrieved from http://www.ncbi.nlm.nih.gov/
doi.org/10.1016/j.brainresrev.2006.11.004 pubmed/11222658
2. S. Ramón y Cajal, The Croonian Lecture. La fine 16. J.H. Maunsell, S. Treue, Feature-based attention in vi-
structure des centres nerveux. Proc. Roy. Soc.Lond. sual cortex. Trends Neurosci. 29(6), 317–322 (2006).
B 55, 444–468 (1894) https://doi.org/10.1016/j.tins.2006.04.001
3. K.H. Britten, M.N. Shadlen, W.T. Newsome, J.A. 17. J.H. Reynolds, L. Chelazzi, Attentional modulation
Movshon, The analysis of visual motion: A compar- of visual processing. Annu. Rev. Neurosci.
ison of neuronal and psychophysical performance. J. 27, 611–647 (2004). https://doi.org/10.1146/
Neurosci. 12(12), 4745–4765 (1992). Retrieved from annurev.neuro.26.041002.131039
https://www.ncbi.nlm.nih.gov/pubmed/1464765 18. S. Yantis, J.T. Serences, Cortical mechanisms of
4. E. Zohary, M.N. Shadlen, W.T. Newsome, Correlated space-based and object-based attentional control.
neuronal discharge rate and its implications for psy- Curr Opin Neurobiol13(2), 187–193 (2003). Re-
chophysical performance. Nature 370(6485), 140– trieved from https://www.ncbi.nlm.nih.gov/pubmed/
143 (1994). https://doi.org/10.1038/370140a0 12744972
5. B.B. Averbeck, P.E. Latham, A. Pouget, Neural cor- 19. M.R. Cohen, J.H. Maunsell, Attention improves per-
relations, population coding and computation. Nat. formance primarily by reducing interneuronal cor-
Rev. Neurosci. 7(5), 358–366 (2006). https://doi.org/ relations. Nat. Neurosci. 12(12), 1594–1600 (2009).
10.1038/nrn1888 https://doi.org/10.1038/nn.2439
6. L.F. Abbott, P. Dayan, The effect of correlated vari- 20. J.F. Mitchell, K.A. Sundberg, J.H. Reynolds, Spatial
ability on the accuracy of a population code. Neural attention decorrelates intrinsic activity fluctuations in
Comput11(1), 91–101 (1999). Retrieved from https:/ macaque area V4. Neuron 63(6), 879–888 (2009).
/www.ncbi.nlm.nih.gov/pubmed/9950724 https://doi.org/10.1016/j.neuron.2009.09.013
7. M.N. Shadlen, W.T. Newsome, The variable dis- 21. M.M. Churchland, B.M. Yu, J.P. Cunningham, L.P.
charge of cortical neurons: Implications for con- Sugrue, M.R. Cohen, G.S. Corrado, et al., Stimu-
nectivity, computation, and information coding. J lus onset quenches neural variability: A widespread
Neurosci18(10), 3870–3896 (1998). Retrieved from cortical phenomenon. Nat. Neurosci. 13(3), 369–378
https://www.ncbi.nlm.nih.gov/pubmed/9570816 (2010). https://doi.org/10.1038/nn.2501
8. H. Sompolinsky, H. Yoon, K. Kang, M. Shamir, Pop- 22. G. Santhanam, B.M. Yu, V. Gilja, S.I. Ryu, A. Afshar,
ulation coding in neuronal systems with correlated M. Sahani, K.V. Shenoy, Factor-analysis methods
noise. Phys. Rev. E Stat. Nonlinear Soft Matter Phys. for higher-performance neural prostheses. J. Neuro-
64(5 Pt 1), 051904 (2001). https://doi.org/10.1103/ physiol. 102(2), 1315–1330 (2009). https://doi.org/
PhysRevE.64.051904 10.1152/jn.00097.2009
9. A.S. Ecker, P. Berens, A.S. Tolias, M. Bethge, The 23. B.M. Yu, J.P. Cunningham, G. Santhanam, S.I.
effect of noise correlations in populations of diversely Ryu, K.V. Shenoy, M. Sahani, Gaussian-process fac-
tuned neurons. J. Neurosci. 31(40), 14272–14283 tor analysis for low-dimensional single-trial anal-
(2011). https://doi.org/10.1523/JNEUROSCI.2539- ysis of neural population activity. J. Neurophys-
11.2011
iol. 102(1), 614–635 (2009). https://doi.org/10.1152/ 36. D.A. Ruff, M.R. Cohen, Attention increases spike
jn.90941.2008 count correlations between visual cortical areas. J.
24. C. Pandarinath, D.J. O’Shea, J. Collins, R. Joze- Neurosci. 36(28), 7523–7534 (2016). https://doi.org/
fowicz, S.D. Stavisky, J.C. Kao, et al., Inferring 10.1523/JNEUROSCI.0610-16.2016
single-trial neural population dynamics using sequen- 37. M.R. Cohen, J.H. Maunsell, A neuronal population
tial auto-encoders. Nat. Methods 15(10), 805–815 measure of attention predicts behavioral performance
(2018). https://doi.org/10.1038/s41592-018-0109-9 on individual trials. J. Neurosci. 30(45),
25. J.W. Pillow, J. Shlens, L. Paninski, A. Sher, A.M. 15241–15253 (2010). https://doi.org/10.1523/
Litke, E.J. Chichilnisky, E.P. Simoncelli, Spatio- JNEUROSCI.2171-10.2010
temporal correlations and visual signalling in a com- 38. J.P. Mayo, J.H. Maunsell, Graded neuronal modula-
plete neuronal population. Nature 454(7207), 995– tions related to visual spatial attention. J. Neurosci.
999 (2008). https://doi.org/10.1038/nature07140 36(19), 5353–5361 (2016). https://doi.org/10.1523/
26. C. Stringer, M. Pachitariu, N. Steinmetz, M. JNEUROSCI.0192-16.2016
Carandini, K.D. Harris, High-dimensional geometry 39. A.C. Snyder, B.M. Yu, M.A. Smith, Distinct popula-
of population responses in visual cortex. Nature tion codes for attention in the absence and presence of
571(7765), 361–365 (2019). https://doi.org/10.1038/ visual stimulation. Nat. Commun. 9(1), 4382 (2018).
s41586-019-1346-5 https://doi.org/10.1038/s41467-018-06754-5
27. N.K. Logothetis, J.D. Schall, Neuronal correlates of 40. S.B. Khanna, A.C. Snyder, M.A. Smith, Distinct
subjective visual perception. Science245(4919), sources of variability affect eye movement prepara-
761–763 (1989). Retrieved from https:// tion. J. Neurosci. 39, 4511 (2019). https://doi.org/
www.ncbi.nlm.nih.gov/pubmed/2772635 10.1523/JNEUROSCI.2329-18.2019
28. K.H. Britten, W.T. Newsome, M.N. Shadlen, S. Cele- 41. M.T. Kaufman, M.M. Churchland, S.I. Ryu, K.V.
brini, J.A. Movshon, A relationship between behav- Shenoy, Cortical activity in the null space: Permitting
ioral choice and the visual responses of neurons in preparation without movement. Nat. Neurosci. 17(3),
macaque MT. Vis Neurosci13(1), 87–100 (1996). Re- 440–448 (2014). https://doi.org/10.1038/nn.3643
trieved from https://www.ncbi.nlm.nih.gov/pubmed/ 42. M.T. Kaufman, M.M. Churchland, S.I. Ryu, K.V.
8730992 Shenoy, Vacillation, indecision and hesitation in
29. H. Nienborg, M.R. Cohen, B.G. Cumming, Decision- moment-by-moment decoding of monkey motor cor-
related activity in sensory neurons: Correlations tex. elife 4, e04677 (2015). https://doi.org/10.7554/
among neurons and with behavior. Annu. Rev. Neu- eLife.04677
rosci. 35, 463–483 (2012). https://doi.org/10.1146/ 43. A. Afshar, G. Santhanam, B.M. Yu, S.I. Ryu,
annurev-neuro-062111-150403 M. Sahani, K.V. Shenoy, Single-trial neural
30. T.B. Crapse, M.A. Basso, Insights into decision correlates of arm movement preparation. Neuron
making using choice probability. J. Neurophysiol. 71(3), 555–564 (2011). https://doi.org/10.1016/
114(6), 3039–3049 (2015). https://doi.org/10.1152/ j.neuron.2011.05.047
jn.00335.2015 44. J.A. Michaels, B. Dann, R.W. Intveld, H. Scherberger,
31. A.G. Bondy, R.M. Haefner, B.G. Cumming, Predicting reaction time from the neural state space of
Feedback determines the structure of correlated the premotor and parietal grasping network. J. Neu-
variability in primary visual cortex. Nat. Neurosci. rosci. 35(32), 11415–11432 (2015). https://doi.org/
21(4), 598–606 (2018). https://doi.org/10.1038/ 10.1523/JNEUROSCI.1714-15.2015
s41593-018-0089-1 45. G.F. Elsayed, A.H. Lara, M.T. Kaufman, M.M.
32. R.D. Lange, R.M. Haefner, Characterizing and inter- Churchland, J.P. Cunningham, Reorganization be-
preting the influence of internal variables on sensory tween preparatory and movement population re-
activity. Curr. Opin. Neurobiol. 46, 84–89 (2017). sponses in motor cortex. Nat. Commun. 7, 13239
https://doi.org/10.1016/j.conb.2017.07.006 (2016). https://doi.org/10.1038/ncomms13239
33. J.L. Herrero, M.A. Gieselmann, M. Sanayei, A. 46. K.C. Ames, S.I. Ryu, K.V. Shenoy, Neural dynam-
Thiele, Attention-induced variance and noise correla- ics of reaching following incorrect or absent motor
tion reduction in macaque V1 is mediated by NMDA preparation. Neuron 81(2), 438–451 (2014). https://
receptors. Neuron 78(4), 729–739 (2013). https:// doi.org/10.1016/j.neuron.2013.11.003
doi.org/10.1016/j.neuron.2013.03.029 47. C.J. Bruce, M.E. Goldberg, M.C. Bushnell, G.B.
34. A.C. Snyder, M.J. Morais, M.A. Smith, Dynamics Stanton, Primate frontal eye fields. II. Physiological
of excitatory and inhibitory networks are differen- and anatomical correlates of electrically evoked
tially altered by selective attention. J. Neurophysiol. eye movements. J Neurophysiol54(3), 714–734
116(4), 1807–1820 (2016). https://doi.org/10.1152/ (1985). Retrieved from http://www.ncbi.nlm.nih.gov/
jn.00343.2016 pubmed/4045546
35. D.A. Ruff, M.R. Cohen, Attention can either increase 48. D.A. Robinson, Eye movements evoked by col-
or decrease spike count correlations in visual cortex. licular stimulation in the alert monkey. Vision
Nat. Neurosci. 17(11), 1591–1597 (2014). https:// Res12(11), 1795–1808 (1972). Retrieved from https:/
doi.org/10.1038/nn.3835 /www.ncbi.nlm.nih.gov/pubmed/4627952
49. P. Thier, R.A. Andersen, Electrical microstimu- 58. J.A. Hennig, M.D. Golub, P.J. Lund, P.T. Sadtler,
lation distinguishes distinct saccade-related areas E.R. Oby, K.M. Quick, et al., Constraints on neural
in the posterior parietal cortex. J. Neurophysiol. redundancy. Elife 7 (2018). https://doi.org/10.7554/
80(4), 1713–1735 (1998). https://doi.org/10.1152/ eLife.36774
jn.1998.80.4.1713 59. E.R. Oby, M.D. Golub, J.A. Hennig, A.D. Degenhart,
50. M.S. Graziano, C.S. Taylor, T. Moore, Complex E.C. Tyler-Kabara, B.M. Yu, et al., New neural ac-
movements evoked by microstimulation of precentral tivity patterns emerge with long-term learning. Proc.
cortex. Neuron34(5), 841–851 (2002). Retrieved from Natl. Acad. Sci. U. S. A. 116(30), 15210–15215
https://www.ncbi.nlm.nih.gov/pubmed/12062029 (2019). https://doi.org/10.1073/pnas.1820296116
51. S.N. Flesher, J.L. Collinger, S.T. Foldes, J.M. Weiss, 60. C. Constantinidis, S. Funahashi, D. Lee, J.D. Murray,
J.E. Downey, E.C. Tyler-Kabara, et al., Intracortical X.L. Qi, M. Wang, A.F.T. Arnsten, Persistent spik-
microstimulation of human somatosensory cortex. Sci ing activity underlies working memory. J. Neurosci.
Transl Med8(361), 361ra141 (2016). https://doi.org/ 38(32), 7020–7028 (2018). https://doi.org/10.1523/
10.1126/scitranslmed.aaf8083 JNEUROSCI.2486-17.2018
52. C.D. Salzman, C.M. Murasugi, K.H. Britten, W.T. 61. M. Lundqvist, P. Herman, E.K. Miller, Working mem-
Newsome, Microstimulation in visual area MT: Ef- ory: Delay activity, yes! Persistent activity? Maybe
fects on direction discrimination performance. J Neu- not. J. Neurosci. 38(32), 7013–7019 (2018). https://
rosci12(6), 2331–2355 (1992). Retrieved from https:/ doi.org/10.1523/JNEUROSCI.2485-17.2018
/www.ncbi.nlm.nih.gov/pubmed/1607944 62. K.K. Sreenivasan, M. D’Esposito, The what, where
53. T. Moore, M. Fallah, Control of eye movements and how of delay activity. Nat. Rev. Neurosci. 20, 466
and spatial attention. Proc. Natl. Acad. Sci. U. S. (2019). https://doi.org/10.1038/s41583-019-0176-7
A. 98(3), 1273–1276 (2001). https://doi.org/10.1073/ 63. E.M. Meyers, Dynamic population coding and its re-
pnas.021549498 lationship to working memory. J. Neurophysiol. 120,
54. T. Moore, M. Fallah, Microstimulation of the frontal 2260 (2018). https://doi.org/10.1152/jn.00225.2018
eye field and its effects on covert spatial attention. J. 64. J.D. Murray, A. Bernacchia, N.A. Roy, C. Constan-
Neurophysiol. 91(1), 152–162 (2004). https://doi.org/ tinidis, R. Romo, X.J. Wang, Stable population cod-
10.1152/jn.00741.2002 ing for working memory coexists with heterogeneous
55. M.M. Churchland, K.V. Shenoy, Delay of movement neural dynamics in prefrontal cortex. Proc. Natl.
caused by disruption of cortical preparatory activ- Acad. Sci. U. S. A. 114(2), 394–399 (2017). https:/
ity. J. Neurophysiol. 97(1), 348–359 (2007). https:// /doi.org/10.1073/pnas.1619449114
doi.org/10.1152/jn.00808.2006 65. E.M. Meyers, A. Liang, F. Katsuki, C. Constantinidis,
56. K.M. Armstrong, J.K. Fitzgerald, T. Moore, Changes Differential processing of isolated object and multi-
in visual receptive fields with microstimulation of item pop-out displays in LIP and PFC. Cereb. Cortex
frontal cortex. Neuron 50(5), 791–798 (2006). https:/ 28(11), 3816–3828 (2018). https://doi.org/10.1093/
/doi.org/10.1016/j.neuron.2006.05.010 cercor/bhx243
57. P.T. Sadtler, K.M. Quick, M.D. Golub, S.M. Chase, 66. A. Parthasarathy, R. Herikstad, J.H. Bong, F.S. Med-
S.I. Ryu, E.C. Tyler-Kabara, et al., Neural constraints ina, C. Libedinsky, S.C. Yen, Mixed selectivity
on learning. Nature 512(7515), 423–426 (2014). morphs population codes in prefrontal cortex. Nat.
https://doi.org/10.1038/nature13665 Neurosci. 20(12), 1770–1779 (2017). https://doi.org/
10.1038/s41593-017-0003-2
Machine Intelligence-Based Epileptic
Seizure Forecasting 19
Vasily Grigorovsky, Uilki Tufa, Daniel Jacobs,
and Berj L. Bardakjian
Abstract man eye and perform identification of seizure

states, and medicine regiments in an auto-
Epilepsy is one of the most common neuro-
mated objective manner. In this chapter, we
logical disorders globally, and the decrease in
will discuss such machine learning algorithms.
quality of life associated with it includes –
We will explore the most widely used algo-
among other things – fear and uncertainty over
rithms and their variations – both in the context
when the next seizure would manifest itself.
of seizure prediction and detection (arguably
The most common way to treat epilepsy is
the most widely used application of machine
by using antiepileptic drugs; however, around
intelligence in epilepsy), as well as in other
30% of all patients develop refractory epilepsy,
applications, such as antiepileptic drug effi-
where medication fails to control seizures, and
cacy. We will also talk about common tech-
patients have to resort to surgical resection of
niques of feature extraction – particularly fo-
epileptogenic zones. While manual techniques
cusing on wavelet phase coherence and cross-
exist to detect epileptic seizures, and come up
frequency coupling. While much of work has
with the appropriate regiment of antiepileptic
been done to improve current machine learn-
drugs, they are generally limited by the skill
ing algorithms in the context of epilepsy, chal-
of the human operator and can be applied
lenges still remain to be solved, and potential
only to a particular application. Arguably, a
future directions for machine intelligence ap-
better approach is to use machine intelligence
plications in epilepsy are discussed at the end
to identify patterns in data unseen to the hu-
of the chapter.
Electronic Supplementary Material: The online version Keywords

6_19) contains supplementary material, which is available Epilepsy · Seizure prediction · Seizure
to authorized users. detection · Machine learning ·
V. Grigorovsky · U. Tufa · D. Jacobs
Cross-frequency coupling · Machine
Institute of Biomaterials and Biomedical Engineering, intelligence · State classification
B. L. Bardakjian ()
Institute of Biomaterials and Biomedical Engineering,
19.1 Introduction
Edward S. Rogers Sr. Department of Electrical &
Epilepsy is a dynamical disease, and its effects
Computer Engineering, University of Toronto, Toronto,
ON, Canada are evident in up to 1% of the population, or over
e-mail: [email protected] 60 million people worldwide. It is characterized

536 V. Grigorovsky et al.
by transient interruptions of brain function caused

19.2 Feature Extraction
by abnormal temporal and spatial coherent firing
of a neuronal population, often referred to as
All machine learning techniques rely on input
a seizure, paroxysmal discharge, or ictal event
data to find underlying patterns and develop data-
[1]. Beyond a number of comorbidities associated
based models. This input data consists of measur-
with epilepsy, patients with epilepsy are usually
able quantities designated as features, and choos-
unable to predict when they will have a seizure
ing appropriate features is one of the main chal-
and thus are often unable to drive, have difficulty
lenges in machine intelligence. Manual feature
engaging in the workforce, are at increased risk
selection and tuning is a task that can be time-
of head injury due to seizure-related fall, and
consuming and often requires expertise in the
typically carry a stigma associated with having
application. Feature engineering is the process of
epilepsy. All of these factors contribute to a re-
finding these features from our knowledge of the
duced quality of life in patients with epilepsy and
origin of scalp EEG recordings and deciphering
are largely attributed to the debilitating as well as
the physiological and pathological basis of their
the unpredictable nature of seizures. Furthermore,
oscillations.
patients with refractory epilepsy are also at an ele-
vated risk of sudden unexpected death in epilepsy
(SUDEP), which might be preventable if one
19.2.1 Rhythms of the Brain
could anticipate a seizure occurrence [2]. Hence,
there exists a need for monitoring systems that
Scalp EEG is a noninvasive recording method
detect preclinical seizure states in the EEG to alert
that has been widely used by neurologists to
patients and caregivers to oncoming seizures.
identify epileptiform activity in patients. Human
The pathophysiology of seizures is an
scalp EEG recordings are measures of electrical
enhanced cortical excitability, leading to
fields with contributions from all transmembrane
paroxysmal depolarization shifts, an enhanced
currents in the brain. EEG reflects the summa-
probability of hypersynchronous activity of small
tion and superposition of similarly oriented, syn-
neuronal networks, and an abnormal spreading of
chronous neuronal and glial electrical activity
this pathological activity along cortico-cortical
favoring superficial sources rather than subcorti-
and cortico-subcortical neuronal connections [3,
cal deeper structures [7]. The nature of volume
4]. Thus, the common feature of antiepileptic
integration in the brain leads to spatial averaging
therapies is the reduction of any pathological
in EEG as compared to local field potential (LFP)
hyperactivity by either enhancing neuronal
recordings which can pick out local activity [7].
inhibition or reducing excitation. Current
Nonetheless, EEG signals show brain rhythms re-
methods for seizure treatment include either the
lating to neuronal network effects and oscillations
use of antiepileptic drugs (AEDs) or surgical
with high temporal resolution, and temporal and
removal of epileptic tissues. While usually the
spectral analysis of these signals forms a large
first treatment option to be used, AEDs require
and important set of features for machine learning
a regiment tailor-made for a given patient and
techniques.
have a wide range of side effects associated with
As information in the brain is transmitted
them [5] – thus being able to predict whether a
using neural coding, spectral information or
given regiment of AEDs will be successful will
rhythms at different frequency ranges recorded
improve epilepsy therapy strategies.
in EEG have been the target for analysis in
In this chapter, we will describe EEG-based
perceptual binding and transient short- and long-
machine learning approaches for classification
range coordination. The rhythms of the brain
and detection of preclinical seizure states in
were noticed by Penttonen and Buzsaki to show
epileptic patients, as well as look at some other
frequency ranges at an arithmetic progression
applications of machine intelligence in context of
on the natural logarithmic scale (Fig. 19.1) [6].
epilepsy.
19 Machine Intelligence-Based Epileptic Seizure Forecasting 537
Fig. 19.1 Brain rhythm

frequency range following
a logarithmic scale. (Figure
adapted from Penttonen
and Buzsáki [6])
Lower-frequency oscillations allow for longer ten there is an overlap between physiological
delays and communication between larger areas. and pathological activity in the range of high-
Higher-frequency oscillations facilitate acute frequency oscillations. Brazdil et al. showed a
and spatially limited communication. These higher specificity in locating the zone using fre-
oscillations are concurrent with one another quency ranges from 600 Hz up to 2000 Hz (see
suggesting that the brain works at different time Fig. 19.2) [11]. These very-high-frequency os-
scales [8]. cillations (VHFO) were shown to be present in
While low-frequency oscillations (LFOs) are patients with focal epilepsy [12]. Patients whose
important – e.g., the shape and synchronicity resected brain regions more closely corresponded
of beta (13–30 Hz) waveforms was shown to to EEG channels containing VHFOs showed sig-
improve detection of Parkinson’s disease patho- nificantly better surgery outcomes indicating that
physiology in noninvasive recordings [9] – in this may be a superior biomarker.
the past decade, higher frequencies have gained To examine the power of different spectral
prominence. High-frequency oscillations (HFOs) bands, including the VHFOs, the Fourier trans-
are defined as frequencies in EEG ranging from form has enabled us to transform EEG recorded
100 to 500 Hz. More specifically, HFOs rang- signals from the time domain to the frequency
ing from 100 to 150 Hz are described as ripple domain. The Fourier transform is given as
and 250 to 500 Hz as fast ripple [10]. HFOs ∞
have been identified occurring during interictal I {f (t)} = fˆ (ξ ) = f (t) e−2πj tξ dt,
epileptiform discharges (IEDs) with fast ripples −∞
more restricted to seizure-onset zone. Jacobs et (19.1)
al. showed analysis of HFO rate independent of
IEDs for identifying seizure-onset zone [10]. Fast where ξ is the frequency in hertz. Applying it to
ripples during IEDs and in absence showed higher discrete data and using a finite window, the short-
sensitivity in finding the seizure-onset zone while term Fourier transform takes the form
keeping a specificity value of 95%.
The challenge of using HFOs in EEG for high-
N

F (ξ, k) = fk w n − k e−j ξ k, (19.2)
lighting the seizure-onset zone (SOZ) is that of- n =1
Fig. 19.2 Localization of epileptogenic zone using of frequency bands (R ripple, FR fast ripple, VFR very
VHFO activity on iEEG data. Each vertical bar shows the fast ripple, UFR ultrafast ripple). Stars identify selected
length of duration exceeding threshold in different ranges regions. (Figure taken from Bradzil et al. 2017)
where fk is the value of the signal at tk = kδt and imaginary wavelet coefficients can be used

and w(n − k) is a window function. The wavelet to extract phase information of specific frequency
transform follows this transform using a wavelet bands in EEG signals.
basis instead of a sinusoidal basis function.
Wavelets are a family of functions used as a basis
for wavelet transforms which have the property 19.2.2 Wavelet Phase Coherence
of integrating to zero and are expressed as
4 Wavelet phase coherence (WPC) is a measure of
5

N
n − n δt phase coherence that uses complex wavelet trans-
W (s, n) = f n ψ ∗ , (19.3) form to extract the phase information of different
n =1
s
frequency bands in EEG data. WPC describes
how the phases of two EEG signals change with
where ψ(s, n) is the wavelet function used with
respect to one another within a time window.
scaling factors s and n. We can convert the scaling
Unlike other coherence measurement, WPC is not
factor s into frequencies by scaling the 6 central related to the power of the frequency bands. The
frequency of the mother wavelet by 1 s . Con-
relative phase difference φ is extracted from
tinuous wavelet transform (CWT) is preferred wavelet coefficients of two signals W1 (s, τ ) and
over short-time Fourier transforms (STFT) for W2 (s, τ ), with s as the wavelet scaling coefficient
two distinct reasons. The chosen mother wavelet and τ as the time shift, as follows:
of the CWT can better extract the preferred fre-
quencies of EEG signal which do not typically Δφ (s, τ ) = arctan
follow sinusoidal functions, and the CWT has bet- ∗
ter temporal resolution increasing with frequency. W1 (s, τ ) W2 (s, τ ) − W1 (s, τ ) W2∗ (s, τ )
Complex wavelet transforms are a type of CWT W1 (s, τ ) W2 (s, τ ) − W1∗ (s, τ ) W2∗ (s, τ )
which uses complex mother wavelets. The real (19.4)
where W∗ indicates the complex conjugate. The gamma code and its role in spatial memory
relative phase coherence is then measured as the [15]. Distinct neural ensembles observed to
7 8 fire in the gamma range were encoded within
ρ (s, τ ) = ej Δφ(s,τ ) (19.5) specific phases of theta cycles cued by positional
information and long-term memory. One of the
and ranges from zero to one, with a value of most common measures of PAC was developed
one indicating complete coherence or a constant by Tort et al. [16]. A variation of the algorithm
phase difference within a time window. uses complex wavelet transforms to extract phase
Wavelet phase coherence (WPC) of high- and amplitude information in contrast to using
frequency oscillations was shown by Cotic et band pass filtering with Hilbert transforms [17].
al. to be a useful feature in the localization of The amplitude of the high-frequency rhythm is
the epileptogenic zone [13]. Although the power computed using (Fig. 19.3)
of HFOs increased during seizures and could
9 : 9 :
roughly locate the epileptogenic zone, WPC was A tˆ, fH = Re W tˆ, fH + j Im W tˆ, fH .
better able to identify electrodes within this zone (19.6)
as confirmed using ROC curve analysis.
The phase of the low frequency can easily be
computed from the analytic wavelet transform
19.2.3 Cross-Frequency Coupling representation.
9 :
We have thus far introduced brain rhythms Im W tˆ, fL
ˆ
φ t , fL = arctan 9 : . (19.7)
and how different regions can show phase Re W tˆ, fL
coherence within specific frequency ranges.
Cross-frequency coupling (CFC) pertains to The mean amplitude is normalized in order to
the communication or brain code observed as a have an amplitude-independent measure of CFC
function of two or more interacting frequencies.
7 8
Phase-amplitude CFC (PAC) has been observed A tˆ, fH j
in humans under a variety of conditions [14]. pj tˆ, fH , fL = N 7 8 . (19.8)
ˆ
k=1 A t , fH k
PAC refers to the relationship where the phase
of a low-frequency oscillation modulates the
The cross-frequency coupling index is then
amplitude of a high-frequency rhythm. The
computed as a measure of entropy normalized to
most popular example of PAC is the theta-
a uniform distribution.
Fig. 19.3 Example wavelet phase coherence between electrodes 1 and 5 during a seizure. (Adapted from Cotic et al.
[13])

N
Table 19.1 Selection of algorithm performance metrics
H tˆ, fH , fL = − pj tˆ, fH , fL log Sensitivity TP
Accuracy T P +T N
T P +F N T P +T N +F P +F N
j =1 Specificity TN
F1 score 2T P
T N +F P 2T P +F P +F N
FP TP
pj tˆ, fH , fL False- F P +T N Precision T P +F P
positive
(19.9) rate

log N − H tˆ, fH , fL expect a uniform distribution when binning the
ICF C ˆt , fH , fL = .
log N amplitude of high-frequency rhythms to phases
(19.10) of low frequencies, there may be an inherent CFC
based on the noise of the data.
PAC has been used as a biomarker of both Once the machine learning model is created,
physiological and pathological conditions. Guir- its performance needs to be evaluated. In case of
gis et al. [18] showed PAC captured seizure dy- a two-state classification (e.g., seizure vs. non-
namics and identified regions of interest for sur- seizure), a number of metrics can be used; how-
gical resection in seven patients (Fig. 19.4). Mod- ever, first we need to introduce the basic termi-
ulation of high-frequency oscillations by delta nology:
activity showed higher specificity in selecting True positive (TP) – The algorithm has classified
the seizure-onset zone (SOZ) as compared with and identified the state.
regions determined by neurologists as well as False positive (FP) – The algorithm has incor-
considering the Engel class of the patient (i.e., rectly identified the state (Type I error).
how seizure-free is the patient after the surgery; True negative (TN) – The algorithm has correctly
EC I–IV denote a progressively worse surgical rejected the state.
resection outcome). Conversely, in Amiri et al. False negative (FN) – The algorithm has incor-
[19], theta modulation of high-frequency oscil- rectly rejected the state (Type II error).
lations was shown to best identify seizure-onset
patterns. From these definitions, several metrics can be
established (see Table 19.1).
Sensitivity and specificity are commonly used
in evaluating algorithms’ performance in general
19.2.4 Model Performance [21]; however, other metrics – especially false-
positive rate and accuracy – are also widely used.
Before a given machine learning algorithm can be While it is not necessary to show all of these met-
trained on a set of features, those features need rics (some of them can be derived from others),
to be tested for reliability. Surrogate analysis is a each measure offers different information on how
common way to assess this reliability of nonlinear the algorithm performs. For example, high sen-
measures and how they differ from noise and sitivity indicates a high probability of correctly
inherent trends in the data. A common way to identifying the diseased state, and high speci-
create surrogate data, described by Theiler et al. ficity indicates a high probability of correctly
[20], is to shuffle phase while having an am- rejecting the diseased state. In classification tasks
plitude adjusted Fourier transform. This method with more than two classes (e.g., interictal, pre-
preserves spectral information while removing ictal, ictal EEG state classification), these mea-
the original temporal information. In the case sures can be used for each individual class in
of cross-frequency coupling, surrogate analysis a one-vs.-all approach – for example, interictal
consists of shuffling the phase information and vs. non-interictal, preictal vs. non-preictal, and
recomputing the CFC index. Although we might so forth.
Fig. 19.4 Localization of epileptogenic zone using eigenvalue decomposition (EVD) to localize epileptogenic
phase-amplitude cross-frequency coupling of iEEG data. zones in patients who underwent surgical resection with
Delta-HFO modulation index (MI) used along with varying outcomes. (Figure adapted from Guirgis et al.
[18])
(A) 1 (B) 1
TPR(th)
TPR(th)
Model A
Model B
Model C
Chance
0 FPR(th) 1 0 FPR(th) 1
Fig. 19.5 (a) Example of receiver operating curve of three different models. (b) Area under the curve shows model A
as the best performing classification model. (Figure used with author’s permission)
These measures can be better visualized as

a receiver operating characteristic (ROC), which 19.3 Seizure Detection
is a plot of sensitivity against false-positive rate and Forecasting
while ranging over values of a parameter of an
algorithm with binary classification such as a The ability to reliably detect, classify, and
threshold [22]. ROCs explore the trade-off be- forecast seizures in epileptic patients can
tween high sensitivity and high specificity. We have a profound impact on state-of-the-art
can compare different classification models using therapies for epilepsy and patients’ quality of
the area under the curve (AUC) and find the best life. Successful classification of EEG signals
parameter to maximize sensitivity and specificity, into a number of states – such as interictal,
giving bias to meet the requirements of classifica- preictal, or potentially several seizure states –
tion problem (see Fig. 19.5). can identify different epilepsy etiologies, predict
potential complications, and aid in classifying
the severity of seizures. Being able to detect the then at both tree-based methods and deep neural
seizure early (as opposed to after the fact) or networks.
even forecast the event before it happens can
provide an alert or therapeutic intervention for
epileptic patients. People with chronic epilepsy 19.3.1 Linear Methods
report decreased quality of life and common
fear of future seizures due to uncertainty [23], The underlying feature of all linear methods is
which an early warning system could reduce or that, as the name implies, at the core they create
eliminate. However, the question whether reliable a boundary to distinguish between two or more
seizure detection and, forecasting are possible has classes (in case of classification tasks) based on
long been left unanswered. While the difficulty some linear combination of input features. For
varies significantly based on the task (detection, example, a logistic regression model applies an
classification, or forecasting), the quality of data, activation function to an otherwise linear summa-
and the overall goal, it was only in the last decade tion of inputs:
that computer algorithms became sophisticated
enough to be able to forecast epileptic seizures z = b + w 1 x 1 + w2 x 2 + · · · = b + x T w
with above chance accuracy (compare Mormann (19.11)
et al. [24] and Kuhlmann et al. [25]).
1
The algorithms that enabled this breakthrough y = , (19.12)
belong to the area of machine intelligence, espe- 1 + exp (−z)
cially deep learning, that train on large amounts
where x is a vector of inputs, w is a weight
of data to extract underlying features and patterns
vector, and b is a bias term. In one case, logistic
which might not be noticeable to the human eye.
regression was used for seizure prediction with
Generally machine learning algorithms can be
EEG data from 9 patients with an average of
split into supervised and unsupervised learning;
320 days of recording and 116 seizures each
in this section we will mostly focus on the former
[26]. The signal energy features from four fre-
category, while still presenting some examples of
quency bands (8–16 Hz, 16–32 Hz, 32–64 Hz,
clustering algorithms used for EEG signal classi-
and 64–128 Hz) were used, and the algorithm
fication. In supervised learning, the algorithm is
showed the average sensitivity for seizure predic-
presented with a training set of inputs and corre-
tion of 0.55 and an average AUC of 0.79. The
sponding outputs, based on which it attempts to
authors have suggested to augment the logistic re-
infer an underlying input-output map – with its
gression by integrating patient-specific circadian
performance evaluated on the never-before-seen
information, which increases average sensitivity
test set of data. Supervised learning could be fur-
to 0.61.
ther broken down into two areas – classification
On its own, logistic regression is only suited
tasks with categorical outputs, such as seizure
for binary classification – e.g., whether an EEG
detection, and regression tasks with numerical an-
signal is a seizure or not. However, it can be
swers, for example, predicting the duration of the
generalized to a multiclass classification using
seizure. The former dominates epilepsy research,
softmax regression, where a softmax function is
as it is important to determine the current and
used to calculate probability of every class occur-
the next state the patient is in; so in this sec-
ring given the input (and the class with the largest
tion, we will exclusively focus on classification
probability is selected):
algorithms. Another way supervised learning can
be divided is into linear models (e.g., logistic exp (zi )
regression and support vector machines) and non- σ (z) = K . (19.13)
linear models (e.g., decision trees and deep neural j =1 exp zj
networks). We will first look at linear models

and how they are used in epilepsy research and

One group utilized softmax regression in a so- k x i , x = x Ti x , (19.15)
called mixture-of-experts model to classify EEG
signals into normal or epileptic using the Univer- where y’ is the predicted class for the input x’ and
sity of Bonn dataset (Bonn dataset), which con- k() is the so-called kernel. Kernels are a transfor-
sists of scalp EEG data obtained from five healthy mation of the (potentially nonlinear) feature space
volunteers and five individuals suffering from associated with a classification problem. A linear
epilepsy [27]. A mixture-of-experts model con- SVM is very similar to the logistic regression,
sists of a population of simple linear classifiers but has a few advantages over it, since SVM
(such as logistic regression) and a gating network (a) ensures that the found solution is as fair as
(which contains a softmax function). The gating possible and (b) less sensitive to outliers com-
network mixes outputs from linear classifiers, and pared to logistic regression. In one case, Bonn
during training, it eventually learns to partition dataset was used to construct features such as
inputs such that each classifier is an “expert” in dominant frequency, mean of power spectrum,
one subset of features. The model used features and coefficient of variation [29]. These features
such as mean, standard deviation, and average were fed into a linear SVM to classify the given
power of wavelet coefficients from six distinct EEG signal as either normal or epileptic. The
frequency bands covering the entire range up to authors found that while each individual feature
86.8 Hz and showed an improvement over a basic had about a 50% accuracy, combining the features
multilayer perceptron neural network (which we led to a 98% accuracy.
will cover in more detail in a later section) with Furthermore, SVMs can be used to extend
an increased accuracy (94.5%), specificity (94%), linear modelling to a nonlinear domain, using
and sensitivity (95%). kernels such as:
Support vector machines (SVMs) are another
d
family of linear models, where the objective polynomial : k x i , x = x Ti x + 1
is to find the optimal hyperplane separating (19.16)
two classes by maximizing the space between
the closest points (or support vectors) of these radial basis f unction :
classes (see Fig. 19.6). A linear SVM is very $ $2 (19.17)
similar to the logistic regression and can be k x i , x = exp −γ $x i − x $ .
adapted from Eqs. (19.11) and (19.12) to look
like this: This allows the capture of some of the non-
linear dynamics of the brain. One group used a

N
patient-specific radial basis function (RBF) SVM
y = wi y i k x i , x + b (19.14) on intracranial EEG data of 19 out of 21 patients
i with epilepsy from Epilepsy Center of Univer-
Fig. 19.6 Hyperplane and support vectors (−1 and 1) in a two-class SVM with linear, polynomial, and Gaussian (RBF)
kernels. (Figure adapted from Ben-Hur et al. [28])
sity of Freiburg dataset (Freiburg dataset), us- prepared training set), while another study
ing features based on correlation patterns and showed that when using permutation entropy
space/time delays to forecast seizures [30]. SVM (a complexity measure based on neighboring
outputs were also averaged over time to reduce values in the time series), whether nonlinear SVM
noise, and the resultant algorithm, depending on outperformed linear one or vice versa depended
alarm threshold values, had a sensitivity of 0.86– on the state associated with the EEG [34].
0.95 and false prediction rate (FPR) of 0.03/h to While SVMs show adequate results for EEG
0.07/h. Additionally, the algorithm spent between classification, work is being done to further im-
3% and 9% of time in the seizure warning state. prove their performance. In the study by Park et
As feature selection is an important element of al. [35], so-called cost-sensitive SVMs (CSVMs)
designing seizure detection and forecasting algo- are proposed, which penalize misclassification of
rithms, another study used RBF SVM to test two preictal data higher than interictal data in an effort
ways of identifying the most important features to address the imbalance of preictal and interictal
for predicting seizures [31]. The authors used a samples in the training set. The authors used the
combined dataset of scalp EEG (sEEG) of 16 algorithm on the Freiburg dataset and found that
patients and intracranial EEG (iEEG) of another it achieved a sensitivity of 97.5% and a FPR of
8 patients to extract absolute and relative spectral 0.27/h for seizure prediction. Another strategy to
power from several frequency bands – delta (0.5– improve SVM performance was to utilize a group
4 Hz), theta (4–8 Hz), alpha (8–15 Hz), beta (15– of different classifiers (an ensemble) each trained
30 Hz), and gamma (30–128 Hz). They compared with a different set of weights – using Bonn
a method of maximum difference of amplitude dataset and extracted Teager energy among other
distribution histogram (MDAD) between preic- features. The algorithm achieved an accuracy of
tal and non-preictal feature samples with mini- 98.72% for seizure detection [36].
mum redundancy maximum relevance (mRMR) As with many machine learning algorithms,
method and found that the former outperformed one concern with SVMs is that the algorithm will
in seizure prediction with average sensitivity of overfit the training set, meaning that it will model
75.8% and FPR of 0.1/h, while mRMR showed not just the underlying pattern of the data but also
sensitivity of 64.4% but marginally lower FPR. the noise specific to the training set – reducing
Several studies have compared the perfor- its performance on the test set. In order to reduce
mance of different commonly used kernels for the chance of overfitting, regularization is used
SVM in the context of seizure detection and where the weights or coefficients of the algorithm
forecasting using EEG data. In a work by Zhang are kept small, which discourages learning a more
and Parhi [32], polynomial and RBF SVM complex model. While there are many ways to
classifiers were compared using iEEG from accomplish that, in one study Kalman filters were
two patients and spectral power-based features used to regularize SVM classifier on coefficients
calculated from 10 frequency bands covering the of autoregressive models (AR) of EEG signals to
range from 3 Hz to 200 Hz. While RBF SVM predict seizures, which achieved FPR of as low as
classifier showed slightly better performance for 0.02/h [37].
predicting a seizure (AUC of 0.9985 compared
to 0.9795 of the polynomial SVM), the second
degree polynomial SVM classifier used fewer 19.3.2 Tree-Based Methods
number of features, potentially increasing the
computational efficiency of classification. Other Another family of machine learning algorithms
studies have compared the performance of are tree-based methods. A decision tree is a
linear and nonlinear SVM classifiers. In Shiao flowchart-like structure, consisting of branches
et al. [33], the authors found that both linear and nodes, traversing which allows the algorithm
and nonlinear SVMs can perform with similar to make a conclusion about the class of a new data
sensitivity and FPR (attributing it to a carefully point based on a recursive analysis of features
Fig. 19.7 Example of a decision tree for classification. that a point belongs to either class in each node of the tree.
(a) Sample training set with two classes. (b) Splitting of (Figure used with author’s permission)
the original space into decision tree nodes. (c) Probability
associated with the data (see Fig. 19.7). In a results were confirmed by a long-term seizure
decision tree, the nodes represent a feature, the advisory system, which was implanted into 15
branches connecting the nodes are a decision, patients with drug-resistant epilepsy for up to
and a terminal node (or a leaf) is the probable 24 months [40]. In that study, features from a
outcome. Thus, each path from the top of the range of frequencies from 8 to 128 Hz, such as
tree to the leaf is a classification rule which average energy, Teager energy, and line length,
the algorithm applies to the input vector. The were used in a combination of decision tree and
trees are constructed by a recursive algorithm of k-nearest neighbors (an algorithm where a class
binary splitting, which divides the training set of a given data point is determined by plurality
data into two along a feature based on some cost vote of k of its neighbors) classifiers. The final
function, with the goal of minimizing the cost. algorithm showed a patient-specific sensitivity of
By splitting the data along each feature to come 54–100% with time spent in “high” alert state
up with least-cost classification rules, decision of between 3% and 41%. While the algorithm
tree-based algorithms are able to successfully showed a large variability in performance de-
capture nonlinear dynamics of the EEG signals pending on the patient, it was one of the first
and have been used for seizure detection and results from a long-term real-life patient trial
forecasting. where the authors found little to no significant
In one study, empirical mode decomposition reduction in clinical effectiveness after 4 months
(EMD) has been used to separate scalp EEG sig- of implantation. Several studies have also com-
nals from the Bonn dataset into mono-rhythmic pared decision trees to other machine learning
intrinsic mode functions (IMFs), and correspond- algorithms. In one, decision trees were compared
ing features such as spectral peaks, entropy, and with SVM classifiers with various kernels (linear,
energy of these IMFs were fed into a decision tree polynomial, RBF) and probabilistic neural net-
algorithm for seizure detection [39]. The algo- works (which will be briefly covered in the next
rithm was able to achieve the accuracy of 95.33%, section) for seizure detection task using features
sensitivity of 98%, and specificity of 97%. These derived from intrinsic time-scale decomposition
Fig. 19.8 Sample strategy for multistate classifier based on random forest. (Figure adapted from Jacobs et al. [38])
(ITD) – an adaptive data-driven method similar low error rate of the random forest. Random
to EMD to decompose a complex signal [41]. forest algorithms have been used extensively for
The authors found that decision trees performed seizure forecasting (a sample strategy for random
slightly better than the rest with accuracy of 96%, forest use shown in Fig. 19.8). In the work by
sensitivity of 99%, and specificity of 99.5%. This Tzimourta et al. [45], energy coefficients, en-
finding was confirmed by another study using tropy, and other frequency-based features were
different set of frequency-related features and extracted from Bonn and Freiburg datasets and
an extended algorithm comparison, which found used with a random forest classifier achieving
decision trees to have average sensitivity of 99% accuracy of 95% with FPR of 0.21/h. In Donos
and specificity of 94% [42]. et al. [46], 11 time- and frequency-domain fea-
An iteration on the decision tree algorithm is tures have been extracted from intracranial EEG
a logistic model tree, where each of the leaves of 8 patients and fed into a random forest clas-
(terminal nodes) of the tree consists of a logistic sifier, which showed 1.75 s median delays of
regression. Logistic model trees have been re- seizure prediction and 0.07/h FPR. As the authors
ported to be accurate classifiers, combining high suggested, “For closed loop stimulation devices,
performance with ease of interpretability [43]. an early detection is necessary if termination of
In one study, they have been used on the Bonn epileptic activity prior to first ictal manifestations
dataset for seizure detection and outperformed is aimed at,” which the short median delay of
both logistic regression and SVM, with an over- seizure-onset prediction enables. However, with
all AUC of 0.988 (compared to 0.932 and 0.52, correct input feature selection, the time of ad-
respectively) [44]. vance seizure forecasting can be extended. In Ja-
In an effort to improve the performance of cobs et al. [38], a global index of cross-frequency
decision trees, an ensemble technique of random coupling computed from scalp EEG was used
forest has been developed. As with real forests, as an input to a multistage state classifier based
a random forest algorithm consists of a number on random forest, and the algorithm achieved a
of trees (in this case, decision tree algorithms). 45 ± 16 second advance alarm with AUC of
In a random forest, each decision tree has access 0.934. Robustness of a classifier to input fea-
only to a random subset of features while making tures is also an important consideration, and in
the decision to split the node and a random subset the same study, the authors found that the per-
of training data points. The random forest, then, formance of random forest did not significantly
takes a majority vote (for the classification task) change with reduced electrode ring configuration.
of all individual tree decisions as the final class. Random forest classifier has also been used for
The large number of classifiers with, ideally, low seizure detection. As an example, in Zhang et al.
correlation between any two trees results in the [47], a combination of variational mode decom-
position (VMD; an extension of EMD technique, While on its own a perceptron is a linear classi-
with an advantage of decomposing a multicom- fier, a system with multiple perceptrons arranged
ponent signal into a number of band-limited in- in several layers becomes nonlinear. A typical
trinsic mode functions non-recursively and syn- multilayer perceptron network (MLP, also called
chronously) and AR was used on the Bonn dataset a feedforward network) has at least three layers –
for feature extraction. These features were then an input layer of features, a hidden layer, and
fed into a three-state random forest classifier, an output layer (a typical MLP is shown in Fig.
which delivered an accuracy of 97.4%. In another 19.9). While the input layer has the same number
study, a random forest classifier was compared of units as input features, and the number of
with both SVM and an existing closed loop neu- units in the output layer is restricted by however
romodulation device for seizure detection and many classes there are in the classification task,
showed better performance compared to the other the number of hidden layers and units in each
two strategies, while maintaining low detection layer is dependent on algorithm design. Too few
delay and good energy efficiency [48]. hidden layers/units lead to poor differentiation of
complex patterns in the data, while too many units
can lead to overfitting, and too many layers can
19.3.3 Deep Neural Networks make training time-consuming – so a careful con-
sideration for MLP parameters is necessary. In the
Artificial neural networks (ANN) are a large fam- work by Sriraam et al. [50], a three-layer MLP
ily of machine learning algorithms inspired by with 10 hidden units was used with spectral power
biological neurons. The simplest ANN is sys- and energy features from scalp EEG of 20 patients
tem of multiple perceptrons, or individual artifi- for seizure detection and achieved a sensitivity of
cial neurons which behave very similarly to the 97.1%, specificity of 97.8%, and FPR of 1/h. In
logistic regression described in an earlier sec- Subasi and Erçelebi [51], a similar MLP with one
tion – the only difference being a slightly dif- hidden layer and 21 hidden units was compared
ferent activation function. In fact, unlike logistic with logistic regression using wavelet-extracted
regression, ANNs can use any activation function, features from 500 scalp EEG segments for seizure
and several have been commonly used. While classification, and it outperformed the latter al-
the activation function cannot be linear (other- gorithm with an accuracy of 92% and AUC of
wise an ANN will collapse into a single per- 0.889. MLPs with more than one hidden layer
ceptron), both sigmoid (with range 0 to 1) and have also been used, for example, in the study
hyperbolic tangent (with range −1 to 1) functions by Abbasi and Esmaeilpour [52], where a neural
have been used. More recently, rectified linear network with two hidden layers (with 4 units in
unit (ReLU) and “leaky ReLU” functions have the first and 5 units in the second hidden layer)
been designed to improve upon some of the is- was used with wavelet-derived features from the
sues with the sigmoid and tanh functions and are Bonn dataset and achieved 98.3% accuracy in
defined as seizure detection.
, While multilayer perceptron network is the
z, z > 0 most common among the simpler ANN designs,
f (z) = , (19.18)
αz, z ≤ 0 there are many iterations that attempt to improve
the algorithm’s performance. Probabilistic neu-
where α is zero for ReLU and a small value (e.g., ral network (PNN) is a neural network with an
0.01) for leaky ReLU. While these activation exponential as an activation function which com-
functions have enjoyed wide adoption as typical putes the distance from the test input to the train-
activations used in ANNs, a few other functions ing input vectors and produces a net output as a
have been occasionally used, such as a radial basis vector of probabilities. PNNs are characterized
function and a nonlinear cube function. by fast training and have been compared with
decision trees and SVM classifiers in Martis et
Fig. 19.9 Example of a multilayer perceptron with one hidden layer and four hidden units. (Figure adapted from
Acharya et al. [49])
al. [41] and Acharya et al. [42] showing compara- work, (3) a layer-recurrent network (with a feed-
ble accuracy, sensitivity, and specificity. Contin- back loop around each layer), and (4) a distributed
uous neural networks are ANNs where each unit time-delay network (where the output of a layer
is described by ordinary differential equations also depends on past outputs) using energy-based
(ODEs), and in one case, they were trained on and complexity-based features extracted from the
Freiburg dataset as well as 90 scalp EEG trials, Freiburg dataset. The comparison showed that in
and the overall correct classification percentage a patient-specific task (i.e., both testing and train-
was 97.2%, using features that, unlike most other ing data came from the same patient), all of ANNs
noncontinuous classifiers, take into account the showed great performance with accuracy of close
continuous nature of EEG signals [53]. Extreme to 100% – with RBF network performing slightly
learning machines (ELM) are a generalized sin- worse than others. However, when the system was
gle hidden layer MLP network where the param- trained on one patient and tested on another, the
eters of hidden units (and not just the weights) performance of ANNs dropped significantly.
are randomly generated. A sparse ELM has been Perhaps the two patients used for compari-
shown to perform comparably to SVM classifiers son had two drastically different epilepsy eti-
and traditional ANN on a seizure detection task ologies (as the authors suggested), or there was
with accuracy of 98.4%, while requiring less stor- not enough data to properly tune ANNs to suc-
age space and training time [54]. cessfully classify EEG signals across different
An early comparison of several types of artifi- patients. However, it is equally likely that ANNs
cial neural networks for EEG state classification used were unable to capture the full complexity
was shown in Costa et al. [55]. In the study, the of the provided EEG signals. Deep learning is
authors investigated (1) a traditional feedforward a subfield of machine learning which is rapidly
network, (2) a radial basis function neural net- gaining prominence due to the ability of deep
neural networks to better capture complexity as- seizure prediction and performed with sensitivity
sociated with real-life data without the necessary of 87.8% and FPR of 0.142/h. In another study,
fine-tuned feature selection. While the precise a sequence of short-time Fourier transforms was
definition of what makes a neural network deep is used with a CNN with three convolutional layers
elusive, the generally accepted criterion is having with max pooling and two MLP layers for seizure
at least three hidden layers. An example of deep prediction with FPR of 0.06/h and sensitivity of
neural network is the multilayer perceptron with 81.4% [57]. In both examples, a two-dimensional
three hidden layers used with bispectral entropy convolutional neural network was used on, a spec-
features for seizure prediction using intracranial trogram image; however, this need not be the
EEG data, where it achieved a test accuracy of case. In Acharya et al. [49], for example, a one-
78.11% [56]. dimensional CNN with five convolutional layers,
One specific class of deep neural network is a five max-pooling layers, and three MLP layers
convolutional neural network (CNN or ConvNet, was used on a normalized EEG trace. The al-
example shown in Fig. 19.10) which was inspired gorithm was used to classify scalp EEG into
by and is highly correlated with the organization normal, preictal, and seizure states and achieved
of the visual cortex [58] and has been extensively an accuracy of 88.7%, sensitivity of 95%, and
used on image classification tasks. In general, specificity of 90%. On the other hand, in the study
ConvNets consist of a feature learning stage and by Wei et al. [60], a multichannel scalp EEG
a classification stage. The feature learning stage data was fed into a three-dimensional CNN with
is comprised of convolutional and pooling layers. nine total layers, and the seizure detection per-
The former consists of filters or kernels, matrices formance was compared with a two-dimensional
that convolve with the image (a spectrogram, a CNN and a SVM-based classifier. With average
matrix of wavelet coefficients, or a compilation of accuracy of 92.4%, the 3-D CNN outperformed
EEG signals for seizure detection and forecasting the other two classifiers. Further comparison of
tasks) to extract spatial features and create a fea- CNNs to other classifiers also showed that CNNs
ture map. The latter, pooling layer, down-samples outperformed SVM and logistic regression clas-
the input data and reduces its dimensions, de- sifiers for seizure prediction [61] and achieved
creasing the necessary computational power as zero-false-alarm seizure prediction in 20 out of
well as extracting dominant features. Two types 21 patients of the Freiburg dataset, while SVMs
of pooling layers exist – a max pooling returns only had 11 such predictions [62].
the maximum value from the subregion of the Another commonly used class of a deep neural
data, while the average pooling returns the av- network is a recurrent neural network (RNN), de-
erage of all values from the subregion. As max signed specifically for sequential data. RNNs take
pooling can also act as a de-noising filter, it is the as an input not only the current training/testing
preferred choice when designing the CNN. Due example but also previous information they have
to existence of the feature learning stage, CNNs encountered – so they are said to have memory.
require little preprocessing or manual feature se- Adding this memory can be advantageous since
lection, unlike other machine learning algorithms. there is information in the sequence itself (e.g.,
Features extracted from the input data are then fed the sequence of interictal → preictal → ictal
into the classification stage, which is typically a EEG states) that other ANNs cannot capture.
multilayer perceptron trained for a classification Recurrent neural networks for seizure prediction
task. were first used in 2000, when an RNN with one
Recently, convolutional neural networks have hidden layer of between 10 and 15 units was
been used for seizure prediction and EEG state used with intracranial and scalp EEG of two
classification. In a work by Khan et al. [59], a patients for seizure prediction [63]. Both EEG
CNN with six convolutional layers (with max time-series data and wavelet-decomposed spec-
pooling) and two dense (or MLP) layers was used tral bands were fed into the RNN which resulted
with wavelet-transformed scalp EEG signals for in up to 15 second early warning of seizure onset.
Fig. 19.10 Example of a convolutional neural network, with three convolutional kernels, three max-pooling layers,
and two fully connected MLP layers. (Figure from Truong et al. [57])
More work has been done since with RNNs, study, a CNN-LSTM hybrid algorithm was used
including classification of epileptic seizures using on scalp EEG of 23 patients, with three frequency
wavelet energy and norm entropy as features, bands covering 0–49 Hz and 2-D projection of
resulting in average accuracy of 99.8% [64], and electrode placements as features [68]. The pro-
using a recurrent cellular neural network (an posed hybrid algorithm achieved sensitivity of
ANN with geometric arrangement of units with 95–100%, FPR of 0.1/h for the same patient,
the restriction that the communication is only al- and 0.8/h for cross-patient trials. Furthermore, it
lowed between neighboring units) on EEG time- proved to be more robust to missing electrodes
series data to successfully detect 100% of seizures than previous algorithms.
with an average detection delay of 7.0 seconds
[65].
Regular recurrent neural networks have some 19.3.4 Improving Model
limitations on their memory, and improved RNNs Performance
have been developed – namely, gated recurrent
unit (GRU) and long short-term memory (LSTM, In the previous subsections, we have outlined the
schematic shown in Fig. 19.11) networks. main classes of machine learning algorithms used
Both networks have units which contain so- for seizure detection and forecasting from in-
called gates, mechanisms regulating the flow of tracranial and scalp EEG signals. However, across
information and allowing the unit to learn which all types of algorithms, some strategies exist to
data in the sequence is important to keep. These further improve classification performance. One
gates improve the performance, for example, of the ways to improve algorithm performance
when an LSTM network was used on frequency- is through using ensemble techniques, where a
domain, time-domain, and cross-correlation combination of weak learners is used to create
features extracted from scalp EEG for seizure an overall strong learner with better performance.
prediction [67]. The algorithm achieved average We have briefly mentioned examples of ensemble
sensitivity of 100% and a false prediction rate of techniques before, such as random forests, or
0.11/h – the authors also noted that increasing ensemble of SVM classifiers in the work by Tang
the window of preictal data available to LSTM and Durand [36]. Ensemble learning can also be
reduced the FPR to as low as 0.03/h. extended to ANNs and deep learning, such as
There has been some effort put into combin- using three groups of five neural networks each
ing recurrent neural networks and ConvNets to for three-way EEG signal classification, which
take advantage of both automated feature learning improved the performance by 10% compared to
and sequential memory in one algorithm. In one
NET OUTPUT
∫
OUTPUT GATE
Π ∫ Σ
Σ ∫ Π
1.0
Π ∫ Σ
FORGET GATE
∫ INPUT GATE
NET INPUT
Fig. 19.11 Schematic of a long short-term memory unit for RNN with internal gates for memory management. (Figure
from Yu et al. [66])
an individual ANN (98.78% vs. 88%) [69]. In done by Brinkmann et al. [72], where several of
another study, an ensemble of so-called pyrami- the top 10 algorithms utilized ensemble learning
dal convolutional networks (CNNs with smaller (see Table 19.2) and showed higher AUC than,
kernel size at each layer) was used with raw EEG for example, a ConvNet; moreover, the first al-
signals and achieved an accuracy of 99.1% for gorithm improved its performance AUC by up to
epilepsy detection task [70]. 10% compared to its base classifiers. In any en-
Ensemble learning is not limited to using mul- semble model, the final decision has to be reached
tiple copies of the same algorithm. In a work by from the combination of individual classifiers –
Abdulhay et al. [71], k-nearest neighbor, RBF- one of the most widely used ways of determining
SVM classifier, and naïve Bayes (a conditional the final decision in a classification is a majority
probability supervised learning method based on vote. However, other ensemble methods exist,
Bayes’ theorem) classifiers were combined into such as weighted average, Platt scaling (combin-
an ensemble model, and the performance for each ing all of the outputs into a probability distribu-
base classifier increased by around 3% for EEG tion over all classes), or Bayesian combination of
state classification. A large study of different en- classifiers.
semble models for seizure forecasting in human
and canine epilepsy in an online competition was
Table 19.2 Details of seizure forecasting classifiers

Selected features used Machine learning algorithm Ensemble method AUC
Spectral power, correlation, Generalized linear model; Weighted average 0.82
distribution statistics, signal SVM classifier; random
variance forest
Log spectral power, SVM Platt scaling 0.8
covariance
Spectral power, correlation, Neural network; k-nearest Bayesian combination 0.79
signal derivative neighbor
Spectral power, statistical SVM; generalized linear Weighted average of rank scores 0.79
measures, covariance model
matrices
Spectral power, signal Convolutional neural N/A 0.78
standard deviation network
Adapted from Brinkmann et al. [72]
Ensemble models are not the only strategy for best ranked method, likely due to more linear and
improving classification results – correctly select- discriminative feature space.
ing features to feed into a machine learning al- Careful consideration for the type of machine
gorithm is equally important. One way to reduce learning algorithm and the feature selection is
the algorithm’s reliance on correctly selected fea- necessary for good classification performance.
tures is to utilize a CNN with its feature learn- In Fig. 19.12, ROC curves show that even for
ing stage, which was covered earlier. Another is the same algorithm, using different features can
to rely on unsupervised learning algorithms to lead to vastly different AUC – in the example, a
automatically identify useful features. Instead of random forest algorithm using time-based and co-
building an input-output map from a training set, modulogram features led to an increase of 0.226
unsupervised algorithms find patterns in the data in AUC compared to power-based features [17].
without being provided the “correct” answers. Deep neural network-based unsupervised learn-
In one study, k-means clustering algorithm was ing algorithms also have been used for feature
used for feature extraction from scalp EEG (Bonn extraction. Autoencoders are a type of unsuper-
dataset) together with an MLP model to achieve vised neural networks with two stages – an en-
an overall accuracy of 98.3%, about 5–8% in- coder and a decoder – which attempt to learn
crease compared to MLP used with manual fea- an identity function by adjusting hidden layer(s)
tures [73]. K-means algorithm finds k number of such that the input and the output are as close
clusters, or collections of data points aggregated to each other as possible – in essence create a
together based on some similarity, by reducing reduced representation of the data which can be
the in-cluster distance between every data point used as features. The underlying type of neural
and the center of the cluster. Another unsuper- network used for an autoencoder can vary, for
vised learning technique for feature extraction example, in one study, a CNN-based autoencoder
is bag-of-words, originally developed for natural feature learning was used with various classifiers
language processing, where each feature vector (a (SVM, decision tree, random forest, MLP) for
so-called bag) is described by the distribution of EEG state classification and showed more than
unique features (“words”) – or how many times 10% improvement in average accuracy compared
each feature has appeared in the input. In the to other, non-machine-learning, feature extraction
study by Martinez-del-Rincon et al. [74], bag-of- techniques [75]. Another study used a stack of
words technique was used with an SVM classifier two autoencoders to extract the features to the
for seizure detection and showed an overall 10% extent that only a supervised learning softmax
improvement in the F1 score over the second- function was needed, and it achieved accuracy of
Fig. 19.12 ROC curves of different machine learning algorithms (SVM and RF) using varying sets of features, used
to predict AED treatment efficacy. (Figure from Colic et al. [17])
94% (around 15% points higher than the next best example of an HMM is predicting the weather
method) and FPR of 0.05/h for seizure forecasting state (rain, cloudy, sunny) based on the type of
[76]. A recent paper improved on that approach clothes people wear without being able to look
by using a deep convolutional autoencoder cou- outside. In context of EEG analysis, HMMs
pled with bidirectional LSTM and showed an in- can identify the underlying EEG state based
creased per-patient prediction accuracy of 99.6% on some observable feature set. In a work by
with false alarm rate of 0.004/h and prediction Baldassano et al. [79], an autoregressive hidden
time of 1 h prior the seizure onset [77]. Markov model was used with intracranial EEG
Occasionally, in addition to feature selection, recordings from six dogs with naturally occurring
unsupervised learning algorithms can also be epilepsy, and the method showed a reduced
used for seizure prediction and forecasting in false-positive rate compared to a previously used
their own right. K-means algorithm has been with random forest classifier with manually selected
entropy-based features extracted from the Bonn features (0.0012/h vs. 0.058/h FPR) with an
dataset for seizure detection and showed a 6% average 12.1 second advance seizure detection.
higher accuracy with 97% less execution time In another study, an HMM with observable
compared to the SVM classifier [78]. Another states that were assumed to be a combination
type of unsupervised learning used for seizure of Gaussian distributions (a Gaussian mixture
detection and forecasting is a hidden Markov model) was used with pediatric scalp EEG data
model (HMM) – a probabilistic algorithm used to predict seizures with sensitivity of 0.95 and
to model a sequence of underlying hidden states specificity of 0.86 [80].
based on observable variables. A very common It is evident from this section that a large
variety of machine intelligence algorithms have
Table 19.3 Summary of strengths and weaknesses of a number of common machine learning algorithms
Algorithm Strengths Weaknesses
Logistic regression Output can be interpreted as probability As a linear model, cannot handle
Easy to train nonlinear relationships in the data
SVM Works well for nonlinear classification Hard to pick the right kernel
Deals well with outliers Memory intensive
Poor performance on noisy data
Decision tree Easy to understand and visualize Can create complex trees that do not
Require less data preprocessing generalize well
Random forest Improves the performance of decision Output can be hard to interpret
trees Predictions are slow to create
Works well in high-dimensional feature Does not work well with sparse datasets
spaces
Deep neural networks Can learn complex input-output Require a lot of data
mapping of the data Computationally expensive
Can perform feature extraction Very hard to interpret the resultant
On large datasets generally outperform classifier itself and the internal workings
most other algorithms of the algorithm
been used for seizure detection, classification,

and forecasting. While some studies and strate- 19.4 Other Applications
gies discussed above have compared their perfor- of Machine Intelligence
mance to other classifiers, an astute reader can with EEG
notice that no one particular method has been
identified as the “gold standard” to be used for In the previous section, we have discussed at
EEG signal classification. In part, it is due to length the application of several types of ma-
the fact that EEG signals are inherently com- chine learning algorithms to seizure prediction
plex due to their nonlinear, dynamic, and non- task. However, while these algorithms are effec-
Gaussian nature, making classification difficult. tive, they are not the only approach – in one
Another reason is the so-called no free lunch case, effective connectivity of brain networks was
theorem which states that there is no one ma- used for seizure prediction, achieving sensitiv-
chine learning model that works best for every ity of 80% and FPR of 0.33/h [81]. Another
problem due to underlying assumptions one has area where machine intelligence performance is
to make during algorithm design. Deep convolu- steadily improving is seizure localization. In one
tional neural networks, for example, can perform study, using intracranial EEG signals, an SVM
better than some other classifiers due to fewer classifier was trained and tested on patients with
number of parameters and CNN’s property of Engel class I to class IV outcomes, demonstrat-
rotational and positional invariance; however, that ing superior performance in the class I patients
same invariance can prove detrimental when the in Fig. 19.13 [82]. The classification using fea-
position or rotation of a feature is important. tures based upon both high-frequency and low-
Furthermore, deep learning models in general are frequency oscillations was best able to identify
not very good at handling imbalanced data, a channels suited for resection. This study demon-
situation frequently encountered in EEG signal strates a novel approach to region of interest
classification. With that in mind, some of the identification and provides a path for developing
strengths and weaknesses of machine learning al- tools to improve outcomes in epilepsy surgery
gorithms discussed in this chapter are presented in [17]. Another SVM classifier was used in iden-
Table 19.3. tifying SOZ based on phase locking value (PLV)
[83]. The study showed that more than 96% of
electrodes identified as the SOZ were within the
Fig. 19.13 SVM-classified region of interest channels resected area, surgical resection resulted in poor control
(shown in brown) coincide with resected area (shown in of seizures (Patients B and C, EC III and IV, respectively).
gray) in a seizure-free patient (Patient A, EC I), while (Figure from Dian et al. [82])
in patients where SVM-identified channels are outside of
resected area in six seizure-free patients. In four magnetoencephalogram (MEG) recordings.

non-seizure-free patients, more than 31% of the MEG is a technique very similar to scalp EEG,
identified SOZ electrodes were outside the re- though better suited to source localization, and
sected area. Furthermore, in the same study the with features such as delta frequency power,
outcome in non-seizure-free patients correlated power ratio, and phase lag index extracted
with the number of non-resected SOZ electrodes from MEG data, both classifiers distinguished
identified. In the study by Tomlinson et al. [84], the resection areas from non-resection areas
an SVM classifier was used on iEEG data from 17 with 59.94% accuracy for SVM and 60.34%
pediatric patients, and it was able to predict sur- for random forest (however, the above method
gical outcome using global synchrony and local was not able to differentiate seizure-free from
heterogeneity features with 94.1% accuracy. not seizure-free patients) [85]. Overall, as with
Both random forest and SVM classifiers were seizure prediction, the accuracy of epileptogenic
used to distinguish between resection and non- source localization techniques varies based on
resection areas of 94 patients, using interictal data modality and features selected. Although
machine learning methods showed improvement study by Colic et al. [17], the normalized power
over manual SOZ identification, they are feature projections did not show any clustering
still facing challenges to properly identify by individual animal subjects and were the
epileptogenic sources especially in noninvasive least useful features in terms of separating
recordings due to low signal-to-noise ratio (SNR). responders and non-responders, while ensemble
empirical mode decomposition (EEMD) time-
based and comodulogram features achieved the
19.4.1 Prediction of Antiepileptic best separability with distinct clusters for each
Drug Treatment Outcomes of the animal subjects. These features were then
used with both SVM and random forest classifiers
Frequently, antiepileptic drug (AED) treatments to predict treatment efficacy of an antiepileptic
produce inconsistent outcomes, so patients may drug, and the results showed that comodulogram
need to go through several drug trials until a suc- features (AUC 0.974) outperformed those of
cessful treatment can be found. There are dozens EEMD time-based (AUC 0.918) and normalized
of commonly used AEDs, and many more ex- power (AUC 0.745) – see Fig. 19.12.
perimental drugs, available to treat the disorder. When the two machine learning methods were
Determining the efficacy of one drug for a spe- evaluated to predict the treatment outcome of four
cific patient often involves a trial-and-error pro- different AEDs, SVM was found to predict the
cedure. There are 20–40% of epileptic patients treatment outcome of outliers found in random
with drug-resistant epilepsy [86], though they forest predictions (see Fig. 19.14). In the same
only become aware of this after having already study by Colic et al. [17], random forest predic-
participated in numerous AED trials. Antiepilep- tion of treatment outcome for ganaxolone applied
tic drugs can also make the seizures worse and on mouse 2 was close to 100%, when it should
more frequent, which are associated with numer- have been closer to 0%, whereas SVMs predicted
ous side effects that can affect patients’ cognition 44%. Similarly, for phenytoin, the prediction for
and functioning [87]. Unsuccessful drug trials mouse 1 was 84% when it should have been closer
and delayed treatments highly impact patients’ to 0%, whereas SVMs predicted 59%. Generally,
quality of life and are expensive for both patients SVMs estimated 90% or greater likelihood scores
and the health-care system. Determining a priori only for successful treatments.
the most effective treatment using machine learn- Patient variability is a serious challenge to se-
ing methods would go a long way in improving lecting treatments for epilepsy. Often antiepilep-
the lives of patients and reducing the financial tic drug treatments are cycled through until an
burden. effective treatment can be found, and with over
While using patients’ scalp or intracranial two dozen commonly prescribed AEDs available,
EEG is the gold standard for epilepsy research, it can be a cumbersome process. There are certain
sometimes it is easier to do preliminary AEDs that have been found to be statistically
assessment on computer or animal models before more likely to lead to a successful treatment out-
transitioning the methodology to humans. One come, and it is those AEDs that typically are
example of this is the use of rodent models of Rett tried first. However, the likelihood of a success-
syndrome – a neurological disorder characterized ful treatment reduces with each round of AED
in part by neural network hyperexcitability application [90], possibly due to patient desen-
and spontaneous epileptiform-like discharges, sitization to AEDs which happens over time. By
similar to epilepsy [88]. In Rett syndrome indicating which patients would be unresponsive
model, an examination of different feature sets to certain AEDs, and what AEDs are most likely
showed that, like other classification tasks, the to be successful – using machine intelligence –
selection of features is vital in achieving class epileptologists could choose the most appropri-
separation and thus has a profound effect on ate therapy for the patient without unnecessary
determining treatment outcome [89]. In the testing of AEDs, and the treatment is more likely
Fig. 19.14 Predicted likelihood of favorable treatment predictions accurately predicted treatment outcome for all
outcome across four commonly used AEDs using SVM AEDs. (b) RFs had comparable prediction results, with
and RF machine learning algorithms. Green bars indicate misclassifications for ganaxolone treatment for mice 2 and
the patient was successfully treated by specific AED, 4 and phenytoin for mouse 1. (Figure from Colic et al.
and brown bars indicate unsuccessful treatment. (a) SVM [17])
to show a positive improvement in a patient’s by relatively low signal-to-noise ratio, which can
quality of life. significantly impair a given classifier’s perfor-
mance – so any classifier should be designed to
be robust to high noise and class balance issues.
19.5 Current Challenges In situations where it is important to understand
and Future Directions how the classifier reached its decision, low inter-
pretability of machine learning algorithms (espe-
In this chapter, we have focused on the use of ma- cially deep neural networks) might prove that it is
chine intelligence for seizure detection and fore- difficult to get the necessary insight.
casting, and prediction of antiepileptic drug treat- Other major challenges of EEG-based ma-
ment outcomes, as well as feature extraction and chine learning algorithms include issues concern-
selection to be used for machine learning algo- ing EEG data, namely amount of data, source of
rithms – including wavelet phase coherence and data, accurate data labels, and artifacts. Due to
cross-frequency coupling. While a lot of progress the constraints of human EEG acquisition, there
has been made in the past several years to improve is typically a relatively small amount of heteroge-
EEG-based techniques with cutting-edge algo- neous data available for a particular task – usually
rithms, several important challenges still remain. on the order of a couple of dozen to a couple
Frequently, EEG data (especially obtained from of hundred EEG segments from 5 to 20 patients.
scalp) is imbalanced, favoring one class (e.g. in- While it might seem like a lot of data for manual
terictal EEG state) over others, and characterized analysis, this amount of data could make it diffi-
cult for machine learning algorithms (especially transform, and neural network-based algorithms,
deep networks) to achieve reliable and highly have been used for artifact correction.
accurate classification. One way to circumvent Specific uses of machine intelligence can also
this issue is to use a technique called transfer have their unique challenges. As an example,
learning, where a model trained in the domain for seizure prediction, it can be complicated to
with a lot of general data (e.g., all of scalp EEG compare patient-specific algorithms that are in
available) is repurposed to a more specific task the 95–100% sensitivity range. For one, patient-
(e.g., antiepileptic drug efficacy). This improved specific algorithms require new training for ev-
the classifier’s performance, since the algorithm ery new patient, so optimally cross-patient algo-
can learn more basic features on a larger dataset. rithms should be prioritized. Another issue is the
One such strategy was used in Liang et al. [91] potential discrepancy between reported bench-
where six available EEG datasets not related to marks and real-life expectations. For example, the
seizure prediction were used as auxiliary infor- best seizure prediction algorithms report around
mation to a one dataset for seizure prediction and 0.05/h false-positive rate, which appears low es-
found that the prediction performance improved. pecially compared to previous methods. How-
A related issue is the source of EEG data – it is ever, that translates to roughly one false alarm
easier to get the data for analysis from animal every day. For some uses, such as warning the per-
models; however, one must be careful to ensure son about the upcoming seizure, this might not be
that features or classes they identified are trans- an acceptable rate; for others, such as neurostim-
ferable to humans. ulation system, it might be within tolerance –
Sometimes, parts of the data are unlabelled, though long-term effects of routine daily neu-
or there is some uncertainty about how reliable rostimulation should probably be investigated.
labels are. This poses an issue for the classifier, Sometimes, parameters that normally are not a
since it is given incorrect or missing training main priority (such as the latency of the algo-
data. In this situation, so-called semi-supervised rithm or its energy efficiency) become crucially
learning techniques can be used, such as semi- important, as they are in mobile seizure prediction
supervised version of extreme learning machines systems. All these challenges – both general and
(ELM) which, despite having unlabelled data, specific – are the reasons why a recent seizure
outperformed a fully supervised ELM model prediction system designed for a wearable device
[92]. The most common issue with real-world achieved mean sensitivity of only 69% [94].
EEG signals is the presence of artifacts. Artifacts At their current stage, machine learning al-
in EEG can be very diverse, from not relevant gorithms can be used to augment existing tech-
physiological signals (e.g., EMG, ECG) to niques, such as providing an opinion on the po-
cable and electrode movement, environmental tential location of the epileptogenic zone, or iden-
interference, and recording equipment; they tifying seizures for further processing. While it is
can be present in multiple electrodes or only not yet clear whether machine intelligence will
in one and can be periodic or irregular. Most completely eliminate the need for manual inter-
of publicly available EEG datasets manually vention, some future directions of EEG-based
remove artifacts ahead of time, which means algorithms can be suggested. One likely potential
that algorithms trained on them will not perform development is integration of more probabilis-
as well on the non-processed data. Islam et al. tic modelling into machine learning algorithms.
[93] presented a thorough review of methods Estimating seizure probability as a way to de-
for artifact detection and removal, but, in short, tect seizures has already been investigated by
artificial neural networks, SVM classifiers, Kuhlmann et al. [95], a circadian probability as
and k-means clustering can be used to detect subclassifier was used in Karoly et al. [26] for
unwanted signals, while other techniques, such as seizure forecasting, and we have briefly described
independent component analysis, EMD, wavelet a probabilistic neural network in Sect. 19.3. A
natural extension to all those is a Bayesian neu- depressive disorder classification task increased
ral network (BNN) or Bayesian deep learning, accuracy and AUC by 10% [99]. In a work by
which, for example, was used recently with scalp Mesejo et al. [100], an evolutionary computa-
EEG for mental fatigue detection [96]. BNN is tion algorithm was combined with an artificial
a neural network that uses a prior probability neuron-glia network (ANGN) – an extension of
distribution on its weights in order to incorporate a regular ANN to include longer-term dependen-
uncertainty about the prediction. This gives an cies for weight adjustments which mirror effects
advantage of BNN to work better on smaller of astrocytes (dominant glial cells in the brain) in
datasets, prevent overfitting, and give an overall biological neural networks. Astrocytes have been
insight over how reliable the given prediction is. shown to be involved in neuronal firing [101],
Another way to incorporate probability into ma- particularly that their activity has an effect on
chine intelligence is to use restricted Boltzmann neuronal codes similar to those seen in the human
machines (RBM). A Boltzmann machine (BM) is brain [102]. These findings make astrocytes an
a type of unsupervised fully connected stochastic attractive target for more biologically inspired
recurrent neural network with a visible input layer machine learning algorithms. While in the study
and at least one hidden layer, while an RBM by Mesejo et al. [100] the resultant algorithm
has a restriction that connections can exist only performed comparably to existing ANNs, intro-
between layers. In context of EEG signals, one ducing more biomimetic algorithms for machine
interpretation is that the units in the visible layer intelligence tasks could result in better perfor-
represent observable attributes, while the hidden mance in complex problems.
layer units act as nonlinear feature detectors, and One final direction of future development is
recently, an RBM-based technique has been suc- adapting alternative sequential models for EEG
cessfully evaluated for detection of epileptogenic analysis. Since EEG data is sequential in nature,
lesions [97]. machine learning algorithms would benefit from
Another potential development is the integra- having memory to be able to capture existing
tion of genetic algorithms with machine learning temporal dependencies within it. We have already
techniques to improve feature or hyperparameter described several variants of recurrent neural net-
(a parameter with a value set before the learn- works – deep neural networks adapted for se-
ing process) selection. Genetic algorithm belongs quential data – and their use in seizure prediction
to a family of evolutionary computation algo- studies. One disadvantage of RNNs, however, is
rithms inspired by biological evolution – mir- that they require a lot of resources (time and
roring the biological inspiration between various computational power) to train properly. Autore-
types of artificial neural networks. In short, the gressive feedforward models, such as a WaveNet
genetic algorithm generates multiple candidate [103] or gated convolutional networks [104], are
solutions with various parameters and after some being developed as an alternative to RNNs. In
training assesses their “fitness.” Each new gen- autoregressive neural networks, instead of relying
eration of algorithms is produced by removing on most of the history of the sequence for making
less fit solutions and introducing small random predictions, the model only uses the finite number
changes (mimicking biological concepts of muta- n of most recent inputs. While theoretically RNNs
tion and crossover) – this eventually creates a sub- should be more flexible, in practice, Bai et al.
set of high-quality optimized solutions to a given [105] showed that autoregressive neural networks
problem. A recent review thoroughly examined a outperform comparable RNNs in a wide variety
number of evolutionary computation algorithms of tasks such as audio synthesis and machine
for EEG feature selection, including the genetic translation while also benefitting from signifi-
algorithm [98], while another study found that cantly easier and faster model training and pre-
using genetic algorithm with an MLP for a major diction.
Acknowledgments The authors would like to acknowl- (available at https://www.kaggle.com/c/seizure-

edge funding from Canadian Institutes of Health Research detection).
and National Sciences and Engineering Research Council.
7. Physicians and researchers working in

epilepsy often review large quantities of
Homework EEG data to identify seizures, which in
some patients may be quite subtle and hard
Conceptual Questions
to detect. Automated algorithms to detect
seizures in large EEG datasets with low false-
1. Given the discussion of feature engineering
positive rates (FPR) and false-negative rates
of both scalp and intracranial EEG data in
(FNR) would greatly assist both clinical care
this chapter, describe some useful features for
and preclinical research. Using a multilayer
seizure detection and prediction.
perceptron, classify windows of human EEG
2. Given a relatively small dataset of 10 patients
data as seizure or non-seizure. Use spectral
with a selection of interictal, preictal, and
power features computed from 1 second
ictal recordings, a) suggest an approach to
windows as inputs to the MLP (see figure
divide the dataset into training and test sets,
below). Divide the data into a training set
and b) provide benefits and drawbacks of
and a testing set using a ratio of 80% to
leaving one or more patients entirely for the
20%, respectively. Use the training set to
test set.
train the MLP and the testing set to find the
3. For the same dataset as in previous question,
FPR and FNR. Compute an ROC curve and
suggest what machine learning algorithm you
the area under the curve to compare network
would use and why. Would your answer
performance.
change if a) it was only two classes; b)
(a) Using a MLP with one hidden layer, and
the dataset contained 1000 patients; c)
gradient descent method with step size of
the algorithm needs to perform EEG state
0.5, alter the number of units in the hidden
classification in real time.
layer (5, 10, 40) and explore whether in-
4. What are common noise sources and artifacts
creased number of hidden units will have
in EEG recordings? Suggest a few ways to
a positive effect on the network perfor-
improve signal quality and eliminate these ar-
mance. What are the pros and cons of
tifacts.
having more hidden units?
5. Frequently, EEG data is imbalanced, favoring
(b) Alter the number of hidden layer (no hid-
one class over others. How does that impact
den layers, 1 hidden layer, or 2 hidden
classification performance? How would you
layers) in the feedforward neural network,
overcome this issue?
using 10 units per hidden layer and gradi-
6. In this chapter, we have briefly covered several
ent descent method with step size of 0.5.
network architectures where the targets are the
Determine whether increased number of
same as their inputs. Name two and explain
hidden layers will have a positive effect
when you would likely use them.
on the network performance. What are
the pros and cons of having more hidden
layers?
Practical Analysis Questions
(c) Would you say that using a convolutional
neural network is preferable over using a
These questions are intended as introductory
multilayer perceptron and why?
guides to your own practical implementation of
8. Using the same approach as in question 6,
the techniques outlined in this chapter.
explore the effect of training parameters.
For questions 7 and 8, use data from UPenn
(a) Learning Rate – Try different step sizes or
and Mayo Clinic’s Seizure Detection Challenge
learning rates (lr = 0.1, 0.5, 1) using gra-
dient descent training function on a neural Neurol. 5, 492–504 (2009). https://doi.org/10.1038/

network with one hidden layer network nrneurol.2009.118
3. F.E. Dudek, T.P. Sutula. Epileptogenesis in the den-
with 40 hidden units. Determine whether
tate gyrus: A critical perspective. Progress Brain
large step size will always expedite learn- Res. 153, 755–773 (2007)
ing. 4. M. Steriade, Corticothalamic networks, oscillations,
(b) Momentum – Investigate the effect of mo- and plasticity. Adv. Neurol. 77, 105–134 (1998)
5. E. St. Louis, Minimizing AED adverse effects:
mentum using a network with 1 hidden
Improving quality of life in the interictal
layer (10 units) and gradient descent with state in epilepsy care. Curr. Neuropharmacol.
momentum (mc = 0.1, 0.5, 0.9). Deter- 7, 106–114 (2009). https://doi.org/10.2174/
mine whether a strong momentum term 157015909788848857
6. M. Penttonen, G. Buzsáki, Natural logarithmic re-
will always expedite learning. lationship between brain oscillators. Thalamus Re-
lat. Syst. 2, 145–152 (2003). https://doi.org/10.1016/
For questions 9 and 10, use data from S1472-9288(03)00007-4
American Epilepsy Society Seizure Prediction 7. G. Buzsáki, C.A. Anastassiou, C. Koch, The origin
of extracellular fields and currents—EEG, ECoG,
Challenge (available at https://www.kaggle.com/
LFP and spikes. Nat. Rev. Neurosci. 13, 407–420
c/seizure-prediction). (2012). https://doi.org/10.1038/nrn3241
8. G. Buzsáki,Rhythms of the Brain (Oxford University
9. Responsive neurostimulation (RNS) presents Press, Oxford/New York, 2006)
9. N. Jackson, S.R. Cole, B. Voytek, N.C.
a possible therapy for abolishing seizures in
Swann, Characteristics of waveform shape
epileptic patients that are drug-resistant and in Parkinson’s disease detected with scalp
ineligible for surgery. Seizures that build and electroencephalography. eNeuro (2019). https:/
generalize beyond the area of origin are very /doi.org/10.1523/ENEURO.0151-19.2019
10. J. Jacobs, P. LeVan, R. Chander, et al., Interic-
difficult to abort; thus electrical stimulation
tal high-frequency oscillations (80-500 Hz) are an
must be applied as early as possible. Using indicator of seizure onset areas independent of
the same algorithmic approach as in ques- spikes in the human epileptic brain. Epilepsia 49,
tion 6, train your system to predict epileptic 1893–1907 (2008). https://doi.org/10.1111/j.1528-
1167.2008.01656.x
seizures in human patients. How does your
11. M. Brázdil, M. Pail, J. Halámek, et al., Very high-
performance (in terms of the AUC metric) frequency oscillations: Novel biomarkers of the
compare to the seizure detection task as well epileptogenic zone: VHF oscillations in epilepsy.
as results shown in Table 19.2? Explain your Ann. Neurol. 82, 299–310 (2017). https://doi.org/
10.1002/ana.25006
results.
12. J. Jacobs, R. Staba, E. Asano, et al., High-frequency
10. Suggest improvements to your seizure oscillations (HFOs) in clinical epilepsy. Prog. Neu-
prediction algorithm. Select a few im- robiol. 98, 302–315 (2012). https://doi.org/10.1016/
provements, and implement them to see j.pneurobio.2012.03.001
13. M. Cotic, O.C. Zalay, Y. Chinvarun, et al., Mapping
how much AUC is increased compared
the coherence of ictal high frequency oscillations
to results in question 8. If you know that in human extratemporal lobe epilepsy. Epilepsia 56,
sequential state changes are characteristic of 393–402 (2015). https://doi.org/10.1111/epi.12918
seizure episodes, how does that change your 14. R.T. Canolty, R.T. Knight, The functional role
of cross-frequency coupling. Trends Cogn. Sci.
suggested improvements?
14, 506–515 (2010). https://doi.org/10.1016/
j.tics.2010.09.001
15. J. Lisman, The theta/gamma discrete phase code
References occuring during the hippocampal phase preces-
sion may be a more general brain coding scheme.
Hippocampus 15, 913–922 (2005). https://doi.org/
1. D.M. Durand, M. Bikson, Suppression and control
10.1002/hipo.20121
of epileptiform activity by electrical stimulation: A
16. A.B.L. Tort, R. Komorowski, H. Eichenbaum, N.
review. Proc. IEEE 89, 1065–1082 (2001). https://
Kopell, Measuring phase-amplitude coupling be-
doi.org/10.1109/5.939821
tween neuronal oscillations of different frequencies.
2. R. Surges, R.D. Thijs, H.L. Tan, J.W. Sander,
J. Neurophysiol. 104, 1195–1210 (2010). https://
Sudden unexpected death in epilepsy: Risk fac-
doi.org/10.1152/jn.00106.2010
tors and potential pathomechanisms. Nat. Rev.
17. S. Colic, R.G. Wither, M. Lang, et al., Prediction tiotemporal correlation structure. Epilepsy Be-
of antiepileptic drug treatment outcomes using ma- hav. 25, 230–238 (2012). https://doi.org/10.1016/
chine learning. J. Neural Eng. 14, 016002 (2017). j.yebeh.2012.07.007
https://doi.org/10.1088/1741-2560/14/1/016002 31. M. Bandarabadi, C.A. Teixeira, J. Rasekhi, A.
18. M. Guirgis, Y. Chinvarun, M. del Campo, et al., Dourado, Epileptic seizure prediction using rel-
Defining regions of interest using cross-frequency ative spectral power features. Clin. Neurophys-
coupling in extratemporal lobe epilepsy patients. iol. 126, 237–248 (2015). https://doi.org/10.1016/
J. Neural Eng. 12, 026011 (2015). https://doi.org/ j.clinph.2014.05.022
10.1088/1741-2560/12/2/026011 32. Z. Zhang, K.K. Parhi, Seizure prediction using poly-
19. M. Amiri, B. Frauscher, J. Gotman, Interictal nomial SVM classification, in 2015 37th Annual
coupling of HFOs and slow oscillations predicts International Conference of the IEEE Engineering
the seizure-onset pattern in mesiotemporal lobe in Medicine and Biology Society (EMBC), (IEEE,
epilepsy. Epilepsia 60, 1160–1170 (2019). https:// Milan, 2015), pp. 5748–5751
doi.org/10.1111/epi.15541 33. H.-T. Shiao, V. Cherkassky, J. Lee, et al., SVM-
20. J. Theiler, S. Eubank, A. Longtin, et al., Test- based system for prediction of epileptic seizures
ing for nonlinearity in time series: The method from iEEG signal. IEEE Trans. Biomed. Eng.
of surrogate data. Physica D: Nonlinear Phenom- 64, 1011–1022 (2017). https://doi.org/10.1109/
ena 58, 77–94 (1992). https://doi.org/10.1016/0167- TBME.2016.2586475
2789(92)90102-S 34. N. Nicolaou, J. Georgiou, Detection of epilep-
21. A.G. Lalkhen, A. McCluskey, Clinical tests: Sen- tic electroencephalogram based on permutation en-
sitivity and specificity. Contin. Educ. Anaesth. tropy and support vector machines. Expert Syst.
Crit. Care Pain 8, 221–223 (2008). https://doi.org/ Appl. 39, 202–209 (2012). https://doi.org/10.1016/
10.1093/bjaceaccp/mkn041 j.eswa.2011.07.008
22. C.D. Brown, H.T. Davis, Receiver operating charac- 35. Y. Park, L. Luo, K.K. Parhi, T. Netoff, Seizure
teristics curves and related decision measures: A tu- prediction with spectral power of EEG using cost-
torial. Chemom. Intell. Lab. Syst. 80, 24–38 (2006). sensitive support vector machines: Seizure predic-
https://doi.org/10.1016/j.chemolab.2005.05.004 tion with spectral power of EEG. Epilepsia 52,
23. S. Beyenburg, A.J. Mitchell, D. Schmidt, et al., 1761–1770 (2011). https://doi.org/10.1111/j.1528-
Anxiety in patients with epilepsy: Systematic review 1167.2011.03138.x
and suggestions for clinical management. Epilepsy 36. Y. Tang, D.M. Durand, A tunable support vector
Behav. 7, 161–171 (2005). https://doi.org/10.1016/ machine assembly classifier for epileptic seizure
j.yebeh.2005.05.014 detection. Expert Syst. Appl. 39, 3925–3938 (2012).
24. F. Mormann, R.G. Andrzejak, C.E. Elger, K. Lehn- https://doi.org/10.1016/j.eswa.2011.08.088
ertz, Seizure prediction: The long and winding road. 37. L. Chisci, A. Mavino, G. Perferi, et al., Real-
Brain 130, 314–333 (2007). https://doi.org/10.1093/ time epileptic seizure prediction using AR models
brain/awl241 and support vector machines. IEEE Trans. Biomed.
25. L. Kuhlmann, K. Lehnertz, M.P. Richardson, et Eng. 57, 1124–1132 (2010). https://doi.org/10.1109/
al., Seizure prediction—Ready for a new era. Nat. TBME.2009.2038990
Rev. Neurol. 14, 618–630 (2018). https://doi.org/ 38. D. Jacobs, T. Hilton, M. del Campo, et al.,
10.1038/s41582-018-0055-2 Classification of pre-clinical seizure states using
26. P.J. Karoly, H. Ung, D.B. Grayden, et al., The circa- scalp EEG cross-frequency coupling features. IEEE
dian profile of epilepsy improves seizure forecast- Trans. Biomed. Eng. 65, 2440–2449 (2018). https://
ing. Brain 140, 2169–2182 (2017). https://doi.org/ doi.org/10.1109/TBME.2018.2797919
10.1093/brain/awx173 39. R.J. Martis, U.R. Acharya, J.H. Tan, et al., Appli-
27. A. Subasi, EEG signal classification using wavelet cation of empirical mode decomposition (EMD) for
feature extraction and a mixture of expert model. automated detection of epilepsy using EEG signals.
Expert Syst. Appl. 32, 1084–1093 (2007). https:// Int. J. Neural Syst. 22, 1250027 (2012). https://
doi.org/10.1016/j.eswa.2006.02.005 doi.org/10.1142/S012906571250027X
28. A. Ben-Hur, C.S. Ong, S. Sonnenburg, et al., Sup- 40. M.J. Cook, T.J. O’Brien, S.F. Berkovic, et al.,
port vector machines and kernels for computational Prediction of seizure likelihood with a long-term,
biology. PLoS Comput. Biol. 4, e1000173 (2008). implanted seizure advisory system in patients
https://doi.org/10.1371/journal.pcbi.1000173 with drug-resistant epilepsy: A first-in-man study.
29. C.H. Seng, R. Demirli, L. Khuon, D. Bolger, Seizure Lancet Neurol. 12, 563–571 (2013). https://doi.org/
detection in EEG signals using support vector ma- 10.1016/S1474-4422(13)70075-9
chines, in 2012 38th Annual Northeast Bioengi- 41. R.J. Martis, U.R. Acharya, J.H. Tan, et al., Applica-
neering Conference (NEBEC), (IEEE, Philadelphia, tion of intrinsic time-scale decomposition (ITD) to
2012), pp. 231–232 EEG signals for automated seizure prediction. Int.
30. J.R. Williamson, D.W. Bliss, D.W. Browne, J.T. J. Neural Syst. 23, 1350023 (2013). https://doi.org/
Narayanan, Seizure prediction using EEG spa- 10.1142/S0129065713500238
42. U.R. Acharya, S.V. Sree, P.C.A. Ang, et al., Ap- Math. Methods Med. 2017, 1–10 (2017). https://
plication of non-linear and wavelet based features doi.org/10.1155/2017/6849360
for the automated identification of epileptic EEG 55. R.P. Costa, P. Oliveira, G. Rodrigues, et al., Epileptic
signals. Int. J. Neural Syst. 22, 1250002 (2012). seizure classification using neural networks with 14
https://doi.org/10.1142/S0129065712500025 features, in Knowledge-Based Intelligent Informa-
43. N. Landwehr, M. Hall, E. Frank, Logistic model tion and Engineering Systems, ed. by I. Lovrek, R.
trees. Mach. Learn. 59, 161–205 (2005). https:// J. Howlett, L. C. Jain, (Springer, Berlin/Heidelberg,
doi.org/10.1007/s10994-005-0466-3 2008), pp. 281–288
44. E. Kabir, Siuly, Y. Zhang, Epileptic seizure detec- 56. E. Bou Assi, L. Gagliano, S. Rihana, et al., Bispec-
tion from EEG signals using logistic model trees. trum features and multilayer perceptron classifier
Brain Informatics 3, 93–100 (2016). https://doi.org/ to enhance seizure prediction. Sci. Rep. 8 (2018).
10.1007/s40708-015-0030-2 https://doi.org/10.1038/s41598-018-33969-9
45. K.D. Tzimourta, A.T. Tzallas, N. Giannakeas, et al., 57. N.D. Truong, A.D. Nguyen, L. Kuhlmann, et al.,
A robust methodology for classification of epileptic Convolutional neural networks for seizure predic-
seizures in EEG signals. Heal. Technol. 9, 135–142 tion using intracranial and scalp electroencephalo-
(2019). https://doi.org/10.1007/s12553-018-0265-z gram. Neural Netw. 105, 104–111 (2018). https://
46. C. Donos, M. Dümpelmann, A. Schulze-Bonhage, doi.org/10.1016/j.neunet.2018.04.018
Early seizure detection algorithm based on intracra- 58. I. Kuzovkin, R. Vicente, M. Petton, et al., Acti-
nial EEG and random Forest classification. Int. J. vations of deep convolutional neural networks are
Neural Syst. 25, 1550023 (2015). https://doi.org/ aligned with gamma band activity of human visual
10.1142/S0129065715500239 cortex. Commun. Biol. 1 (2018). https://doi.org/
47. T. Zhang, W. Chen, M. Li, AR based quadratic 10.1038/s42003-018-0110-y
feature extraction in the VMD domain for the 59. H. Khan, L. Marcuse, M. Fields, et al., Focal on-
automated seizure detection of EEG using ran- set seizure prediction using convolutional networks.
dom forest classifier. Biomed. Signal Process. Con- IEEE Trans. Biomed. Eng. 65, 2109–2118 (2018).
trol 31, 550–559 (2017). https://doi.org/10.1016/ https://doi.org/10.1109/TBME.2017.2785401
j.bspc.2016.10.001 60. X. Wei, L. Zhou, Z. Chen, et al., Automatic seizure
48. F. Manzouri, S. Heller, M. Dümpelmann, et al., detection using three-dimensional CNN based on
A comparison of machine learning classifiers for multi-channel EEG. BMC Med. Inform. Decis.
energy-efficient implementation of seizure detec- Mak. 18 (2018). https://doi.org/10.1186/s12911-
tion. Front. Syst. Neurosci. 12 (2018). https:// 018-0693-8
doi.org/10.3389/fnsys.2018.00043 61. P. Mirowski, D. Madhavan, Y. LeCun, R. Kuzniecky,
49. U.R. Acharya, S.L. Oh, Y. Hagiwara, et al., Deep Classification of patterns of EEG synchroniza-
convolutional neural network for the automated de- tion for seizure prediction. Clin. Neurophysiol.
tection and diagnosis of seizure using EEG signals. 120, 1927–1940 (2009). https://doi.org/10.1016/
Comput. Biol. Med. 100, 270–278 (2018). https:// j.clinph.2009.09.002
doi.org/10.1016/j.compbiomed.2017.09.017 62. P.W. Mirowski, Y. LeCun, D. Madhavan, R.
50. N. Sriraam, S. Raghu, K. Tamanna, et al., Au- Kuzniecky, Comparing SVM and convolutional net-
tomated epileptic seizures detection using multi- works for epileptic seizure prediction from intracra-
features and multilayer perceptron neural network. nial EEG, in 2008 IEEE Workshop on Machine
Brain Informatics 5 (2018). https://doi.org/10.1186/ Learning for Signal Processing, (IEEE, Cancun,
s40708-018-0088-8 2008), pp. 244–249
51. A. Subasi, E. Erçelebi, Classification of EEG sig- 63. A. Petrosian, D. Prokhorov, R. Homan, et al., Re-
nals using neural network and logistic regression. current neural network based prediction of epileptic
Comput. Methods Prog. Biomed. 78, 87–99 (2005). seizures in intra- and extracranial EEG. Neurocom-
https://doi.org/10.1016/j.cmpb.2004.10.009 puting 30, 201–218 (2000). https://doi.org/10.1016/
52. R. Abbasi, M. Esmaeilpour, Selecting statistical S0925-2312(99)00126-5
characteristics of brain signals to detect epileptic 64. S. Raghu, N. Sriraam, G.P. Kumar, Classification of
seizures using discrete wavelet transform and per- epileptic seizures using wavelet packet log energy
ceptron neural network. International Journal of In- and norm entropies with recurrent Elman neural net-
teractive Multimedia and Artificial Intelligence 4, 33 work classifier. Cogn. Neurodyn. 11, 51–66 (2017).
(2017). https://doi.org/10.9781/ijimai.2017.456 https://doi.org/10.1007/s11571-016-9408-y
53. M. Alfaro-Ponce, A. Argüelles, I. Chairez, Pat- 65. L. Vidyaratne, A. Glandon, M. Alam, K.M.
tern recognition for electroencephalographic sig- Iftekharuddin, Deep recurrent neural network
nals based on continuous neural networks. Neu- for seizure detection, in 2016 International Joint
ral Netw. 79, 88–96 (2016). https://doi.org/10.1016/ Conference on Neural Networks (IJCNN), (IEEE,
j.neunet.2016.03.004 Vancouver, 2016), pp. 1202–1207
54. Y. Wang, Z. Li, L. Feng, et al., Automatic detec- 66. Z. Yu, D.S. Moirangthem, M. Lee, Continuous
tion of epilepsy and seizure using multiclass sparse timescale long-short term memory neural
extreme learning machine classification. Comput. network for human intent understanding. Front.
Neurorobot. 11 (2017). https://doi.org/10.3389/ 79. S. Baldassano, D. Wulsin, H. Ung, et al., A novel

fnbot.2017.00042 seizure detection algorithm informed by hidden
67. K.M. Tsiouris, V.C. Pezoulas, M. Zervakis, et al., A Markov model event states. J. Neural Eng. 13,
long short-term memory deep learning network for 036011 (2016). https://doi.org/10.1088/1741-2560/
the prediction of epileptic seizures using EEG sig- 13/3/036011
nals. Comput. Biol. Med. 99, 24–37 (2018). https:// 80. C.E. Solorzano-Espindola, B. Tovar-Corona, A.
doi.org/10.1016/j.compbiomed.2018.05.019 Anzueto-Rios, Pediatric seizure forecasting using
68. P. Thodoroff, J. Pineau, A. Lim, Learning robust nonlinear features and Gaussian mixture hidden
features using deep learning for automatic seizure Markov models on scalp EEG signals, in 2018 15th
detection (2016). arXiv:160800220 [cs] International Conference on Electrical Engineering,
69. M. Li, W. Chen, T. Zhang, Classification of epilepsy Computing Science and Automatic Control (CCE),
EEG signals using DWT-based envelope analysis (IEEE, Mexico City, 2018), pp. 1–6
and neural network ensemble. Biomed. Signal Pro- 81. M. Hejazi, A. Motie Nasrabadi, Prediction of
cess. Control 31, 357–365 (2017). https://doi.org/ epilepsy seizure from multi-channel electroen-
10.1016/j.bspc.2016.09.008 cephalogram by effective connectivity analysis us-
70. I. Ullah, M. Hussain, E.-H. Qazi, H. Aboalsamh, An ing granger causality and directed transfer function
automated system for epilepsy detection using EEG methods. Cogn. Neurodyn. (2019). https://doi.org/
brain signals based on deep learning approach. Ex- 10.1007/s11571-019-09534-z
pert Syst. Appl. 107, 61–71 (2018). https://doi.org/ 82. J.A. Dian, S. Colic, Y. Chinvarun, et al., Identi-
10.1016/j.eswa.2018.04.021 fication of brain regions of interest for epilepsy
71. E. Abdulhay, V. Elamaran, M. Chandrasekar, et surgery planning using support vector machines,
al., Automated diagnosis of epilepsy from EEG in 2015 37th Annual International Conference
signals using ensemble learning approach. Pat- of the IEEE Engineering in Medicine and Bi-
tern Recogn. Lett. (2017). https://doi.org/10.1016/ ology Society (EMBC), (IEEE, Milan, 2015),
j.patrec.2017.05.021 pp. 6590–6593
72. B.H. Brinkmann, J. Wagenaar, D. Abbot, et al., 83. B. Elahian, M. Yeasin, B. Mudigoudar, et al., Identi-
Crowdsourcing reproducible seizure forecasting in fying seizure onset zone from electrocorticographic
human and canine epilepsy. Brain 139, 1713–1722 recordings: A machine learning approach based on
(2016). https://doi.org/10.1093/brain/aww045 phase locking value. Seizure 51, 35–42 (2017).
73. U. Orhan, M. Hekim, M. Ozer, EEG signals clas- https://doi.org/10.1016/j.seizure.2017.07.010
sification using the K-means clustering and a mul- 84. S.B. Tomlinson, B.E. Porter, E.D. Marsh, Interic-
tilayer perceptron neural network model. Expert tal network synchrony and local heterogeneity pre-
Syst. Appl. 38, 13475–13481 (2011). https://doi.org/ dict epilepsy surgery outcome among pediatric pa-
10.1016/j.eswa.2011.04.149 tients. Epilepsia 58, 402–411 (2017). https://doi.org/
74. J. Martinez-del-Rincon, M.J. Santofimia, X. del 10.1111/epi.13657
Toro, et al., Non-linear classifiers applied to EEG 85. I.A. Nissen, C.J. Stam, E.C.W. van Straaten, et al.,
analysis for epilepsy seizure detection. Expert Syst. Localization of the epileptogenic zone using inter-
Appl. 86, 99–112 (2017). https://doi.org/10.1016/ ictal MEG and machine learning in a large cohort
j.eswa.2017.05.052 of drug-resistant epilepsy patients. Front. Neurol. 9
75. T. Wen, Z. Zhang, Deep convolution neural (2018). https://doi.org/10.3389/fneur.2018.00647
network and autoencoders-based unsupervised 86. J. Jacobs, M. Zijlmans, R. Zelmann, et al., Value
feature learning of EEG signals. IEEE Access of electrical stimulation and high frequency oscil-
6, 25399–25410 (2018). https://doi.org/10.1109/ lations (80-500 Hz) in identifying epileptogenic ar-
ACCESS.2018.2833746 eas during intracranial EEG recordings. Epilepsia
76. M.-P. Hosseini, H. Soltanian-Zadeh, K. Elisevich, D. 51, 573–582 (2010). https://doi.org/10.1111/j.1528-
Pompili, Cloud-based deep learning of big EEG data 1167.2009.02389.x
for epileptic seizure prediction, in 2016 IEEE Global 87. E. Ben-Menachem, J.W. Sander, M. Privitera, F.
Conference on Signal and Information Processing Gilliam, Measuring outcomes of treatment with
(GlobalSIP), (IEEE, Washington, DC, 2016), pp. antiepileptic drugs in clinical trials. Epilepsy Be-
1151–1155 hav. 18, 24–30 (2010). https://doi.org/10.1016/
77. H. Daoud, M.A. Bayoumi, Efficient epileptic seizure j.yebeh.2010.04.001
prediction based on deep learning. IEEE Trans. 88. R.G. Wither, S. Colic, C. Wu, et al., Daily rhythmic
Biomed. Circuits Syst. 13, 804–813 (2019). https:/ behaviors and thermoregulatory patterns are dis-
/doi.org/10.1109/TBCAS.2019.2929053 rupted in adult female MeCP2-deficient mice. PLoS
78. G. Zhu, Y. Li, P. Wen, et al., Unsupervised classi- One 7, e35396 (2012). https://doi.org/10.1371/
fication of epileptic EEG signals with multi scale journal.pone.0035396
K-means algorithm, in Brain and Health Informat- 89. L. van der Maaten, G. Hinton, Visualizing Data
ics, ed. by K. Imamura, S. Usui, T. Shirao, et al., using t-SNE. J. Mach. Learn. Res. 9, 2579–2605
(Springer, Cham, 2013), pp. 158–167 (2008)
90. M. Brodie, S. Barry, G. Bamagous, J. Norrie, P. 98. B. Nakisa, M.N. Rastgoo, D. Tjondronegoro, V.
Kwan, Patterns of treatment response in newly di- Chandran, Evolutionary computation algorithms
agnosed epilepsy. Neurology 78(20), 1548–1554 for feature selection of EEG-based emotion
(2012) recognition using mobile sensors. Expert Syst.
91. J. Liang, R. Lu, C. Zhang, F. Wang, Predicting Appl. 93, 143–155 (2018). https://doi.org/10.1016/
seizures from electroencephalography recordings: j.eswa.2017.09.062
A knowledge transfer strategy, in 2016 IEEE In- 99. T.T. Erguzel, S. Ozekes, O. Tan, S. Gultekin, Fea-
ternational Conference on Healthcare Informatics ture selection and classification of electroencephalo-
(ICHI), (IEEE, Chicago, 2016), pp. 184–191 graphic signals: An artificial neural network and
92. Q. She, B. Hu, H. Gan, et al., Safe semi-supervised genetic algorithm based approach. Clin. EEG Neu-
extreme learning machine for EEG signal classifica- rosci. 46, 321–326 (2015). https://doi.org/10.1177/
tion. IEEE Access 6, 49399–49407 (2018). https:// 1550059414523764
doi.org/10.1109/ACCESS.2018.2868713 100. P. Mesejo, O. Ibáñez, E. Fernández-Blanco, et al.,
93. M.K. Islam, A. Rastegarnia, Z. Yang, Methods for Artificial neuron–glia networks learning approach
artifact detection and removal from scalp EEG: based on cooperative coevolution. Int. J. Neural
A review. Neurophysiol. Clin./Clin. Neurophys- Syst. 25, 1550012 (2015). https://doi.org/10.1142/
iol. 46, 287–305 (2016). https://doi.org/10.1016/ S0129065715500124
j.neucli.2016.07.002 101. G.G. Somjen, H. Kager, W.J. Wadman, Computer
94. I. Kiral-Kornek, S. Roy, E. Nurse, et al., Epileptic simulations of neuron-glia interactions mediated by
seizure prediction using big data and deep ion flux. J. Comput. Neurosci. 25, 349–365 (2008).
learning: Toward a Mobile system. EBioMedicine https://doi.org/10.1007/s10827-008-0083-9
27, 103–111 (2018). https://doi.org/10.1016/ 102. V. Grigorovsky, B.L. Bardakjian, Low-to-high
j.ebiom.2017.11.032 cross-frequency coupling in the electrical rhythms
95. L. Kuhlmann, A.N. Burkitt, M.J. Cook, et al., as biomarker for Hyperexcitable neuroglial
Seizure detection using seizure probability estima- networks of the brain. IEEE Trans. Biomed. Eng.
tion: Comparison of features used to detect seizures. 65, 1504–1515 (2018). https://doi.org/10.1109/
Ann. Biomed. Eng. 37, 2129–2145 (2009). https:// TBME.2017.2757878
doi.org/10.1007/s10439-009-9755-5 103. A. van den Oord, S. Dieleman, H. Zen, et al.,
96. R. Chai, Y. Tran, G.R. Naik, et al., Classification WaveNet: A generative model for raw audio (2016).
of EEG based-mental fatigue using principal com- arXiv:160903499 [cs]
ponent analysis and Bayesian neural network, in 104. Y.N. Dauphin, A. Fan, M. Auli, D. Grangier, Lan-
2016 38th Annual International Conference of the guage modeling with gated convolutional networks
IEEE Engineering in Medicine and Biology Society (2016). arXiv:161208083 [cs]
(EMBC), (IEEE, Orlando, 2016), pp. 4654–4657 105. S. Bai, J.Z. Kolter, V. Koltun, An empirical
97. Y. Zhao, B. Ahmed, T. Thesen, et al., A non- evaluation of generic convolutional and recur-
parametric approach to detect epileptogenic lesions rent networks for sequence modeling (2018).
using restricted Boltzmann machines, in Proceed- arXiv:180301271 [cs]
ings of the 22nd ACM SIGKDD International Con-
ference on Knowledge Discovery and Data Mining –
KDD’16, (ACM Press, San Francisco, 2016), pp.
373–382
Retinal Prosthesis
20
James Weiland and Mark S. Humayun
Abstract
20.1 Introduction
Retinal prostheses can provide a sense of
sight to people with severe visual impairment A microelectronic retinal prosthesis is a
due to retinal photoreceptor degeneration. bioelectronic system designed to address one
Several devices have been evaluated in of medicine’s most vexing problems; loss of
humans, and some devices have received sight due to photoreceptor degeneration. Other
regulatory approval. Clinical research studies causes of blindness have been more successfully
have shown that people with retinal implants addressed. Cataracts (opacity in the lens) are
have improved navigation skills but cannot routinely treated by removal of the aged opaic
read letters in a normal way (rather it takes lens and replacing it with an artificial lens.
them several seconds to recognize a letter). Glaucoma is managed initially with topical eye
Improvements in visual acuity may be possible drops, but the loss of photoreceptors cannot
through denser electrode arrays or image currently be treated. Retinal prosthetic systems,
processing strategies that yield more focus, based on the principle of electrical activation of
natural responses from the retina. nerve cells by a device implanted near the retina
(Fig. 20.1), have advanced from experimental
Keywords implants to medical devices with regulatory
Retinal prosthesis · Blindness · Retinal approval [1]. This chapter will cover the state
degeneration · Photoreceptors · Artificial of the art in retinal prostheses. A discussion
vision of normal eye anatomy will set the physical
constraints on the implantable device. A review
of retina disease will demonstrate that in a large
number of blind, the retina remains populated
Electronic Supplementary Material: The online version with electrically excitable cells even when the
6_20) contains supplementary material, which is available light sensitive cells are virtually absent. The main
to authorized users. results from retinal prosthesis human trials will
be reviewed, followed by discussion of technical
J. Weiland()
Biomedical Engineering and Ophthalmology, University
advances in cameras, image processing, and
of Michigan, Ann Arbor, MI, USA electrode arrays.
M. S. Humayun
Ophthalmology, Biomedical Engineering, and Integrative
Anatomical Sciences, Ginsburg Institute for Biomedical
Therapeutics, USC Roski Eye Institute, University of
Southern California, Los Angeles, CA, USA

568 J. Weiland and M. S. Humayun
Fig. 20.1 Retinal prosthesis array locations. (a) A di- the space previously occupied by photoreceptors (prior to
agram for the human visual system (b) An epiretinal degeneration) (d) A suprachoroidal array is between the
array rests on the surface of the retina, near the ganglion sclera and choroid, and the choroid separates the array
cells. (c) A subretinal array is underneath the retina, in from the retina. (From E. Zrenner [41]. Illustration by:
Credit: V. Altounian/Science Translational Medicine)
of human eye is on average 2.5 cm, shown in

20.2 Basic Anatomy of the Eye the cross section in Fig. 20.2. The eye’s shape
and Retina is maintained in part by a complex intraocular
fluid flow mechanism that maintains the eye
The anatomical information below is derived pressure. The cornea and sclera form most
from two sources, except where noted: Wolff’s of the outer layer of the eye, with the cornea
Anatomy of the Eye and Adler’s Physiology of in the front of the eye and the sclera on the
the Eye: Clinical Applications. The diameter sides and back of the eye. Both cornea and
20 Retinal Prosthesis 569
Fig. 20.2 Cross section of

the eye
sclera are composed mostly of collagen, but the vitreous cavity since at this point the sclera can
regular arrangement and diameter of the fibers be incised without damaging the retina or ciliary
in the cornea allows transparency while irregular body. The retina is a multilayer neural tissue (Fig.
collagen fiber patterns make the sclera opaque. 20.3). Between the retina and the choriocapillaris
The next layer of the eye is the uvea, consisting of is the retinal pigment epithelium, which regulates
the choriocapillaris, the ciliary body, and the iris. the exchanges of nutrients and waste between the
The uveal tissue is highly vascularized. In fact, the retina and choriocapillaris. Photoreceptors are the
choriocapillaris, the blood vessels which supply light sensing cells of the retina. The photoreceptor
nutrients to the outer retina, has the highest per outer segments are next to the RPE. Two types
unit volume of blood in the human body. Behind of photoreceptors are in the human retina: rods
the iris is the crystalline lens. The lens and the and cones. The rods detect very dim light and
cornea form the optical system of the eye. The are used in night vision. The cones, which are
vitreous cavity, space between the lens and the further subdivided into red, green, and blue cones,
retina, is filled with the vitreous. The vitreous operate at ambient daylight levels and mediate
is clear, composed of 99% physiological saline color vision. The nuclei of the photoreceptor cells
and 1% hyaluronic acid. The vitreous cavity has are in the outer nuclear layer. The outer plexi-
a volume of 6 cm3 . The geometric axis of the form layer has synapses between the outer nuclear
eye connects the opposite ends of the sphere. The layer and the bipolar cells. Horizontal cells form
visual axis of the eye connects the fovea of the lateral connections in the outer plexiform layer.
retina (detailed below) to the fixation point in the Horizontal and bipolar cell somata are in the
visual field. inner nuclear layer. The bipolar cells, in turn,
The retina lines the back half of the eye, as the synapse in the inner plexiform layer to ganglion
innermost layer. It terminates at the ora serrata, cells. Amacrine cells form lateral connections in
3–5 mm before the insertion of the ciliary body the ganglion cell layer. Amacrine cell somata are
(Fig. 20.2). This 3–5 mm space is called the pars found in both the inner nuclear layer and ganglion
plana and is the preferred surgical approach to the cell layer; in the latter case, amacrine cells are
Fig. 20.3 Cross section of the retina
called “displaced” amacrine cells. Finally, the cells respond to a transition from light to dark.
axons of the ganglion cells form the nerve fiber Other types of bipolar cell responses exist. The
layer. The nerve fibers coalesce at the optic disk to depolarization of the bipolar cell leads to a
form the optic nerve, which transmits information response in the ganglion cell. Instead of a graded
to the brain. In primates, ganglion cell axons are potential, ganglion cells fire action potentials. In
unmyelinated until the optic disk. general, the number and rate of action potentials
Signal transmission through the retina is well corresponds to the strength of depolarization of
studied. An excellent online resource for all the bipolar cell. Thus, this synapse performs a
things retina and vision is webvision (http:// type of analog-to-digital conversion. There are at
webvision.med.utah.edu/), maintained by Kolb, least 18 types of ganglion cells in human retina.
Fernandez, Nelson, and Jones. A photon incident Similar to bipolar cells, ganglion cells respond
upon the photoreceptor outer segment initiates in an on and off fashion. Some ganglion cells
the process called phototransduction. The result respond to the direction of motion of a light
of the process is hyperpolarization of the pattern. Amacrine cells modulate the synapse
photoreceptor. These sensory neurons are unusual between bipolar and ganglion cells and can inhibit
in that they hyperpolarize in response to stimuli. ganglion cell activity.
The photoreceptors pass this signal to the bipolar Glial cells in the retina include astrocytes
cells, which respond with graded potentials, and Mueller cells. Astrocytes are similar to
meaning the degree of depolarization of the those found in other areas of the CNS, while
bipolar cell will be in roughly proportional with Mueller cells are particular to the retina.
the photoreceptor hyperpolarization. Horizontal Mueller cells have a physiological function
cells modulate the photoreceptor and bipolar that includes buffering extracellular potassium.
cells through inhibitory connections. The well- Mueller cell end plates form part of the external
studied “center-surround” organization of the and internal limiting membranes as well as
retina begins at this stage. ON bipolar cells the internal limiting membrane. The external
respond to the onset of light and OFF bipolar limiting membrane separates the photoreceptor
outer segments from the outer nuclear layer and

serves as selective diffusion barrier between
these two areas. The internal limiting membrane
(ILM) forms the barrier between the retina and
the vitreous. The ILM also contains collagen
fibrils and proteoglycans from the vitreous. Both
astrocytes and Mueller cells will hypertrophy in
response to injury or as consequence of retinal
degeneration.
Cells of the primate retina vary in density,
structure, and function depending on the position
relative to the fovea (Fig. 20.4). The fovea is the
part of the primate retina considered the center of
the visual field and is capable of the highest acuity
vision. In mammals, only primates have a fovea,
although most mammals do have a specialized
central area with better visual acuity than the
peripheral retina. When the eye moves to direct
gaze on an object, it is positioning the eye so that
the fovea can gather detailed information. The
fovea has only cone photoreceptors. In the fovea,
the other layers of the retina are laterally dis-
placed so that light scattering does not affect the
quality of the image. Each foveal cone connects
to a single bipolar cell which connects to a single
ganglion cell. The structure and function of the
retina change with increasing eccentricity from Fig. 20.4 Top: The back of the eye as viewed through the
the fovea (moving peripherally away from the pupil. Circle 1 represents the fovea (shown in histological
fovea on the retina). Rods become more promi- cross section in the bottom frame). At the fovea, only
photoreceptors are present. Circle 2 is the macula, the
nent. The dendritic tree and receptive field size central 15◦ of vision
for a ganglion cell increases. In contrast to the
fovea, thousands of rods send convergent input to
a peripheral single ganglion cell. The density of power. The fovea projects to a disproportionately
cones in the periphery is 100 times less than the large area of visual cortex (compared to other
cone density in the fovea. As such, visual acuity areas of the retina), a mechanism called cortical
is lower in the periphery. magnification where a large number of cortical
If only a small central part of the retina can neurons process information from small part of
provide high acuity vision, then how is it possi- the visual field. If the entire retina required this
ble to know someone’s face in detail, appreciate much cortical area, the optic nerve and cortex
art and sunsets, and in general have a detailed would be enormous.
understanding of our surroundings? The answer The role then of the peripheral retina is to
lies in the movement of the eye over a scene and identify gross targets and movement and to cue
the coordination of eye and head movement with the oculomotor system to direct eye gaze to this
the visual system, to create overall perception area. Models of human attention have been de-
and understanding. But why not simply have a veloped that successfully predict where a person
high resolution retina over the entire visual field? will look when presented with an image. These
The reason only the fovea has high acuity vision models are based on movement, intensity, color,
relates to the efficient use of cortical processing and orientation. In general, if an object stands out
from the background, this draws the attention of macula only 4% of photoreceptors remained
the visual system. but 80% of inner retina and 30% of ganglion
cells remained [4]. In extramacular regions, only
40% of inner retina remained. In AMD, 90%
20.2.1 Eye Disease of the inner retina cells remained compared to
age-matched controls [5]. Thus, by measures
Incurable eye diseases result in blindness of cell counting, the inner retina in RP and
for 100,000 individuals each year. A retinal AMD appears to be less affected by disease
prosthesis will require the presence of a relatively compared to the photoreceptors. Recent studies
high percentage of healthy cells in the inner using computational phenotyping, however,
retina. Therefore, diseases that affect the outer suggest that the inner retina undergoes significant
retina are potentially treatable with a retinal remodeling during retinal degeneration [6].
prosthesis. The two most common outer retinal Further, Mueller cell hypertrophy leads to a
degenerative diseases are age-related macular seal between the subretinal space and the inner
degeneration (AMD) and retinitis pigmentosa retina. In summary, RP and AMD do not result in
(RP). AMD is more prevalent, but RP is more complete degeneration of the retina, but, at least
severe. in the case of RP, it may not be correct to assume
AMD results from abnormal aging of the RPE that the information processing circuitry of the
and retina [2]. There are approximately 700,000 inner retina is intact.
new AMD patients in the US each year, 10%
of which will result in legal blindness. Common
symptoms include the formation of yellow de- 20.2.2 Retinal Prosthesis
posits (drusen) under the RPE and proliferation
of leaky blood vessels in the subretinal space. An electronic retinal prosthesis must perform sev-
The types and severity of AMD are defined by eral basic functions in order to replace the sense of
these symptoms. Persons with AMD will start to vision. First, it must detect light in the nearby en-
have distorted central vision and if not treated vironment of the implant patient. The light must
could lose most vision in the central 20–30◦ . A be converted to an electrical stimulus. Next, the
number of treatments such as intravitreal injec- artificial electrical stimulus must be delivered to
tions of anti-VEGF (vascular endothelial growth the retina and evoke a response. As a system, the
factor) drugs have shown effectiveness in treating prosthesis must be safe and effective, acceptable
neovascular AMD, no cure exists. to the patient, and functional in the human body
Retinitis pigmentosa is a collective name for for decades.
almost 200 genetic defects that results in photore- Several artificial vision approaches have been
ceptor loss [3]. The overall incidence of RP is 1 proposed. While this chapter will focus on retinal
in 4000 live births. The rod-cone variant of RP prostheses, other approaches bear mentioning.
attacks the rods first resulting in night blindness. Visual cortical prostheses have been pursued by
Cone vision is lost first in the mid-periphery a number of investigators since the 1960s [7, 8].
followed by complete peripheral vision loss. In This approach has the potential to aid individ-
some cases, RP patients can maintain near normal uals with blindness due to a wider variety of
visual acuity in the fovea and parafovea, but have disease, since the condition of the retina is not
no vision in other parts of the retina. There is no a factor. Diseases such as glaucoma and diabetic
treatment or cure for RP. retinopathy, which damage the inner retina and
Postmortem evaluation of retina with RP or optic nerve, would potentially be treatable. The
AMD has shown significantly better survival of fovea projection to the visual cortex is greatly
cells in the neural retina (ganglion cell layer expanded, so a cortical prosthesis can put more
and inner nuclear layer) versus the sensory electrodes in cortex that represents the fovea than
retina (photoreceptors). In severe RP, in the a retinal prosthesis can put in the actual fovea.
Thus, a cortical prosthesis has potential for higher The advantages of the epiretinal approach in-
acuity vision provided electrodes can be placed clude the following: (1) the surgery is less com-
deep into the calcarine fissure without severing plex and (2) the retinal array can be larger (vs.
blood vessels which could lead to mortality and a subretinal approach) The disadvantages of this
potentially morbidity. Hence, a cortical prosthesis approach include the following: (1) retinal arrays
will have a higher threshold for safety than a have proven to separate from the retina [10],
retinal prosthesis, due to the higher rate of mor- causing increased perceptual thresholds and (2)
bidity and mortality associated with brain surgery activation of axons of passage leads to distorted
vs. eye surgery. Additionally, a cortical visual perceptions [11].
prosthesis will have to account for the significant The subretinal approach to the retinal pros-
signal processing that occurs in both the retina thesis involves implanting a stimulating electrode
and lateral geniculate nucleus. Positioning elec- array between the remnant retina and retinal pig-
trodes precisely in primary visual cortex is also ment epithelium [12, 13]. This is accomplished
a challenge. Recently, a visual cortex prostheses surgically via an incision of the sclera, chorio-
(Orion, Second Sight Medical Products, Sylmar, capillaris, and RPE. There are distinct advantages
CA) has begun a feasibility trial in humans. At and disadvantages to the subretinal prosthesis
the time of this chapter writing, no peer-reviewed approach. Advantages include closer proximity
report has described the outcome from the Orion to surviving neurons at the earliest point in the
feasibility trial. visual pathway (i.e., bipolar cell). Stimulation
An optic nerve prosthesis was tested in two at the bipolar cell level avoids the problem of
subjects. A remarkable amount of visual function axon stimulation. Placing an electrode interface
was apparent even with only four electrodes avail- in the subretinal space will use the retina to hold
able to stimulate the optic nerve [9]. The patient the electrode in close proximity to the electrode.
scanned the scene in front of them with a head- The disadvantages include the limited subretinal
mounted video camera to distinguish light from space, which limits the size of the array, since
dark. A significant amount of time (30–40 sec- a larger array will increase the risk of retinal
onds) was required to make pattern discrimina- detachment. Another issues with subretinal arrays
tions, but a high rate of accuracy was achieved. is the need to power the implant will require
Nevertheless, access to a retinotopic map in the either a cable will have to traverse the sclera
optic nerve was difficult with a cuff electrode, and choroid or an external projector to send high
since the cuff electrode was on the outside of the intensity light into the eye (if the subretinal array
nerve and focal stimulation would have required has microphotodiodes). A cable traversing the
an electrode penetrating into the nerve. highly vascularized choroid, over the long term,
Retinal implants have distinct advantages over increases the likelihood of subretinal hemorrhage
optic nerve and cortical implants with regard to and total or local retinal detachment. In the latter
surgical implantation and access to target nerve case, the subretinal fluid would increase the dis-
cells. The retina lines the back of the eye and tance between the underlying electrode and the
the mapping of the retina to a physical loca- retinal neurons and therefore increase the current
tion in space is well known. The types of reti- requirements.
nal implants are defined mainly by the anatom- A third approach in terms of electrode array
ical location of the stimulating electrode array. position is suprachoroidal transretinal stimula-
The array can be on the epiretinal surface or in tion (STS) [14, 15]. Placing the stimulating elec-
the subretinal space (Fig. 20.1). The epiretinal trode in the suprachoroidal space (between the
surface is the internal limiting membrane. The sclera and choroid) may allow for a less-invasive
subretinal space is between the retina and the method to achieve functional percepts. The only
RPE, in the place of the photoreceptors, which intraocular component would be a return elec-
are absent in the patients who would receive an trode in the vitreous cavity, to ensure that stimulus
implant. current passes through the retina. This approach
has several advantages. First of all, the surgery is read large letters and demonstrated visual acuity
less complicated. Second, the electrodes are less of approximately 20/540 using a Landolt C test
invasive to the retina. Third, the electrodes are [12]. In contrast to a passive subretinal device
relatively easy to remove or replace if damaged. [16], the active device clearly shows that chip-
However, because the electrodes are further from mediated vision is possible if a subretinal chip is
the target neurons, they should need to deliver adequately powered. Using this device, patients
higher currents and the current spread should be could identify letters and common objects, but
greater, limiting the resolution. An STS system only when in context. This means that the subjects
has been implanted in blind by two groups. Func- could identify letters if they were told that they
tional testing suggests that electrically elicited were looking at letters and objects if they were
percepts are possible, but with stimulus currents given additional information, such as objects on a
generally greater than those reported with epireti- dining table (e.g., knife and plate).
nal or subretinal implants. The ARGUS I device was a 16-electrode
device, with electronics based on cochlear
implant technology. Six subjects were implanted
20.2.3 Clinical Studies in a feasibility study. Subjects were reported
to have thresholds well within published safe
This section will review current status of retinal limits for platinum metal. Perceptual threshold
prostheses that have received regulatory approval correlated with separation (i.e., lift-off) between
or are in active clinical trials. Chronic implan- the electrode array and the retina [17]. In addition,
tation refers to devices left in the test subjects increasing frequency of pulses lowered the charge
for some period of time. For a device to proceed per pulse in a predictable way [18]. Subjects
to this step requires considerable engineering in could perform select visual tasks using the device,
order to manufacture the device such that regu- such as identifying objects like a knife, a plate, or
latory agencies will permit surgical implantation. a cup from a set and detecting the direction of a
All studies were performed with approval of the moving bar [19]. In one subject, a grating visual
appropriate governmental body. acuity measure was used to demonstrate that the
Retina Implant, GmbH has developed an sub- best visual acuity using the retinal prosthesis
retinal microphotodiode array, powered and pro- was the maximum allowable by the spacing of
grammed by an extraocular source (via a cable electrodes on the array (i.e., 20/4000). A 10-year
across the eyewall). The first version of this sys- follow-up study in one patient showed that the
tem used a percutaneous connector for power. The ability to create the perception of light remained
wireless version developed later used an implant after a decade of implantation [20].
behind the ear, wirelessly powered, to provide The ARGUS II has 60 electrodes, and the en-
voltage and configuration data for the subretinal tire device fits inside the orbit. An external cam-
circuit. The device has 1500 microphotodiodes era unit delivers image information wirelessly to
each with amplification and stimulation circuitry the implant [21]. The implanted electronic com-
as well as an electrode for output. The light de- ponents are sutured to the sclera under the rectus
tected by the photodiode serves as a scalar for muscles, and a ribbon cable enters the eye via a
stimulus voltage output. The initial series of im- pars plana incision and is tacked to the retinal
plants had a multiwire cable running from the surface resulting in the placement of 60 electrodes
eye to a transcutaneous connector behind the ear, onto the epiretinal surface. The ARGUS II retinal
while a more recent set of experimental implants prosthesis was evaluated in a multicenter clinical
has an implanted electronic module for power trial. Thirty subjects were enrolled between June
generation (via an inductive link). One advantage 2007 and August 2009. All subjects were able
that this device has is that the imaging functional- to perceive light during electrical stimulation. On
ity of the implant is in the eye and hence coupled object localization tests using a target of a 7 cm
with eye movements. The best patient was able to white square on a black LCD screen at 30 cm dis-
tance, 27 out of 28 subjects (96%) performed bet- Bionic Vision Australia and Nidek are two
ter in localizing the object with System ON ver- research/industry teams that each has advanced
sus OFF, and no subjects performed significantly suprachoroidal implant approaches. Both have
better with the System OFF [22]. In a test that tested implants in patients and demonstrated
requires motion detection as well as orientation the ability to evoke phosphenes [15, 27]. As
of objects, using a target of a white bar moving expected, visual acuity is low and thresholds
across a black LCD screen, 16 out of 28 subjects are high. However, the subjects could use the
(57%) perform this test better with System ON perceived light to detect actions. This approach
versus OFF. The best result visual acuity result to represents an attractive alternative, if intraocular
date reported in a peer-reviewed journal is 20 of approaches continue to show limited ability to
1200 using grating acuity [23], but recently vision generate form vision.
of 20 of 480 has been demonstrated using an
ETDRS eye chart (personal communication Dr.
Yoon, Seoul Korea). Letter reading was tested in 20.3 Retinal Prostheses Research
22 of 30 subjects. Six of these subjects were able
to identify any letter of the alphabet at a 63.5% Retinal implants have successfully restored the
success rate (vs. 9.5% with the system off). In perception of light in humans with almost no
all 22 subjects, a small set of eight letters was vision. Patients with implants report improved
identified 72.5% correctly, vs. 16.8% with the mobility and enjoy “being visual” again. How-
system off. Subjects were free to take as much ever, the improvements are small to date, and
time as needed to make a judgment. Subjects better technology is needed to allow more func-
provided answers after 100 seconds in the full tion to be provided by retinal implants. Here,
alphabet and 44 seconds in the limited letter set we review potential technological innovations in
[24]. Follow-up studies on this cohort suggest ac- camera, image/video processing, and electrode
ceptable long-term safety [21], and some benefit array, which may improve overall performance.
from the amount of restored vision [25]. Based
on the clinical trial results, Argus II received a
CE Mark in 2011, allowing sale in Europe, and 20.3.1 Camera
a Humanitarian Device Exemption from the US
Food and Drug Administration in 2013, allowing Imagers for retinal prosthesis have been imple-
sale in the US. mented as external cameras mounted on glasses,
A subretinal implant system named Prima has implanted cameras in the place of the crystalline
been tested in patients with Age-Related Macular lens [28], and microphotodiode arrays implanted
Degeneration. The implanted part of this system under the retina [26, 29]. Implanting the imager
is relatively simple, and it consists of an array in the eye offers the clear advantage of placing
of infrared (IR) sensitive microphotodiodes on a the imager direction under control of eye move-
single silicon chip, about 2 mm in diameter [26]. ment. Implanted cameras are made possible by
An external IR projector systems is needed to ex- the rapid reduction in size in the commercial
cite the microphotodiode to produce stimulus cur- imager chips. Even so, any implanted device will
rent. The microphotodiode circuit is connected likely eliminate some common camera functions
to an electrode with deliver the photocurrent to to achieve low power consumption. Features such
the retina, cause excitation, and the perception as automatic gain control, low light sensitivity,
of light. Extensive preclinical testing in rodent and sophisticated image processing (see next sec-
models support the ability of such system to action) may need to be traded for efficient oper-
tivate a degenerated retina and achieve visual ation (reduced power and size). An implanted
acuity [13]. As of the writing of this chapter, the camera that has a custom design may be able to
Prima clinical results were not published in peer- include many features by reducing the number
reviewed format. of pixels. Retinal prosthesis will have at most
several thousand channels, still well below com- processor will include decimation and some type
mon megapixel cameras. One such approach is to of enhancement. Decimation will be necessary
use an event based, neuromorphic camera [30]. since commercially available cameras typically
However, a custom camera design will add cost have at least 320 × 240 pixels, two orders of
to a retinal implant system. magnitude higher than the number of electrodes
in a retinal prosthesis. It is known that the retina
performs gain control, edge enhancement, motion
20.3.2 Image Processing detection, so a first generation image processor
should anticipate the need to replicate some of
Image processing strategies for a retinal prosthe- these functions.
sis will depend on the system implementation. Silicon retinas based on the biology of the
Implanted imagers will have less flexibility to visual system have been realized [31]. This in-
implement complex algorithms. For the follow- volves designing a computer chip to emulate reti-
ing discussion, we will set aside this consid- nal processing. Such systems may represent the
eration and discuss possible algorithms without best compromise between computational capa-
regard for the computational power requirements bility and low power operation, since such cus-
to implement such algorithms. An image process- tom, hardware solutions can be very efficient.
ing algorithm will transform a frame of video On the other hand, the programmability of these
into a set of stimulus commands for the im- chips is limited. One such system has 5760 pho-
planted stimulator. Alternatively, image process- toreceptor elements and 3600 ganglion cell out-
ing maybe used to detect and highlight important puts [32]. Both outer and inner retina circuitry
parts of an image. With such little experience with were modeled in silicon. Functions performed by
retinal prostheses in people, the best processing this chip include luminance adaptation, bandpass
strategy cannot be known at this time. Patient in- spatiotemporal filtering, temporal adaptation, and
teraction in fine-tuning the device will be needed contrast gain control. The chip was fabricated in
to optimize the perception. Both the software and 0.35 μm CMOS, is 3.5 × 3.3 mm2 , and con-
hardware for image processing must be devel- sumes 62.7 mW of power. If a similar chip was
oped in concert with the implanted portion of developed for a 1000 channel stimulator, the size
the device. Therefore, the initial image processor and power consumption could be reduced sub-
must be flexible to account for the current lack of stantially.
knowledge concerning pattern stimulation of the Another type of image processing for retinal
retina and the likely variability in patient response prosthesis involves the use of computer vision
due to different disease states. algorithms to detect/enhance important parts of
The implementation of the image processing the scene captured by the camera. The determi-
system can be on platforms ranging from general nation of what is important is the difficult part of
purpose processors to custom chips with hard- such algorithms, but some recent progresses have
wired processing schemes. Regardless of this, been made. Using models of the primate vision
several systems requirements will be common. system coupled with testing in human observers,
Real-time operation is necessary, since the sub- “saliency” algorithms have been developed that
jects will be correlating camera direction with detect the most salient part of an image [33].
the location of the perception and the stimulus These algorithms are based on color opponency,
must update fast enough to create the percep- contrast differences, orientation, and movement.
tion of where the camera is pointed. The system Using a series of test images, good correspon-
must be portable. Conceivably, a laptop computer dence was noted between what humans looked
could be worn in backpack, but a belt-worn sys- at in the scene (recorded via eye tracking) and
tem would be preferable and glasses mounted what the algorithm predicted would be the most
system ideal. Thus, some custom hardware will salient region. It is important to note that the
be required. The main algorithms of the image algorithms do not identify specifics objects or
people, instead they act similarly to the peripheral platinum is the small, but finite, dissolution rate
retina by highlighting parts of a scene that might under repeated stimulation [37]. Considering that
be important to examine closer. Such algorithms a retinal electrode array must use a thin film of
could be employed in a retinal prosthesis that cov- metal (to remain thin and flexible overall), it is
ered the central vision (but not peripheral vision). doubtful that a submicron thin film of platinum
The user could be alerted to the presence of im- will withstand years of pulsing [38]. Thus, any
portant objects or obstacles in the periphery. With platinum electrode for chronic stimulation must
these algorithms, normally sighted test subjects in have enough material to last under these condi-
a simulated prosthetic vision environment were tions.
able to improve their performance in visually Safe stimulation limits are defined by the
guided tasks, although in some instances it took amount of charge applied before hydrolysis and
longer for them to complete the tasks [34]. By gas evolution. Estimates of safe stimulation
providing additional information through com- limits for platinum range from 0.1 mC/cm2
puter vision algorithms, the user must then pro- to 0.35 mC/cm2 . Alternatives to Pt include
cess this information which adds to the cognitive materials such as iridium oxide and titanium
load of interpreting input from the artificial de- nitride. Both of these have been shown to have
vice. higher safe stimulation limits than Pt (IrOx:
1–4 mC/cm2 : TiN 0.9–22 mC/cm2 ) but have
not been implemented in medical devices as
20.3.3 Retinal Stimulating Electrode extensively as Pt. The reason for this is that
Arrays most neural stimulating devices in usage today
use large electrodes and have a relatively low-
Implementation of a stable electrode-retina in- charge-density requirement. In contrast, retinal
terface will have a number of challenges. Many stimulating arrays have electrodes as small as
of these challenges stem from the fact that the 50 × 50 microns [29]. Clinical retinal stimulating
retina is spherical and microelectrode technology arrays have used platinum gray (a high surface
typically produces planar structures. Even if a area platinum used by Argus II), titanium nitride
spherical structure can be produced, the eye cur- (Alpha-IMS), and iridium oxide (Prima).
vature is not consistent and even a separation of One deficiency with current retinal implants is
200 μm can be significant. If the curvature mis- the narrow field of view provided, due to the elec-
match causes a device to mechanically pressure trode array size. Surgical constraints limit the size
the retina, then the retina can be damaged [35]. of an eyewall incision to about 5 mm. Above that,
If the curvature mismatch results in separation of the risk of catastrophic loss of eye pressure during
the electrodes form the retina, then more current surgery increases. The Argus II array is 20◦ (di-
may be needed to activate the retina, as noted agonally), while the prima and Alpha-IMS arrays
earlier. Therefore, the ideal retinal stimulating are less than 15◦ . The suprachoroidal approach
electrode would have the flexibility to match the can overcome this limitation, since no eyewall
curvature of the retina without placing significant incision is required, at the cost of higher stim-
mechanical pressure on the retina. General re- ulus thresholds as described earlier. To achieve
quirements for neurostimulation electrodes, such both low thresholds (intraocular) and wide field,
as materials, have been covered in detail in an two approaches have been proposed. For subreti-
excellent review by Cogan [36], and below we nal arrays, a multi-implant approach may allow
discuss electrode materials in the context of reti- peripheral perception. For example, in the case
nal prostheses. of the Prima system, multiple microphotodiode
Platinum is the electrode material most fre- arrays would be implanted. For epiretinal arrays,
quently used for neural stimulation, but other several groups have created prototype “wide-field
materials have been evaluated. One problem with arrays” that have flexibility that allow unfold-
ing of the array after implantation. Ameri et al. 4. Name one advantage of an epiretinal
described a foldable polyimide retinal electrode approach. Name on advantage of a subretinal
array that could cover 34◦ of visual field [39]. approach.
The design featured “arms” around a central disk. 5. What feature of foveal structure will make
The arms were folded over the disk for insertion. normal visual acuity (20/20) difficult to
The passive array was successfully inserted in achieve with electronic retinal implants?
cadaveric pig eye, demonstrating the feasibility 6. Name two diseases that can be treated with
of this approach. A flexible retinal prosthesis, retinal implants. What type of disease is not
including light sensitive elements, was developed treatable?
by Ferlauto and colleagues. This device is curved 7. What is a “silicon retina”? Why is such a de-
to match the eye shape and can be folded for vice more useful for epiretinal vs. subretinal
insertion but still requires an insertion larger than implants.
5 mm. Accelerated aging of this system suggests 8. List two challenges is making a curved elec-
a lifetime greater than 2 years [40]. trode array
9. Why is it difficult to provide peripheral vision
with retinal prostheses?
20.4 Conclusion 10. How can computer vision improve the perfor-
mance of someone with a retinal prostheses?
Retinal prostheses are medical devices approved
by regulatory agencies and available for patients
with severe retinal degeneration. However, the References
quality of vision provided is limited. Improve-
ments in technology will be needed to increase 1. L. Yue, J.D. Weiland, B. Roska, M.S. Humayun,
Retinal stimulation strategies to restore vision: Fun-
upon the results from clinical trials of experimen- damentals and systems. Prog. Retin. Eye Res. 53, 21–
tal devices. For more information on clinical trials 47 (2016)
of retinal prosthesis, visit www.clinicaltrials.gov. 2. R. Ratnapriya, E.Y. Chew, Age-related macular
degeneration—Clinical review and genetics update.
Clin. Genet. 84(2), 160–166 (2013)
Acknowledgments Dr. Weiland has a research collabo-
3. D.T. Hartong, E.L. Berson, T.P. Dryja, Retinitis pig-
ration with Second Sight Medical Products, Inc. Dr. Hu-
mentosa. Lancet 368(9549), 1795–1809 (2006)
mayun receives royalties from patents licensed to Second
4. A. Santos, M.S. Humayun, E. de Juan Jr., R.J. Green-
Sight Medical Products, Inc.
burg, M.J. Marsh, I.B. Klock, A.H. Milam, Preser-
vation of the inner retina in retinitis pigmentosa. A
morphometric analysis. Arch. Ophthalmol. 115(4),
Homework 511–515 (1997)
5. S.Y. Kim, S. Sadda, M.S. Humayun, E. de Juan Jr.,
B.M. Melia, W.R. Green, Morphometric analysis of
1. List the three main functions of an electronic the macula in eyes with geographic atrophy due to
retinal prosthesis. age-related macular degeneration. Retina 22(4), 464–
2. The photoreceptors and bipolar cells are 470 (2002)
6. B. Jones, R. Pfeiffer, W. Ferrell, C. Watt, M. Marmor,
sometimes referred to as analog cells and
R. Marc, Retinal remodeling in human retinitis pig-
retinal ganglion cells are sometimes referred mentosa. Exp. Eye Res. 150, 149–165 (2016)
to as digital cells, in terms of how they 7. G.S. Brindley, W.S. Lewin, The sensations produced
respond to stimuli. Explain why these terms by electrical stimulation of the visual cortex. J. Phys-
iol. 196(2), 479–493 (1968)
are used to describe each type of cell.
8. E. Fernández, B. Greger, P.A. House, I. Aranda, C.
3. Why is a variable density of electrodes, with Botella, J. Albisua, C. Soto-Sánchez, A. Alfaro, R.A.
high density in the center and low density in Normann, Acute human brain responses to intracor-
the periphery, a desirable design for retinal tical microelectrode arrays: Challenges and future
prospects. Front. Neuroeng. 7, 24 (2014)
prostheses?
9. M.E. Brelén, F. Duret, B. Gérard, J. Delbeke, C.
Veraart, Creating a meaningful visual perception in
blind volunteers by optic nerve stimulation. J. Neural chronically implanted with Epiretinal prosthesis Ar-
Eng. 2(1), S22 (2005) gus I. Ophthalmology 122(12), 2545–2552 (2015).
10. N.Z. Gregori, N.F. Callaway, C. Hoeppner, A. Yuan, e2541
A. Rachitskaya, W. Feuer, H. Ameri, J.F. Arevalo, 21. A.C. Ho, M.S. Humayun, J.D. Dorn, L. da Cruz,
A.J. Augustin, D.G. Birch, Retinal anatomy and elec- G. Dagnelie, J. Handa, P.O. Barale, J.A. Sahel, P.E.
trode Array position in retinitis Pigmentosa patients Stanga, F. Hafezi, A.B. Safran, J. Salzmann, A. San-
after Argus II implantation: An international study. tos, D. Birch, R. Spencer, A.V. Cideciyan, E. de Juan,
Am J. Ophthalmol. 193, 87–99 (2018) J.L. Duncan, D. Eliott, A. Fawzi, L.C. Olmos de
11. M. Beyeler, D. Nanduri, J.D. Weiland, A. Rokem, Koo, G.C. Brown, J.A. Haller, C.D. Regillo, L.V. Del
G.M. Boynton, I. Fine, A model of ganglion axon Priore, A. Arditi, D.R. Geruschat, R.J. Greenberg,
pathways accounts for percepts elicited by retinal I.I.S.G. Argus, Long-term results from an Epiretinal
implants. Sci. Rep. 9(1), 9199 (2019) prosthesis to restore sight to the blind. Ophthalmol-
12. K. Stingl, K.U. Bartz-Schmidt, D. Besch, C.K. Chee, ogy 122(8), 1547–1554 (2015)
C.L. Cottriall, F. Gekeler, M. Groppe, T.L. Jack- 22. A.K. Ahuja, J.D. Dorn, A. Caspi, M.J. McMahon, G.
son, R.E. MacLaren, A. Koitschev, A. Kusnyerik, Dagnelie, L. Dacruz, P. Stanga, M.S. Humayun, R.J.
J. Neffendorf, J. Nemeth, M.A. Naeem, T. Peters, Greenberg, I.I.S.G. Argus, Blind subjects implanted
J.D. Ramsden, H. Sachs, A. Simpson, M.S. Singh, with the Argus II retinal prosthesis are able to improve
B. Wilhelm, D. Wong, E. Zrenner, Subretinal visual performance in a spatial-motor task. Br. J. Ophthal-
implant alpha IMS–clinical trial interim report. Vis. mol. 95(4), 539–543 (2011)
Res. 111(Pt B), 149–160 (2015) 23. M.S. Humayun, J.D. Dorn, L. da Cruz, G. Dagnelie,
13. H. Lorach, G. Goetz, Y. Mandel, X. Lei, T.I. Kamins, J.A. Sahel, P.E. Stanga, A.V. Cideciyan, J.L. Duncan,
K. Mathieson, P. Huie, R. Dalal, J.S. Harris, D. D. Eliott, E. Filley, A.C. Ho, A. Santos, A.B. Safran,
Palanker, Performance of photovoltaic arrays in-vivo A. Arditi, L.V. Del Priore, R.J. Greenberg, I.I.S.G.
and characteristics of prosthetic vision in animals Argus, Interim results from the international trial
with retinal degeneration. Vis. Res. 111, 142–148 of second Sight’s visual prosthesis. Ophthalmology
(2015) 119(4), 779–788 (2012)
14. N. Barnes, A.F. Scott, P. Lieby, M.A. Petoe, C. Mc- 24. L. da Cruz, B.F. Coley, J. Dorn, F. Merlini, E. Filley,
Carthy, A. Stacey, L.N. Ayton, N.C. Sinclair, M.N. P. Christopher, F.K. Chen, V. Wuyyuru, J. Sahel, P.
Shivdasani, N.H. Lovell, Vision function testing for Stanga, M. Humayun, R.J. Greenberg, G. Dagnelie,
a suprachoroidal retinal prosthesis: Effects of image I.I.S.G. Argus, The Argus II epiretinal prosthesis
filtering. J. Neural Eng. 13(3), 036013 (2016) system allows letter and word reading and long-term
15. T. Endo, T. Fujikado, M. Hirota, H. Kanda, T. Mo- function in patients with profound vision loss. Br. J.
rimoto, K. Nishida, Light localization with low- Ophthalmol. 97(5), 632–636 (2013)
contrast targets in a patient implanted with a 25. D.R. Geruschat, T.P. Richards, A. Arditi, L. da Cruz,
suprachoroidal–transretinal stimulation retinal pros- G. Dagnelie, J.D. Dorn, J.L. Duncan, A.C. Ho, L.C.
thesis. Graefes Arch. Clin. Exp. Ophthalmol. 256(9), Olmos de Koo, J.A. Sahel, An analysis of observer-
1723–1729 (2018) rated functional vision in patients implanted with the
16. A.Y. Chow, N.S. Peachey, The subretinal microphoto- Argus II retinal prosthesis system at three years. Clin.
diode array retinal prosthesis. Ophthalmic Res. 30(3), Exp. Optom. 99(3), 227–232 (2016)
195–196 (1998) 26. K. Mathieson, J. Loudin, G. Goetz, P. Huie, L. Wang,
17. M. Mahadevappa, J.D. Weiland, D. Yanai, I. Fine, T.I. Kamins, L. Galambos, R. Smith, J.S. Harris,
R.J. Greenberg, M.S. Humayun, Perceptual thresh- A. Sher, D. Palanker, Photovoltaic retinal prosthesis
olds and electrode impedance in three retinal prosthe- with high pixel density. Nat. Photonics 6(6), 391
sis subjects. IEEE Trans. Neural Syst. Rehabil. Eng. (2012)
13(2), 201–206 (2005) 27. M.A. Petoe, C.D. McCarthy, M.N. Shivdasani, N.C.
18. D. Nanduri, I. Fine, A. Horsager, G.M. Boynton, M.S. Sinclair, A.F. Scott, L.N. Ayton, N.M. Barnes, R.H.
Humayun, R.J. Greenberg, J.D. Weiland, Frequency Guymer, P.J. Allen, P.J. Blamey, Determining the con-
and amplitude modulation have different effects on tribution of retinotopic discrimination to localization
the percepts elicited by retinal stimulation. Invest. performance with a suprachoroidal retinal prosthe-
Ophthalmol. Vis. Sci. 53(1), 205–214 (2012) sis. Invest. Ophthalmol. Vis. Sci. 58(7), 3231–3239
19. J.D. Weiland, D. Yanai, M. Mahadevappa, R. (2017)
Williamson, B.V. Mech, G.Y. Fujii, J. Little, R.J. 28. N.R. Stiles, B.P. McIntosh, P.J. Nasiatka, M.C. Hauer,
Greenberg, E. de Juan Jr., M.S. Humayun, Visual task J.D. Weiland, M.S. Humayun, J. Tanguay, R. Ar-
performance in blind humans with retinal prosthetic mand, An intraocular camera for retinal prostheses:
implants. Conf. Proc. IEEE Eng. Med. Biol. Soc. 6, Restoring sight to the blind, in Optical Processes
4172–4173 (2004) in Microparticles and Nanostructures: A Festschrift
20. L. Yue, P. Falabella, P. Christopher, V. Wuyyuru, Dedicated to Richard Kounai Chang on His Retire-
J. Dorn, P. Schor, R.J. Greenberg, J.D. Weiland, ment from Yale University. World Scientific, (2011),
M.S. Humayun, Ten-year follow-up of a blind patient pp. 385–429. ISBN 109814295779
29. E. Zrenner, K.U. Bartz-Schmidt, H. Benav, D. Besch, 35. B.C. Basinger, A.P. Rowley, K. Chen, M.S. Humayun,
A. Bruckmann, V.P. Gabel, F. Gekeler, U. Grepp- J.D. Weiland, Finite element modeling of retinal
maier, A. Harscher, S. Kibbel, J. Koch, A. Kusnyerik, prosthesis mechanics. J. Neural Eng. 6(5), 055006
T. Peters, K. Stingl, H. Sachs, A. Stett, P. Szurman, B. (2009)
Wilhelm, R. Wilke, Subretinal electronic chips allow 36. S.F. Cogan, Neural stimulation and recording elec-
blind patients to read letters and combine them to trodes. Annu. Rev. Biomed. Eng. 10, 275–309 (2008)
words. Proc. Biol. Sci. 278(1711), 1489–1497 (2011) 37. D.B. Hibbert, K. Weitzner, B. Tabor, P. Carter, Mass
30. M. Giulioni, X. Lagorce, F. Galluppi, R.B. Benosman, changes and dissolution of platinum during electrical
Event-based computation of motion flow on a neurostimulation in artificial perilymph solution. Biomate-
morphic analog neural platform. Front. Neurosci. 10, rials 21(21), 2177–2182 (2000)
35 (2016) 38. C. Boehler, F. Oberueber, S. Schlabach, T. Stieglitz,
31. M. Firouzi, J. Conradt, Asynchronous event-based M. Asplund, Long-term stable adhesion for conduct-
cooperative stereo matching using neuromorphic sil- ing polymers in biomedical applications: IrOx and
icon retinas. Neural. Process. Lett. 43(2), 311–326 nanostructured platinum solve the chronic challenge.
(2016) ACS Appl. Mater. Interfaces 9(1), 189–197 (2016)
32. K.A. Zaghloul, K. Boahen, A silicon retina that repro- 39. H. Ameri, T. Ratanapakorn, S. Ufer, H. Eckhardt,
duces signals in the optic nerve. J. Neural Eng. 3(4), M.S. Humayun, J.D. Weiland, Toward a wide-field
257 (2006) retinal prosthesis. J. Neural Eng. 6(3), 035002 (2009)
33. L. Itti, C. Koch, E. Niebur, A model of saliency-based 40. L. Ferlauto, M.J.I.A. Leccardi, N.A.L. Chenais,
visual attention for rapid scene analysis. IEEE Trans. S.C.A. Gilliéron, P. Vagni, M. Bevilacqua, T.J.
Pattern Anal. Mach. Intell. 11, 1254–1259 (1998) Wolfensberger, K. Sivula, D. Ghezzi, Design and
34. N. Parikh, L. Itti, M. Humayun, J. Weiland, Per- validation of a foldable and photovoltaic wide-field
formance of visually guided tasks using simulated epiretinal prosthesis. Nat. Commun. 9(1), 992 (2018)
prosthetic vision and saliency-based cues. J. Neural 41. E. Zrenner, Fighting blindness with microelectronics.
Eng. 10(2), 026017 (2013) Sci. Transl. Med 5(210. ps16) (2013)
Retinal Bioengineering
21
Robert A. Linsenmeier and John B. Troy
Abstract record massed activity of some types of retinal

neurons, including photoreceptors and bipolar
The retina is a small piece of the central ner-
cells, in the electroretinogram (ERG) in hu-
vous system responsible for the first steps in
mans as well as animals. ERG analyses have
vision, so understanding how it works has
led to models of signal processing prior to
great importance for daily life. In addition,
the RGCs. Engineering methods in combi-
features of the retina make it attractive as a
nation with physiology have thus elucidated
model neural system. The only input to the
the basic features of the retinal network that
retina is light, which can be easily manip-
allow the convergence of signals from many
ulated, and recordings have been made for
millions of photoreceptors to yield the center-
many decades from individual output cells of
surround organization and response properties
the retina, the retinal ganglion cells (RGCs),
of the primary types of RGCs in cats and
allowing application of linear (and to some ex-
primates. However, some of the approximately
tent nonlinear) systems analysis methods that
20 types of RGCs that send parallel signals to
define the transfer functions of the retina. The
the brain are still poorly understood. Recent
small, non-spiking photoreceptors and retinal
work has used isolated retinas and multielec-
interneurons make recordings from these ear-
trode arrays to record from many retinal gan-
lier stages difficult in mammals, but this has
glion cells simultaneously. Specific contribu-
been partially surmounted by the ability to
tions of interneurons to the retinal circuits have
also been addressed with new methods, some
of which are reviewed here. Another aspect
Electronic Supplementary Material: The online version of retinal bioengineering concerns the reti-
of this chapter (https://doi.org/10.1007/978-3-030-43395- nal microenvironment. Diffusion models and
to authorized users. spatially precise intraretinal measurements of
oxygen and pH provide information about reti-
R. A. Linsenmeier () nal metabolism that is useful in understanding
Department of Biomedical Engineering, Northwestern
University, Evanston, IL, USA
dysfunction of the retina in some diseases.
Department of Neurobiology, Northwestern University,
Keywords
Evanston, IL, USA
Department of Ophthalmology, Northwestern University, Retina · Retinal ganglion cell ·
Chicago, IL, USA Photoreceptor · Microelectrode array ·
Systems analysis · Electroretinogram ·
J. B. Troy Retinal model · Bioengineering ·
Department of Biomedical Engineering, Northwestern
Microenvironment
University, Evanston, IL, USA

582 R. A. Linsenmeier and J. B. Troy
21.1 Introduction etiology of disease and assist in providing treat-

ments for some of the many blinding diseases.
The retina is a tiny piece of neural tissue, weigh- Diagnosing the problems of the diseased retina
ing less than 100 mg in humans, whose impor- and repairing or providing substitutes for its func-
tance for bioengineers is out of proportion to its tions are obviously within the purview of design-
size. It has long served as a model for understand- oriented neural engineers. However, there is also
ing complex parts of the nervous system, and it a large body of work by retinal bioengineers
has attracted a great deal of attention from neu- in measuring and modeling normal retinal func-
roscientists of all types, including bioengineers. tion. Understanding retinal neural mechanisms
The quantitative and systems approaches of engi- will provide information for the design of artifi-
neering have been central to the understanding of cial retinas.
retinal function for more than five decades. Many This chapter focuses on aspects of retinal bio-
of the retina’s properties hold up well in vitro, engineering related to mathematical modeling of
and it is accessible to microelectrodes both in vivo neural responses and the modeling of the retinal
and in vitro. It has a modest number of principal microenvironment, and on experiments that pro-
cell types, and the total number of output neurons vide inputs to that modeling or are used to vali-
(retinal ganglion cells – RGCs) in each eye is date the modeling. While vertebrate retinas share
1.1 ± 0.4 million in humans [1], and much less many similarities and a great deal of physiologi-
in non-primate mammalian species, numbers that cal information has come from lower vertebrates,
are almost manageable by comparison with the this chapter is largely concerned with mammals.
outputs of other parts of the central nervous sys- Even across mammals, there are enough differ-
tem. The retina can be studied while it responds ences that one has to be careful in generalizing.
to its natural input, patterns of light, which can The topics covered here are at the intersection
be controlled easily. For deeper neural structures, of bioengineering and neuroscience and there has
one often has to make the choice between study- been such a rich interplay that there is no clear line
ing responses to electrical stimulation, which is between engineers and biologists. This chapter
unnatural, or responses to natural inputs from covers a lot of ground. The references included
other locations in the nervous system that may here should not be considered exhaustive but are
be difficult to control or completely characterize. the ones that summarize areas and provide leads
The retina is also simpler than many areas of the into the rest of the literature.
brain because there is almost no feedback from This chapter omits a consideration of visual
the brain to the retina. In short, no other region of prostheses [2–7], which are covered in a separate
comparable complexity provides the advantages chapter. It also omits topics at the intersection
for study that the retina does. of bioengineering and optics, which comprise
The retina is also important for its role in another exciting type of retinal bioengineering.
human lifestyles and performance, coupled with It is now possible to image individual retinal
its sensitivity to disease. A large part of the human photoreceptors in the human eye with adaptive
brain is devoted to visual processing, and all of optics [8–10]. Also, optical imaging of the lam-
this relies on the transduction and initial visual inated retinal structure via optical coherence to-
processing steps that occur in the retina. Both mography (OCT) [11–14] has become an essen-
our ability to receive information about the world tial diagnostic tool in a remarkably short period of
and our mobility within it are ordinarily strongly time. Scanning laser ophthalmoscopy (SLO) [15–
dependent on vision. Unfortunately, the retina is 17] is less prevalent, but can also be important
rather fragile, and a number of genetic, vascular, in the identification of pathology. Laser-based
and metabolic diseases interfere with its function. techniques are used for both diagnostic appli-
Just as engineers can contribute to understanding cations and therapeutic ones, for instance, pan-
normal retinal function, they can help unravel the retinal photocoagulation [18] and photodynamic
21 Retinal Bioengineering 583
therapy [19]. Yet another intersection between vi- 21.2 The Neural Structure
sion and engineering is in drug delivery, to avoid and Function of the Retina
frequent injections into the eye and obtain con-
tinuous dosing [20, 21], a particularly important This section on retinal structure and function is
goal now that agents against vascular endothelial necessarily brief, and more comprehensive views
growth factor (VEGF) have been shown to be of the retina can be found in many books and
effective in several diseases. chapters [27–30], and a web textbook [31].
Molecular and cellular therapies, particularly The retina is the innermost of three layers
for photoreceptor degenerations, are among the comprising the posterior part of the eye (Fig.
most exciting innovations today, and engineers 21.1). An outer layer, the sclera is a fibrous coat
are engaged in using these therapies. Gene trans- that provides most of the structural rigidity of
fer to repair or rescue photoreceptors that would the eye and merges with the cornea in the front
otherwise degenerate has become a reality in cer- of the eye. However, the eye maintains its shape
tain cases [22, 23]. In addition, after photorecep- only because secretion of fluids from the ciliary
tors are lost, the transfection of cells with pho- body (just behind the lens) keeps the intraocular
toreceptive proteins, an application of the larger pressure at about 15 mm Hg above atmospheric.
area of optogenetics [24], can make second- and Inside the sclera is the choroid, which is primarily
third-order neurons light sensitive [25, 26]. vascular, and supplies much of the nutrition that
Fig. 21.1 Structure of the three coats of the vertebrate through the optic nerve. The completely separate choroidal
eye, the sclera on the outside, the choroid in the middle, circulation is fed largely by short posterior ciliary arteries
and the retina adjacent to the vitreous humor. The major that penetrate the sclera near the optic nerve. These in turn
arteries and veins are also shown. The central retinal artery distribute into a capillary bed called the choriocapillaris,
enters through the optic nerve and feeds the capillaries and the choroid is then drained by the vortex veins. (Copy-
of the retinal circulation within the retina. The central right 2019 by Robert Morreale, CMI, all rights reserved)
retinal vein drains the retinal circulation, leaving the eye
the retina, especially the photoreceptors, requires.

The retina itself is adjacent to the microcircu-
lation of the choroid. In humans, the retina is
only about 250 μm thick on the average. It is
thinner in the fovea and in the periphery, and
thicker adjacent to the fovea. The retinas of all
vertebrates share basic structural and physiolog-
ical similarities, but the description here concen-
trates on the retina of mammals, especially cats
and primates, which serve as the most relevant
models for understanding the human retina (Fig.
21.2), although mice have also gained importance
because of the availability of transgenic models.
21.2.1 Photoreceptors
At the back of the retina are the photoreceptor

cells, which contain many stacked disks in their Fig. 21.2 Cell types and lamination of the mammalian
outer segments. The disk membranes contain the retina. At the top are the photoreceptors, which comprise
visual pigment, which absorbs light and begins about half of the retinal thickness. The thinner photore-
ceptors are rods and the thicker ones are cones. They
the process of transducing it into electrical sig- are intermixed in most of the retina. At the outer plex-
nals. In vertebrates, light leads to a hyperpolar- iform layer (OPL), rods contact rod bipolars (RB) and
ization of the photoreceptors, as described more cones contact ON and OFF cone bipolars (OFF B and
fully below. Photoreceptors in humans fall into ON B). Horizontal cells (H) also make synapses in the
OPL, receiving input from cones and providing feedback
two classes, rods and cones. Rods mediate vision to other cones. Horizontal cell processes are also found
over about 6 log units of illumination, from the in the rod-RB synaptic complex. Cell bodies of bipolar
threshold of less than 0.001 quanta per second cells, horizontal cells, and amacrine cells (represented
per rod up to about the illumination at dawn and by one subtype, the AII) are found in the inner nuclear
layer. Connections of bipolars and amacrine cells to retinal
twilight [29] (scotopic conditions). The ampli- ganglion cells (RGCs) are found in the inner plexiform
tude of rod responses then saturates, and cones layer in separate sublaminae for the ON and OFF systems.
gradually take over and are responsible for vision RGC cell bodies are found in the ganglion cell layer and
under the rest of the approximately 10 log units their axons run in the nerve fiber layer. The connections
that are shown are the principal ones needed to explain
of illumination over which we have vision [29, the circuitry of Fig. 21.3. Not shown are interplexiform
34] (photopic conditions). Still, in order to cover cells, whose cell bodies are in the inner nuclear layer, and
this entire range adequately, both rods and cones project from the inner plexiform to the outer plexiform, the
(and subsequent neurons) must adapt, or adjust Müller cells, the principal glial cells of the retina, which
span all the layers except the outer and inner segments, and
their sensitivity, as mean illumination changes, astrocytes in the nerve fiber layer. (Modified from Wassle
because the dynamic range at any given time for and Boycott [33])
a rod or a cone is only about 2 log units [29]. The
transition region where rods and cones may both
be involved is called the mesopic range. tached. All the pigments respond to light over a
There are four standard visual pigments in wavelength range of more than half the complete
humans, one in rods and the other three in cones. visual spectrum (400–750 nm) but the slight dif-
All have the same light-absorbing component, ferences in the proteins give the rods and each of
the chromophore retinal, which is derived from the three types of cones, called short-, middle-,
vitamin A, but they vary slightly in the protein, and long-wavelength (or S, M, and L) cones, an
called opsin, to which the chromophore is at- absorbance maximum at different wavelengths.
Rods absorb maximally at 500 nm, and the S, project to higher structures ([36, 37]). The most
M, and L cones absorb maximally at 419, 530, important of these are (1) the lateral geniculate
and 568 nm, respectively, although these peaks nucleus of the thalamus, which is the major relay
vary slightly among individuals. Comparison of station for signals that travel to visual cortex to
the outputs of different cones by second-order mediate visual perception, and (2) the superior
neurons is required to extract a wavelength signal colliculus in the midbrain, which uses visual in-
and discriminate color. While full color vision put to guide eye movements. The brain regions
requires all three cone types, many humans, espe- that mediate the pupil response and the synchro-
cially males, function reasonably well with only nization of the circadian clock to light receive
one or two cone types. Thus, it is vision at high input from a relatively small number of RGCs
illuminations, rather than color vision, that is the that contain the visual pigment melanopsin. This
critical function of cones in humans. makes them intrinsically photosensitive [40–42],
although these RGCs also receive input from
bipolars fed by photoreceptors.
21.2.2 Retinal Circuits There are several pathways by which photore-
ceptor signals reach the ganglion cells. Mammals
Photoreceptors make their synapses in the outer have two main classes (and several subtypes) of
plexiform layer (Fig. 21.2). At this location, rods cone bipolar cells, depolarizing (ON) and hyper-
and cones project to separate subtypes of bipo- polarizing (OFF), that receive signals from the
lar cells, which carry the signals forward, and cones, and another type called rod bipolars, which
to horizontal cells, which then make lateral in- connect only to the rods. Rod bipolars are all ON
hibitory connections back to other photorecep- bipolar cells. As with other retinal neurons, ON
tors. The cell bodies of bipolar cells, horizontal and OFF refer to the stimulus polarity that depo-
cells, and amacrine cells (along with Müller cells, larizes the cell. An increase in illumination leads
the principal glial cells of the retina) form the to a depolarization of ON bipolars and a hyper-
inner nuclear layer, and the outputs of bipolar and polarization of OFF bipolars, while a decrease in
amacrine cells provide the input to several classes illumination hyperpolarizes ON and depolarizes
of RGCs at the inner plexiform layer. The RGC OFF bipolars. All photoreceptors hyperpolarize
bodies along with some “displaced” amacrine with illumination, so they can be regarded as “off”
cells are located in the ganglion cell layer (GCL). cells, although that terminology is never used
Because of the different requirements for visual for them. Connections from cones to OFF bipo-
information going to different locations in the lars therefore preserve the sign of the responses
brain, RGCs are of several different physiological (i.e., both cones and OFF bipolars hyperpolarize
types, which are correlated with different anatom- upon illumination) and are fundamentally excita-
ical types [35–37]. Each of the major types of tory, while connections from cones and rods to
RGCs tiles the retina, providing several overlap- their respective ON bipolars require a sign rever-
ping representations of the visual world [33, 38]. sal, which implies an inhibitory connection (Fig.
Because of the need to transmit signals over long 21.3). The different properties of bipolar cells re-
distances, RGCs and some amacrine cells [39] sult from differences in their glutamate receptors.
fire action potentials. Other retinal neurons do not Photoreceptor to horizontal cell connections are
support action potentials but instead control their excitatory, so horizontal cells are another type
transmitter release by graded potential changes. of OFF cell. At the inner plexiform layer, where
Each RGC sends an unmyelinated axon toward bipolars connect to RGCs, the ON/OFF separa-
the optic disc (also called the optic nerve head) tion of bipolar cell response types is preserved
in the nerve fiber layer. The axons then pass by segregated excitatory connections of ON cone
through a modified part of the sclera called the bipolar cells to ON-center RGCs, and OFF cone
lamina cribrosa at the optic disc. Past the lam- bipolars to OFF-center RGCs [32]. These excita-
ina cribrosa, the axons become myelinated and tory connections occur in separate sublamina of
Fig. 21.3 Cellular connections underlying the center and cone pathways, the on-off dichotomy arises at the OPL and
surround pathways of cone driven RGCs (left) and rod- is preserved in the IPL. In the rod system, the situation
driven RGCs (right) in the mammalian retina. The ovals at is more complex because all rod bipolars are depolarizing
the top represent a spot centered on the receptive field of (ON). The AII amacrine cells project to OFF-center RGCs
cones or rods and also on the relevant bipolars or RGCs. through sign reversing synapses, and to ON-center RGCs
Shaded cells hyperpolarize in response to light, while clear through a gap junction with ON bipolars. (Modified from
cells depolarize. Thus, if a shaded cell connects to a clear Schiller [32], representing the work of many investigators)
cell, there is a sign-reversing or inhibitory synapse. In the
the inner plexiform layer, so both the “axons” of bipolar (Figs. 21.2 and 21.3) [33, 47]. Because
bipolar cells and dendrites of RGCs have to find the rod system is designed to detect low levels of
the correct sublamina (Figs. 21.2 and 21.3) [33, light, RGCs always collect information from
43]. While this is clearly the basic pattern of con- many rods, with the number being larger at
nectivity, there is some evidence for more com- greater eccentricities. The connections shown
plexity [44, 45]. Further, ON- and OFF-center in Fig. 21.3 appear to be the most important ones
RGCs are not exactly mirror images of each other, under very strong and very dim illumination, but
but they have some distinct characteristics [46]. at intermediate levels of illumination, other signal
In most of the retina, there is convergence from pathways exist [48–50].
cones onto RGCs with at least a few cones con-
nected through bipolar cells to RGCs. However,
in primates, most of the RGCs with receptive 21.2.3 Receptive Fields
fields in the fovea (midget ganglion cells) each
connect to only one cone to preserve the high One of the important concepts for understanding
spatial acuity afforded by the cone mosaic. the retina is the idea of a receptive field. The
In general, rod and cone signals converge onto receptive field of a neuron in the visual system is
the same RGCs, but the pathways are different. defined to be that portion of visual space within
Surprisingly, rod bipolars do not contact RGCs which light will influence the neuron’s behav-
directly. Interposed in the pathway from rod ior. This part of visual space directly maps to a
bipolars to ON-center RGCs is an AII amacrine particular region of the retina, so the receptive
cell connecting to ON cone bipolar cells via gap field can also be discussed in terms of an area
junctions. Interposed in the pathway from rod or distance on the retinal surface. The receptive
bipolars to OFF-center RGCs is the same AII field size is most often specified in terms of the
amacrine cell connecting via chemical synapses visual angle, as indicated in Fig. 21.4. One degree
to the OFF-center ganglion cell and OFF cone of visual angle is about 0.294 mm on the retinal
that influences its membrane potential. Bipolar

and ganglion cell receptive fields are somewhat
larger, because of convergence of signals from
the cells preceding them. As noted above, many
bipolars and RGCs are named for the influence
of increased light falling in the middle of the
receptive field, as shown in the top panels of
Fig. 21.5, which illustrates firing patterns and a
histogram of firing frequency for an ON-center
ganglion cell. As also shown in Fig. 21.5, a
reduction in illumination in the middle of the
receptive field of an ON-center ganglion cell
leads to the opposite effect as an increase in
illumination: a suppression of the response or
reduction in firing. Receptive fields of many
RGCs also have a “surround” region, or the
“surround mechanism,” as first proposed by
Kuffler [53]. Light falling outside of the middle
of the receptive field region in a larger, concentric
region has the opposite effect as light falling in
the middle of the receptive field, antagonizing the
effect of light on the center (bottom panels of Fig.
21.5). ON-center RGCs have OFF-surrounds and
OFF-center cells have ON-surrounds. In primates
and humans, color processing is added to some
of the ON- and OFF-center cells.
Fig. 21.4 Illustration of the concepts of retinal eccen- Figure 21.6 shows the receptive field organi-
tricity and visual angle. The fovea is defined to be at an zation of the midget cells in central retina (also
eccentricity of zero degrees. The eye is viewing an object called P-type because they project to the parvo-
that subtends about ten degrees of visual angle, centered at
an eccentricity of about 20◦ . This is a top view of a left eye,
cellular layers of the lateral geniculate nucleus
because the blind spot (optic disk) is nasal to the fovea. (LGN)) and the parasol cells (also called M-
(Modified from Cornsweet [51]) type because of their projection to magnocellular
layers of the LGN). Many midget RGCs have red-
green opponency, with center signals dominated
surface in humans. For reference, a US quarter
by either M (“green”) or L (“red”) cones and
held at arm’s length roughly subtends 2.4 degrees
the surround being nonselective [54, 55]. Another
of visual angle. The concept of eccentricity is
class, the “small bistratified” cells, have blue-
also important. If one looks straight at an object,
yellow color opponency, with blue (S-type) cones
it is said to be at a visual eccentricity of zero
contributing to the center and a combination of
degrees. If one moves the quarter horizontally by
cones contributing to the surround [56, 57]. Both
5 quarter diameters but still gazes straight ahead,
the center and surround of the parasol RGCs re-
the quarter is now at an eccentricity of about 12
ceive input from M and L cones, and therefore can
degrees off the visual axis of the eye.
signal luminance, but not color. As suggested in
To a first approximation, most retinal receptive
this figure, P-type receptive fields are smaller than
fields are circular, but their form and size change
those of M-type cells. After the LGN, both P and
as visual signals advance through the retina from
M streams then project to different sublayers of
photoreceptors to RGCs. Photoreceptors have
the thalamic input layer of visual cortex (layer 4).
simple small receptive fields, since it is mainly
From there, the signals at least partially diverge,
the light falling on a particular photoreceptor
On-centre with M signals going more toward the dorsal

1 2 3 4 5 stream of further visual processing, associated
with motion and location of objects, and P signals
going to a ventral stream associated with form
vision.
Surrounds are sometimes said to be “in-
A hibitory,” but this is incorrect in the sense of
synaptic inhibition, and surrounds are really
“antagonistic” to the centers. For instance, the
B increased activity that occurs in the surround
pathway of an OFF-center ganglion cell when the
surround is illuminated depolarizes the ganglion
C cell, which is not an inhibitory action. Also,
50/sec
the surround pathway of an ON-center ganglion
cell exerts a net inhibitory effect on firing when
sec
illumination is increased, but a net excitatory
Off-centre effect when illumination is decreased. The
1 2 3 4 5 6 7
center and surround strengths are relatively well-
balanced, as described below, so diffuse flashes,
which stimulate both the center and surround,
cause only a small change in firing of RGCs, as
A shown in the bottom panels of Fig. 21.5.
The first quantitative description of the recep-
B tive field of RGCs suggested that the influence
of light was not uniform across the center (or sur-
round), but in each case had a Gaussian weighting
C
(Fig. 21.7), so that a stimulus in the middle of the
50/sec
receptive field center would have a larger effect
sec than one near the edge of the center [58, 59].
This idea is still accepted as a good approxima-
Fig. 21.5 Simulations of typical responses of cat RGCs
tion for the roughly 40–60% of the RGCs in cat
to flashing stimuli. Top panels: Responses of an on-center
ganglion cell. The pictures show the spatial configuration [37] and a larger fraction in primate [60] whose
of the stimulus with respect to the center (inner circle) axons project to the lateral geniculate nucleus,
and surround (outer circle) of the receptive field; trace but, as described below, more detailed analyses
A shows the time course of the stimulus; B shows the
show the limitations of the Gaussian model. Other
spike patterns in two repetitions of the stimulus; C shows
peristimulus time histograms (PSTHs) of the firing rate RGCs (see below) have receptive fields that can-
averaged over several presentations. The odd-numbered not be described easily by the center-surround
panels at the top show times when the cell is subjected to model. Refinements of the basic Gaussian Center-
a uniform gray background. In period two, the firing rate
Surround Model have come from engineering
is increased by presentation of a centered bright spot of
light. In period four, the centered stimulus is made dimmer approaches, as described later.
than the background, causing the firing rate to decrease. It appears that horizontal cell feedback to pho-
The surround was not activated in this set of stimuli, but toreceptors in the outer plexiform layer is respon-
if an annulus of light brighter than the background had
sible for at least part of the surround signal [61–
been presented to activate the surround, a response similar
to that in panel four would have been observed. Lower 66], which comes along with the center signal to
panels: Responses of an off-center ganglion cell. The plan RGCs through bipolar cells. Part of the surround
of the figure is the same but now the dimming of a spot of at least some RGCs also appears to be created
activates the center (panel two), while the dimming of
through amacrine cell inputs to RGCs at the inner
the surround suppresses firing (panel four). Stimulation
of both center and surround evokes only small transient plexiform layer [65, 67].
response from the cell (panel six). (Enroth-Cugell and
Robson [52])
Fig. 21.6 Receptive field organization of parvocellular medium (M) or long (L) wavelength cones. Surrounds
(P-type), magnocellular (M-type), and small bistratified have mixed L and M input, and may also have some input
RGCs in the primate retina. Plus means that increasing from S cones. In M-type cells, L and M cone signals both
illumination on a particular cone type projecting to the contribute to the center and surround, so these cells code
ganglion cell increases the firing rate, and minus means luminance rather than color. The small bistratified, blue-
that stimulation of that cone type decreases firing rate. yellow opponent cells have centers that are formed by
Cells in the top row are ON-center and those in the bottom input from short wavelength (S) cones and their surrounds
row are OFF-center. P-type cells in the central retina have have mixed L and M input. There are no comparable OFF-
the spectral properties indicated, with dominant input from center cells
21.2.4 Eccentricity and Acuity eye is constructed so that the acuity that would be
predicted by the photoreceptor spacing, without
Retinal structure and function vary considerably considering optical blur, is almost the same. That
with retinal eccentricity. On the optic axis of is, retinal anatomy is well matched to the best
primates and humans is the fovea, a region about that the optics can do. Because these two ways
5◦ in diameter ([68]), in which the retina is thin- of looking at acuity give essentially the same
ner. Here the second- and third-order neurons are answer, we will discuss only the more intuitive
pushed aside, presumably for optical clarity, and concepts based on detector spacing.
the photoreceptors extend long lateral processes If one had a pattern of dark and light lines of
out to the bipolar cells that serve them. A smaller high contrast, i.e., a grating pattern, the minimum
region, about 600 μm in diameter, in the middle detectable spacing between light lines would be
of the fovea contains only cones. This area has the spacing where two light lines were detected
the best acuity and is ordinarily used for tasks by two different photoreceptors with another pho-
like reading. No other animals except birds of toreceptor receiving less light between them. The
prey have foveas, but many vertebrates have some minimal detectable line spacing is then twice the
degree of specialization for higher acuity on the spacing between centers of the detector elements,
optic axis and in a horizontal streak [69]. which is about 2.5 μm in the fovea. (Photorecep-
The concept of acuity is important in subse- tors are tightly packed, so this is also the diameter
quent sections. The most obvious way to specify of one photoreceptor.) The maximum resolution
acuity is to consider the minimum spacing that in cycles of the grating that could be resolved per
is required between two points or lines in order degree is the inverse of this minimal spacing in
that they can be seen as distinct objects rather degrees. Therefore, if there are 300 μm across
than as a single object. This limit is set by the the retinal surface per degree of visual angle, the
point spread function of the optics [68], but the resolution limit or best acuity should be about:
Fig. 21.7 Difference of

Gaussians receptive field
model, showing the Center mechanism
sensitivity distributions
across the receptive field
center, and across the
receptive field surround,
which is shown below the
horizontal line because it
produces antagonistic
responses. The rows of
responses show
hypothetical responses at Surround mechanism
three locations in the
receptive field for an
ON-center cell for the
receptive field center and center
surround, and component
representations of the
ganglion cell membrane
potential (sum) and firing
rate (response). The
“response” rows show surround
truncated versions of the component
membrane potential
because firing rate cannot
be negative. (Rodieck and
Stone [58])
sum
response
center type no response on-off type surround type

response response response
R = 300 μm/deg · (1 c/(2 · element spacing in μm)) from this region is not lost. More peripherally,
cone density falls and rod density rises, and the
= 60 c/degree
two are intermixed in most of the retina, so acuity
(21.1) decreases rapidly outside the fovea The periph-
eral retina is, however, important for both spatial
This is not far from the actual acuity of a vision and motion detection.
person with good vision and a small pupil ([68]). Acuity is often specified in terms of the visual
A combination of factors gives the fovea the best ability of a person relative to the ability of a
acuity. First, the photoreceptors are smaller there, “normal” observer. The familiar “20/20” (or in
so a larger number can be packed in. Second, metric units 6/6) vision means that an individual
foveal cones project through 1:1 connections (i.e., (numerator) can see at 20 feet what a “normal”
no convergence) through bipolar cells to midget person (denominator) can see at 20 feet. The
RGCs. In fact, there is divergence, with a single features of the test stimulus on a standard eye
foveal cone typically projecting to both one ON- chart that are just barely distinguishable on the
center and one OFF-center midget ganglion cell. “20/20” row of the chart (e.g., the gap that makes
Third, the representation of this region in visual a “C” different from an “O” or one arm of an
cortex is large, so the detailed retinal information
“E”) subtend 1 minute of arc for an observer RPE is also important for recycling bleached vi-
who is 20 feet away. A person with 20/100 vision sual pigment, pumping fluid from the retina into
needs to be at 20 feet from the object to resolve the choroid, and performing phagocytosis of tips
what a “normal” person can see at 100 feet. If of photoreceptor outer segments [77].) Choroidal
the poor acuity is due to optical imperfections blood flow is controlled by the autonomic ner-
in the eye, such as myopia (nearsightedness) or vous system, and there is almost no matching of
astigmatism, it can usually be corrected to 20/20 its flow rate to the metabolic needs of the outer
(or better) vision with lenses or corneal surgery. retina, even though this circulation is of critical
If poor acuity is due to disease of the retina or importance to the photoreceptors. The choroidal
brain, the same system of acuity designations is circulation exhibits some autoregulation in re-
used, but the vision cannot be corrected optically. sponse to changes in arterial pressure but less in
A person is legally blind if vision in the best eye, response to changes in intraocular pressure [71].
when best corrected, is no better than 20/200 or if The retinal circulation supplies the inner half
the visual field is less than 20◦ in diameter. This of the retina in many animals (cat, primate, rat),
is still useful vision for mobility but is frequently but others rely exclusively on the choroid (rabbit,
called “low vision” rather than blindness. guinea pig). In humans, the central retinal artery
enters the eye at the optic nerve head and then
branches to form four retinal arteries that travel
21.3 Vasculature of the Retina superficially in the retina. These in turn branch
into a relatively typical arteriolar and capillary
Many retinal diseases are fundamentally vascular network that forms two layers in most of the
or have a vascular component, so it is important to inner retina, with the innermost layer in the inner
consider the dual circulation of the retina. There plexiform layer, and the deeper layer extending
are a number of more complete reviews of this as far as the outer plexiform layer. The retinal
subject [70–73]. Metabolically, it is often useful circulation is similar to brain circulation, with a
to think of the retina as two domains. The outer flow rate of about 40 ml/100 g-min [78]. The
retina, consisting primarily of photoreceptors, is retinal circulation has tight junctions between
supplied mainly by the choroidal circulation. The capillary endothelial cells that form the blood
inner retina, consisting primarily of the second- retinal barrier. In contrast to the choroidal circu-
and third-order neurons and glia, is supplied by lation, it has no autonomic control but good au-
the retinal circulation. Diseases that affect the toregulation with respect to arterial pressure [72].
vasculature typically cause blindness by affecting Retinal blood flow is also controlled by blood
either the inner or outer retina initially, but rarely gases [72] and changing metabolic demand in
both, and then there may be secondary effects on the inner retina, particularly greater retinal neural
the other region. activity [70, 79, 80].
The choroidal circulation is behind the retina,
separated from it by the retinal pigment epithe-
lium (RPE) (Figs. 21.1 and 21.8). The choroidal 21.4 Major Retinal Diseases
circulation has a very high flow rate, on the or-
der of 1000 ml/100 g-min [74], but in cat and It is estimated that there are nearly 15 million
primate, this is normally sufficient to supply only blind or visually impaired people in the USA,
the photoreceptors, not other retinal neurons [75, and for people over the age of 65, more than 10%
76]. The flow rate is high because the vessels of the population is legally blind. Age-related
are large and the resistance is low. Choroidal macular degeneration, glaucoma, and diabetic
capillaries are fenestrated, and the blood-retinal retinopathy are the most prevalent retinal diseases
barrier, analogous to the blood-brain barrier, is causing visual impairment [81]. Glaucoma and
provided by the tight junctions between RPE cells diabetic retinopathy are especially significant
rather than by the choroidal endothelium. (The because they affect many working age adults.
Fig. 21.8 Schematic of the relationship of the choriocap- by the retinal pigment epithelial (RPE) cells. The retinal
illaris and the retinal capillaries to the retinal neurons. circulation only occupies the inner half of the retina,
The choriocapillaris is separated from the photoreceptors leaving the photoreceptors in an avascular region that is
more than 100 μm thick
A few diseases with lower prevalence also need actly, and some in which an animal appears to
to be considered because of the potential for be a good model for the human disease [86, 87].
bioengineering solutions. In this discussion, In some cases, it is clear why the genetic defect
we move from genetic disorders to vascular kills photoreceptors. Gene transfer to the subreti-
disorders. nal space has been able to restore photoreceptor
structure, retinal electrical responses, and vision
in dogs [88, 89] and humans [22, 90–92]. Patients
21.4.1 Retinitis Pigmentosa with RP are also likely to be the first to benefit
from both electronic visual prostheses [2, 3, 5, 93]
The most common hereditary cause of blindness and optoelectonic approaches [25, 94].
is called retinitis pigmentosa (RP). It is estimated
to affect about 1 in 4000 [82] to 1 in 3000 [83].
There are more than 50 genetic defects in pho- 21.4.2 Macular Degeneration
toreceptor or RPE proteins that lead to loss of
photoreceptors. These may be autosomal or sex- Age-related macular degeneration (AMD or
linked and can be dominant or recessive. Many of ARMD) is a more prevalent photoreceptor
these diseases cause loss of rods first, with cone degeneration and is the leading cause of blindness
degeneration following later (rod-cone degenera- in the USA [81]. In one large study, some level
tion), but a few act in the reverse manner (cone- of AMD developed over 15 years in 14% of
rod degeneration). The inner retina seems largely individuals who were 43–86 with no AMD at
normal during the time when photoreceptors are the beginning of the study, and 8% of individuals
degenerating, but later there is often a loss and/or over 75 developed the more severe “wet,” or
disorganization of inner retinal neurons as well exudative form of AMD [95]. A hallmark of the
[84, 85]. dry form of macular degeneration is the presence
The time course of RP is variable, with some of extracellular deposits called drusen between
types leading to blindness in adolescence, while the RPE and choroid. Drusen contain both
others progress more slowly. There are several lipids and proteins, and are probably produced
cases in which the genetic defect is known ex- by the RPE [96]. Some drusen can be present
with no apparent symptoms; larger drusen are usually exhibit a loss of sensitivity (elevation
associated with localized photoreceptor loss of threshold) first in the mid periphery of the
[97]. Dry AMD sometimes proceeds to wet nasal visual field (temporal retina), and the loss
AMD. In the wet form, which is responsible gradually progresses closer to the central visual
for 75% of cases with severe visual loss, there field.
is choroidal neovascularization (CNV), in which While high IOP is the single most important
choroidal vessels proliferate, break through the risk factor, for glaucoma, some individuals with
RPE, and enter the retina and vitreous [98]. The elevated pressure (ocular hypertension) do not
vessels in all neovascularizations of the retina exhibit the retinal symptoms of glaucoma. Others,
are abnormal and may bleed or cause traction whose IOP is “normal,” still have visual loss
on the retina, resulting in retinal detachment [101]. These individuals are classified as having
or other damage. Laser treatment is sometimes normal or low tension glaucoma.
effective against CNV [19], but the remarkable The usual strategy in glaucoma involves drugs
breakthrough in recent years is that blockers of to reduce the inflow of aqueous humor, even if the
vascular endothelial growth factor, VEGF, given pressure is already “normal” based on population
intravitreally, can not only halt the progression of data. The inflow across the ciliary epithelium is
wet AMD but can improve vision [99]. controlled by several different pumps and chan-
nels, so there are a variety of drugs that can
be effective [104]. When drugs are ineffective,
21.4.3 Glaucoma surgery, sometimes including insertion of a shunt
[105–107], is performed to increase the drainage
Glaucoma is a slow neurodegenerative disease of fluid.
that primarily affects RGCs rather than photore-
ceptors. The principal risk factor for glaucoma
is elevated intraocular pressure (IOP), from its 21.4.4 Diabetic Retinopathy
normal value of about 15 mm to 2 SD higher than
the mean, about 22 mmHg [100]. Glaucoma is In both insulin-dependent (type 1) and non-
estimated to occur in 0.8–3 in 100 Caucasians, insulin-dependent (type 2) diabetes, elevated
but the incidence is higher in African Americans blood glucose over many years can lead to
[101]. The elevation of IOP is generally caused microvascular complications in several organs,
by a decrease in the conductance (c – called including the eye, the kidney, and peripheral
the “outflow facility”) to flow of the aqueous nerves. Almost all diabetic patients with a
humor out of the eye. The amount of fluid that disease duration greater than 20 years show some
leaves depends on this conductance and on the signs of retinopathy [108]. The earliest clinical
pressure between the inside of the eye (intraocular signs of retinopathy are microaneurysms and
pressure – IOP) and venous pressure. However, capillary leakage in the retinal circulation that
the inflow of fluid is driven by active transport, are especially apparent in fluorescein angiograms
which does not depend on pressure, so the IOP [108]. At the microvascular level, there is a loss
rises until it is high enough to make the outflow of pericytes and endothelial cells [109], the two
equal the inflow. cell types that comprise the blood-retinal barrier,
High IOP probably causes damage to the and plugging of capillaries with leukocytes and
RGCs by reducing retinal or optic nerve blood platelets [110–112]. Clinically, the next stage is
flow [102] and/or by compressing optic nerve further fluid leakage including hemorrhage, and
fibers as they pass through the lamina cribrosa. capillary nonperfusion in patches across the retina
Axonal compression blocks axonal transport and [108]. With the loss of retinal capillaries in this
causes retrograde degeneration of the RGCs “background” or “preproliferative stage” comes
[68, 103]. As the disease progresses, there is loss of visual function [113]. As the capillary
a loss of visual function. Glaucoma patients dropout progresses, there is proliferation of new,
abnormal tufts of blood vessels, promoted in part rhages in the retina, presumably because venous
by vascular endothelial growth factor (VEGF). pressure is elevated upstream of the occlusion,
The new vessels can grow out into the vitreous, but in some cases, venous occlusions may resolve
bleed, collapse, and shrink, and cause traction on without permanent visual loss [122]. Venous oc-
the retina. Since the retinal attachment to the back clusions may, however, lead to neovasculariza-
of the eye is tenuous, this traction can detach the tion in either the retina or the iris [123, 124],
retina from the RPE, leading to blindness [114, probably because VEGF is produced in the retina
115]. and diffuses to the anterior part of the eye [125],
Another serious consequence of diabetes that and can also cause macular edema, presumably
probably results from VEGF and the consequent because VEGF causes the retinal vasculature to
leakiness of retinal vessels is macular edema, leak fluid. The anti-VEGF compounds, as well as
which distorts vision by disrupting the highly or- laser treatment [126, 127], are useful under these
dered photoreceptor mosaic. Placement of many conditions.
small laser burns (panretinal photocoagulation) is
often used to treat both macular edema and pro-
liferative retinopathy. Also, the same anti-VEGF 21.4.6 Retinal Detachment
agents that are used in AMD appear to be useful
[116, 117]. However, both photocoagulation and A frequent result of proliferative diabetic
anti-VEGF agents are used only relatively late retinopathy is detachment of the retina. There are
and are not completely effective. Furthermore, other causes for detachment as well, including
while the vascular symptoms can be treated, sep- trauma, severe myopia (in which the eye is too
arate neural loss, probably unrelated to vascular large and the retina is stretched), detachments of
damage [118, 119], is untreatable. the vitreous from the retina, and retinal holes of
idiopathic origin [128]. In all cases, fluid gains
access to the subretinal space between the retina
21.4.5 Vascular Occlusive Disease and RPE, lifting the retina off and sometimes
detaching large areas. The photoreceptors, being
Like the brain, the retina is susceptible to vas- separated from the choroid and deprived of their
cular occlusive events that occur from thrombi main source of nutrition [129, 130], undergo
or atherosclerosis in either the arteries or veins apoptotic cell death unless the retina is reattached
[120]. These produce the retinal equivalent of by one of several surgical procedures that bring
strokes. The most serious type of occlusion is the retina and eye wall closer together [128].
one that affects the central retinal artery, because The quality of vision following these procedures
this prevents circulation to the entire inner retina. depends on the time between detachment and
Occlusion of a branch artery produces a scotoma reattachment and on whether the detachment had
(blind spot) in the region supplied by that vessel reached the central part of the retina.
because there is no redundancy in the retinal
circulation, unlike that provided by the Circle
of Willis for the brain. Experimentally-produced 21.5 Engineering Contributions
occlusions lead to irreversible damage to the pri- to Understanding Retinal
mate retina if they last more than about 2 hours Physiology
[121], which is a much longer window for re- and Pathophysiology
covery than one has for the brain, possibly be-
cause the vitreous and choroid provide a small Engineering approaches to understanding retinal
reservoir of glucose. Many treatments have been function date to the 1960s. Engineering methods
attempted, but none are accepted as a standard have been used by physiologists, psychophysi-
in arterial occlusion. Venous occlusions are more cists, and biomedical engineers who have con-
common. They often produce multiple hemor- structed mathematical models of the retina with
several goals in mind. These goals include pro- [131–134] and tiger salamander retinas [135–
viding a compact representation of a great deal 137], models that attempt to explain general prop-
of data, extracting parameters that characterize erties of retinal responses [138, 139], models of
retinal function and then investigating how those spatiotemporal transfer properties of horizontal
parameters vary with light and with retinal ec- cells [140, 141], models of direction selectivity
centricity, explaining and using the waveforms [142, 143], and models of light and dark adapta-
of the electroretinogram for diagnostic purposes, tion [34, 144].
and creating systems models whose transfer func-
tions are similar to those of the retina in order to
understand stages of signal transformation in the 21.5.1 Photoreceptor Models
retina or to create artificial vision systems. There
are models of many aspects of retinal function, Models of photoreceptor function are intended
and even more of visual function, and it will not to quantify the relation between incident light
be possible to review all of this work. Here we and photoreceptor hyperpolarization. There are
review some of the major analytical threads that many reviews of the physiology and biophysics
constitute retinal bioengineering. Section 21.5.1 of photoreceptors [145–147]. In darkness, pho-
discusses models of how light is transduced by the toreceptors have resting potentials that are depo-
photoreceptors into an electrical signal that can larized relative to those of many other neurons
be recorded from the photoreceptors themselves, (c. −30 mV), because their outer segments have
or within the retina, or at the cornea in the elec- a cation-selective channel with a high Na+ and
troretinogram (ERG). Section 21.5.2 covers the Ca+2 conductance. More of these channels are
smaller effort that has gone into modeling aspects open in the dark than in the light, keeping the
of the electroretinogram that are generated by cell depolarized. The number of channels that
second-order neurons. There is less of this work, are open, and therefore the current entering the
and little work on modeling bipolar cell responses cell, is determined by the level of cGMP, be-
in mammals, because it is difficult to record from cause binding of cGMP to the channel holds it
these small interneurons. Section 21.5.3 covers open. Absorption of a photon causes activation of
some of the many models describing the receptive rhodopsin, which then activates a G-protein (one
field and response properties of single RGCs or that requires GTP binding for activity) bound to
ensembles of RGCs in cats and monkeys. Sec- the photoreceptor discs, called transducin. Trans-
tion 21.6 then addresses a completely different ducin in turn activates a phosphodiesterase, which
category of models that characterize the retinal breaks down cGMP. Decreased cGMP closes the
microenvironment, focusing on nutrient and ionic channels, which decreases the inflow of Na+ and
balance. Ca+2 and causes hyperpolarization. (The synthe-
Models in these areas are all based on data, sis of cGMP from GTP is controlled by light only
rather than being completely theoretical. This indirectly, when levels of Ca+2 in the cytoplasm
does not mean that they are necessarily the best decrease and increase the activity of guanylate cy-
models or unique models, but they are constrained clase [148].) The light-evoked hyperpolarization
by the data and at least have descriptive validity. increases in amplitude with increasing illumina-
These areas are also ones in which sustained tion, up to a saturating value. The dependence
effort and refinement of models has taken place of response amplitude on illumination has often
over many years. These particular models, based been characterized by Eq. 21.2, which is some-
on data from nonhuman primates, cats, rabbits, times called the Naka–Rushton equation [149]
and more recently mice, also have led to conclu- when it is used in vision:
sions pertinent to the human retina.
Some models are omitted here, including most I
R = Rmax (21.2)
models of information transfer through the catfish I +σ
where R is the response amplitude at intensity out of the inner segments [150]. The current
I, Rmax is a maximum amplitude, and σ is the flowing through individual photoreceptors can
illumination at half-saturation. Sometimes this be determined by sucking outer segments of
does not rise steeply enough to fit the data, so a isolated photoreceptors into a pipette and forcing
modified form is used in which n is greater than the receptor current to flow through the electrode
1.0. [145, 151]. This clever method has been useful
in recording from photoreceptors that are too
In small to be impaled by a standard intracellular
R = Rmax (21.3)
In + σn microelectrode.
In the intact retina, most of the return current
Amplitude-intensity data from higher order to complete the loop flows through adjacent ex-
visual neurons can also be well described by one tracellular fluid, but some of the receptor current
of these equations. In addition to an increase in flows out of the retina and across the wall of the
response amplitude with illumination, the time eye before returning. The voltage drop associated
course of the photoreceptor hyperpolarization with this larger current loop produces a negative-
also speeds up with increasing illumination, and going signal as large as several hundred micro-
the leading edge continues to become steeper volts that can be recorded between the vitreous
even after amplitude saturation occurs (Fig. 21.9). humor or cornea and a reference electrode. This
The current that flows into the outer segments makes it possible to record photoreceptor activity
is completed by current (mostly K+ ) flowing
Fig. 21.9 Responses of membrane current of a primate and are the inverse of the voltage changes that would
rod (top) and cone (bottom) outer segment, each in re- be observed with an intracellular electrode if the outer
sponse to several brief stimuli of different intensities. segment could be isolated from the rest of the neuron. The
Responses were recorded by sucking the outer segment cone responses are characteristically faster. Amplitude as
of isolated photoreceptors into a pipette and recording the a function of intensity for the two photoreceptors is plotted
current. The inward current in darkness of about −35 pA at the right. The half-saturating intensity for the cones was
is reduced by light. The rod reaches saturation, with all 100 times that required for the rods. (Oyster [68]) (after
channels closed, in the top two traces. All traces are Baylor [145])
essentially the impulse responses of the photoreceptors
Fig. 21.10 Left: Electroretinograms (ERGs) in response and faster at higher intensities. Right: Fits of the photore-
to flashes of several intensities in the dark-adapted human ceptor model (P3(t)) described in the text (with n = 4;
retina, showing the a-wave, originating in the photore- tp = 189 ms) to the early part of the ERGs. The intensities
ceptors as a suppression of the dark current, and the b- used for the right half of the figure were 2–4 log scotopic
wave, originating from the bipolars. Responses are larger td-sec, which were higher than those on the left. (Hood and
Birch [153])
from the surface of the human eye as part of the and then coupled these individual equations into
electroretinogram (ERG). The ERG manifesta- an overall model [146, 156].
tion of the photoreceptor signal is often called P3,
or PIII, because it was the third component of the 21.5.1.1 Input-Output Analysis of Rod
ERG to disappear following treatment with ether Responses
or anoxia [152]. If the stimulus is very bright, The input-output analysis by Hood and Birch
the initial part of P3 is observed in almost pure that describes the data [155] consists of an n-
form as the “a-wave” of the ERG (Fig. 21.10). stage low-pass filter for r(t), the impulse-response
For up to about 15 msec after a brief flash, the a- function of the photoreceptors, in which the time-
wave is a reasonably good reflection of the light- to-peak of the response is tp .
dependent current in the outer segment. Even
(n−1)
though the ERG is a complex set of potential t t
changes, its clear advantage is that it can be used r(t) = exp 1−
tp tp
to study retinal electrophysiology in the intact (21.4)
human eye, and potentially to investigate how
disease processes affect different types of retinal Here each response is normalized to a peak
neurons. response of 1.0, so it does not depend on illu-
The same models of photoreceptor activity mination. The a-wave, called P3(i,t) in this anal-
apply both to the signals from individual rods ysis, depends on r(t) and on the intensity of a
and to the leading edge of the a-wave of the brief flash of energy, i (where i is in scotopic
ERG, as long as one uses diffuse light, which troland-seconds, a measure of light incident at the
stimulates many photoreceptors. There have been cornea). P3(i,t) involves a second stage, which is
two different approaches to modeling the onset of a saturating exponential nonlinearity:
photoreceptor activity that eventually converged

to the same mathematical form. One model fit- − ln 2 · i · r(t)
P 3 (i, t) = 1 − exp RmP 3
ted families of a-wave responses to brief flashes σ
of different intensities to an input-output analy- (21.5)
sis having a few characteristic parameters [153–
155]. The other attempted to characterize each of This introduces two new variables. RmP3 is
the known steps in transduction by an equation the maximum amplitude of P3 in response to
Fig. 21.11 a: The first

40 msec of human ERGs in
response to brief flashes
ranging from 402 to
128,000
photoisomerizations per
rod. Each is the average of
four stimulus presentations
except at the highest
intensity, which is an
average of two. b: Fits
(dotted line) of the model
described in the text to the
early parts of these
responses. The same
parameters, A = 8.7 sec−2
and teff = 2.7 msec, were
used to fit all responses.
(Breton et al. [157])
bright flashes. It is assumed to be just the sum 21.5.1.2 Biochemically Based Analysis
of maximum responses of individual photorecep- of Rod Responses
tors. The other new parameter is the semisatura- Lamb and Pugh ([146, 156]) derived an alternate
tion constant, σ , which reflects the sensitivity of model that was based on the biochemical steps
the photoreceptor to light. It is the value of i · r(t) in transduction. In this very detailed model, also
at which P3(i,t) = RmP3 /2. These two equations presented in a simplified form by Breton et al.
were used to fit families of a-waves, setting tp and [157], the dynamics of five major processes were
n fixed and extracting RmP3 and σ . The chosen considered: (1) activation of rhodopsin by light,
value of n was 4, implying a four-stage filter. (2) activation of transducin by rhodopsin, (3)
The parameter tp is not directly observable in activation of PDE by activated transducin, (4)
a-wave recordings, because other waves intrude hydrolysis of cGMP by activated PDE, and (5)
before t reaches tp , so tp was set for human a-wave channel closure caused by the fall in cGMP. Other
recordings to be the value observed in primate models had taken similar approaches [158, 159],
rods, 189 msec. Fits of this model for the human but Lamb and Pugh started at the most molec-
a-wave are shown in Fig. 21.11. ular level. Their model was the first to explic-
itly consider that rhodopsin diffuses in the disc nored originally because the responses modeled
membrane to cause activation of many transducin were current rather than voltage responses, (2)
molecules, which converts the step activation of allowing the isomerizations to take place over a
rhodopsin by a flash into a ramp increase in short interval rather than all at t = 0, and (3)
transducin activity. Formally, it also allowed for recognizing that for high intensity flashes, the
the longitudinal diffusion of cGMP in the cyto- response time course will be on the time scale of
plasm, although only isotropic conditions (i.e., teff .
illumination of the whole outer segment) were
considered. It turned out that the overall gain 21.5.1.3 Responses to Steps of Light
of transduction was an important parameter that The analyses described so far were only intended
came from the model. This parameter has been to address the initial hyperpolarization of the pho-
called A in subsequent work and was the product toreceptor in response to very brief flashes. They
of the gains of steps 2 through 5 above. “A” did not address the recovery of the response as
was related to the characteristic time constant rhodopsin and the subsequent steps are inacti-
of transduction, τ φ , by A = τ φ −2 . In terms of vated, or responses to steps rather than flashes of
timing, processes 2 and 4 above were found to light. In addition, the reproducibility of responses
contribute substantially to the time course of the to single photons, which underlies the very high
flash responses, and the others were very fast by sensitivity of the retina, was not modeled. Cover-
comparison. A small delay called teff was also ing activation and inactivation from brief flashes
needed to account for the onset of a noticeable to steps over the full range from single photon
change in PDE activity. The overall response was responses to saturation is clearly a challenging
then: problem, but it has been addressed [161, 162].
The many required differential equations are too
1 numerous to reproduce here, as they include more
R (Φ, t) = 1− exp − Φ · A · [t−teff ] 2
Rm
2 than 40 parameters, based on data from many
(21.6) sources. The overall model includes kinetics of
the response to single photons, the calcium de-
where is the intensity in isomerizations of pendence of cGMP synthesis, the inactivation of
rhodopsin, rather than scotopic troland-seconds. rhodopsin by arrestin and rhodopsin kinase, and
These two intensity units are related by a constant an integration of the single flash responses to give
that depends on the optics and light-capturing ef- step responses. The match between the model and
ficiency of rods, and this is different for different step responses in the salamander retina, just one
animals. While the Lamb and Pugh, and Hood of the types of comparisons between data and
and Birch formulations look different, Hood and model provided by Hamer et al. [161], is shown
Birch [153] showed that they had very similar in Fig. 21.12.
forms if t < tp , which are the only times at which While step responses of photoreceptors can
either model can be applied. The Lamb and Pugh be obtained experimentally from isolated pho-
model was originally applied to salamander rod toreceptors, another approach is needed to derive
responses, but it does a good job of fitting a- the photoreceptor response in the intact retina.
waves from human ERGs as well [157]. For hu- When one uses a test flash to evoke an ERG
man rods, the amplification constant is about 100 response from the retina, one usually obtains a
times higher, which means that the responses complex waveform. However, if a test flash is
develop about 100 times faster than in poikilo- followed at different intervals by a brief, bright
thermic salamanders. The fits to the ERG a-wave “probe” flash designed to drive the photoreceptor
are even better if one rectifies certain simplify- current all the way to saturation (the “paired-
ing assumptions that were made originally [160]. flash technique”), one can determine how far the
These are (1) taking into account the photore- photoreceptors were from saturation before the
ceptor membrane time constant, which was ig- probe flash, and this allows reconstruction of the
pro-23-his mutation in rhodopsin, the decrease

in amplitude of the a-wave cannot be explained
completely by loss of photoreceptors or disks, but
must involve a decrease in the gain of transduc-
tion [164]. These patients also have a delayed
recovery of rod responses. This is not always
the case, and in a different condition, Stargardt’s
disease, the amplitude decreases but recovery is
normal [170].
Fig. 21.12 Photocurrent responses to steps of light of 21.5.2 Post-Receptor ERG Analyses
increasing intensity from a newt rod (red traces) and the
model of Hamer et al. described in the text (blue traces). 21.5.2.1 B-Wave Analyses
The model recovers more rapidly than the actual responses
If the photoreceptor models accurately described
at the end of stimulation, but a better match (dashed lines)
can be made by incorporating reversibility of rhodopsin the time course of P3 for times up to the time-
binding to arrestin. (Hamer et al. [161]) to-peak of the response, then the model fits
could be subtracted from the entire ERG to
reveal the time course of the remaining ERG
complete photoreceptor response to the test flash components. After the a-wave, the ERG is
(Fig. 21.13) [164–166]. Subsequent work on a dominated by the b-wave (also called P2 or
strain of mouse that is missing post-receptoral re- PII), which arises largely from the activity of
sponses (the nob mouse) validated this approach bipolar cells [171]. That “photoreceptor-free”
[167]. An extension of the paired flash method led ERG waveform could then be used to derive a
to the development of a descriptive equation that model of the second level of retinal processing.
characterized the complete time course of rod re- Such an analysis was done by Hood and Birch
sponses in mice, but this did not link biochemical [153, 172] and led to a three-stage model for
steps to their electrical consequences [168]. This the b-wave, rather than the two stage model for
model was similar in form to Eq. 21.5 above, but rods. Unfortunately, the photoreceptor voltage
included the dynamics of what is supposed to be responses seen in the mammalian ERG depart
the underlying single photon response. from the predictions of Eqs. 21.5 and 21.6,
which are based on the photoreceptor current.
21.5.1.4 Diagnostic Value of a-Wave The voltage shows a partial rebound that is not
The ERG has always had some diagnostic value, present in the photoreceptor current [173–175].
because it is the only objective measure of reti- In some species, the rebound occurs because
nal neural function available for use in humans. voltage-dependent inner segment currents are
Until the models discussed in Sect. 21.5.1.2 al- activated by hyperpolarization [176–178], but in
lowed a deeper understanding of the waveforms, mammals, it is more likely a capacitive current
however, most of the conclusions about retinal [179, 180]. The rebound means that a subtraction
function from ERGs were simply based on the of the photoreceptor activity from the ERG based
amplitude or timing of components in the re- on photocurrent (Eqs. 21.5 and 21.6) subtracts
sponses to flashes of light. These may or may too much, even if one only considers times
not have been optimal for revealing particular shorter than the time-to-peak of the photoreceptor
disease processes. With more quantitative mod- impulse response.
els of the ERG, more detailed conclusions have Consequently, another approach is needed to
become possible, particularly with respect to remodel the dynamics of the retina’s second stage.
tinitis pigmentosa [169]. For example, it appears To quantitatively address responses of second-
that in photoreceptor degeneration caused by a order cells in the ERG, Robson and Frishman
Fig. 21.13 The paired flash technique for revealing the occurs at t = 0. The ERG in response to one probe flash,
entire time course of photoreceptor responses. a: Illus- labeled Am , is shown as a solid curve, and those at other
tration of the method with hypothetical data. The top test-probe intervals are dashed. Am0 is the probe flash
trace shows an ERG in response to the “test flash” of amplitude when given alone. The curve connecting the
moderate intensity followed after approximately 200 ms data points is the reconstructed response of the rod to the
by a “probe flash” designed to saturate the response of test flash. b: Reconstructions of rod responses using the
the rods. The presence of the b-wave makes it impossible method in part A. The test flashes were 11 scotopic td-sec
to determine the time course or amplitude of the photore- (squares), 44 scotopic td-sec (circles), and 320 scotopic td-
ceptor’s response to the test flash alone. The lower graph sec (triangles). Probe flashes in all cases were 1.2 × 104
shows amplitudes of the responses to the probe flashes scotopic td-sec. (Pepperberg et al. [163])
given at different times before and after a test flash, which
[171, 181, 182] blocked activity of neurons after tamatergic agent, 2-aminophosphonobutyric acid
the bipolar cells in cats, with intravitreal appli- (APB), which acts at the outer plexiform layer,
cations of the glutamate antagonist N-methyl- one obtains a purely photoreceptor response. By
DL-aspartic acid (NMDLA). NMDLA blocks ac- subtracting the APB-altered response from the
tivity at the inner plexiform layer, but not at NMDA-altered response, one can isolate the b-
glutamatergic synapses at the outer plexiform wave (and anything else that might come from the
layer, so it should simplify the ERG if there middle of the retina). When this was done, a good
are any components from amacrine or RGCs. fit to the rising side of the rod b-wave could be
Then, if one blocks the rod b-wave with a glu- obtained with a six-stage process, of which three
are carried forward from the main activation steps across the whole retina. For a major functional
in the photoreceptor, and the others are associated deficit, ophthalmoscopic inspection might pro-
with the response of the bipolars. On-bipolars vide enough additional information to identify the
respond to glutamate with a G-protein mediated site of the problem, but for more subtle changes,
cascade that was expected to introduce three more this is not the case. One might expect that local
stages. The rising side of the isolated b-wave stimulation of different parts of the retina with
could be fitted, therefore, by: a spot of light would be able to elicit a corneal
ERG from just the part of the retina stimulated.
Rb (t) = k · I · (t − td )5 (21.7) However, in practice, when a bright flash is pre-
sented on a dark background, light scatters away
where Rb (t) is the b-wave as a function of time, from the intended location of the stimulus and
k is a constant, I is illumination, and td is a brief produces responses across the retina, so localized
delay, less than 5 msec, which is primarily retinal, flashes do not allow one to isolate responses from
but includes any small delay caused by filtering different regions. Studies were carried out in the
during the recording. At present, there is no model 1950s that showed that a perfectly normal ERG
that describes the complete time course of the b- could be generated by flashing a light at the optic
wave. disk, which, of course, has no photoreceptors,
This discussion of the ERG is necessarily sim- and these studies emphasized the contribution of
plified and has not taken into account the other scattered light [187]. One successful solution has
components that exist. For instance, at very low been the multifocal ERG [188]. In this technique,
levels of illumination, one observes a different one presents a grid of approximately 100 flashing
negative going wave than the a-wave, called the elements to a region 20–50◦ in diameter (Fig.
scotopic threshold response that arises in the in- 21.14). The elements are hexagons of varying
ner retina [183, 184]. Even at higher levels of size, so the grid looks like a distorted honeycomb.
illumination, the a-wave may not be entirely due Each element turns on and off with a pseudoran-
to photoreceptors but may contain some inner dom sequence called an m sequence [190], uncor-
retinal contribution [185]. Retinal ganglion cell related with the behavior of any other element.
behavior is not present in typical flash responses Because there is a reasonably high mean level
but can contribute to ERGs if different stimu- of illumination (typically in the photopic range)
lus conditions, particularly patterned stimuli, are in all elements over time, light scattered from
used [186]. Finally, for animals with significant bright to dark regions has less influence on the
numbers of cones, like primates, care must be cells in the darker regions than it would if the
taken in analyzing the ERG to separate the faster dark regions were dark-adapted. This means that
cone-mediated components from the slower rod- signals are generated reasonably specifically in
mediated ones. the part of the retina corresponding to each ele-
ment. The local responses from each element are
21.5.2.2 Computing ERGs from Specific invisible until a cross correlation is done between
Retinal Areas the voltage and the pattern of stimulation for each
The ERG provides an objective electrophysiolog- element, but then one can extract components of
ical test of retinal function, but one of its disad- the signal that are correlated with the activity of
vantages as a diagnostic tool has been the inability each retinal element. One can look at different
to determine what region of the retina generates orders (kernels) of these multifocal responses, but
the signal. The ERG represents summed activity the first-order ones look very much like miniature
across the retina, so if the temporal half of the ERGs. There is no new mechanistic model pro-
retina were severely damaged, the ERG a- and b- vided by this technique, but it is an application of
wave amplitudes might be approximately halved engineering methods to derive more information.
relative to normal, but one could not infer from This has received wide application across a spec-
the ERG that the damage was in the temporal trum of retinal diseases [189].
retina as opposed to a general loss of function
Fig. 21.14 Left: Stimulus pattern used to elicit the mul- patches of the fovea, are about three orders of magnitude
tifocal ERG. This pattern has 241 elements that can be smaller than those that are obtained when a full field flash
turned on and off independently, and subtend the central is given (Figs. 21.10 and 21.11). The right panel is a 3D
23 degrees of the visual field. The center panel shows map of the ERGs from each region. The black arrow shows
miniature ERG responses at each location obtained by that there is a small response from the blind spot. The red
cross-correlation of the voltage with the stimulus pattern. arrow illustrates a region that was known to have an RPE
Note that the largest signals, which come from small defect and generated smaller responses. (Lai et al. [189])
Another approach to noninvasively isolate the been feasible to record from them in vivo for
responses of small patches of retina involves the 70 years. A great deal of this work was originally
use of multiple electrodes in a contact lens placed on the cat retina and then on primates in vivo,
on the cornea [191, 192]. The signals recorded but most recent studies of RGCs are in vitro with
by each electrode are slightly different, like the single electrodes or multielectrode recordings in
responses recorded with multiple EEG electrodes mice and primates. Ganglion cell responses are
on the scalp, and with an electrical model of the only present in the ERG under special stimulation
eye, one can solve an inverse problem to deter- conditions, and even then it is not possible to
mine where different responses were generated. separate the responses of different types of RGCs,
It is then possible to use more traditional diffuse so essentially everything we know about ganglion
flash stimuli, rather than the multifocal stimulus. cell behavior comes from animal studies. There
are several reviews of the vast ganglion cell lit-
erature in cats and nonhuman primates [36, 37,
21.5.3 Ganglion Cell Models 57, 193–196]. The discussion below focuses on
models. Recent reviews cover many aspects of the
At the other end of the retina from photoreceptors benefits and limitations of different ganglion cell
are RGCs. Most ganglion cell models to this point models and experimental strategies [197, 198].
have treated the retina as a black box receiving
light inputs and generating neural outputs. Re- 21.5.3.1 Systems Analysis
membering that the RGCs are the only output Systems analysis techniques began to be applied
from the retina to the brain, it is important to to the retina by Enroth-Cugell and Robson [199].
characterize the retinal output and understand the This engineering approach dominated much of
several parallel channels of information that com- retinal physiology, not just the thinking of en-
prise this output, and this does not require specific gineers. They used stationary “grating” patterns
understanding or modeling of the photorecep- whose contrast was sinusoidally modulated in
tors or interneurons. Also, because RGCs are the one dimension in space, and temporally varied
only retinal cell type (apart from a minority of either sinusoidally or as a square wave in time.
amacrine cells) that fire action potentials and can Grating patterns are characterized by a luminance
be studied with extracellular recordings, it has profile L:
L (x, t) = Lmean + L1 sin (2π kx + ϕ) ∗ M(t) RGCs projecting through the lateral geniculate to
(21.8) visual cortex, could be discriminated by whether
the light distribution in the receptive field was
where L1 is the sine wave amplitude (Lmax −Lmean ), reported on linearly or nonlinearly by the RGCs.
x is distance in visual angle in degrees, k is For the X cells, the linearity was quite remark-
spatial frequency, usually expressed in cycles able. It was possible to position a high spatial
per degree of visual angle, φ is the phase of frequency stationary grating on the receptive field
the grating with respect to the receptive field, so that contrast reversal of the light and dark bars
and M is the sinusoidal or square wave temporal led to no response from the cell, even though
reversal. Grating patterns are shown in Fig. 21.15. the photoreceptors and bipolar cells must all have
Alternately, “drifting” gratings have been used, been producing responses (Fig. 21.15). Shifting
in which the temporal modulation is caused the phase of the grating with respect to the recep-
by a continuous variation in spatial phase at a tive field away from this “null position” yielded
frequency of f Hz: large responses from the cell at the fundamental
frequency of contrast reversal. For Y cells, there
L (x, t) = Lmean + L1 sin (2π kx − f t) (21.9) was no null position; all positions of the grating
evoked responses from the cell indicating that
Contrast refers to the amplitude of the sine summation of light was nonlinear (Fig. 21.15).
wave divided by the mean illumination, the As also shown in Fig. 21.15, the X-Y distinction
Rayleigh contrast [34]: proved to be a fundamental property of the cells,
independent of adaptation level [200]. X and Y
C = L1 /Lmean = (Lmax − Lmin ) / (Lmax + Lmin ) cells differ not only in spatial summation, but in
receptive field size [205, 206], and soma and den-
= (Lmax − Lmin ) / (2Lmean )
dritic field size [205–208], with Y cell receptive
(21.10)
fields being about three times as broad as X cells
at any eccentricity. The conduction velocity of Y
where L is mean, maximum, or minimum lu-
cell axons is also faster because they have larger
minance of the pattern. Sinusoidal patterns have
axons [205, 209].
become standard for this field because arbitrary
Hochstein and Shapley [210, 211] further an-
patterns can be represented by the Fourier sum
alyzed Y cells and showed that both X and Y
of such patterns. To the extent that the retina
cells had a linear response at the fundamental fre-
operates linearly, retinal responses to arbitrary
quency of temporal modulation whose amplitude
stimuli can be predicted by knowing the spatial
depended on the phase of the grating with respect
and temporal tuning curves of RGCs. Further, un-
to the receptive field (Fig. 21.16). In addition, Y
like flashing spots, gratings are effective stimuli
cells had an additional nonlinear response that
for probing all levels of the visual system, includ-
could be characterized as a second harmonic that
ing visual cortical neurons and psychophysical
was independent of phase [211] and was most
analyses of human performance [201]. An alter-
pronounced at high spatial frequencies. For Y
native to the use of sinusoidal gratings that does
cells, the second harmonic was at least twice
not assume linearity is the use of pseudorandom
as large as the fundamental at some spatial fre-
[202] or white noise stimuli [203, 204], which can
quency, and for X cells, the second harmonic
reveal nonlinear behavior, and we will consider
was always less than the fundamental, providing
the insights that these have provided below.
quantitative support for a true dichotomy between
these cell types [211] rather than a range of prop-
21.5.3.2 X and Y Cells in Cat
erties. This work led to an important modification
Enroth-Cugell and Robson [52, 199] discovered
of the center-surround model of RGCs to include
that two prominent classes of RGCs in the cat
small, nonlinear subunits (Fig. 21.17; [210, 212])
retina, which are believed to make up most of the
that may arise from the behavior of amacrine cells
Fig. 21.15 Difference in spatial summation between X the grating in odd symmetry. The bottom four histograms
and Y type cat retinal RGCs. At the top are sinusoidal are from a Y cell at two backgrounds separated by four log
grating patterns positioned in odd symmetry (spatial phase units, showing that the Y cell generates frequency doubled
of zero degrees) and even symmetry (spatial phase of 90◦ ) responses at both backgrounds when the grating is in odd
on the receptive field. The grating contrast reversed with symmetry. The contrasts were as follows: a: 0.2; b: 0.2;
the timing shown at the bottom of the figure. The top four c: 0.7; d: 0.3; e: 0.07; f: 0.03; g: 0.4; and h: 0.2. The
histograms are from an X cell at two background levels spatial frequency was chosen to be above the peak of the
separated by three log units, illustrating that summation is contrast sensitivity curve for the fundamental response.
linear at both backgrounds, because there is no response to (Linsenmeier and Jakiela [200])
[213]. The work on cat RGCs also clearly showed cat X cells, i.e., their contrast sensitivities as a
that there were parallel streams of information function of spatial frequency, and the transfer
leaving the retina other than the on-off dichotomy functions of the linear part of cat Y cell receptive
that had been identified by Kuffler [53] and work fields were analyzed later [206, 210]. Contrast
on other species continues to explore the different sensitivity is the reciprocal of the contrast needed
parallel pathways. to evoke a small fixed response from the cell at
the fundamental frequency of contrast reversal
21.5.3.3 Difference of Gaussians Model or grating movement. This measure was adopted
of the Receptive Field rather than response amplitude for two reasons.
Enroth-Cugell and Robson [199] also quantita- First, Enroth-Cugell and Robson were interested
tively described the spatial transfer functions of in linear behavior, so they wished to remain in
Fig. 21.16 Spatial phase

dependence of the linear
(fundamental) and
nonlinear (second
harmonic) responses of an
X cell (top) and a Y cell
(bottom) to
contrast-reversing gratings,
as shown at the top. X cells
have negligible second
harmonic responses, while
Y cells have nonlinear
responses that are present
at all contrasts, and which
are larger than responses of
the linear center receptive
field mechanism at high
spatial frequencies.
(Enroth-Cugell and
Robson [52])
Fig. 21.17 Modification of the difference of Gaussians half-wave or full-wave rectified response that appears as
receptive field model to account for the nonlinear re- a frequency doubling in response to stationary gratings,
sponses of Y cells. The data are consistent with the exis- and may appear as an elevation of the mean rate of firing
tence of a number of subunits, each smaller than the center, in response to a drifting grating. (Hochstein and Shapley
within the receptive field. Each subunit generates either a [210])
the linear part of the response versus contrast

relationship. The responses they recorded of 10–
15 impulses/sec in amplitude [214] allowed them
to ensure that the response amplitudes were in
that linear range. Second, they wanted to be able
to relate their findings to measures of human
visual performance, which were beginning to use
systems analysis techniques at about the same
time. It is feasible to determine the minimum
contrast at which a person sees a grating (i.e.,
the contrast sensitivity), but not the sizes of the
neural responses in the human retina or brain.
The results of measuring contrast sensitivity as
a function of spatial frequency were interpreted
as the spatial frequency domain representation of
the spatial “Difference of Gaussians” model [58].
The point weighting function, expressed in radial
coordinates, assumes a linear addition of center
(c) and surround (s) and is given by:
cW (r) = Wc (r)–Ws (r)

= Kc exp −(r/rc )2 –Ks exp −(r/rs )2
(21.11)
The corresponding spatial frequency represen-
tation is:
Fig. 21.18 Panel a: The characteristic parameters, ra-
dius and peak sensitivity for the center mechanism in
S (ν) = Sc (ν) –Ss (ν) the Difference of Gaussians model in spatial coordinates.
Similar parameters define the surround. Panel b: Symbols
= Kc π r c 2 exp –(π r c ν)2 (21.12)
show the response of an on-center X cell to gratings of
different spatial frequencies at a temporal drift rate of 2 Hz.
–Ks π r s 2 exp –(π r s ν)2
Fits of the Difference of Gaussians model to these data
yielded the solid curve, which was comprised of the spatial
where W is the sensitivity as a function of radial frequency tuning curves for the center (C) and surround
position, and S is the contrast sensitivity (the (S). The C curve and the solid curve are the same at high
reciprocal of the contrast required for a particular spatial frequencies, because high spatial frequencies are
invisible to the surround. The receptive field profile in a
small response amplitude) at spatial frequency ν. was generated from the parameters obtained for this cell.
The K’s and r’s are the maximum sensitivities (Linsenmeier et al. [206])
and characteristic radii (at K/e) of the center and
surround, as shown in Fig. 21.18. This model fits
the spatially linear parts of the responses of both cells identified as X would include not only P
X and Y cells [199, 206, 211]. cells, but some M cells as well. In addition, as
As noted in Sect. 21.2.3, monkey RGCs pro- noted earlier, some P cells and small bistratified
jecting to the LGN are generally designated P primate RGCs have color opponency [56, 195,
(or midget) and M (or parasol) rather than X and 216, 217]. The Difference of Gaussians model
Y for which they are not generally considered developed for cat RGCs also describes concen-
exact homologs. All P cells have linear spatial trically organized primate RGCs [218]. There is
summation, but M cells may have spatially linear evidence from experiments with chromatic stim-
or nonlinear behavior [195, 215]. Thus, primate uli that the surround mechanism of at least some P
180◦ out of phase and can be regarded as sub-

tractive, but this is true for only some temporal
frequencies. In order to deal with the limitations
of the original Difference of Gaussians Model,
several investigators [214, 219–223] used models
that can be called “Gaussian Center-Surround
Models,” which allowed the temporal phases of
both center and surround to vary with temporal
frequency. These have five to eight parameters,
rather than the four parameters of the Difference
of Gaussians model. The response in the Gaussian
Center-Surround Model of Frishman et al. [223]
had six parameters, allowing center and surround
responsivity to vary with temporal frequency, ω,
Fig. 21.19 Relation between the peak sensitivity and size R (ν, ω) = Rc (ν, ω) + Rs (ν, ω) (21.13)
of the center and surround for primate M and P RGCs. The
slope of the line for centers of P and M cells and surrounds
of P and M cells is about −2 on this log-log plot, indicating R is responsivity of the cell or of the center
that peak sensitivity is inversely proportional to the area of or surround, a new term that means amplitude
the center or surround. (Croner and Kaplan [218]) divided by contrast. It is used only when the
response is small enough that it is in the linear
cells is absent in the middle of the receptive field part of the response versus contrast function and
[216], which is not predicted by the model, and is functionally equivalent to sensitivity. R can be
that circular symmetry is an oversimplification expressed in terms of magnitude and phase of the
[38, 54]. However, for both cat X and Y and center and surround components:
primate P and M cells, this model is valuable
| R (ν, ω)| eiP (ν,ω)
because it allows an analysis of how the different
2
receptive field parameters depend on eccentricity, = | Rc (0, ω)| eiPc (ω)−[π ν rc (ω)]
and how they depend on each other. For instance, 2
the larger the receptive field center, the lower the + | Rs (0, ω)| eiPs (ω)−[π ν rs (ω)]
peak sensitivity under photopic conditions (Fig. (21.14)
21.19) [206, 218], and this tradeoff works in such
a way that the integral under the center Gaussian Here the quantities in the absolute value sym-
is almost independent of center radius. Also, de- bols represent the strengths of the center and
spite adjustments in all the individual parameters surround. It turned out that not only temporal
characterizing the receptive field, the integrated phase but also center and surround radii, and
strength of the surround relative to the center center and surround strength, had to be allowed to
tends to be fairly tightly constrained (average of vary with temporal frequency [221, 223]. When
0.73 in cat [206] and 0.55 in monkey [218]). The this was combined with the fact that center and
Difference of Gaussians model also fits receptive surround strength vary with spatial frequency, the
fields of cat and primate LGN cells [219, 220]. overall behavior of RGCs depended strongly on
temporal frequency. This can be seen in both tem-
21.5.3.4 Gaussian Center-Surround poral frequency tuning curves at selected spatial
Models frequencies (Fig. 21.20) and spatial frequency
The Difference of Gaussians model works when tuning curves at selected temporal frequencies,
the center and surround responses are temporally which were fitted by Eq. 21.14 (Fig. 21.21).
Fig. 21.20 Dependence of the temporal tuning curve of similar temporal tuning curves for spatial frequencies at
cat X cells on the spatial properties of the stimulus. On the peak of the spatial tuning curve, which may involve
the left are the amplitude and phase of the responses for some surround, and at a spatial frequency above the peak,
17 on-center X cells when the stimulus was a diffuse where the response is solely due to the center. Responsivity
field (i.e., zero spatial frequency) that stimulated both is response divided by contrast. (Frishman et al. [223])
center and surround. In the center and right panels are
21.5.3.5 More Complex Retinal In the work discussed so far, the stimuli were
Ganglion Cell Models modulated at one temporal frequency at a time. A
Unfortunately, while models can be fitted to more general approach is to use white noise or a
individual spatial and temporal frequency tuning sum of discrete temporal frequencies as stimuli. It
curves to investigate the parameter space, this is then possible to use first-order responses (i.e.,
does not mean that there is a comprehensive those response components at the input tempo-
systems model that can predict spatiotemporal ral frequencies) as an alternative way of inves-
behavior completely, even for X cells. Another tigating linear behavior. By measuring second-
model represented each stage of processing and higher-order components present in the re-
by cable equations, and either a feedforward sponses, one could also investigate nonlinear be-
or feedback loop was used to represent the havior. Victor and Shapley [202, 228, 229] took
interaction of center and surround [224]. While this approach and used a sum of six or eight
this model did fit data reasonably well, it did not sinusoids that were nearly incommensurate in
take advantage of the existing Gaussian models. temporal frequency (i.e., no individual test fre-
Extensions of the Gaussian analyses have been quency was an integer multiple of another and
made to investigate ganglion cell properties at could not be created by a sum or difference of two
different adaptation levels [222, 225–227], but others). This series of studies cannot be reviewed
there is no comprehensive model. completely here, but it supported most of the
fundamental conclusions about X and Y cells
Fig. 21.21 Dependence of the spatial tuning curve of frequencies (2, 40, 52, and 60 Hz). Solid lines are fits to
cat X cells on the temporal frequency for four temporal the Gaussian Center-Surround model described in the text.
(Frishman et al. [223])
outlined above. One striking new result of their at high temporal frequencies made the cell almost
work, however, was the finding of a “contrast “ignore” increases in contrast at low temporal
gain control” as shown in Fig. 21.22 [202]. On frequencies. This behavior was observed more
the right are responses of a Y cell to individual strongly in Y cells than in X cells, but occurred
sinusoidal stimuli at different contrasts, showing in both. As shown in the lower part of the figure,
the intuitive result that the shape of the temporal the temporal phase of the response components
tuning curve is independent of contrast. However, also shifted with contrast.
on the left, a sum of sinusoids was used, and in
this case, the responses at low temporal frequency 21.5.3.6 Multielectrode Recordings
increase little with contrast, and it is only those at The work discussed so far was nearly all per-
higher temporal frequencies that grow with con- formed by recording responses from one cell at a
trast. Thus, the presence of stimulus components time. Early work to investigate correlations of re-
Fig. 21.22 One manifestation of the contrast gain con- a degree in diameter, positioned to produce a maximal fun-
trol. On the left are responses elicited from a Y cell damental response and reversing in contrast sinusoidally.
when six stimuli were presented simultaneously at differ- Each curve represents a different contrast (0.0125, 0.025,
ent temporal frequencies (shown on abscissa). The points 0.05, and 0.10 per sinusoid from bottom to top). On the
show the amplitude and phase of the fundamental at each right are responses of the same cell when each temporal
temporal frequency. The stimulus was a stationary bar, half frequency was presented separately. (Shapley and Victor
[202])
sponses of adjacent or nearly adjacent RGCs was for salamander, mouse, rabbit, and primate retina.
done by Mastronarde [230–232], who used two The basic design is shown in Fig. 21.23. The
electrodes. His work showed that the discharges retina is stimulated with an array containing many
of like type (e.g., pairs of ON-center) retinal pixels of temporal white noise, and spikes are
RGCs with overlapping receptive field centers collected on hundreds of electrodes, whose sig-
had positive correlations in their firing patterns, nals are multiplexed. Receptive fields can be char-
while the discharges of RGCs with unlike over- acterized by spike-triggered averaging (STA), a
lapping receptive fields (e.g., an ON-center and key analytical tool in this approach. STA analyzes
OFF-center pair) were correlated negatively. spike trains to determine which pixels were reli-
In the mid-1990s, it became possible to extract ably bright or dim before a spike from a particular
the retina and lay it on a bed of electrodes, so neuron, and therefore which pixels are in the
that the discharges of multiple RGCs could be cell’s receptive field. Investigators can thereby
recorded simultaneously [233], and subsequent map the receptive fields of all the cells of a
improvements of the technique have been used particular subtype in a patch of retina. Using this
Fig. 21.23 Multielectrode recording. The left panel temporal pattern of other pixels. Cross correlation of spike
shows a retina placed ganglion cell side down on an array trains and the stimulus pattern allows the definition of
of electrodes like that shown in the upper right. A lens receptive fields of many cells simultaneously. Positions
is used to create an image of the stimulus on the retina. and shapes of receptive fields of cells recorded from one
The stimulus is composed of many pixels, the luminance retina are shown at the lower right for ON and OFF M cells
of which varies over several levels stepwise with a pattern (top) and ON and OFF P cells (bottom). (Litke et al. [234])
governed by white noise so that is uncorrelated with the
approach, Gauthier et al. [38] further explored the timing of adjacent RGCs that can be seen in
concept that each type of ganglion cell type tiles cross correlations [235, 236], probably because of
the retina, so that several parallel representations common inputs rather than connections between
of the visual scene are transmitted to the brain. the RGCs themselves. The question of whether
The new result was that each type of ganglion cell these correlations are useful or important could be
(M and P, ON and OFF center) is used optimally. answered only with a model [237]. The filters in
The receptive fields were not exactly circular, the full model are shown in Fig. 21.25. These only
but an “indentation” in the receptive field of one roughly correspond to physical entities but do
cell was matched by a “protrusion” in the recep- mimic responses of real cells. Each ganglion cell
tive field of the adjacent cell; they interlocked, is represented by a spatiotemporal filter, which is
as shown for one class of cells in Fig. 21.24. essentially a receptive field, followed by a non-
This meant that there was little overlap, and little linearity to represent the spike generation mecha-
space in the visual field that was not covered. nism, followed by a Poisson process to create the
The shapes were not randomly distributed. Math- actual spike timing. There is also a “post-spike
ematical rotation of each receptive field about filter” that feeds back to modify the nonlinear-
its center always led to more overlap and less ity, essentially accounting for voltage-dependent
coverage [38]. conductances that come into play after spikes.
Multielectrode recordings have also furthered Spiking of one ganglion cell is coupled to other
our understanding of the correlations between cells to modify their firing, and this brings in the
spike trains of neighboring cells. For P cells, there synchrony. The model can be tuned to match the
is some level of synchronization between spike firing rates of cells and the correlations observed
Fig. 21.24 Detailed receptive field center shapes for all all cells, about 36% of the peak response for this particular
the ON-center M (parasol) and ON-center P cells (midget) sample. Small white areas represent overlap in the recep-
in a patch of primate retina. Because the responsivity of tive fields. With receptive fields mapped precisely, they are
cells is maximal in the middle of the receptive field and not oval, but mesh to leave few gaps between cells. Holes
then falls off gradually, there is no absolute measure of in the midget array probably reflect failure to record from
size, so the boundaries of the receptive fields were defined some cells rather than the absence of cells. (Gauthier et al.
by setting a specific response level that was uniform for [38])
between pairs of cells in a patch of retina. But, up much of the projection to higher visual cen-
the coupling can also be left out of the model to ters. The RGCs accounting for the balance of
explore its effect. The coupling had no effect on the retinal output typically have axons that all
averaged responses, so a traditional analysis us- conduct more slowly, but they are heterogenous in
ing repeated presentations and poststimulus time terms of other properties. Some project to visual
histograms would not reveal any role for the cou- cortex, but many do not, and appear to subserve
pling. However, coupling did give a population roles other than perception. In cat, these were
response in spike timing to a single presentation lumped together in a diverse group called W
that was less noisy than the population response cells by Stone and Fukuda [209]. One of their
where no correlations existed. Further, the full approximately seven types of W cells is the highly
model with coupling contained about 20% more linear “Q cell” [214, 240], also called sluggish-
information, in the sense that it was more accurate sustained [205] or tonic W cells [209, 241]. Their
in decoding spike trains to provide a representa- spatial summation is similar to that of X cells,
tion of the original stimulus than a model without but they have receptive field centers similar in
coupling, as shown in Fig. 21.25. It was still better size to Y cells, and have lower peak sensitivity.
than a model that left out both coupling and the All other W cells appear to have nonlinear spatial
post-spike filter (Poisson model) and an earlier summation [241, 242]. Phasic W cells (sluggish
model with linear decoding [238]. In general, the transient cells [205]) have spatial summation sim-
multielectrode work and models that use it have ilar to Y cells, but poor sensitivity to gratings,
given us a clearer picture of the information that and most can be characterized by a Difference
the retina sends to the brain. of Gaussians model. Directionally selective cells
and ON-OFF RGCs have receptive fields that are
21.5.3.7 Other Types of Retinal not well described by a Difference of Gaussians
Ganglion Cells model [241, 243]. Our understanding of the re-
The X and Y cells comprise 40–60% of the RGCs ceptive field properties of some of the more rare
in cat [37], and M, P, and bistratified cells com- ganglion cell types advanced significantly over
prise about 75% of the retinal RGCs in primates the past decade [244–247], but there is still more
[239]. As noted above, these are all concentri- to do. There are at least 17 morphologically dis-
cally center-surround organized, and they make tinct types of retinal RGCs in primates [239, 248]
Fig. 21.25 Panel A illustrates a full coupled model of identify the pattern of illumination from spike trains, with
ganglion cell spiking designed to investigate the impor- light input at the top, p(s), and spike trains at the bottom.
tance of correlations in the timing of spikes between adja- Panel C shows that the full model, with coupling, had a
cent RGCs. The model is shown for two RGCs, with filters higher signal to noise ratio than the same model without
between the light input and the spike output as described coupling, or than a Poisson model missing coupling and
in the text. Panel B shows a test of the model’s ability to the post-spike filter, or an earlier linear decoding model.
(Pillow et al. [237])
and arguments have been made for more than The strategy used by Real et al. in the work on
30 physiological types in mouse [249, 250]. This tiger salamander [252] was to build increasingly
does not necessarily mean that mice have more complex models, constrained by some known fea-
types than primates (or for that matter cats or tures of retinal anatomy and cellular responses,
other mammals) but reflects the fact that enough but with some novel features. These were fit-
cells have been studied in mice that one can ted to data on RGC spike timing that had been
differentiate these many types reliably. Mammals recorded in multielectrode array recordings in
are generally believed to have many RGCs that response to a set of 0.066 mm wide vertical bars
are at least similar across species. in which the intensity of each one flickered in-
dependently, but repeatedly over gray levels at
60 Hz. The model parameters were obtained by
21.5.4 Retinal Connectivity Models optimizing fits to STAs for 80% of the data, and
the quality of fits was judged by investigating the
A newer line of work than the black box RGC variance from actual responses when the mod-
models referred to in Sects. 21.5.3.2, 21.5.3.3, els were tested with the other 20% of the data.
21.5.3.4, and 21.5.3.5 still require ganglion cell One advantage of using salamander was that in
responses as their main source of data but have addition to testing the models against RGC data,
revealed more about the connections of first- and it was possible to record from bipolar cells with
second-order neurons to RGCs, so they are called intracellular recordings to determine if they had
retinal connectivity models here. This is a grow- the features predicted by the models. As noted
ing area, partly enabled by multielectrode record- above, there are many types of RGCs, so the
ings, and always using isolated retinas. For ex- models had to account for at least some differ-
ample, such models have enabled a better un- ent types. Figure 21.26 shows the second model
derstanding of how different types of primate and the final one; all the models were of essen-
cones connect to RGCs [54, 55, 251]. Only two tially two stages: a bipolar cell module (BCM)
examples of retinal connectivity models will be incorporating photoreceptor responses and hori-
described, one from the tiger salamander retina zontal cell responses and a ganglion cell mod-
[252], and one from mouse retina [63]. ule (GCM). The first model was an LN (linear-
A LNSN model B LNFDSNF model

stimulus
stimulus
D
1
0.4
Filter strength
Time (s)
BCM
0
0.2
1
0
GCM 0.2 0 0.2
Space (mm)
1
output output
Pooling weights
C 0.6 0.5
0
Explained Variance (E.V.)
0.5
Space
0.3
0.4
Time (s)
0.2
LNFDSNF
LNFSNF
LNSNF
0
LNSN
0.5 0 0.5
0
LN
Space (mm)
Fig. 21.26 Models of retinal circuitry in the tiger sala- on the average (black bars). Models are named for the
mander retina. One of the simpler models is shown in sequence of elements, where L = linear, N = nonlinear
a, and the most complex model is shown in b. Stimuli (rectifying), F = feedback, S = summation, and D = delay.
were narrow vertical bars as shown at the top. The models Panel d shows, for the LNSN model, the spatiotemporal
had a bipolar cell module (BCM) and a ganglion cell response of an individual BCM (top), the weighting (S)
module (GCM) which varied in complexity across models. of these BCMs feeding into an ON-center ganglion cell
c shows how much of the spike train of real ganglion cells (middle), and the derived spatiotemporal response of the
could be explained by different models. Increasing the ganglion cell (bottom), which has the expected response
complexity of the model improved the ability to explain features of a ganglion cell. (Real et al. [252])
variance in spike trains for individual cells (dots) and
nonlinear) model (not shown), in which bipo- cell behavior increased for all ganglion cells with
lar cell responses are linear transfer functions the complexity of the model, but without an in-
with different temporal properties followed by a crease in the number of parameters, as shown
rectifying nonlinearity. The LNSN model (Fig. in Fig. 21.26c (gray lines), so that on average,
21.26a) retained these features and added a non- only 26% of the variance was explained by the
linearity (N) before summation (S) of the bipolar LN model, but 42% was accounted for by the
cell outputs. These additions and forcing the bipo- LNFDSNF model. The improvement for individ-
lar cell “modules” to be the same allowed a reduc- ual ganglion cells was quite variable. As an exam-
tion in the total number of parameters relative to ple of the way the models were developed, Fig.
the LN model from 187 to 68. The fifth model 21.26d shows, at the bottom, the spatiotemporal
tested (Fig. 21.26b) added feedback (F) at two behavior of an OFF ganglion cell derived from
stages, and a delay (D) in the surround relative to the STA that was the input to the modeling; at
the center, and led to the final LNFDSNF model. the top, the best spatiotemporal filter representing
The ability of the models to account for ganglion the BCM feeding into this ganglion cell; and in
the middle, the weighting and spatial positions RGCs and three in OFF RGCs. Figure 21.27c, d
of the multiple bipolar cell modules (represented illustrates one of these effects for each type of cell
by dots) that gave the ganglion cell response at in response to large spots. In Fig. 21.27c, raster
the bottom. Even the best models left room for displays show the timing of ganglion cell spikes
improvement, because, for instance, the model before (black), during (orange), and after (dark
did not allow spatiotemporal differences among blue) the application of PSEM ligand in response
the bipolars feeding into particular ganglion cells. to two levels of stimulus contrasts for one cell.
Nevertheless, by recording from bipolar cells, Orange throughout the figure shows responses in
the authors were able to show that bipolars had the absence of H cells. As one might expect, the
characteristics that the model predicted. role of horizontal cells in the response of this ON
The exact role of horizontal cells (H cells) class of RGCs was to make the response more
in shaping the responses of retinal cells has transient, without altering the peak, suggesting
been unclear, except for the knowledge that that the H cell input was slow or delayed. In Fig.
they contribute to the surround. Pharmacological 21.27d, a more transient class of OFF cells had the
suppression of the responses of H cells is possible peak off response enhanced without H cells in the
with certain glutamate antagonists, but this circuit (arrows). For both of these effects, H cells
also suppresses off bipolar cells and therefore reduced spiking in an approximately subtractive
does not specifically reveal the role of H cells. way, but in others, the role of horizontal cells was
Drinnenberg et al. [63] found that they could use to sharpen the response of RGCs and increase
viral delivery to transfect a chloride channel gain. None of the effects were observed in all cells
called PSAM (pharmacologically selective of a particular class even in the same retina but
actuator molecule) selectively into mouse H were robust enough to allow the formulation of a
cells, and then, when PSAM ligand was supplied model (Fig. 21.27e) in which three filters (blue,
to the isolated retina at 3 μM to open those pink, green) plus rectifiers represent the effect of
channels, feedback from all H cells to cones H cells in different pathways in the inner retina.
could be blocked. This is shown in Fig. 21.27a, Potentially all of these could sum at the ganglion
b, where pink denotes PSAM-expressing H cells cell level, which is represented at the bottom of
and green shows cones, which were transfected Fig. 21.27e by a straight pathway and a derivative
with GCaMP6s, a fluorescent calcium reporter. pathway, accounting for sustained and transient
Figure 21.27b shows responses of GCaMP6s RGCs, and then by the usual rectifier due to the
to a small spot of light (gray) illustrating the spike mechanism. Figure 21.27h shows what
expected decrease in cone synapse Ca+2 when each of these filters does separately. Drinnenberg
cones hyperpolarize during illumination. The et al. found that all six of the effects of PSEM
response to a larger spot (black) shows lateral could be accounted for by different combinations
inhibition of the cone due to H cell feedback. of filters in the retinal circuit, as illustrated in Fig.
When H cells were clamped at a hyperpolarized 21.27f for the sustained ON cells of Fig. 21.27c
potential by opening the PSAM-channel with the and in Fig. 21.27g for the transient OFF cells
PSAM ligand, their responses could no longer be of Fig. 21.27d. The model recapitulated all the
modulated by light, and the cone response was the features of the responses in terms of increases
same for large and small spots, that is, it no longer and decreases in responses and in gain changes,
exhibited a surround. This allowed Drinnenberg and suggested that RGCs that were not affected
et al. [63] go on to investigate RGC responses by PSEM may have received input from only the
with and without H cell contributions. As in fastest pathways in Fig. 21.27e. Drinnenberg et
other recent work, many RGCs were recorded al. went on to successful tests of other predictions
simultaneously with multielectrode arrays. They of the model.
found six separate effects of H cells, three in ON
Fig. 21.27 Role of horizontal cells in retinal circuits. a: plots of individual spikes during several stimulus presen-
Retinal structure, highlighting cones (green) and horizon- tations are collected into the histograms below. Responses
tal (H) cells genetically targeted to contain PSEM sensitive before and after PSEM are shown in black; responses in
Cl− channels (pink). b: Cone synaptic terminal responses the presence of PSEM are shown in orange. e: Retinal
to large and small spots of light. On the left, the large spot model showing filters (colored blocks) between cones and
produces a smaller response due to H cell feedback; on the ganglion cells, each of which is affected to some extent
right, both spots produce the same response when PSEM by eliminating H cell feedback, as shown by the modeled
ligand is applied to the retina, presumably clamping the filters in panel h. f and g: Modeled ganglion cell responses,
H cell near the Cl− equilibrium potential and preventing using appropriate combinations of filters as shown below
feedback. c and d: Responses of an ON-center and an the responses, and matching the behavior of the ganglion
OFF-center ganglion cell. Stimulus traces to the left show cells shown in panels c and d. (Drinnenberg et al. [63])
the part of the response affected by PSEM ligand. Raster
and H+ ) and gases (O2 and NO in particular).

21.6 Engineering and the Retinal Diffusion models can then be fitted to the data
Microenvironment to understand both the fluxes of these substances
through the retina and cellular metabolism. This
A completely different set of engineering
work is important because alterations in the
approaches has been used to study the retinal
microenvironment, caused either by vascular
microenvironment and retinal metabolism. The
dysfunction or cellular dysfunction, are often
microenvironment refers to the composition of
the aspect of disease that leads to retinal cell
the extracellular space surrounding the neurons,
death. In addition, these measurements can
in terms of ion distributions, nutrient and waste
often give a different kind of insight into retinal
product concentrations, and extracellular volume.
cell physiology. However, unlike the modeling
These properties can be studied with intraretinal
discussed earlier, where electrophysiological data
microelectrodes sensitive to ions (e.g., K+ , Ca+ ,
provided almost all of the information on which

the models were constructed, the microelectrode
techniques are not the only way to study the
microenvironment. A full understanding, which
we will not attempt here, requires the use of many
complimentary techniques, including recordings
of retinal activity, biochemical measurements
of various metabolites, optical measurements
of intracellular ion concentrations and retinal
vascular oxyhemoglobin saturation, histological
measurements of cytochrome oxidase and
of the uptake of a non-metabolizable sugar
(deoxyglucose), and measurements of blood Fig. 21.28 A profile of oxygen tension across the cat
flow. The microenvironment also includes retina during dark adaptation. The recording was made
molecules used to signal between cells, such with an oxygen microelectrode that was first advanced
through the retina in steps to the choriocapillaris and then
as neurotransmitters and paracrine substances was withdrawn continuously at 2 μm/sec to the vitreous.
like melatonin, but in general, there are no Evidence of retinal capillaries is visible as peaks in the
techniques available to measure these with spatial inner half of the retina. The correspondence to retinal
and temporal precision. layers is shown at the bottom
circulation. The gradient of O2 across the retina

21.6.1 Oxygen of a cat under dark-adapted conditions is shown
in Fig. 21.28. There is a deep trough in the outer
One of the important constituents of the
half of the retina because O2 supplied from both
microenvironment is O2 , and there are several
circulations is consumed by the photoreceptors.
reviews of this subject [253–256]. Normally,
In the inner retina, there are typically peaks and
the metabolism of the retina is limited by the
valleys depending on proximity of the electrode
availability of O2 , which cannot be stored in
to capillaries of the retinal circulation.
tissue. Hypoxia, the lack of O2 , clearly plays
Oxygen moves only by simple diffusion, and
a role in diabetic retinopathy, retinopathy of
it diffuses equally well through membranes as
prematurity, and retinal vascular occlusive
through intracellular and extracellular space, so
disease, and may be involved in any situation
the tissue can be modeled as homogeneous. In the
where blood flow is compromised. Oxygen
most general terms, O2 diffusion is described by:
partial pressure, PO2 , can be measured with
O2 -sensitive polarographic electrodes, which
Dk∇ 2 P + Q = k · ∂P /∂t (21.15)
chemically reduce O2 and yield a current
proportional to the concentration of O2 at the where D is the diffusion coefficient of O2
tip of the electrode. Microelectrodes have a (cm2 /sec), k is the O2 solubility (ml O2 -ml
spatial resolution approaching one μm and have retina−1 -mm Hg−1 ), P is the partial pressure
response times of milliseconds [257]. They can (mm Hg or torr), Q is the utilization of O2 (ml
be used to map the PO2 as a function of position O2 − 100 g−1 -min−1 ), and ∇P is the second
across the retina (PO2 profiles), and this has been spatial derivative of P (mm Hg/cm2 ). This
done in several species. The animals fall into two equation only applies in a region that can be
categories: those with both a choroidal circulation assumed to have a homogeneous value of Q, so
and a retinal circulation, such as human, monkey, the challenge is to define a region where this
cat, pig, and rat, and those whose retinas have no can be applied, and specify appropriate boundary
retinal circulation, including rabbit and guinea conditions. Most analyses performed to date have
pig, and therefore rely on only the choroidal
attempted to fit data such as that shown in Fig. geometry of the circulation. The earliest retinal
21.28 to the diffusion equation and extract a value model, by Dollery et al. [258], was a simulation of
for Q under steady state conditions (darkness the outer retina as one layer before any intraretinal
or steady illumination), so the right side of the PO2 recordings were available. These authors
equation is set to zero. reached the somewhat surprising conclusion that
Equation 21.15 can be applied to the outer while there was adequate O2 at the boundaries of
half of the retina, which can be considered to the outer retina, the PO2 was likely to be almost
be an avascular slab of tissue, with O2 supply zero somewhere in the tissue. The first intrareti-
only from the boundaries at the choriocapillaris nal measurements were made in cats [259], and
and about half way through the retina, where revealed steep gradients of O2 in the outer retina
the retinal circulation begins. The curvature of during light adaptation, but under this condition,
the retina is negligible with respect to its thick- PO2 was not unusually low. Measurements in the
ness. In this slab, O2 is assumed to diffuse only dark-adapted cat retina, however, a condition in
in one dimension, along the photoreceptors; any which the metabolic rate was known to be higher
lateral gradients are expected to be very small. from prior work [260, 261], supported the idea
Using these geometrical simplifications, one can that part of the outer retina had a very low PO2
fit models with different numbers of layers to O2 [76, 262]. Subsequent work [263] led to a model
profiles in order to determine how many layers are for O2 diffusion in the outer retina (Fig. 21.29)
needed to fit the data and to extract values for Q. that had three layers rather than the one used by
Again, this is a very unusual aspect of the retina. Dollery et al. [258], and all subsequent models
Other parts of the CNS are not amenable to such derive from this one [265–273]. The solution to
models because of the much more complicated Eq. 21.15 under these conditions is:
P1 (x) = a1 x + b1 0 ≤ x ≤ L1
P2 (x) = (Q2 /2Dk) x 2 + a2 x + b2 L1 ≤ x ≤ L2 (21.16)
P3 (x) = a3 x + b3 L2 ≤ x ≤ L
where the constants ai and bi for each of the three

layers are determined from the boundary con- corresponded to the cell bodies of photorecep-
ditions [263]. The boundary conditions include tors in the outer nuclear layer. Mitochondria are
specified PO2 values at the choroid (x = 0) and present only in the inner segments [274], so this
at the outer-inner retinal border (x = L), about model agreed with the anatomy. The value of Q2
half way through the retina, as well as matching was very high, on the order of 20 ml O2 /100g-
of PO2 s and O2 fluxes at L1 and L2 , the borders min [76, 266], which is about five times the
between layers. O2 consumption of brain tissue [275]. This high
The fits of this model to data yielded values for consumption, in combination with the relatively
the PO2 s at the choroid and inner retinal boundary long distance between the inner segments and the
(PC and PL ), the locations of the boundaries, L1 choriocapillaris, which is the closest circulation,
and L2 , and a value for Q2 /Dk in the middle layer, is responsible for the very low PO2 s observed in
the only layer in which consumption was found the inner segment layer. However, because most
to be necessary. The initial fitting of the model of the outer retina uses no O2 , the QO2 averaged
to PO2 profiles placed the boundaries between across the outer retina is not remarkably high. The
regions at locations where it appeared that the out- model was originally applied to cat retina, but it
ermost layer was the outer segments, the middle also describes oxygenation of primate [75, 276]
layer was the inner segments, and the third layer and rat [268, 273, 277]. In rat, the normal min-
Fig. 21.29 The structure

of the oxygen model used
to describe the profile of
oxygen in the outer half of
the retina. Only the middle
layer, corresponding to the
photoreceptor inner
segments, has a non-zero
oxygen consumption. The
parameters that are
adjustable during fitting
are PC , PL , L1 , L2 , and Q2.
(Linsenmeier and
Pournaras [264];
Linsenmeier and Zhang
[254])
imum PO2 is higher, which the modeling shows choroid and provide enough O2 in the choroid
is a result of shorter inner segments and a higher to supply the entire retina. In this case, another
inner retinal boundary PO2 [277]. layer representing the inner retina can be added
The low PO2 in the inner segment layer under to the three-layer model described above [266],
normal conditions suggested that photoreceptors although one then relies on values being the same
would be at risk if arterial PO2 were reduced or as would exist if the circulation were normal. A
if choroidal blood flow were reduced by elevated diffusion model of the inner retina also applies
intraocular pressure [278], or if the retina were when the inner retina is avascular, as in guinea
detached [129]. In fact, all these conditions do pig [256, 280, 281] and rabbit [282] The inner
reduce photoreceptor O2 consumption [76, 130, retina in these animals receives very little O2 ,
278]. In addition, drusen under the retina in AMD however, and has low O2 consumption, so they are
also limit O2 diffusion to the inner segments [254] not good models for human retinal metabolism.
and may account for the loss of photoreceptors A third approach to understand the inner retina is
specifically over drusen. to try to couple the amount of O2 extracted from
In general, the same equations cannot be ap- the retinal circulation, measured with oximetry,
plied to the inner retina, because there are vessels to the distribution of O2 in the inner retina [267,
embedded in the tissue, reflected in the peaks 269, 272], but it is difficult to assess the validity
in the inner retina in Fig. 21.28, that make it of these models.
impossible to reduce the geometry to a one di-
mensional problem. The three-dimensional vas-
cular geometry is difficult to measure, and there 21.6.2 Ion Distribution
are no three-dimensional data to use in fits to
a three-dimensional model. Cringle et al. [268, The tip of a microelectrode can be filled with
279] attempted to circumvent this problem and a resin that is selectively permeable to a partic-
analyze the metabolism of the inner retina of ular ion, allowing the recording of the Nernst
rats by using an eight-layer model, five for the potential for that ion across the resin. When this
inner retina, avoiding the layers containing retinal electrode is placed in the retina, measurements
capillaries, but the errors in this are unknown. of extracellular ion concentrations can be made
Another strategy for extending the model into the with 1 μm resolution. A great deal of information
inner retina is to block the circulation of the inner leading to understanding of retinal neural activity
retina so that all of the O2 is derived from the has come from studies of K+ in the retina [283–
291], because light-evoked changes in K+ alter actually the extrusion of the ion from cells and its
the membrane potentials of Muller cells and RPE appearance extracellularly, and “consumption” is
cells, creating several ERG components. Mea- the uptake of the ion by cells. Ions may, of course
surements of Ca+2 have been important in re- diffuse in or out of the retina as well, and, as
vealing photoreceptor transduction mechanisms with the O2 models, this is covered by setting the
[292–294]. Measurements of H+ are the only boundary conditions so that there are fluxes that
way to obtain information about retinal glycolytic depend on the gradients between the tissue and
metabolism on a detailed spatial scale [295–302]. the boundaries.
Unfortunately, only a few studies have coupled
ion measurements to quantitative diffusion mod- 21.6.2.1 H+ Distribution
els. There are only two mathematical models of and Production
K+ fluxes [289, 303]. Modeling the transport of Using Eq. 21.17 as a basis, an analysis of H+ dif-
ions through the retina is complicated. The tissue fusion and production was done for the cat retina
cannot be treated as homogeneous, because ions [299, 300]. Like the O2 diffusion model described
diffuse only through ECF, and require facilitated above, the pH model was one-dimensional, ap-
or active transport across membranes. In order plied only to the avascular outer retina, and re-
to describe extracellular transport, the concepts quired the same three layers to fit the data. A
of tortuosity of the extracellular space, λ, and H+ profile across the retina, obtained with an
fraction of the total volume that is extracellular, ion-selective H+ electrode, and the correspond-
α, have to be introduced, so the general equation ing fitted model are shown in Fig. 21.30. The
developed for ion diffusion in the brain by Nichol- curvature of the profile is opposite to that of
son and coworkers [304–306] includes correc- the O2 profile, because H+ is produced, while
tions for these factors: O2 is consumed. Two layers, the inner segments
and outer nuclear layer, were found to produce

D/λ2 ∇ 2 C + Q/α = ∂C/∂t (21.17) H+ . H+ production is believed to reflect the high
rate of glycolytic ATP production in the retina,
Values for α are on the order of 0.1, and λ which is found even under aerobic conditions for
is on the order of 1.5. Once these modifications reasons that are not entirely clear. The H+ model
have been made to the diffusion equation, one can is not completely satisfactory, because the H+
attempt to define production rates and fluxes in production rates were far below those that were
the extracellular space. For ions, which are not expected on the basis of lactate production in
actually produced or consumed, “production” is the outer retina [307], even though there should
Fig. 21.30 Gradients and modeling of hydrogen ion in layers, of which layers 2 and 3 produced H+ and the
the cat retina. The profile was recorded with an ion- outer segments (layer 1) did not. Values of H+ production
selective H+ microelectrode. For the model fitted to this derived from this model are believed to be underestimates
data, the outer half of the retina was comprised of three of actual H+ production. (Modified from Padnick-Silver
and Linsenmeier [299])
be a 1:1 stoichiometry between lactate and H+ . in a way that was consistent with an increase
This meant that some H+ is probably cleared or in ECF in the subretinal space (but not the rest
buffered so rapidly that it is never seen as H+ , of the retina) [314, 315, 317]. Pharmacological
and the values of production derived from the experiments suggested that this hydration of the
model underestimate total H+ production. Buffer- subretinal space was probably initiated by the
ing of H+ appeared to be of great importance in light-induced decrease in [K+ ] in the subretinal
preventing the retina from becoming very acidic space [288, 290]. The decrease in [K+ ] reduces
[298, 308, 309]. Animals in which isoforms of the activity of a Na/K/Cl transporter at the api-
carbonic anhydrase have been knocked out have cal membrane of the RPE. That transporter is a
abnormal light responses [310]. Whether changes major driving force for water transport out of the
in pH play a role in disease is not yet known, but retina, so water transport decreases and the space
there are alterations of pH gradients in diabetic hydrates. The model that was developed for the
cats and rats [311, 312]. subretinal space of chick retina [316] was:
21.6.2.2 Retinal Extracellular Volume ∂C (x, t) ∂ 2 C (x, t) C (x, t) ∂α

= Ds • −
Another line of investigation has explored ∂t ∂x 2 α ∂t
whether retinal extracellular volume changes (21.18)
under any physiological or pathological con-
ditions. In brain and brain slices, extracellular where Ds is an apparent diffusion coefficient for
volume (α) decreases during hypoxia [306] and the subretinal space that includes the tortuosity
during stimulation [313]. This would affect the and α is again the ECF volume. The first term on
diffusion of all molecules in the extracellular the right represents the diffusion of TMA+ into
space. The technique of measuring changes in α a region of lower concentration, and the second
in the brain [304] involved using a micropipette term reflects the change in volume with time, to
to introduce an impermeant cation, such as be obtained by fitting data to the model. Figure
tetramethylammonium (TMA+ ), into the ECF. Its 21.31 shows two situations. In A, a step change
concentration was then followed over time with of volume (delta volume) was used as the input to
“K+ ” microelectrodes, which, in the presence the model and the resulting concentration change
of TMA+ , become TMA+ electrodes, because (delta concentration) showed a steep decrease
they are almost 200 times more sensitive to followed by a recovery. The recovery was due to
TMA+ than to K+ [314]. Because TMA+ is not diffusion of TMA+ from the inner retina into this
produced or consumed and does not enter cells, increased volume. An example is shown in Fig.
its concentration changes are caused by ECF 21.31b. A curve of dC/dt similar to those actually
volume changes and by diffusion of TMA+ away observed is shown as delta C. When the calculated
from the injection pipette. If one holds constant step response in A was deconvolved from this, the
the amount of TMA+ injected, then differences resulting delta volume was computed as the solid
in the concentration versus time curves before line. Here a 7% concentration change, which was
and after a manipulation, such as hypoxia, reveal the magnitude of the change observed, implied
differences in volume under the two conditions. a 20% increase in α. In cat, a similar model
A modification of this approach was taken in suggested that illumination could increase α by
the isolated frog [315] and chick retinas [292, 60% on average [314, 318]. These are very large
316, 317] and the intact cat retina [314, 318]. changes and would require shrinkage of RPE
In this work, a uniform initial concentration of cells or photoreceptors. A limitation of the model
TMA+ could be achieved by adding it to the is that it assumed that the light-evoked volume
bathing solution of the isolated retina, or injecting change was sustained during illumination, and
enough in the cat vitreous to achieve an equi- that the transient nature of the TMA concentration
librium ECF concentration of about 5 mM. Dur- change was due solely to diffusion of TMA into
ing illumination [TMA+ ] was found to change the subretinal space. If the model had allowed for
Fig. 21.31 Model of the extracellular volume change centration decreases because TMA+ is diluted and then
of the subretinal space in the isolated chick retina-RPE- recovers because TMA+ diffuses into the outer retina from
choroid preparation during illumination. Tetramethylam- the inner retina. b. The diffusion response in a was decon-
monium ion (TMA+ ) was added to the bathing solution volved from a curve simulating an actual concentration
to produce a concentration of TMA+ that was initially change during illumination (delta C), yielding a derived
uniform across the retina. (a) The delta volume trace volume change. While this is a simulation, the delta C
represents a step increase in subretinal extracellular vol- curve closely matches actual concentration changes with
ume, and the lower trace shows how TMA+ concentration light. (Govardovskii et al. [316])
would be expected to change in response to this. The con-
recovery of the volume during sustained illumi- ions entering or leaving the subretinal space. The
nation, the derived volume changes would have sign of the concentration change and the amount
been smaller. This could well be true, as pumping of ions turned out to be the same for Na+ and
rates adjust during maintained illumination, and K+ , but for Cl− , a small decrease in concentration
the estimated changes should probably be re- coupled with a larger increase in volume, implied
garded as upper limits. The failure to account for that the amount of Cl− was increasing in the ECF.
possible transience in the volume change could
also explain the apparent inconsistency that the
derived value of α in cat retina did not recover 21.7 Opportunities
after the end of sustained illumination. Using
the TMA approach, hypoxemia was found to de- Retinal bioengineering has contributed greatly to
crease the ECF volume by as much as a factor of our overall understanding of the retina, explaining
four [318], which is also probably an upper limit. retinal function in terms of quantitative models
of the electroretinogram, photoreceptor function,
21.6.2.3 Net Changes in Ion ganglion cell function, and transport of O2 and
Distribution with Light ions. Retinal models of these processes may not
The concentration changes for different ions are be exact descriptions at present and will evolve.
not affected by the changes in volume; the micro- However, the tradition in retinal research is not
electrodes measure them accurately. But the mea- to propose simply theoretical models but ones
sured changes in ion concentration do not directly that are strongly based on, and constrained by,
indicate the amount of the ions entering or leaving data. These have provided descriptions of retinal
the subretinal space, because water changes as function in terms of parameters with clear physi-
well. So far, the only attempt to describe all ological meaning.
the ionic and volume changes in the subretinal Nevertheless, many opportunities are open.
space during illumination [292] was done for the First, we cannot quite predict the full optic
isolated chick retina-RPE. This allowed the con- nerve response that would result from an
centration changes to be converted to amounts of arbitrary visual stimulus with properties chosen
from the full range of luminance, contrast, have predicted that something as simple as O2
color, and spatiotemporal properties. It would inspiration, if maintained for long enough, would
also be useful to understand how the brain’s have substantial benefits in retinal detachment
interpretation of the retinal signal would be and vascular occlusive diseases [254].
influenced by different degrees of failure or
loss in the ganglion cell output. Understanding Acknowledgments The work of RAL’s laboratory was
these system properties might be necessary to supported largely by NIH R01 EY05034.
optimize the design of visual prostheses or to
design robotic systems with good vision. Second,
while there are descriptive models of ganglion Homework
cell behavior in terms of spatial and temporal
properties of the receptive field, we are at an 1. Calculate the relative gNa /gK (or PNa /PK if
early stage of linking that behavior to the role of you prefer) for a photoreceptor whose resting
each anatomical cell type and synapse between potential in the dark is −30 mV. Make reason-
photoreceptors and RGCs. Models of bipolar, able assumptions for ENa and EK (or Na+ and
horizontal cell, and amacrine cell behavior are K+ concentrations) and assume that Cl− is at
beginning to provide this, as indicated in Sect. equilibrium. How does this differ from most
21.5.4, and it may be that further applications neurons at rest?
of the “chemigenetic” technique of knocking 2. The dark current of photoreceptors is about
out different types of cells will prove extremely −30 pA. Assume all the current is carried by
fruitful in this regard, as long as this does not Na+ . All the Na+ has to be pumped out of the
lead to adaptive changes within the retina that inner segment (IS) to maintain the normally
would confuse us. Similarly, we know a great low intracellular [Na+ ]i . The pump exchanges
deal about the microenvironment at a descriptive 3 Na+ for 2 K+ as usual, and each pump cycle
level, but linkage with the underlying metabolic (i.e., 3 Na+ ) requires one molecule of ATP.
processes at the cellular or biochemical level (a) What is the usage of ATP/min in the dark
has only been attempted occasionally [319]. for an individual rod? (This is not the
Third, optical tests are currently the most only function requiring ATP but it is by
frequently used for detecting ocular disease far the largest in the dark-adapted retina.
but are still based largely on retinal anatomy. Actually about 85% of the current is due
Noninvasive retinal electrophysiological and to Na+ , and 15% is due to Ca+2 , but Ca+2
psychophysical measurements could be more is pumped out by a secondary active trans-
sensitive to early functional deficits, before porter that moves Ca+2 out and Na+ in
anatomy is compromised. These tests have come in the outer segment, which makes the
a long way in terms of understanding their basis in load of Na+ higher than assumed in the
retinal cellular behavior, but more work remains problem statement.)
to make them specific. Fourth, repair of the (b) There are 180,000 rods per mm2 at the
diseased retina is generally not possible, although peak of rod density. The IS are about
for some genetic and age-related diseases, there 25 μm long. As noted in the text, other
are several promising approaches (gene transfer, layers of the outer retina use no oxygen,
stem cells, retinal prostheses). However, in so this ATP usage is over a volume of
many cases, the best we can do at present is 1 mm2 × 25 μm. Roughly what is the
hold the line against further loss with VEGF oxygen usage of the IS, in μM-ml−1 -
antagonists, steroids, or laser photocoagulation. min−1 of IS volume (essentially per gram
Finally, a further understanding of the retinal since tissue density is about 1.05 g/ml),
microenvironment can potentially lead to an if all of the metabolism is oxidative
understanding of the etiology of vascular and metabolism (1 glucose + 6 O2 → 6
metabolic diseases, and to improved treatments. CO2 + 6 H2 O). Also assume that 36 moles
For example, measurements and modeling of O2 of ATP are produced per mole of glucose.
(After you do the calculation, you will 6. The chapter shows difference of Gaussian re-
be able to compare this with the typical ceptive field profiles for selected cat retinal
oxygen consumption of the brain, which ganglion cells, but as noted in Fig. 21.19,
is around 2 μmoles O2 -ml−1 -min−1 or primate retinal ganglion cells can be charac-
as it is often expressed, 4 ml O2 -100g−1 - terized in the same way. Receptive fields vary
min−1 .) a great deal across the retina.
3. The ganglion cell center and surround are usu- (a) For the P (midget) cells with the smallest
ally viewed as being antagonistic to each other, and largest receptive field centers, plot the
but this is actually true only for certain stim- sensitivity of center and surround of the
ulus conditions, as in Fig. 21.18. Under what receptive field in spatial coordinates, as in
conditions do the center and surround of gan- Fig. 21.18a. For the larger P cell, also show
glion cells add rather than subtract? Justify the surround sensitivity multiplied by 10.
your answer. The cells in Fig. 21.19 were recorded be-
4. Gauthier et al. [38] hypothesized that the tween about 1 and 35 degrees of eccentric-
receptive fields of primate retinal ganglion ity. For convenience, the centers and sur-
cells were arranged to tile the retina (or visual rounds of P cells from Croner and Kaplan
world). They suggested that the interdigitation [218] are shown separately below.
of adjacent receptive fields was not random (b) In response to a large (or diffuse) stimulus,
but was nearly optimal, with minimal gaps both center and surround will be maxi-
between ganglion cells and minimum overlap mally activated. The area under the cen-
of receptive fields. Using RF data like those ter curve represents this “integrated center
shown in Fig. 21.24, suggest a method to test strength” and is Kc rc 2 . The integrated sur-
this hypothesis. round strength is Ks rs 2 . For these two cells,
5. Figure 21.14 shows that the small ERG sig- compare the integrated center strengths.
nals that comprise the multifocal ERG vary Also, what is the strength of the surround
in amplitude across the visual field. In fact, relative to the strength of the center? From
the stimulus elements are not equal in size, your graphs of the center and surround, the
and the smaller elements in the middle of the answers to these questions may surprise
stimulus array (left) produce the largest re- you, but they seem to reveal a logic about
sponses (right). Generate at least one testable the way ganglion cell receptive fields vary
hypothesis about why this might be true, rec- with eccentricity.
ognizing that the ERG comes largely from (c) What is the highest spatial frequency that
photoreceptors (here cones) and bipolar cells. each of these cells can detect? (In the
units of the figures and the equation in
the text, this is where contrast sensitivity
falls to 0.01, meaning that 100% contrast 6. J.D. Weiland, M.S. Humayun, Retinal
is needed.) prosthesis. I.E.E.E. Trans. Biomed. Eng. 61(5),
1412–1424 (2014). https://doi.org/10.1109/
7. Which eye diseases could a retinal prosthe-
TBME.2014.2314733
sis be used to treat and why? What is the 7. J.D. Weiland, S.T. Walston, M.S. Humayun, Electri-
definition of legal blindness in the USA in cal stimulation of the retina to produce artificial vi-
terms of visual acuity? No currently available sion. Annu. Rev. Vis. Sci. 2, 273–294 (2016). https:/
/doi.org/10.1146/annurev-vision-111815-114425
retinal prosthesis has succeeded in providing
8. T.Y. Chui, H. Song, S.A. Burns, Adaptive-optics
this minimal level of acuity. Why do you think imaging of human cone photoreceptor distribution.
that this is the case and what has limited our J. Opt. Soc. Am. A Opt. Image Sci. Vis. 25(12),
ability to reach this standard? 3021–3029 (2008). doi:174847 [pii]
9. Y. Kitaguchi, S. Kusaka, T. Yamaguchi, T. Mihashi,
8. Barlow and Levick in Fig. 7 of their 1965
T. Fujikado, Detection of photoreceptor disruption
paper “The mechanisms of directionally se- by adaptive optics fundus imaging and Fourier-
lective units in the rabbit’s retina” (Journal of domain optical coherence tomography in eyes with
Physiology 178, 477–504) proposed a model occult macular dystrophy. Clin. Ophthalmol. 5, 345–
351 (2011). https://doi.org/10.2147/OPTH.S17335
for the receptive field of a rabbit retinal gan-
10. A. Roorda, D.R. Williams, The arrangement of
glion cell that has directional selectivity. It is the three cone classes in the living human eye.
known now that retinal ganglion cells with Nature 397(6719), 520–522 (1999). https://doi.org/
similar receptive field properties exist in most, 10.1038/17383
11. J. Fujimoto, E. Swanson, The development,
if not all, vertebrate retinas, including those of
commercialization, and impact of optical coherence
the primate. Suggest a model for the creation tomography. Invest. Ophthalmol. Vis. Sci. 57(9),
of directional selectivity based on retinal cir- OCT1–OCT13 (2016). https://doi.org/10.1167/
cuitry involving bipolar and amacrine cells. iovs.16-19963
12. M.L. Gabriele, G. Wollstein, H. Ishikawa, L. Kage-
mann, J.A. Xu, L.S. Folio, J.S. Schuman, Optical
coherence tomography: History, current status, and
References laboratory work. Invest. Ophthalmol. Vis. Sci. 52(5),
2425–2436 (2011). https://doi.org/10.1167/iovs.10-
1. C.A. Curcio, K.A. Allen, Topography of ganglion 6312
cells in human retina. J. Comp. Neurol. 300(1), 5– 13. M.R. Hee, J.A. Izatt, E.A. Swanson, D. Huang, J.S.
25 (1990). https://doi.org/10.1002/cne.903000103 Schuman, C.P. Lin, C.A. Puliafito, J.G. Fujimoto,
2. H. Benav, K.U. Bartz-Schmidt, D. Besch, A. Bruck- Optical coherence tomography of the human retina.
mann, F. Gekeler, U. Greppmaier, A. Harscher, S. Arch. Ophthalmol. 113(3), 325–332 (1995)
Kibbel, A. Kusnyerik, T. Peters, H. Sachs, A. Stett, 14. V.J. Srinivasan, B.K. Monson, M. Wojtkowski,
K. Stingl, B. Wilhelm, R. Wilke, W. Wrobel, E. R.A. Bilonick, I. Gorczynska, R. Chen, J.S.
Zrenner, Restoration of useful vision up to letter Duker, J.S. Schuman, J.G. Fujimoto, Characteriza-
recognition capabilities using subretinal micropho- tion of outer retinal morphology with high-speed,
todiodes. Conf. Proc. IEEE Eng. Med. Biol. Soc. ultrahigh-resolution optical coherence tomography.
2010, 5919–5922 (2010). https://doi.org/10.1109/ Invest. Ophthalmol. Vis. Sci. 49(4), 1571–1579.
IEMBS.2010.5627549 doi:49/4/1571 [pii] (2008). https://doi.org/10.1167/
3. J. Dowling, Current and future prospects for opto- iovs.07-0838
electronic retinal prostheses. Eye (Lond.) 23(10), 15. A. Kanamori, A. Nagai-Kusuhara, M.F. Escano, H.
1999–2005. doi:eye2008385 [pii] (2009). https:// Maeda, M. Nakamura, A. Negi, Comparison of con-
doi.org/10.1038/eye.2008.385 focal scanning laser ophthalmoscopy, scanning laser
4. C.M. Rountree, J.B. Troy, L. Saggere, polarimetry and optical coherence tomography to
Microfluidics-based subretinal chemical discriminate ocular hypertension and glaucoma at
Neuromodulation of photoreceptor degenerated an early stage. Graefes Arch. Clin. Exp. Ophthal-
retinas. Invest. Ophthalmol. Vis. Sci. 59(1), 418–430 mol. 244(1), 58–68 (2006). https://doi.org/10.1007/
(2018). https://doi.org/10.1167/iovs.17-23142 s00417-005-0029-0
5. E.J. Tehovnik, W.M. Slocum, S.M. Smirnakis, 16. A. Roorda, Applications of adaptive optics
A.S. Tolias, Microstimulation of visual cortex to scanning laser ophthalmoscopy. Optom. Vis. Sci.
restore vision. Prog. Brain Res. 175, 347–375. 87(4), 260–268 (2010). https://doi.org/10.1097/
doi:S0079-6123(09)17524-6 [pii] (2009). https:// OPX.0b013e3181d39479
doi.org/10.1016/S0079-6123(09)17524-6 17. P.F. Sharp, A. Manivannan, H. Xu, J.V. Forrester,
The scanning laser ophthalmoscope–a review of its
role in bioscience and medicine. Phys. Med. Biol. 27. J. Dowling, The Retina: An Approachable Part of the
49(7), 1085–1096 (2004) Brain (Belknap PRess, Cambridge, MA, 1987)
18. A.S. Neubauer, M.W. Ulbig, Laser treatment in dia- 28. R.H. Masland, The neuronal organization of the
betic retinopathy. Ophthalmologica 221(2), 95–102. retina. Neuron 76(2), 266–280 (2012). https://
doi:000098254 [pii] (2007). https://doi.org/10.1159/ doi.org/10.1016/j.neuron.2012.10.002
000098254 29. R.W. Rodieck, First Steps in Seeing (Sinauer Asso-
19. C.D. Regillo, Update on photodynamic therapy. ciates, Sunderland, 1998)
Curr. Opin. Ophthalmol. 11(3), 166–170 (2000) 30. B.A. Wandell, Foundations of Vision (Sinauer Asso-
20. J.L. Dumouchel, N. Chemuturi, M.N. Milton, G. ciates, Sunderland, 1995)
Camenisch, J. Chastain, M. Walles, V. Sasseville, M. 31. H. Kolb, R. Nelson, E. Fernandez, B.W. Jones,
Gunduz, G.R. Iyer, U.A. Argikar, Models and ap- Webvision: The organization of the retina and visual
proaches describing the metabolism, transport, and system (2019). http://webvision.med.utah.edu. Ac-
toxicity of drugs administered by the ocular route. cessed 24 Jan 2019
Drug Metab. Dispos. 46(11), 1670–1683 (2018). 32. P.H. Schiller, The ON and OFF channels of the vi-
https://doi.org/10.1124/dmd.118.082974 sual system. Trends Neurosci. 15(3), 86–92 (1992).
21. G.A. Rodrigues, D. Lutz, J. Shen, X.D. Yuan, H. doi:0166-2236(92)90017-3 [pii]
Shen, J. Cunningham, H.M. Rivers, Topical drug 33. H. Wassle, B.B. Boycott, Functional architecture of
delivery to the posterior segment of the eye: Ad- the mammalian retina. Physiol. Rev. 71(2), 447–480
dressing the challenge of preclinical to clinical trans- (1991)
lation. Pharm. Res. 35(12) (2018). https://doi.org/ 34. R.M. Shapley, C. Enroth-Cugell, Visual adaptation
10.1007/s11095-018-2519-x and retinal gain controls. Prog. Retin. Res. 3, 263–
22. A.M. Maguire, K.A. High, A. Auricchio, J.F. 346 (1984)
Wright, E.A. Pierce, F. Testa, F. Mingozzi, J.L. Ben- 35. D.M. Dacey, B.B. Peterson, F.R. Robinson, P.D.
nicelli, G.S. Ying, S. Rossi, A. Fulton, K.A. Mar- Gamlin, Fireworks in the primate retina: In vitro
shall, S. Banfi, D.C. Chung, J.I. Morgan, B. Hauck, photodynamics reveals diverse LGN-projecting gan-
O. Zelenaia, X. Zhu, L. Raffini, F. Coppieters, E. glion cell types. Neuron 37(1), 15–27 (2003).
De Baere, K.S. Shindler, N.J. Volpe, E.M. Surace, doi:S0896627302011431 [pii]
C. Acerra, A. Lyubarsky, T.M. Redmond, E. Stone, 36. R.M. Shapley, B.B. Lee, E. Kaplan, New views of
J. Sun, J.W. McDonnell, B.P. Leroy, F. Simonelli, primate retinal function. Prog. Retin. Res. 9, 273–
J. Bennett, Age-dependent effects of RPE65 gene 336 (1990)
therapy for Leber’s congenital amaurosis: A phase 1 37. J.B. Troy, T. Shou, The receptive fields of cat reti-
dose-escalation trial. Lancet 374(9701), 1597–1605. nal ganglion cells in physiological and pathological
doi:S0140-6736(09)61836-5 [pii] (2009). https:// states: Where we are after half a century of re-
doi.org/10.1016/S0140-6736(09)61836-5 search. Prog. Retin. Eye Res. 21(3), 263–302 (2002).
23. K. Stieger, B. Lorenz, Gene therapy for vision loss – doi:S1350946202000022 [pii]
Recent developments. Discov. Med. 10(54), 425– 38. J.L. Gauthier, G.D. Field, A. Sher, M. Greschner,
433 (2010) J. Shlens, A.M. Litke, E.J. Chichilnisky, Recep-
24. K. Deisseroth, G. Feng, A.K. Majewska, G. Miesen- tive fields in primate retina are coordinated to
bock, A. Ting, M.J. Schnitzer, Next-generation op- sample visual space more uniformly. PLoS Biol.
tical technologies for illuminating genetically tar- 7(4), e1000063. doi:08-PLBI-RA-0586 [pii] (2009).
geted brain circuits. J. Neurosci. 26(41), 10380– https://doi.org/10.1371/journal.pbio.1000063
10386. doi:26/41/10380 [pii] (2006). https://doi.org/ 39. S.A. Bloomfield, Effect of spike blockade on the
10.1523/JNEUROSCI.3863-06.2006 receptive-field size of amacrine and ganglion cells in
25. M.M. Doroudchi, K.P. Greenberg, J. Liu, K.A. the rabbit retina. J. Neurophysiol. 75(5), 1878–1893
Silka, E.S. Boyden, J.A. Lockridge, A.C. Arman, (1996)
R. Janani, S.E. Boye, S.L. Boye, G.M. Gordon, 40. S. Hattar, H.W. Liao, M. Takao, D.M. Berson, K.W.
B.C. Matteo, A.P. Sampath, W.W. Hauswirth, A. Yau, Melanopsin-containing retinal ganglion cells:
Horsager, Virally delivered channelrhodopsin-2 Architecture, projections, and intrinsic photosensi-
safely and effectively restores visual function tivity. Science 295(5557), 1065–1070 (2002). https:/
in multiple mouse models of blindness. Mol. /doi.org/10.1126/science.1069609. 295/5557/1065
Ther.doi:mt201169 [pii] (2011). https://doi.org/ [pii]
10.1038/mt.2011.69 41. A. Sand, T.M. Schmidt, P. Kofuji, Diverse types
26. B. Lin, A. Koizumi, N. Tanaka, S. Panda, R.H. of ganglion cell photoreceptors in the mammalian
Masland, Restoration of visual function in reti- retina. Prog. Retin. Eye Res. 31(4), 287–302 (2012).
nal degeneration mice by ectopic expression of https://doi.org/10.1016/j.preteyeres.2012.03.003
melanopsin. Proc. Natl. Acad. Sci. U. S. A. 105(41), 42. T.M. Schmidt, S.K. Chen, S. Hattar, Intrinsically
16009–16014. doi:0806114105 [pii] (2008). https:// photosensitive retinal ganglion cells: Many
doi.org/10.1073/pnas.0806114105 subtypes, diverse functions. Trends Neurosci.
34(11), 572–580 (2011). https://doi.org/10.1016/
j.tins.2011.07.001
43. R. Nelson, E.V. Famiglietti Jr., H. Kolb, Intracellular 57. D.M. Dacey, Parallel pathways for spectral
staining reveals different levels of stratification for coding in primate retina. Annu. Rev. Neurosci.
on- and off-center ganglion cells in cat retina. J. 23, 743–775 (2000). https://doi.org/10.1146/
Neurophysiol. 41(2), 472–483 (1978) annurev.neuro.23.1.743
44. J.H. Belgum, D.R. Dvorak, J.S. McReynolds, Sus- 58. R.W. Rodieck, J. Stone, Analysis of receptive fields
tained and transient synaptic inputs to on-off gan- of cat retinal ganglion cells. J. Neurophysiol. 28(5),
glion cells in the mudpuppy retina. J. Physiol. 340, 832–849 (1965)
599–610 (1983) 59. R.W. Rodieck, Quantitative analysis of cat retinal
45. E.P. Chen, R.A. Linsenmeier, Centre components ganglion cell response to visual stimuli. Vis. Res.
of cone-driven retinal ganglion cells: Differential 5(11), 583–601 (1965)
sensitivity to 2-amino-4-phosphonobutyric acid. J. 60. J.J. Nassi, E.M. Callaway, Parallel processing strate-
Physiol. 419, 77–93 (1989) gies of the primate visual system. Nat. Rev.
46. E.J. Chichilnisky, R.S. Kalmar, Functional asym- Neurosci. 10(5), 360–372 (2009). https://doi.org/
metries in ON and OFF ganglion cells of pri- 10.1038/nrn2619
mate retina. J. Neurosci. 22(7), 2737–2747 (2002). 61. N. Babai, W.B. Thoreson, Horizontal cell feed-
doi:20026215 22/7/2737 [pii] back regulates calcium currents and intracellular
47. H. Kolb, E.V. Famiglietti, Rod and cone pathways calcium levels in rod photoreceptors of salamander
in the inner plexiform layer of cat retina. Science and mouse retina. J. Physiol. 587(Pt 10), 2353–
186(4158), 47–49 (1974) 2364. doi:jphysiol.2009.169656 [pii] (2009). https:/
48. S.A. Bloomfield, R.F. Dacheux, Rod vision: Path- /doi.org/10.1113/jphysiol.2009.169656
ways and processing in the mammalian retina. Prog. 62. J.D. Crook, M.B. Manookin, O.S. Packer, D.M.
Retin. Eye Res. 20(3), 351–384 (2001). doi:S1350- Dacey, Horizontal cell feedback without cone type-
9462(00)00031-8 [pii] selective inhibition mediates “red-green” color op-
49. M.B. Manookin, D.L. Beaudoin, Z.R. Ernst, L.J. ponency in midget ganglion cells of the primate
Flagel, J.B. Demb, Disinhibition combines with ex- retina. J. Neurosci. 31(5), 1762–1772 (2011). https:/
citation to extend the operating range of the OFF vi- /doi.org/10.1523/JNEUROSCI.4385-10.2011
sual pathway in daylight. J. Neurosci. 28(16), 4136– 63. A. Drinnenberg, F. Franke, R.K. Morikawa, J. Jut-
4150. doi:28/16/4136 [pii] (2008). https://doi.org/ tner, D. Hillier, P. Hantz, A. Hierlemann, R.A. da
10.1523/JNEUROSCI.4274-07.2008 Silveira, B. Roska, How diverse retinal functions
50. R.G. Smith, M.A. Freed, P. Sterling, Microcircuitry arise from feedback at the first visual synapse.
of the dark-adapted cat retina: Functional architec- Neuron 99(1), 117 (2018). https://doi.org/10.1016/
ture of the rod-cone network. J. Neurosci. 6(12), j.neuron.2018.06.001
3505–3517 (1986) 64. S.C. Mangel, Analysis of the horizontal cell contri-
51. T.N. Cornsweet, Visual Perception (Academic Press, bution to the receptive field surround of ganglion
New York, 1970) cells in the rabbit retina. J. Physiol. 442, 211–234
52. C. Enroth-Cugell, J.G. Robson, Functional charac- (1991)
teristics and diversity of cat retinal ganglion cells. 65. M.J. McMahon, O.S. Packer, D.M. Dacey, The
Basic characteristics and quantitative description. classical receptive field surround of primate
Invest. Ophthalmol. Vis. Sci. 25(3), 250–267 (1984) parasol ganglion cells is mediated primarily
53. S.W. Kuffler, Discharge patterns and functional or- by a non-GABAergic pathway. J. Neurosci.
ganization of mammalian retina. J. Neurophysiol. 24(15), 3736–3745 (2004). https://doi.org/10.1523/
16(1), 37–68 (1953) JNEUROSCI.5252-03.2004
54. G.D. Field, J.L. Gauthier, A. Sher, M. Greschner, 66. O.S. Packer, J. Verweij, P.H. Li, J.L. Schnapf,
T.A. Machado, L.H. Jepson, J. Shlens, D.E. Gun- D.M. Dacey, Blue-yellow opponency in primate S
ning, K. Mathieson, W. Dabrowski, L. Paninski, cone photoreceptors. J. Neurosci. 30(2), 568–572.
A.M. Litke, E.J. Chichilnisky, Functional connectiv- doi:30/2/568 [pii] (2010). https://doi.org/10.1523/
ity in the retina at the resolution of photoreceptors. JNEUROSCI.4738-09.2010
Nature 467(7316), 673–677 (2010). https://doi.org/ 67. S.A. Bloomfield, D. Xin, Surround inhibition of
10.1038/nature09424 mammalian AII amacrine cells is generated in the
55. L.E. Wool, J.D. Crook, J.B. Troy, O.S. Packer, Q. proximal retina. J. Physiol. 523(Pt 3), 771–783
Zaidi, D.M. Dacey, Nonselective wiring accounts (2000)
for red-green opponency in midget ganglion cells 68. C.W. Oyster, The Human Eye: Structure and Func-
of the primate retina. J. Neurosci. 38(6), 1520–1540 tion (Sinauer Associates, Sunderland, 1999)
(2018). https://doi.org/10.1523/JNEUROSCI.1688- 69. R.W. Rodieck, The Vertebrate Retina (Freeman, San
17.2017 Francisco, 1973)
56. D.M. Dacey, Circuitry for color coding in the pri- 70. A. Bill, G.O. Sperber, Control of retinal and
mate retina. Proc. Natl. Acad. Sci. U. S. A. 93(2), choroidal blood flow. Eye 4(Pt 2), 319–325 (1990)
582–588 (1996) 71. J.W. Kiel, The Ocular Circulation, Integrated Sys-
tems Physiology: From Molecule to Function to
Disease (Morgan & Claypool Life Sciences, San Lucas, Neural reprogramming in retinal degenera-
Rafael, 2010) tion. Invest. Ophthalmol. Vis. Sci. 48(7), 3364–3371
72. C.J. Pournaras, E. Rungger-Brandle, C.E. Riva, (2007). https://doi.org/10.1167/iovs.07-0032
S.H. Hardarson, E. Stefansson, Regulation of reti- 86. M.M. LaVail, Analysis of neurological mutants with
nal blood flow in health and disease. Prog. Retin. inherited retinal degeneration. Friedenwald lecture.
Eye Res. 27(3), 284–330 (2008). https://doi.org/ Invest. Ophthalmol. Vis. Sci. 21(5), 638–657 (1981)
10.1016/j.preteyeres.2008.02.002 87. M. Menotti-Raymond, K.H. Deckman, V. David, J.
73. L. Schmetterer, J. W. Kiel (eds.), Ocular Blood Flow Myrkalo, S.J. O’Brien, K. Narfstrom, Mutation dis-
(Springer, Berlin/Heidelberg, 2012) covered in a feline model of human congenital reti-
74. A. Alm, A. Bill, The oxygen supply to the retina. II. nal blinding disease. Invest. Ophthalmol. Vis. Sci.
Effects of high intraocular pressure and of increased 51(6), 2852–2859 (2010). https://doi.org/10.1167/
arterial carbon dioxide tension on uveal and retinal iovs.09-4261
blood flow in cats. A study with radioactively la- 88. G.M. Acland, G.D. Aguirre, J. Bennett, T.S. Ale-
belled microspheres including flow determinations man, A.V. Cideciyan, J. Bennicelli, N.S. Dejneka,
in brain and some other tissues. Acta Physiol. Scand. S.E. Pearce-Kelling, A.M. Maguire, K. Palczewski,
84(3), 306–319 (1972) W.W. Hauswirth, S.G. Jacobson, Long-term restora-
75. G. Birol, S. Wang, E. Budzynski, N.D. Wangsa- tion of rod and cone vision by single dose rAAV-
Wirawan, R.A. Linsenmeier, Oxygen distribution mediated gene transfer to the retina in a ca-
and consumption in the macaque retina. Am. J. nine model of childhood blindness. Mol. Ther.
Physiol. 293(3), H1696–H1704 (2007) 12(6), 1072–1082 (2005). https://doi.org/10.1016/
76. R.A. Linsenmeier, R.D. Braun, Oxygen distribution j.ymthe.2005.08.008
and consumption in the cat retina during normoxia 89. K. Narfstrom, M.L. Katz, M. Ford, T.M. Redmond,
and hypoxemia. J. Gen. Physiol. 99(2), 177–197 E. Rakoczy, R. Bragadottir, In vivo gene therapy in
(1992) young and adult RPE65-/- dogs produces long-term
77. O. Strauss, The retinal pigment epithelium in vi- visual improvement. J. Hered. 94(1), 31–37 (2003)
sual function. Physiol. Rev. 85(3), 845–881 (2005). 90. J.W.B. Bainbridge, M.S. Mehat, V. Sundaram,
https://doi.org/10.1152/physrev.00021.2004 S.J. Robbie, S.E. Barker, C. Ripamonti, A. Geor-
78. J. Ahmed, M.K. Pulfer, R.A. Linsenmeier, Measure- giadis, F.M. Mowat, S.G. Beattie, P.J. Gardner,
ment of blood flow through the retinal circulation K.L. Feathers, V.A. Luong, S. Yzer, K. Balaggan,
of the cat during normoxia and hypoxemia using A. Viswanathan, T.J.L. de Ravel, I. Casteels, G.E.
fluorescent microspheres. Microvasc. Res. 62(2), Holder, N. Tyler, F.W. Fitzke, R.G. Weleber, M. Nar-
143–153 (2001) dini, A.T. Moore, D.A. Thompson, S.M. Petersen-
79. J. Kiryu, S. Asrani, M. Shahidi, M. Mori, R. Zeimer, Jones, M. Michaelides, L.I. van den Born, A. Stock-
Local response of the primate retinal microcircu- man, A.J. Smith, G. Rubin, R.R. Ali, Long-term
lation to increased metabolic demand induced by effect of gene therapy on Leber’s congenital amau-
flicker. Invest. Ophthalmol. Vis. Sci. 36(7), 1240– rosis. N. Engl. J. Med. 372(20), 1887–1897 (2015).
1246 (1995) https://doi.org/10.1056/NEJMoa1414221
80. T. Vo Van, C.E. Riva, Variations of blood flow at 91. M.E. Pennesi, R.G. Weleber, P. Yang, C. Whitebirch,
optic nerve head induced by sinusoidal flicker stimu- B. Thean, T.R. Flotte, M. Humphries, E. Chegarnov,
lation in cats. J. Physiol. 482(Pt 1), 189–202 (1995) K.N. Beasley, J.T. Stout, J.D. Chulay, Results at 5
81. N. Congdon, B. O’Colmain, C.C. Klaver, R. Klein, years after gene therapy for RPE65-deficient reti-
B. Munoz, D.S. Friedman, J. Kempen, H.R. Taylor, nal dystrophy. Hum. Gene Ther. 29(12), 1428–1437
P. Mitchell, Causes and prevalence of visual impair- (2018). https://doi.org/10.1089/hum.2018.014
ment among adults in the United States. Arch. Oph- 92. S. Russell, J. Bennett, J.A. Wellman, D.C. Chung,
thalmol. 122(4), 477–485 (2004). https://doi.org/ Z.F. Yu, A. Tillman, J. Wittes, J. Pappas, O. Elci,
10.1001/archopht.122.4.477 S. McCague, D. Cross, K.A. Marshall, J. Walshire,
82. J.R. Heckenlively, J. Bouchman, L. Friedman, Di- T.L. Kehoe, H. Reichert, M. Davis, L. Raffini, L.A.
agnosis and classification of retinitis pigmentosa, in George, F.P. Hudson, L. Dingfield, X.S. Zhu, J.A.
Retinitis pigmentosa, ed. by J. R. Heckenlively, (J.B. Haller, E.H. Sohn, V.B. Mahajan, W. Pfeifer, M.
Lippincott, Philadelphia, 1988) Weckmann, C. Johnson, D. Gewaily, A. Drack, E.
83. R.A. Saleem, M.A. Walter, The complexities of oc- Stone, K. Wachtel, F. Simonelli, B.P. Leroy, J.F.
ular genetics. Clin. Genet. 61(2), 79–88 (2002) Wright, K.A. High, A.M. Maguire, Efficacy and
84. M.S. Humayun, M. Prince, E. de Juan Jr., Y. Bar- safety of voretigene neparvovec (AAV2-hRPE65v2)
ron, M. Moskowitz, I.B. Klock, A.H. Milam, Mor- in patients with RPE65-mediated inherited retinal
phometric analysis of the extramacular retina from dystrophy: A randomised, controlled, open-label,
postmortem eyes with retinitis pigmentosa. Invest. phase 3 trial. Lancet 390(10097), 849–860 (2017).
Ophthalmol. Vis. Sci. 40(1), 143–148 (1999) https://doi.org/10.1016/s0140-6736(17)31868-8
85. R.E. Marc, B.W. Jones, J.R. Anderson, K. Ki- 93. G.J. Chader, J. Weiland, M.S. Humayun, Artifi-
nard, D.W. Marshak, J.H. Wilson, T. Wensel, R.J. cial vision: Needs, functioning, and testing of a
retinal electronic prosthesis. Prog. Brain Res. 175, 105. C. Camras, C. Toris, Advances in glaucoma man-
317–332. doi:S0079-6123(09)17522-2 [pii] (2009). agement: Risk factors, diagnostic tools, thera-
https://doi.org/10.1016/S0079-6123(09)17522-2 pies and the role of prostaglandin analogs. Fore-
94. H. Tomita, E. Sugano, H. Isago, M. Tamai, Chan- word. Surv. Ophthalmol. 53(Suppl 1), S1–S2.
nelrhodopsins provide a breakthrough insight into doi:S0039-6257(08)00138-0 [pii] (2008). https://
strategies for curing blindness. J. Genet. 88(4), 409– doi.org/10.1016/j.survophthal.2008.08.013
415 (2009) 106. D.S. Minckler, R.A. Hill, Use of novel devices for
95. R. Klein, B.E. Klein, M.D. Knudtson, S.M. Meuer, control of intraocular pressure. Exp. Eye Res. 88(4),
M. Swift, R.E. Gangnon, Fifteen-year cumula- 792–798. doi:S0014-4835(08)00390-4 [pii] (2009).
tive incidence of age-related macular degeneration: https://doi.org/10.1016/j.exer.2008.11.010
The beaver dam eye study. Ophthalmology 114(2), 107. S. Mosaed, L. Dustin, D.S. Minckler, Comparative
253–262. doi:S0161-6420(06)01478-3 [pii] (2007). outcomes between newer and older surgeries for
https://doi.org/10.1016/j.ophtha.2006.10.040 glaucoma. Trans. Am. Ophthalmol. Soc. 107, 127–
96. C.A. Curcio, M. Johnson, J.D. Huang, M. Rudolf, 133 (2009)
Aging, age-related macular degeneration, and 108. R. Engerman, D. Finkelstein, G. Aguirre, K.R. Did-
the response-to-retention of apolipoprotein B- die, R.R. Fox, R.N. Frank, S.D. Varma, Ocular com-
containing lipoproteins. Prog. Retin. Eye Res. 28(6), plications. Diabetes 31(Suppl 1 Pt 2), 82–88 (1982)
393–422. doi:S1350-9462(09)00057-3 [pii] (2009). 109. T.S. Kern, R.L. Engerman, Capillary lesions de-
https://doi.org/10.1016/j.preteyeres.2009.08.001 velop in retina rather than cerebral cortex in diabetes
97. P.T. Johnson, G.P. Lewis, K.C. Talaga, M.N. Brown, and experimental galactosemia. Arch. Ophthalmol.
P.J. Kappel, S.K. Fisher, D.H. Anderson, L.V. John- 114(3), 306–310 (1996)
son, Drusen-associated degeneration in the retina. 110. R. Chibber, B.M. Ben-Mahmud, S. Chibber, E.M.
Invest. Ophthalmol. Vis. Sci. 44(10), 4481–4488 Kohner, Leukocytes in diabetic retinopathy. Curr.
(2003) Diabetes Rev. 3(1), 3–14 (2007)
98. A. Abdelsalam, L. Del Priore, M.A. Zarbin, Drusen 111. D.L. Hatchell, S.H. Sinclair, Role of leukocytes in
in age-related macular degeneration: Pathogenesis, diabetic retinopathy, in Physiology and Pathophysi-
natural course, and laser photocoagulation-induced ology of Leukocyte Adhesion, ed. by D. N. Granger,
regression. Surv. Ophthalmol. 44(1), 1–29 (1999). G. W. Schmid-Schonbein, (Oxford University Press,
doi:S0039625799000727 [pii] New York, 1995), pp. 458–466
99. C. Campa, S.P. Harding, Anti-VEGF compounds in 112. S. Schroder, W. Palinski, G.W. Schmid-Schonbein,
the treatment of neovascular age related macular Activated monocytes and granulocytes, capillary
degeneration. Curr. Drug Targets 12(2), 173–181 nonperfusion, and neovascularization in diabetic
(2011). doi:BSP/CDT/E-Pub/00173 [pii] retinopathy. Am. J. Pathol. 139(1), 81–100 (1991)
100. R.L. Stamper, S.S. Sanghvi, Intraocular pressure: 113. A. Harris, O. Arend, R.P. Danis, D. Evans, S. Wolf,
Measurement, regulation, and flow relationships, in B.J. Martin, Hyperoxia improves contrast sensitiv-
Duane’s Foundations of Clinical Ophthalmology, ity in early diabetic retinopathy. Br. J. Ophthalmol.
ed. by W. Tasman, E. A. Jaeger, vol. 2, (Lippincott- 80(3), 209–213 (1996)
Raven, Philadelphia, 1996), pp. 1–31 114. T.N. Crawford, D.V. Alfaro 3rd, J.B. Kerrison, E.P.
101. J.W. Sassani, Glaucoma, in Duane’s Foundations Jablon, Diabetic retinopathy and angiogenesis. Curr.
of Clinical Ophthalmology, ed. by W. Tasman, E. Diabetes Rev. 5(1), 8–13 (2009)
A. Jaeger, vol. 3, (Lippincott-Raven, Philadelphia, 115. R.N. Frank, Diabetic retinopathy. N. Engl. J. Med.
1996), pp. 1–30 350(1), 48–58 (2004)
102. G. Garhofer, G. Fuchsjager-Mayrl, C. Vass, B. 116. P. Mitchell, F. Bandello, U. Schmidt-Erfurth, G.E.
Pemp, A. Hommer, L. Schmetterer, Retrobulbar Lang, P. Massin, R.O. Schlingemann, F. Sutter, C.
blood flow velocities in open angle glaucoma and Simader, G. Burian, O. Gerstner, A. Weichselberger,
their association with mean arterial blood pres- The RESTORE study Ranibizumab monotherapy
sure. Invest. Ophthalmol. Vis. Sci. 51(12), 6652– or combined with laser versus laser monother-
6657. doi:iovs.10-5490 [pii] (2010). https://doi.org/ apy for diabetic macular edema. Ophthalmology
10.1167/iovs.10-5490 118(4), 615–625 (2011). https://doi.org/10.1016/
103. H.A. Quigley, S.J. McKinnon, D.J. Zack, M.E. j.ophtha.2011.01.031
Pease, L.A. Kerrigan-Baumrind, D.F. Kerrigan, R.S. 117. A. Salam, R. Mathew, S. Sivaprasad, Treatment of
Mitchell, Retrograde axonal transport of BDNF in proliferative diabetic retinopathy with anti-VEGF
retinal ganglion cells is blocked by acute IOP ele- agents. Acta Ophthalmol. (2011). https://doi.org/
vation in rats. Invest. Ophthalmol. Vis. Sci. 41(11), 10.1111/j.1755-3768.2010.02079.x
3460–3466 (2000) 118. D.A. Antonetti, A.J. Barber, S.K. Bronson, W.M.
104. C.B. Toris, Pharmacotherapies for glaucoma. Curr. Freeman, T.W. Gardner, L.S. Jefferson, M. Kester,
Mol. Med. 10(9), 824–840 (2010). doi:CMM # 79 S.R. Kimball, J.K. Krady, K.F. LaNoue, C.C. Nor-
[pii] bury, P.G. Quinn, L. Sandirasegarane, I.A. Simp-
son, Diabetic retinopathy: Seeing beyond glucose-
induced microvascular disease. Diabetes 55(9), 133. P.Z. Marmarelis, K.I. Naka, Nonlinear analysis and
2401–2411 (2006) synthesis of receptive-field responses in the catfish
119. A.J. Barber, E. Lieth, S.A. Khin, D.A. Antonetti, retina. I. Horizontal cell leads to ganglion cell chain.
A.G. Buchanan, T.W. Gardner, Penn State Retina J. Neurophysiol. 36(4), 605–618 (1973b). https://
Res G, Neural apoptosis in the retina during experi- doi.org/10.1152/jn.1973.36.4.605
mental and human diabetes – Early onset and effect 134. P.Z. Marmarelis, K.I. Naka, Nonlinear analysis and
of insulin. J. Clin. Invest. 102(4), 783–791 (1998) synthesis of receptive-field responses in the catfish
120. G.C. Brown, Arterial occlusive disease, in Vireoreti- retina. II. One-input white-noise analysis. J Neu-
nal Disease: The Essentials, ed. by C. D. Regillo, G. rophysiol. 36(4), 619–633 (1973c). https://doi.org/
C. Brown, H. W. Flynn, (Thieme, New York, 1999), 10.1152/jn.1973.36.4.619
pp. 97–115 135. G.P. Das, P.J. Vance, D. Kerr, S.A. Coleman, T.M.
121. S.S. Hayreh, T.A. Weingeist, Experimental occlu- McGinnity, J.K. Liu, Computational modelling of
sion of the central artery of the retina. IV: Retinal salamander retinal ganglion cells using machine
tolerance time to acute ischaemia. Br. J. Ophthalmol. learning approaches. Neurocomputing 325, 101–
64(11), 818–825 (1980) 112 (2019)
122. J.G. Clarkson, Central retinal vein occlusion, in 136. M.N. Geffen, S.E. de Vries, M. Meister, Retinal
Retina, ed. by S. J. Ryan, vol. 2, 2nd edition edn. ganglion cells can rapidly change polarity from
(Mosby, St. Louis, 1994), pp. 1379–1385 off to on. PLoS Biol. 5(3), e65. doi:06-PLBI-
123. S.S. Hayreh, P. Rojas, P. Podhajsky, P. Montague, RA-1074R3 [pii] (2007). https://doi.org/10.1371/
R.F. Woolson, Ocular neovascularization with reti- journal.pbio.0050065
nal vascular occlusion-III. Incidence of ocular neo- 137. B. Roska, E. Nemeth, L. Orzo, F.S. Werblin, Three
vascularization with retinal vein occlusion. Ophthal- levels of lateral inhibition: A space-time study of
mology 90(5), 488–506 (1983) the retina of the tiger salamander. J. Neurosci. 20(5),
124. C.J. Pournaras, Retinal oxygen distribution. Its role 1941–1951 (2000)
in the physiopathology of vasoproliferative microan- 138. M.C. Citron, V.Z. Marmarelis, Applications of
giopathies. Retina 15(4), 332–347 (1995) minimum-order wiener modeling to retinal ganglion
125. A.P. Adamis, D.T. Shima, M.J. Tolentino, E.S. cell spatiotemporal dynamics. Biol. Cybern. 57(4–
Gragoudas, N. Ferrara, J. Folkman, P.A. D’Amore, 5), 241–247 (1987)
J.W. Miller, Inhibition of vascular endothelial 139. M.N. Oguztoreli, Modelling and simulation of ver-
growth factor prevents retinal ischemia-associated tebrate retina. Biol. Cybern. 37(1), 53–61 (1980)
iris neovascularization in a nonhuman primate. 140. M.H. Foerster, W.A. van de Grind, O.J. Grusser,
Arch. Ophthalmol. 114(1), 66–71 (1996) Frequency transfer properties of three distinct types
126. P.A. Keane, S.R. Sadda, Retinal vein occlusion and of cat horizontal cells. Exp. Brain Res. 29(3–4),
macular edema – Critical evaluation of the clinical 347–366 (1977)
value of ranibizumab. Clin. Ophthalmol. 5, 771– 141. D. Tranchina, J. Gordon, R. Shapley, Spatial and
781 (2011). https://doi.org/10.2147/OPTH.S13774. temporal properties of luminosity horizontal cells
opth-5-771 [pii] in the turtle retina. J. Gen. Physiol. 82(5), 573–598
127. A.M. Shah, N.M. Bressler, L.M. Jampol, Does (1983)
laser still have a role in the management of reti- 142. P. Antinucci, R. Hindges, Orientation-selective
nal vascular and neovascular diseases? Am J. Oph- retinal circuits in vertebrates. Front. Neural
thalmol. doi:S0002-9394(11)00326-6 [pii] (2011). Circuit. 12, 11 (2018). https://doi.org/10.3389/
https://doi.org/10.1016/j.ajo.2011.04.015 fncir.2018.00011
128. R. Michels, C. Wilkinson, T. Rice, Retinal Detach- 143. W. Wei, Neural mechanisms of motion processing in
ment (Mosby, St. Louis, 1990) the mammalian retina. Annu. Rev. Vis. Sci. 4, 165–
129. R.A. Linsenmeier, L. Padnick-Silver, Metabolic de- 192 (2018). https://doi.org/10.1146/annurev-vision-
pendence of photoreceptors on the choroid in the 091517-034048
normal and detached retina. Invest. Ophthalmol. Vis. 144. J. Chen, M.L. Woodruff, T. Wang, F.A. Concepcion,
Sci. 41(10), 3117–3123 (2000) D. Tranchina, G.L. Fain, Channel modulation
130. S. Wang, R.A. Linsenmeier, Hyperoxia improves and the mechanism of light adaptation in
oxygen consumption in the detached feline retina. mouse rods. J. Neurosci. 30(48), 16232–16240.
Invest. Ophthalmol. Vis. Sci. 48(3), 1335–1341 doi:30/48/16232 [pii] (2010). https://doi.org/
(2007) 10.1523/JNEUROSCI.2868-10.2010
131. P.Z. Marmarelis, K. Naka, Experimental analysis of 145. D.A. Baylor, Photoreceptor signals and vision. Proc-
a neural system: Two modeling approaches. Kyber- tor lecture. Invest. Ophthalmol. Vis. Sci. 28(1), 34–
netik 15(1), 11–26 (1974) 49 (1987)
132. P.Z. Marmarelis, K.I. Naka, Nonlinear analysis and 146. T.D. Lamb, E.N. Pugh Jr., Phototransduction, dark
synthesis of receptive-field responses in the catfish adaptation, and rhodopsin regeneration the proctor
retina. 3. Two-input white-noise analysis. J. Neu- lecture. Invest. Ophthalmol. Vis. Sci. 47(12), 5137–
rophysiol. 36(4), 634–648 (1973a). https://doi.org/ 5152 (2006)
10.1152/jn.1973.36.4.634
147. K.W. Yau, Phototransduction mechanism in retinal ducibility of vertebrate rod single-photon responses.
rods and cones. The Friedenwald lecture. Invest. J. Gen. Physiol. 122(4), 419–444 (2003). https://
Ophthalmol. Vis. Sci. 35(1), 9–32 (1994) doi.org/10.1085/jgp.200308832
148. X. Zhang, R.H. Cote, cGMP signaling in vertebrate 163. D.R. Pepperberg, D.G. Birch, D.C. Hood, Elec-
retinal photoreceptor cells. Front Biosci 10, 1191– troretinographic determination of human rod flash
1204 (2005) response in vivo. Methods Enzymol. 316, 202–223
149. K.I. Naka, W.A. Rushton, S-potentials from lumi- (2000)
nosity units in the retina of fish (Cyprinidae). J. 164. D.G. Birch, D.C. Hood, S. Nusinowitz, D.R. Pepper-
Physiol. 185(3), 587–599 (1966) berg, Abnormal activation and inactivation mecha-
150. W.A. Hagins, R.D. Penn, S. Yoshikami, Dark cur- nisms of rod transduction in patients with autosomal
rent and photocurrent in retinal rods. Biophys. dominant retinitis pigmentosa and the pro-23-his
J. 10(5), 380–412 (1970). https://doi.org/10.1016/ mutation. Invest. Ophthalmol. Vis. Sci. 36(8), 1603–
S0006-3495(70)86308-1 1614 (1995)
151. D.A. Baylor, T.D. Lamb, K.W. Yau, The membrane 165. D.R. Pepperberg, D.G. Birch, K.P. Hofmann, D.C.
current of single rod outer segments. J. Physiol. 288, Hood, Recovery kinetics of human rod phototrans-
589–611 (1979) duction inferred from the two-branched alpha-wave
152. R. Granit, The components of the retinal action po- saturation function. J. Opt. Soc. Am. A Opt. Image
tential in mammals and their relation to the discharge Sci. Vis. 13(3), 586–600 (1996)
in the optic nerve. J. Physiol. 77(3), 207–239 (1933) 166. D.R. Pepperberg, D.G. Birch, D.C. Hood, Photore-
153. D.C. Hood, D.G. Birch, Computational models of sponses of human rods in vivo derived from paired-
rod-driven retinal activity. IEEE Engineering in flash electroretinograms. Vis. Neurosci. 14(1), 73–
Medicine and Biology Magazine (February 1995), 82 (1997)
pp. 59–66 167. J.J. Kang Derwent, S.M. Saszik, H. Maeda, D.M.
154. D.A. Baylor, B.J. Nunn, J.L. Schnapf, The photocur- Little, M.T. Pardue, L.J. Frishman, D.R. Pepperberg,
rent, noise and spectral sensitivity of rods of the Test of the paired-flash electroretinographic method
monkey Macaca fascicularis. J. Physiol. 357, 575– in mice lacking b-waves. Vis. Neurosci. 24(2), 141–
607 (1984) 149. doi:S0952523807070162 [pii] (2007). https://
155. D.C. Hood, D.G. Birch, A quantitative measure of doi.org/10.1017/S0952523807070162
the electrical activity of human rod photoreceptors 168. J.R. Hetling, D.R. Pepperberg, Sensitivity and ki-
using electroretinography. Vis. Neurosci. 5(4), 379– netics of mouse rod flash responses determined in
387 (1990) vivo from paired-flash electroretinograms. J. Phys-
156. T.D. Lamb, E.N. Pugh Jr., A quantitative account of iol. 516(Pt 2), 593–609 (1999)
the activation steps involved in phototransduction in 169. A. Gal, E. ApfelstedtSylla, A.R. Janecke, E. Zren-
amphibian photoreceptors. J. Physiol. 449, 719–758 ner, Rhodopsin mutations in inherited retinal dys-
(1992) trophies and dysfunctions. Prog. Retin. Eye Res.
157. M.E. Breton, A.W. Schueller, T.D. Lamb, E.N. Pugh 16(1), 51–79 (1997). https://doi.org/10.1016/s1350-
Jr., Analysis of ERG a-wave amplification and kinet- 9462(96)00021-3
ics in terms of the G-protein cascade of phototrans- 170. J.J. Kang Derwent, D.J. Derlacki, J.R. Hetling, G.A.
duction. Invest. Ophthalmol. Vis. Sci. 35(1), 295– Fishman, D.G. Birch, S. Grover, E.M. Stone, D.R.
309 (1994) Pepperberg, Dark adaptation of rod photoreceptors
158. W.H. Cobbs, E.N. Pugh Jr., Kinetics and compo- in normal subjects, and in patients with Stargardt dis-
nents of the flash photocurrent of isolated retinal ease and an ABCA4 mutation. Invest. Ophthalmol.
rods of the larval salamander, Ambystoma tigrinum. Vis. Sci. 45(7), 2447–2456 (2004)
J. Physiol. 394, 529–572 (1987) 171. J.G. Robson, L.J. Frishman, Response linearity and
159. S. Forti, A. Menini, G. Rispoli, V. Torre, Kinetics of kinetics of the cat retina: The bipolar cell com-
phototransduction in retinal rods of the newt Triturus ponent of the dark-adapted electroretinogram. Vis.
cristatus. J. Physiol. 419, 265–295 (1989) Neurosci. 12(5), 837–850 (1995)
160. A.V. Cideciyan, S.G. Jacobson, An alternative pho- 172. D.C. Hood, D.G. Birch, A computational model of
totransduction model for human rod and cone ERG the amplitude and implicit time of the b-wave of the
a-waves: Normal parameters and variation with age. human ERG. Vis. Neurosci. 8(2), 107–126 (1992)
Vis. Res. 36(16), 2609–2621 (1996) 173. G.B. Arden, Voltage gradients across the receptor
161. R.D. Hamer, S.C. Nicholas, D. Tranchina, T.D. layer of the isolated rat retina. J. Physiol. 256(2),
Lamb, J.L. Jarvinen, Toward a unified model of ver- 333–360 (1976)., 331
tebrate rod phototransduction. Vis. Neurosci. 22(4), 174. J.J. Kang Derwent, R.A. Linsenmeier, Intraretinal
417–436. doi:S0952523805224045 [pii] (2005). analysis of the a-wave of the electroretinogram
https://doi.org/10.1017/S0952523805224045 (ERG) in dark-adapted intact cat retina. Vis. Neu-
162. R.D. Hamer, S.C. Nicholas, D. Tranchina, P.A. Lieb- rosci. 18(3), 353–363 (2001)
man, T.D. Lamb, Multiple steps of phosphorylation 175. D.M. Schneeweis, J.L. Schnapf, Photovoltage of
of activated rhodopsin can account for the repro- rods and cones in the macaque retina. Science
268(5213), 1053–1056 (1995)
176. S. Barnes, B. Hille, Ionic channels of the inner 191. Z. Derafshi, B.E. Kunzer, E.M. Mugler, N.
segment of tiger salamander cone photoreceptors. J. Rokhmanova, D.W. Park, H. Tajalli, K. Shetty,
Gen. Physiol. 94(4), 719–743 (1989) Z. Ma, J.C. Williams, J.R. Hetling, Corneal
177. G.L. Fain, F.N. Quandt, B.L. Bastian, H.M. Ger- potential maps measured with multi-electrode
schenfeld, Contribution of a caesium-sensitive con- electroretinography in rat eyes with experimental
ductance increase to the rod photoresponse. Nature lesions. Invest. Ophthalmol. Vis. Sci. 58(7), 2863–
272(5652), 466–469 (1978) 2873 (2017). https://doi.org/10.1167/iovs.16-
178. Y. Kamiyama, T. Ogura, S. Usui, Ionic current 20726
model of the vertebrate rod photoreceptor. Vis. Res. 192. A.N. Selner, Z. Derafshi, B.E. Kunzer, J.R. Hetling,
36(24), 4059–4068 (1996) Three-dimensional model of electroretinogram field
179. J.G. Robson, L.J. Frishman, The rod-driven a-wave potentials in the rat eye. I.E.E.E. Trans. Biomed.
of the dark-adapted mammalian electroretinogram. Eng. 65(12), 2781–2789 (2018). https://doi.org/
Prog. Retin. Eye Res. 39, 1–22 (2014). https:// 10.1109/TBME.2018.2816591
doi.org/10.1016/j.preteyeres.2013.12.003 193. E.M. Callaway, Structure and function of parallel
180. J.G. Robson, L.J. Frishman, Corrigendum to “The pathways in the primate early visual system. J.
rod-driven a-wave of the dark-adapted mammalian Physiol. 566(Pt 1), 13–19 (2005). https://doi.org/
electroretinogram” [Progress in retinal and eye re- 10.1113/jphysiol.2005.088047
search, volume 39, march 2014, pages 1–22]. Prog. 194. D.M. Dacey, Primate retina: Cell types, circuits and
Retin. Eye Res. 59, 202 (2017). https://doi.org/ color opponency. Prog. Retin. Eye Res. 18(6), 737–
10.1016/j.preteyeres.2017.05.002 763 (1999)
181. J.G. Robson, L.J. Frishman, Photoreceptor and bipo- 195. E. Kaplan, The receptive-field structure of retinal
lar cell contributions to the cat electroretinogram: A ganglion cells in cat and monkey, in Vision and Vi-
kinetic model for the early part of the flash response. sual Dysfunction, The Neural Basis of Visual Func-
J. Opt. Soc. Am. A Opt. Image Sci. Vis. 13(3), 613– tion, ed. by G. Leventhal, vol. IV, (CRC Press, Boca
622 (1996) Raton, 1991), pp. 10–40
182. J.G. Robson, L.J. Frishman, Dissecting the dark- 196. M. Meister, M.J. Berry 2nd, The neural code of the
adapted electroretinogram. Doc. Ophthalmol. 95(3– retina. Neuron 22(3), 435–450 (1999). doi:S0896-
4), 187–215 (1998) 6273(00)80700-X [pii]
183. P.A. Sieving, L.J. Frishman, R.H. Steinberg, Sco- 197. P.J. Vance, G.P. Das, D. Kerr, S.A. Coleman, T.M.
topic threshold response of proximal retina in cat. McGinnity, T. Gollisch, J.K. Liu, Bioinspired ap-
J. Neurophysiol. 56(4), 1049–1061 (1986) proach to modeling retinal ganglion cells using sys-
184. P.A. Sieving, C. Nino, Scotopic threshold response tem identification techniques. IEEE Trans. Neural
(STR) of the human electroretinogram. Invest. Oph- Netw. Learn. Sys. 29(5), 1796–1808 (2018). https://
thalmol. Vis. Sci. 29(11), 1608–1614 (1988) doi.org/10.1109/tnnls.2017.2690139
185. J.G. Robson, S.M. Saszik, J. Ahmed, L.J. Frish- 198. S. Wienbar, G.W. Schwartz, The dynamic receptive
man, Rod and cone contributions to the a-wave of fields of retinal ganglion cells. Prog. Retin. Eye
the electroretinogram of the macaque. J. Physiol. Res. 67, 102–117 (2018). https://doi.org/10.1016/
547(Pt 2), 509–530 (2003). https://doi.org/10.1113/ j.preteyeres.2018.06.003
jphysiol.2002.030304 199. C. Enroth-Cugell, J.G. Robson, The contrast sensi-
186. V. Porciatti, Electrophysiological assessment of tivity of retinal ganglion cells of the cat. J. Physiol.
retinal ganglion cell function. Exp. Eye Res. 187(3), 517–552 (1966)
141, 164–170 (2015). https://doi.org/10.1016/ 200. R.A. Linsenmeier, H.G. Jakiela, Non-linear spatial
j.exer.2015.05.008 summation in cat retinal ganglion cells at different
187. R.M. Boynton, L.A. Riggs, The effect of stimulus background levels. Exp. Brain Res. 36(2), 301–309
area and intensity upon the human retinal response. (1979)
J. Exp. Psychol. 42(4), 217–226 (1951) 201. R. Shapley, P. Lennie, Spatial frequency anal-
188. E.E. Sutter, D. Tran, The field topography of ERG ysis in the visual system. Annu. Rev. Neu-
components in man – I. The photopic luminance re- rosci. 8, 547–583 (1985). https://doi.org/10.1146/
sponse. Vis. Res. 32(3), 433–446 (1992). doi:0042- annurev.ne.08.030185.002555
6989(92)90235-B [pii] 202. R.M. Shapley, J.D. Victor, The effect of contrast on
189. T.Y. Lai, W.M. Chan, R.Y. Lai, J.W. Ngai, the transfer properties of cat retinal ganglion cells. J.
H. Li, D.S. Lam, The clinical applications of Physiol. 285, 275–298 (1978)
multifocal electroretinography: A systematic 203. E.J. Chichilnisky, A simple white noise analysis of
review. Surv. Ophthalmol. 52(1), 61–96. doi:S0039- neuronal light responses. Network 12(2), 199–213
6257(06)00174-3 [pii] (2007). https://doi.org/ (2001)
10.1016/j.survophthal.2006.10.005 204. M.C. Citron, R.C. Emerson, W.R. Levick, Nonlinear
190. E.E. Sutter, Imaging visual function with the mul- measurement and classification of receptive fields in
tifocal m-sequence technique. Vis. Res. 41(10– cat retinal ganglion cells. Ann. Biomed. Eng. 16(1),
11), 1241–1255 (2001). doi:S0042-6989(01)00078- 65–77 (1988)
5 [pii]
205. B.G. Cleland, W.R. Levick, Brisk and sluggish con- 222. A.M. Derrington, P. Lennie, The influence of tem-
centrically organized ganglion cells in the cat’s poral frequency and adaptation level on receptive
retina. J. Physiol. 240(2), 421–456 (1974) field organization of retinal ganglion cells in cat. J.
206. R.A. Linsenmeier, L.J. Frishman, H.G. Jakiela, C. Physiol. 333, 343–366 (1982)
Enroth-Cugell, Receptive field properties of x and 223. L.J. Frishman, A.W. Freeman, J.B. Troy, D.E.
y cells in the cat retina derived from contrast sen- Schweitzer-Tong, C. Enroth-Cugell, Spatiotemporal
sitivity measurements. Vis. Res. 22(9), 1173–1183 frequency responses of cat retinal ganglion cells. J.
(1982) Gen. Physiol. 89(4), 599–628 (1987)
207. B.B. Boycott, H. Wassle, The morphological types 224. E.P. Chen, A.W. Freeman, A model for spatiotem-
of ganglion cells of the domestic cat’s retina. J. poral frequency responses in the X cell pathway of
Physiol. 240(2), 397–419 (1974) the cat’s retina. Vis. Res. 29(3), 271–291 (1989).
208. L. Peichl, H. Wassle, Size, scatter and coverage of doi:0042-6989(89)90076-X [pii]
ganglion cell receptive field centres in the cat retina. 225. L.H. Chan, A.W. Freeman, B.G. Cleland, The rod-
J. Physiol. 291, 117–141 (1979) cone shift and its effect on ganglion cells in the
209. J. Stone, Y. Fukuda, Properties of cat retinal gan- cat’s retina. Vis. Res. 32(12), 2209–2219 (1992).
glion cells: A comparison of W-cells with X- and doi:0042-6989(92)90085-W [pii]
Y-cells. J. Neurophysiol. 37(4), 722–748 (1974) 226. J.B. Troy, D.L. Bohnsack, L.C. Diller, Spatial prop-
210. S. Hochstein, R.M. Shapley, Linear and nonlinear erties of the cat X-cell receptive field as a function
spatial subunits in Y cat retinal ganglion cells. J. of mean light level. Vis. Neurosci. 16(6), 1089–1104
Physiol. 262(2), 265–284 (1976a) (1999)
211. S. Hochstein, R.M. Shapley, Quantitative analysis 227. J.B. Troy, J.K. Oh, C. Enroth-Cugell, Effect of am-
of retinal ganglion cell classifications. J. Physiol. bient illumination on the spatial properties of the
262(2), 237–264 (1976b) center and surround of Y-cell receptive fields. Vis.
212. G.W. Schwartz, H. Okawa, F.A. Dunn, J.L. Morgan, Neurosci. 10(4), 753–764 (1993)
D. Kerschensteiner, R.O. Wong, F. Rieke, The spa- 228. J.D. Victor, R.M. Shapley, Receptive field mecha-
tial structure of a nonlinear receptive field. Nat. Neu- nisms of cat X and Y retinal ganglion cells. J. Gen.
rosci. 15(11), 1572–1580 (2012). https://doi.org/ Physiol. 74(2), 275–298 (1979)
10.1038/nn.3225 229. J.D. Victor, R.M. Shapley, B.W. Knight, Nonlinear
213. L.J. Frishman, R.A. Linsenmeier, Effects of picro- analysis of cat retinal ganglion cells in the frequency
toxin and strychnine on non-linear responses of Y- domain. Proc. Natl. Acad. Sci. U. S. A. 74(7), 3068–
type cat retinal ganglion cells. J. Physiol. 324, 347– 3072 (1977)
363 (1982) 230. D.N. Mastronarde, Correlated firing of cat retinal
214. C. Enroth-Cugell, J.G. Robson, D.E. Schweitzer- ganglion cells. I. Spontaneously active inputs to
Tong, A.B. Watson, Spatio-temporal interactions in X- and Y-cells. J. Neurophysiol. 49(2), 303–324
cat retinal ganglion cells showing linear spatial sum- (1983a)
mation. J. Physiol. 341, 279–307 (1983) 231. D.N. Mastronarde, Correlated firing of cat retinal
215. F.M. de Monasterio, Properties of concentrically ganglion cells. II. Responses of X- and Y-cells to
organized X and Y ganglion cells of macaque retina. single quantal events. J. Neurophysiol. 49(2), 325–
J. Neurophysiol. 41(6), 1394–1417 (1978b) 349 (1983b)
216. F.M. de Monasterio, Center and surround mecha- 232. D.N. Mastronarde, Interactions between ganglion
nisms of opponent-color X and Y ganglion cells of cells in cat retina. J. Neurophysiol. 49(2), 350–365
retina of macaques. J. Neurophysiol. 41(6), 1418– (1983c)
1434 (1978a) 233. M. Meister, J. Pine, D.A. Baylor, Multi-neuronal
217. F.M. De Monasterio, P. Gouras, Functional proper- signals from the retina: Acquisition and analysis. J.
ties of ganglion cells of the rhesus monkey retina. J. Neurosci. Methods 51(1), 95–106 (1994). doi:0165-
Physiol. 251(1), 167–195 (1975) 0270(94)90030-2 [pii]
218. L.J. Croner, E. Kaplan, Receptive fields of P and M 234. A.M. Litke, N. Bezayiff, E.J. Chichilnisky, W. Cun-
ganglion cells across the primate retina. Vis. Res. ningham, W. Dabrowski, A.A. Grillo, M. Grivich,
35(1), 7–24 (1995). doi:0042698994E0066T [pii] P. Grybos, P. Hottowy, S. Kachiguine, R.S. Kalmar,
219. E. Kaplan, R.M. Shapley, X and Y cells in the lateral K. Mathieson, D. Petrusca, M. Rahman, A. Sher,
geniculate nucleus of macaque monkeys. J. Physiol. What does the eye tell the brain?: Development
330, 125–143 (1982) of a system for the large-scale recording of retinal
220. Y.T. So, R. Shapley, Spatial properties of X and Y output activity. IEEE Trans. Nucl. Sci. 51, 1434–
cells in the lateral geniculate nucleus of the cat and 1440 (2004)
conduction velocities of their inputs. Exp. Brain Res. 235. M. Greschner, J. Shlens, C. Bakolitsa, G.D. Field,
36(3), 533–550 (1979) J.L. Gauthier, L.H. Jepson, A. Sher, A.M. Litke, E.J.
221. S. Dawis, R. Shapley, E. Kaplan, D. Tranchina, The Chichilnisky, Correlated firing among major gan-
receptive field organization of X-cells in the cat: glion cell types in primate retina. J. Physiol. 589(Pt
Spatiotemporal coupling and asymmetry. Vis. Res. 1), 75–86. doi:jphysiol.2010.193888 [pii] (2011).
24(6), 549–564 (1984). doi:0042-6989(84)90109-3 https://doi.org/10.1113/jphysiol.2010.193888
[pii]
236. J. Shlens, G.D. Field, J.L. Gauthier, M.I. Grivich, sustained retinal ganglion cells. Eur. J. Neurosci.
D. Petrusca, A. Sher, A.M. Litke, E.J. Chichilnisky, 22(8), 2016–2026. doi:EJN4390 [pii] (2005). https:/
The structure of multi-neuron firing patterns /doi.org/10.1111/j.1460-9568.2005.04390.x
in primate retina. J. Neurosci. 26(32), 8254– 248. G.D. Field, E.J. Chichilnisky, Information process-
8266. doi:26/32/8254 [pii] (2006). https://doi.org/ ing in the primate retina: Circuitry and coding.
10.1523/JNEUROSCI.1282-06.2006 Annu. Rev. Neurosci. 30, 1–30 (2007). https://
237. J.W. Pillow, J. Shlens, L. Paninski, A. Sher, A.M. doi.org/10.1146/annurev.neuro.30.051606.094252
Litke, E.J. Chichilnisky, E.P. Simoncelli, Spatio- 249. T. Baden, P. Berens, K. Franke, M. Roman Ro-
temporal correlations and visual signalling in a com- son, M. Bethge, T. Euler, The functional diver-
plete neuronal population. Nature 454(7207), 995– sity of retinal ganglion cells in the mouse. Na-
U937 (2008). https://doi.org/10.1038/nature07140 ture 529(7586), 345–350 (2016). https://doi.org/
238. D.K. Warland, P. Reinagel, M. Meister, Decod- 10.1038/nature16468
ing visual information from a population of retinal 250. J.R. Sanes, R.H. Masland, The types of retinal gan-
ganglion cells. J. Neurophysiol. 78(5), 2336–2350 glion cells: Current status and implications for neu-
(1997) ronal classification. Annu. Rev. Neurosci. 38, 221–
239. D. Dacey, Origins of perception: Retinal ganglion 246 (2015). https://doi.org/10.1146/annurev-neuro-
cell diversity and the creation of parallel visual path- 071714-034120
ways, in The Cognitive Neurosciences, ed. by M. 251. G.D. Field, M. Greschner, J.L. Gauthier, C. Rangel,
Gazzaniga, (MIT Press, Cambridge, MA, 2004), pp. J. Shlens, A. Sher, D.W. Marshak, A.M. Litke, E.J.
281–301 Chichilnisky, High-sensitivity rod photoreceptor in-
240. J.B. Troy, D.E. Schweitzer-Tong, C. Enroth-Cugell, put to the blue-yellow color opponent pathway in
Receptive-field properties of Q retinal ganglion cells macaque retina. Nat. Neurosci. 12(9), 1159–1164
of the cat. Vis. Neurosci. 12(2), 285–300 (1995) (2009). https://doi.org/10.1038/nn.2353
241. M.H. Rowe, J.F. Cox, Spatial receptive-field struc- 252. E. Real, H. Asari, T. Gollisch, M. Meister, Neu-
ture of cat retinal W cells. Vis. Neurosci. 10(4), 765– ral circuit inference from function to structure.
779 (1993) Curr. Biol. 27(2), 189–198 (2017). https://doi.org/
242. J.B. Troy, G. Einstein, R.P. Schuurmans, J.G. Rob- 10.1016/j.cub.2016.11.040
son, C. Enroth-Cugell, Responses to sinusoidal grat- 253. C. Caprara, C. Grimm, From oxygen to ery-
ings of two types of very nonlinear retinal ganglion thropoietin: Relevance of hypoxia for retinal de-
cells of cat. Vis. Neurosci. 3(3), 213–223 (1989) velopment, health and disease. Prog. Retin. Eye
243. N.M. Grzywacz, F.R. Amthor, Robust directional Res. 31(1), 89–119 (2012). https://doi.org/10.1016/
computation in on-off directionally selective gan- j.preteyeres.2011.11.003
glion cells of rabbit retina. Vis. Neurosci. 24(4), 254. R.A. Linsenmeier, H.F. Zhang, Retinal oxygen:
647–661. doi:S0952523807070666 [pii] (2007). From animals to humans. Prog. Retin. Eye Res.
https://doi.org/10.1017/S0952523807070666 58, 115–151 (2017). https://doi.org/10.1016/
244. J.D. Crook, B.B. Peterson, O.S. Packer, F.R. Robin- j.preteyeres.2017.01.003
son, P.D. Gamlin, J.B. Troy, D.M. Dacey, The 255. P.A. Roberts, E.A. Gaffney, P.J. Luthert, A.J.E. Foss,
smooth monostratified ganglion cell: Evidence for H.M. Byrne, Mathematical and computational mod-
spatial diversity in the Y-cell pathway to the lat- els of the retina in health, development and disease.
eral geniculate nucleus and superior colliculus in Prog. Retin. Eye Res. 53, 48–69 (2016b). https://
the macaque monkey. J. Neurosci. 28(48), 12654– doi.org/10.1016/j.preteyeres.2016.04.001
12671. doi:28/48/12654 [pii] (2008). https://doi.org/ 256. D.Y. Yu, S.J. Cringle, Oxygen distribution and con-
10.1523/JNEUROSCI.2986-08.2008 sumption within the retina in vascularised and avas-
245. G.D. Field, A. Sher, J.L. Gauthier, M. Greschner, cular retinas and in animal models of retinal disease.
J. Shlens, A.M. Litke, E.J. Chichilnisky, Spatial Prog. Retin. Eye Res. 20(2), 175–208 (2001)
properties and functional organization of small bis- 257. G. Schneiderman, T.K. Goldstick, Oxygen electrode
tratified ganglion cells in primate retina. J. Neu- design criteria and performance characteristics: Re-
rosci. 27(48), 13261–13272. doi:27/48/13261 [pii] cessed cathode. J. Appl. Physiol. 45(1), 145–154
(2007). https://doi.org/10.1523/JNEUROSCI.3437- (1978)
07.2007 258. C.T. Dollery, C.J. Bulpitt, E.M. Kohner, Oxygen
246. D. Petrusca, M.I. Grivich, A. Sher, G.D. Field, J.L. supply to the retina from the retinal and choroidal
Gauthier, M. Greschner, J. Shlens, E.J. Chichilnisky, circulations at normal and increased arterial oxy-
A.M. Litke, Identification and characterization of gen tensions. Investig. Ophthalmol. 8(6), 588–594
a Y-like primate retinal ganglion cell type. J. Neu- (1969)
rosci. 27(41), 11019–11027. doi:27/41/11019 [pii] 259. V.A. Alder, S.J. Cringle, I.J. Constable, The retinal
(2007). https://doi.org/10.1523/JNEUROSCI.2836- oxygen profile in cats. Invest. Ophthalmol. Vis. Sci.
07.2007 24(1), 30–36 (1983)
247. G.M. Zeck, Q. Xiao, R.H. Masland, The spatial 260. W. Sickel, Retinal metabolism in dark and light,
filtering properties of local edge detectors and brisk- in Physiology of Photoreceptor Organs. Handbook
of Sensory Physiology, ed. by F. MGF, vol. VII/2, Molecular, Cellular and Medical Aspects, ed. by
(Springer, Berlin, 1972), pp. 227–727 G. J. Siegel, B. W. Agranoff, R. W. Albers, S. K.
261. R. Zuckerman, J.J. Weiter, Oxygen transport in Fisher, M. D. Uhler, 6th edn., (Lippincott-Raven,
the bullfrog retina. Exp. Eye Res. 30(2), 117–127 Philadelphia, 1999), pp. 637–669
(1980) 276. D.Y. Yu, S.J. Cringle, E.N. Su, Intraretinal oxygen
262. R.A. Linsenmeier, Effects of light and darkness distribution in the monkey retina and the response
on oxygen distribution and consumption in the cat to systemic hyperoxia. Invest. Ophthalmol. Vis. Sci.
retina. J. Gen. Physiol. 88(4), 521–542 (1986) 46(12), 4728–4733 (2005)
263. L.M. Haugh, R.A. Linsenmeier, T.K. Goldstick, 277. J.C. Lau, R.A. Linsenmeier, Oxygen consump-
Mathematical models of the spatial distribution of tion and distribution in the Long-Evans rat retina.
retinal oxygen tension and consumption, includ- Exp. Eye Res. 102, 50–58 (2012). https://doi.org/
ing changes upon illumination. Ann. Biomed. Eng. 10.1016/j.exer.2012.07.004
18(1), 19–36 (1990) 278. C.M. Yancey, R.A. Linsenmeier, Oxygen distribu-
264. R. Linsenmeier, C. Pournaras, Consommation et tion and consumption in the cat retina at increased
diffusion de l’oxygene retinien, in Pathologies Vas- intraocular pressure. Invest. Ophthalmol. Vis. Sci.
culaires Oculaires, ed. by C. Pournaras, (Masson, 30(4), 600–611 (1989)
Paris, 2008), pp. 99–107 279. S.J. Cringle, D.Y. Yu, A multi-layer model of retinal
265. R. Avtar, D. Tandon, Mathematical modelling of oxygen supply and consumption helps explain the
intraretinal oxygen partial pressure. Trop. J. Pharm. muted rise in inner retinal PO2 during systemic
Res. 7(4), 1107–1116 (2008) hyperoxia. Comp. Biochem. Physiol. 132(1), 61–66
266. R.D. Braun, R.A. Linsenmeier, T.K. Goldstick, Oxy- (2002)
gen consumption in the inner and outer retina of 280. S. Cringle, D.Y. Yu, V. Alder, E.N. Su, P. Yu, Oxygen
the cat. Invest. Ophthalmol. Vis. Sci. 36(3), 542–554 consumption in the avascular guinea pig retina. Am.
(1995) J. Phys. 271(3 Pt 2), H1162–H1165 (1996)
267. P. Causin, G. Guidoboni, F. Malgaroli, R. Sacco, A. 281. D.Y. Yu, S.J. Cringle, V.A. Alder, E.N. Su, P.K.
Harris, Blood flow mechanics and oxygen transport Yu, Intraretinal oxygen distribution and choroidal
and delivery in the retinal microcirculation: regulation in the avascular retina of guinea pigs. Am.
Multiscale mathematical modeling and numerical J. Phys. 270(3 Pt 2), H965–H973 (1996)
simulation. Biomech. Model. Mechanobiol. 15(3), 282. S.J. Cringle, D.Y. Yu, Intraretinal oxygenation and
525–542 (2016). https://doi.org/10.1007/s10237- oxygen consumption in the rabbit during systemic
015-0708-7 hyperoxia. Invest. Ophthalmol. Vis. Sci. 45(9),
268. S.J. Cringle, D.Y. Yu, P.K. Yu, E.N. Su, Intraretinal 3223–3228 (2004)
oxygen consumption in the rat in vivo. Invest. Oph- 283. L.J. Frishman, R.H. Steinberg, Light-evoked in-
thalmol. Vis. Sci. 43(6), 1922–1927 (2002) creases in [K+ ]o in proximal portion of the dark-
269. J.L. Olson, M. Asadi-Zeydabadi, R. Tagg, adapted cat retina. J. Neurophysiol. 61(6), 1233–
Theoretical estimation of retinal oxygenation 1243 (1989)
in chronic diabetic retinopathy. Comput. Biol. 284. L.J. Frishman, F. Yamamoto, J. Bogucka, R.H.
Med. 58, 154–162 (2015). https://doi.org/10.1016/ Steinberg, Light-evoked changes in [K+ ]o in prox-
j.compbiomed.2014.12.021 imal portion of light-adapted cat retina. J. Neuro-
270. P.A. Roberts, E.A. Gaffney, P.J. Luthert, A.J. Foss, physiol. 67(5), 1201–1212 (1992)
H.M. Byrne, Retinal oxygen distribution and the role 285. C.J. Karwoski, H.K. Lu, E.A. Newman, Spatial
of neuroglobin. J. Math. Biol. 73(1), 1–38 (2016a). buffering of light-evoked potassium increases by
https://doi.org/10.1007/s00285-015-0931-y retinal Muller (glial) cells. Science 244(4904), 578–
271. M.W. Roos, Theoretical estimation of retinal oxy- 580 (1989)
genation during retinal artery occlusion. Physiol. 286. C.J. Karwoski, L.M. Proenza, Relationship between
Meas. 25(6), 1523–1532 (2004) Muller cell responses, a local transretinal potential,
272. M.W. Roos, Theoretical estimation of retinal and potassium flux. J. Neurophysiol. 40(2), 244–259
oxygenation during retinal detachment. Comput. (1977)
Biol. Med. 37(6), 890–896 (2007). https://doi.org/ 287. E.A. Newman, D.A. Frambach, L.L. Odette, Con-
10.1016/j.compbiomed.2006.09.005 trol of extracellular potassium levels by retinal glial
273. D.Y. Yu, S.J. Cringle, Outer retinal anoxia during cell K+ siphoning. Science 225(4667), 1174–1175
dark adaptation is not a general property of mam- (1984)
malian retinas. Comp. Biochem. Physiol. 132(1), 288. B. Oakley 2nd, Potassium and the photoreceptor-
47–52 (2002) dependent pigment epithelial hyperpolarization. J.
274. Q.V. Hoang, R.A. Linsenmeier, C.K. Chung, C.A. Gen. Physiol. 70(4), 405–425 (1977)
Curcio, Photoreceptor inner segments in monkey 289. H. Shimazaki, B. Oakley 2nd, Reaccumulation of
and human retina: Mitochondrial density, optics, and [K+ ]o in the toad retina during maintained illumi-
regional variation. Vis. Neurosci. 19(4), 395–407 nation. J. Gen. Physiol. 84(3), 475–504 (1984)
(2002) 290. R.H. Steinberg, B. Oakley 2nd, G. Niemeyer, Light-
275. D.D. Clarke, L. Sokoloff, Circulation and energy evoked changes in [K+ ]o in retina of intact cat eye.
metabolism of the brain, in Basic Neurochemistry: J. Neurophysiol. 44(5), 897–921 (1980)
291. R. Wen, B. Oakley 2nd, K+ -evoked Muller cell de- 306. M.E. Rice, C. Nicholson, Diffusion characteristics
polarization generates b-wave of electroretinogram and extracellular volume fraction during normoxia
in toad retina. Proc. Natl. Acad. Sci. U. S. A. 87(6), and hypoxia in slices of rat neostriatum. J. Neuro-
2117–2121 (1990) physiol. 65(2), 264–272 (1991)
292. A.V. Dmitriev, V.I. Govardovskii, H.N. Schwahn, 307. L. Wang, P. Tornquist, A. Bill, Glucose metabolism
R.H. Steinberg, Light-induced changes of extra- in pig outer retina in light and darkness. Acta Phys-
cellular ions and volume in the isolated chick iol. Scand. 160(1), 75–81 (1997)
retina-pigment epithelium preparation. Vis. Neu- 308. N. Wangsa-Wirawan, L. Padnick-Silver, E. Budzyn-
rosci. 16(6), 1157–1167 (1999) ski, R. Linsenmeier, pH regulation in the intact cat
293. R.P. Gallemore, J.D. Li, V.I. Govardovskii, R.H. outer retina. ARVO abstract. Invest. Ophthalmol.
Steinberg, Calcium gradients and light-evoked cal- Vis. Sci. 42(4), S367 (2001)
cium changes outside rods in the intact cat retina. 309. T.J. Wolfensberger, A.V. Dmitriev, V.I. Govar-
Vis. Neurosci. 11(4), 753–761 (1994) dovskii, Inhibition of membrane-bound carbonic an-
294. G.H. Gold, J.I. Korenbrot, Light-induced calcium hydrase decreases subretinal pH and volume. Doc.
release by intact retinal rods. Proc. Natl. Acad. Sci. Ophthalmol. 97(3–4), 261–271 (1999)
U. S. A. 77(9), 5557–5561 (1980) 310. J.M. Ogilvie, K.K. Ohlemiller, G.N. Shah, B. Ul-
295. G. Birol, E. Budzynski, N.D. Wangsa-Wirawan, masov, T.A. Becker, A. Waheed, A.K. Hennig,
R.A. Linsenmeier, Retinal arterial occlusion leads to P.D. Lukasiewicz, W.S. Sly, Carbonic anhydrase
acidosis in the cat. Exp. Eye Res. 80(4), 527–533 XIV deficiency produces a functional defect in the
(2005) retinal light response. Proc. Natl. Acad. Sci. U.
296. A.V. Dmitriev, S.C. Mangel, Circadian clock regu- S. A. 104(20), 8514–8519 (2007). https://doi.org/
lation of pH in the rabbit retina. J. Neurosci. 21(8), 10.1073/pnas.0702899104
2897–2902 (2001). doi:21/8/2897 [pii] 311. E. Budzynski, N. Wangsa-Wirawan, L. Padnick-
297. A.V. Dmitriev, S.C. Mangel, Retinal pH reflects Silver, D. Hatchell, R. Linsenmeier, Intraretinal pH
retinal energy metabolism in the day and night. in diabetic cats. Curr. Eye Res. 30(3), 229–240
J. Neurophysiol. 91(6), 2404–2412 (2004). https:// (2005)
doi.org/10.1152/jn.00881.2003 312. A.V. Dmitriev, D. Henderson, R.A. Linsenmeier,
298. B. Oakley 2nd, R. Wen, Extracellular pH in the Development of diabetes-induced acidosis in the rat
isolated retina of the toad in darkness and during retina. Exp. Eye Res. 149, 16–25 (2016). https://
illumination. J. Physiol. 419, 353–378 (1989) doi.org/10.1016/j.exer.2016.05.028
299. L. Padnick-Silver, R.A. Linsenmeier, Quantification 313. I. Dietzel, U. Heinemann, G. Hofmeier, H.D. Lux,
of in vivo anaerobic metabolism in the normal cat Transient changes in the size of the extracellular
retina through intraretinal pH measurements. Vis. space in the sensorimotor cortex of cats in relation
Neurosci. 19(6), 793–806 (2002) to stimulus-induced changes in potassium concen-
300. L. Padnick-Silver, R.A. Linsenmeier, Effect of hy- tration. Exp. Brain Res. 40(4), 432–439 (1980)
poxemia and hyperglycemia on pH in the intact 314. J.D. Li, V.I. Govardovskii, R.H. Steinberg, Light-
cat retina. Arch. Ophthalmol. 123(12), 1684–1690 dependent hydration of the space surrounding pho-
(2005) toreceptors in the cat retina. Vis. Neurosci. 11(4),
301. F. Yamamoto, G.A. Borgula, R.H. Steinberg, Effects 743–752 (1994b)
of light and darkness on pH outside rod photorecep- 315. B. Huang, C.J. Karwoski, Light-evoked expansion
tors in the cat retina. Exp. Eye Res. 54(5), 685–697 of subretinal space volume in the retina of the frog.
(1992) J. Neurosci. 12(11), 4243–4252 (1992)
302. F. Yamamoto, R.H. Steinberg, Effects of systemic 316. V.I. Govardovskii, J.D. Li, A.V. Dmitriev, R.H.
hypoxia on pH outside rod photoreceptors in the cat Steinberg, Mathematical model of TMA+ diffusion
retina. Exp. Eye Res. 54(5), 699–709 (1992) and prediction of light-dependent subretinal hydra-
303. L.L. Odette, E.A. Newman, Model of potas- tion in chick retina. Invest. Ophthalmol. Vis. Sci.
sium dynamics in the central nervous system. 35(6), 2712–2724 (1994)
Glia 1(3), 198–210 (1988). https://doi.org/10.1002/ 317. J.D. Li, R.P. Gallemore, A. Dmitriev, R.H. Stein-
glia.440010305 berg, Light-dependent hydration of the space sur-
304. C. Nicholson, J.M. Phillips, Ion diffusion modified rounding photoreceptors in chick retina. Invest.
by tortuosity and volume fraction in the extracellular Ophthalmol. Vis. Sci. 35(6), 2700–2711 (1994a)
microenvironment of the rat cerebellum. J. Physiol. 318. W. Cao, V. Govardovskii, J.D. Li, R.H. Steinberg,
321, 225–257 (1981) Systemic hypoxia dehydrates the space surrounding
305. C. Nicholson, M.E. Rice, Diffusion of ions and photoreceptors in the cat retina. Invest. Ophthalmol.
transmitters in the brain cell microenvironment, in Vis. Sci. 37(4), 586–596 (1996)
Volume Transmission in the Brain: Novel Mecha- 319. H. Okawa, A.P. Sampath, S.B. Laughlin, G.L. Fain,
nisms for Neural Transmission, ed. by K. Fuxe, L. ATP consumption by mammalian rod photorecep-
F. Agnati, (Raven Press, New York, 1991), pp. 279– tors in darkness and in light. Curr. Biol. 18(24),
294 1917–1921 (2008)
Neural Tissue Engineering
22
Johnathan G. Lyon, Lohitash Karumbaiah,
and Ravi V. Bellamkonda
Abstract hydrogels, delivery of neurotrophic factors

and stem cells, genetic approaches to
Tissue engineering is the use of engineering
tissue engineering, immunomodulation, and
methods to replace, replicate, or improve
electrical stimulation.
biological tissues. Neural tissue engineering
involves the integrated use of biomaterials, Keywords
cellular engineering, and drug delivery
technologies with the purpose of protect- Tissue engineering · Biomaterials ·
ing, repairing, or regenerating cells and Traumatic brain injury · Nerve injury ·
tissues of the nervous system. Through the Regeneration · Central nervous system ·
introduction of biochemical, topographic, Peripheral nervous system ·
immunomodulatory, and other types of cues, Neurodegeneration · Spinal cord injury ·
tissues can be therapeutically controlled to Cellular engineering · Immunomodulation
direct growth and tissue function in order
to overcome biological constraints on tissue
repair and regeneration. These strategies can
be applied when injury or disease occurs in 22.1 Introduction
the brain, spinal cord, for damaged peripheral
nerves, or to improve chronic functionality of Tissue engineering is the use of engineering
implantable neural interfaces. In this chapter, methods to replace, replicate, or improve
we present an overview of neural tissue biological tissues. Tissue engineering evolved out
engineering using examples of therapeutic of the field of material science, as biologically
systems including nerve conduits, implantable mimicking or biologically supporting material
chemistries were discovered that could support
and sustain cellular growth at tissue scales.
of this chapter (https://doi.org/10.1007/978-3-030-43395- Broadly speaking, a suitable material substrate
6_22) contains supplementary material, which is available can provide a scaffolding to cells and tissues and
to authorized users. be tailored to provide a wide variety of effects
J. G. Lyon · R. V. Bellamkonda () from supporting specific multicellular structures,
Department of Biomedical Engineering, Pratt School of to coincident immune system modulation,
Engineering, Duke University, Durham, NC, USA to supporting sustained release of growth-
e-mail: [email protected]; [email protected]
supportive chemical agents.
L. Karumbaiah However, before we proceed with examples of
Regenerative Bioscience Center, University of Georgia,
tissue engineering approaches in nervous system,
Athens, GA, USA
e-mail: [email protected] it is important to first understand the underlying

640 J. G. Lyon et al.
tissue physiology of the brain, and what the par- Glial cells also come in many varieties and
ticular engineering challenges are that this unique are typically classified as astrocyte, microglia,
physiology elicits. oligodendrocyte, or ependymal cell. Astrocytes
are particularly important in neural tissue engi-
neering as they play roles in maintaining neural
22.1.1 Tissues of the Nervous System tissue structure and have profound responses to
traumatic injury or inflammation. Astrocytes play
The physiology of the nervous system can be a critical role in maintaining the unique vascu-
divided into three distinct superstructures—the lature of the brain, known as the blood–brain
brain, the spinal cord, and the peripheral nervous barrier (BBB) [3], which is a selective barrier that
system (PNS)—each with its own distinctive tis- protects the brain and prevents direct contact with
sue makeup. Together, the brain and spinal cord blood and its constituents. In the brain, instead
are also known as the central nervous system of blood, the cerebrospinal fluid (CSF) is the
(CNS). The PNS can be further subdivided into extracellular fluid substrate, and under homeosta-
the somatic and autonomic nervous systems. The sis is connected outside the brain via regulated
somatic nervous system includes sensory neurons transport in the BBB, the CSF-producing choroid
such as the dorsal root and the cranial ganglia plexus cells in the ventricles, and the local lym-
which provide sensory information to the CNS, phatic and glymphatic systems recently discov-
while the autonomic nervous system, which is ered in the meninges [4–6]. The understood role
further divided into the sympathetic, parasym- of these systems is to simply allow neurons to
pathetic, and enteric nervous systems, controls function undisturbed, and breaches of these dur-
stress response, maintains homeostasis, and reg- ing injury or pathology have considerable effects
ulates gut function. on normal nervous system tissue and function.
After an injury to the CNS, astrocytes be-
22.1.1.1 Cells and Tissues of the Brain come reactive in response to the inflammation
At a cellular level, the nervous system is and trauma to the CNS leading to a situation
especially complex. In the brain, the tissue known as “reactive astrogliosis” [7]. Reactive
is mainly comprised of neurons and glia, astrocytes, which are identified histologically by
although recent efforts focused on unraveling the relative upregulation of glial fibrillary acidic
transcriptomic and functional maps have revealed protein (GFAP) and other intermediate filament
a staggering diversity and complexity of these proteins such as vimentin, produce a host of bene-
cells [1, 2]. Broadly, neurons can be classified ficial molecules such as proteases and protease in-
based on anatomic morphology, electrophys- hibitors, neurotrophic factors, and cytokines that
iological features, interconnectivity, locality, facilitate remodeling of the lesion site and pre-
transcriptomic phenotype, or neurotransmitter vent neuronal dysfunction [8]. In addition to the
production profile (e.g., cells that arise in the production of beneficial molecules, reactive as-
cerebellum, are GABAergic/inhibitory, have a trocytes are responsible for the formation of “glial
large number of dendritic spines, and express scar” consisting of chondroitin sulfate proteo-
Purkinje cell protein 4 are considered Purkinje glycans (CSPGs) [9], which are components of
neurons). The cellular diversity can also be the neural extracellular matrix (ECM) that have
region-dependent, as the various substructures of long been believed to inhibit neural regeneration.
the brain (e.g., cerebellum, thalamus, substantia A more detailed analysis of CSPG linked gly-
nigra, etc.) all have specific functional roles cosaminoglycans (GAGs) has revealed the pres-
that require a particular cellular makeup and ence of neurite growth-promoting and inhibitory
connective architecture. For tissue engineering CS-GAGs, which might influence the growth per-
purposes, it is important to understand this missiveness of glial scar [10–12].
uniqueness and how it can impact tissue Glial cells also play an important role in regu-
functionality. lating the CNS immune system. Due to the highly
22 Neural Tissue Engineering 641
regulated nature of the brain’s blood supply, the ing mostly of regularly structured descending/as-
CNS is considered an “immune-privileged” zone, cending neurons, spinal interneurons, and spinal
and thus, it incurs an altered profile of resident nerves.
and infiltrating immune cells relative to the sys- The spinal cord, like the brain, possesses
temic immune system. In the brain, a population meningeal layers and is saturated with CSF. Much
of resting microglial cells that are spread through- like the brain, the spinal cord is rendered immune
out the brain serve instead as the primary immune privileged via the blood–spinal cord barrier,
sentinel and are responsible for monitoring injuri- which is similar to the BBB, yet is considered
ous events [13]. Infection or injury to the CNS re- morphologically and functionally distinct [21].
sults in the activation of resting microglia. These
“reactive microglia” are known to be involved in 22.1.1.3 Tissue and Cells of the
a wide range of processes that include regulation Peripheral Nervous System
of astrocytic differentiation, antigen presentation, The cellular makeup of the PNS is radically dis-
and immune response by local secretion of pro- tinct from the CNS. Peripheral nerves are a net-
inflammatory cytokines [14–16]. work of nervous tissue bundles that connect the
The glial cells known as oligodendrocytes are CNS to the body’s nonneural tissues, organs,
important in CNS regeneration in cases where and extremities, and provide the underlying sen-
loss or recovery of myelination is a factor, as sory and control relay system of the body. These
these cells are responsible for maintaining an in- nerves are encased in layers of connective tissue
sulating layer of myelin around CNS axons [17]. that form the inner endoneurium, perineurium,
These cells can be found migrating throughout and the outer epineurium.
the brain, though they originate in specific regions Another major difference from the CNS is
the spinal cord, ganglionic eminence, and ventral that the PNS has only one class of glial cell,
forebrain [18, 19]. Injury to the CNS typically re- Schwann cells, which are responsible for a variety
sults in the simultaneous destruction of oligoden- of functions. Myelinating Schwann cells, akin to
drocytes and myelin, leaving behind a sharply de- oligodendrocytes of the CNS, provide axons with
marcated focal demyelinated lesion. This leads to an insulating layer of myelin [22]. This process
a subsequent remyelination of axons, facilitated is brought about by binding of Schwann cells to
by the triggering of oligodendrocyte progenitor NRG1, a Schwann cell growth, maturation, motil-
cells to a regenerative phenotype by factors se- ity, and myelin thickness regulating factor, which
creted by reactive astrocytes and microglia [20]. is presented to them by NRG1-expressing axons
[23]. In instances of injury to the PNS, Schwann
22.1.1.2 Tissue and Cells of the Spinal cells facilitate regeneration of the transected pe-
Cord ripheral nerve by forming the necessary cellular
The spinal cord is a structured tube of nervous substrate [24, 25], secreting trophic factors [26],
tissue that extends further into the body from and forming the basal lamina [27].
the brainstem. The spinal column is made of 31 Importantly, unlike the CNS, the PNS is in
segments of vertebrae divided into the cervical, direct contact with the systemic blood supply
thoracic, lumbar, and sacral regions. In each seg- and immune system, and is not considered to be
ment, particular sensory and motor nerve roots “immune privileged.”
emerge bilaterally from the column and serve
as the connection between the PNS, extending
further into the body. The PNS connects to the 22.1.2 Targets of Tissue Engineering
CNS via spinal roots: dorsal roots (afferent) or Approaches in the Nervous
ventral roots (efferent). Though distinct from the System
brain, the spinal cord mainly shares the same
cellular milieu; however, it contains significantly Neural tissue engineering may be able to ad-
less regional variety than in the brain, consist- dress challenges for which there may be no other
medical solution. These challenges fall into three occupied by cellular and myelin debris. These
categories: injury, disease, and implant integra- lesion sites further support degradation of injured
tion. In tissue engineering, we often look at the nerves as they become local sources of myelin
resultant pathophysiology from these challenges inhibitors such as myelin-associated glycopro-
as a way to identify specific biological responses tein, oligodendrocyte myelin glycoprotein, and
that can serve as therapeutic targets that may be an oligodendrocyte-associated neurite growth
corrected, manipulated, or replaced, with the goal inhibitor, Nogo-A [32, 33]. Secondary events in
of reducing or repairing the consequent tissue SCI involve invasion of the lesion site by reactive
damage. astrocytes and other immune cells that begin
forming an “astroglial” scar that is also associated
22.1.2.1 Nervous System Injuries with increased secretion of growth-promoting
Traumatic brain injuries (TBIs) result from and inhibitory molecules called CSPGs. Together,
contact injuries to the head leading to contusion, these molecules can form a significant physical
intracranial hemorrhage, and axonal damage [28, and molecular barrier to nerve regeneration post-
29]. Survivors of TBI experience several mild CNS injury. Recent reports demonstrating robust
to severe disabilities ranging from loss of motor axonal growth post-spinal cord injury (SCI),
function, cognitive disorders, and depression. and despite the presence of glial scar containing
TBI is also known to accelerate the onset of growth permissive and inhibitory CSPGs [34],
several neurodegenerative disorders such as confirm these previous observations and question
Alzheimer’s and Parkinson’s diseases. the rather oversimplified characterization of glial
The pathophysiology of TBI is characterized scar-associated CSPGs as inhibitors of nerve
by a sequence of primary and secondary events. regeneration.
Primary events are triggered by acute excitotoxic- Injury to the CNS results in the failure of
ity and ischemia resulting from loss of blood flow. transected axons regenerating past the lesion site.
Coincident lactic acid accumulation results in an This phenomenon was illustrated by Ramón y Ca-
anaerobic environment and leads to membrane jal [35], where the ends of transected axons were
permeation, depletion of ATP, failure of ion chan- described as forming “dystrophic” end-bulbs that
nels, and the onset of edema [30]. The secondary were thought to be incapable of regenerating.
stage of TBI is characterized by continued nerve However, subsequent studies have shown that
damage, cellular degradation, and excitotoxicity. neurons explanted from the injured CNS had the
Eventually, the structural degradation of nucleo- ability to grow into peripheral nerve bridges, in-
somal DNA by caspases and other endonucleases dicating that transected CNS axons possibly do
leads to the triggering of apoptosis that culmi- not readily regenerate within the lesioned CNS
nates in long-term cellular dysfunction and tissue due to the prevalence of a nonpermissive native
loss depending on TBI severity. environment [36]. Further studies have led to the
Spinal cord injuries (SCIs) result from trau- understanding that these “end-bulbs,” although
matic blows or penetrating wounds to the spinal dystrophic, are nevertheless highly active struc-
cord, most often resulting in a crush or contusion tures that are trapped in a nonpermissive environ-
of the cord. A majority of patients that survive ment [37] that have the capacity to be coaxed into
SCI suffer paralysis and lifelong disability rang- a regenerative state.
ing from bladder and bowel dysfunction to respi- Peripheral nerve injuries (PNIs) account for
ratory and heart complications, depending upon nearly 2.8% of all hospital trauma cases [38]
the severity of the injury. Although the patho- and mainly result from injuries such as vehicle
physiology of SCI has been well understood, to collisions, gunshot wounds, and other penetrating
date there is no effective treatment for the condi- trauma that induce mechanical stretch, laceration,
tion [31]. compression, and deformation of nerves in the
In primary stages of SCI, the environment periphery [39] or the stretching or crushing of
surrounding the injury forms a lesion site that is nerves by fractured bones resulting from falls
[40]. Besides accident-related trauma, PNIs nerve regeneration [45, 46]. In crush injuries
can arise from surgical injuries inflicted as a where the endoneurium is not completely
consequence of surgical tumor removal, or due destroyed, sprouting neurofibrils from the
to improper care during the perioperative period, proximal end form an axon that is guided by
or can arise as a consequence of the adverse the bands of Büngner along the endoneurial
neurotoxic effects of chemotherapeutic drugs tube to the target organ. This process is severely
[41]. Diabetic neuropathies are also increasingly impaired when a nerve is sectioned, due to the
common, nontraumatic, peripheral neuropathies presence of gap-containing fibroblasts and scar-
that occur in 5% of patients suffering from associated CSPGs that negatively affect nerve
diabetes mellitus type-1 and in 20% suffering regeneration.
from type-2 [42]. These types of neuropathies Injuries to peripheral nerve are considered eas-
also occur in patients suffering from other forms ier to treat than CNS injuries due to the secre-
of acquired diabetes, and when left untreated, tion of aligned ECM and neurotrophic factors
result in loss of sensation, impair muscle function, by Schwann cells and macrophages that enable
and lead to a tangled mass of nerves that cause axonal elongation, eventually restoring neuronal
severe pain, thereby accounting for a number of function. Spontaneous regeneration of PNS in-
amputations. juries takes place when nerve gaps are <10 mm. In
Injuries to the peripheral nerve are typically these cases, end-to-end and end-to-side suturing
classified according to severity as either neu- can be done to facilitate repair. In nerve gaps
rapraxia, axonotmesis, or neurotmesis [43]. Neu- >10 mm, however, nerve ends cannot be sutured
rapraxia is a mild block without any loss of nerve due to increased tension on the nerve. In such
continuity that leads to only transient loss of nerve cases, nerve gaps can be bridged by the use of
function. Axonotmesis results from axonal tran- autologous nerve grafts and nerve conduits made
section and disruption of the surrounding myelin, from biological or synthetic materials. Autolo-
with little or no disruption of the epineurium gous nerve grafting is a procedure that involves
and the perineurium. Injuries of this nature have explantation of a nerve segment from the patient’s
an excellent prognosis of complete recovery due own body (e.g., from the sural nerve) and im-
to the remaining epineurial and perineurial net- planted into the nerve gap. Although autologous
works. Neurotmesis involves complete transec- nerve grafts are the current “gold standard” in
tion of the nerve without any preservation of treating peripheral nerve injuries, inherent disad-
the mesenchymal network. Injuries of this nature vantages such as donor site morbidity, presence
have a poor prognosis and can only be treated via of painful neuromas, and the need for secondary
surgical intervention and proper management of surgery make the use of nerve conduits fabricated
nerve inhibitory molecules. from natural and synthetic materials an attractive
When peripheral nerves are subjected to treatment alternative.
chronic compression or transection, Schwann
cells are damaged, leading to demyelination of 22.1.2.2 Degenerative Disease
the nerve segment. Depending on the severity of the Nervous System
of the injury, degeneration of the distal end Diseases of the nervous system where rampant
begins, leading to the onset of a characteristic neurodegeneration occurs may be addressed
active degenerative process (known as Wallerian through the development of tissue engineering
degeneration) within the first 24 h after injury strategies that protect or regenerate nervous
(Fig. 22.1) [44]. Myelin debris at the lesion tissues. There exist numerous human pathologies
site induces the accumulation of macrophages where neurodegeneration is rampant, either
and Schwann cells that produce, along with as an indirect effect of injurious breach of
other growth factors, a basal lamina consisting the BBB or through some unique etiology.
of aligned matrix proteins called the bands The most common of these disorders include
of Büngner, which forms the foundation for Alzheimer’s disease, Parkinson’s disease, prion
Fig. 22.1 Schematic

diagram of the regenerative
sequence of injured
peripheral nerve. Initially,
plasma collects and
develops into a fibrin clot.
This fibrin matrix provides
a platform upon which
Schwann cells and
fibroblasts are able to
migrate out of either nerve
stump. These cells are
responsible for the
formation of the
extracellular matrix that
directs neurite extension.
(Reprinted from
Mukhatyar et al. [44].
Copyright (2009) with
permission from
Wiley-VCH Verlag GmbH
& Co. KGaA)
disease, Huntington’s disease, and multiple is well understood, and even so, many of these
sclerosis. The exact progression of many of diseases have complicated etiologies that further
these diseases is active area of research, and produce challenges to engineering therapeutic
the details are beyond the scope of this chapter. solutions.
However, what is important to note for tissue
engineering consideration is that each of these 22.1.2.3 Neural Device Integration
diseases has a specific way in which the nervous With the emergence of neural interfaces and
system cellular milieu is affected, and therein neural implants such as deep brain stimula-
lies a specific set of tissue engineering targets, tors, vagal nerve stimulators, or implantable
whether those targets are simply to ameliorate a brain–computer interfaces, tissue engineering
particular symptom or to reengineer the nervous approaches can help to reduce the consequent
tissue in order to protect from or correct the foreign body inflammatory response, improve
underlying disease. For instance, in Parkinson’s integration, or even be used to support innervation
disease, tremors or bradykinesia occurs due to and afferent/efferent nervous connectivity of
the death of a particular subset of dopaminergic implantable organs or nonnervous tissues. Tissue
cells, and these symptoms in some cases are engineering the integration of devices with
diminished by the use of pharmaceuticals or the nervous system has the parallel goals of
deep brain stimulation in the thalamus [47]. A minimized tissue trauma, maximized efficiency
tissue engineering approach could potentially of transmission/signal propagation, and access
replace function by implantation of Parkinson’s to a more complex and precise density of sense
resistant dopaminergic cells, or through gene and control. Many implantable systems can be
therapy that blocks the pathway by which the cells considered a form of local, chronic injury (e.g.,
undergo apoptosis prior to the development of small stab wounds for penetrating electrodes),
symptoms. Again, this only works if the etiology and can incur mechanical disruption of organized
Fig. 22.2 Schematic illustration and the working model of chronic electrode failure. (Reprinted from Saxena et al. [49]
Copyright (2013) with permission from Elsevier)
nervous tissues, breaches of the BBB, and an ulation to decrease the overall inflammatory re-
undesired influx of inflammatory or immune sponse, or by the design of tissue engineered elec-
cells [48, 49]. trode coatings that facilitate improved tissue inte-
In the case of insertable electrodes, the work- gration and prolonged electrode function [50].
ing model is that penetrating electrodes incur a
chronic BBB breach (Fig. 22.2). This in turn leads
to a feedback loop wherein infiltrating myeloid
cells produce neurotoxic and pro-inflammatory 22.2 Tissue Engineering
cytokines that induce reactive gliosis, leading to Technologies
further production of pro-inflammatory factors
and increased BBB permeability. The accumula- 22.2.1 Material Infrastructure for
tion of these neurotoxic factors eventually leads Regeneration
to a chronic inflammatory state that induces neu-
rodegeneration and loss of electrode functional- The primary approach to neural tissue engineer-
ity. Tissue engineering strategies have the poten- ing is to develop a post-injury environment that
tial to improve these situations, either via en- can facilitate structural regeneration, as well as
hancing acute or chronic tissue tolerance to an to provide the necessary signaling cues to pro-
implanted foreign body, directing immunomod- mote cellular tropism and growth, and eventual
regeneration of cells and tissues. Below, we begin exploited to enhance nerve regeneration. The
our discussion of tissue engineering technologies design criteria for these conduits involve size
with the technologies that provide an infrastruc- requirements such as length and diameter,
tural basis for repair strategies. permeability, topography, swelling ratio,
degradation rate, size of degradation products,
22.2.1.1 Nerve Conduits for PNS Repair and their clearance [71, 72]. Conduits made from
Nerve conduits are typically tubular in design to biodegradable polymers can be designed with
enable bridging of a nerve gap and to facilitate internal scaffolding to enhance the endogenous
a closed environment for the proliferation of cells regenerative potential by bolstering fibrin cable
and accumulation of growth factors. Conduits can formation, and in the case of longer nerve
be designed to present aligned topographies and gaps, to substitute fibrin cable activity by
trophic factors released by nano−/microparticles providing internally aligned topographical cues
embedded in 3D gels made from natural and and micropatterned substrates to encourage
synthetic polymers. Silicone conduits were the Schwann cell and axonal migration (Fig. 22.3)
first generation of nerve conduits used to bridge [73–75]. Additionally, biodegradable polymer
nerve gaps [51]. However, their poor porosity, conduits can be used as reservoirs of growth
rigidity, and inability to allow for swelling led to factors which can be released in a sustained
the exertion of compressive forces and scarring of manner along with the slow degradation of the
the regenerating nerve ultimately leading to their polymer.
failure [52]. Nerve conduits made from biodegradable
Next-generation nerve conduits made use of polymers such as poly(L-lactic acid) (PLLA),
biologically derived materials (e.g., collagen) polyglycolic acid (PGA), and poly(lactic-
as conduits and hydrogel substrates [53, 54]. co-glycolic) acid (PLGA) have been used
Conduits containing collagen hydrogels were to successfully bridge 8–10 mm nerve gaps
found to perform better than hollow or saline- [76–80]. Porous PLLA conduits were also
filled conduits, and performed the same as reported to match autograft performance in
autografts when used to bridge short nerve a 10 mm sciatic nerve defect [78]. Although
gaps [55]. More recently, Bozkurt et al. [56] biodegradable polymer conduits provide control
showed that microstructured collagen conduits over design, inherent disadvantages such as
seeded with Schwann cells that possessed aligned cellular incompatibility, mismatch with neuronal
topography resembling the endoneurial tubes tissue, toxic by-products of degradation, and
induced significant Schwann cell migration, rapid degradation of constructs [72, 81] led to the
formation of bands of Büngner, and axonal use of biocompatible polymers such as poly(3-
regeneration. Collagen, however, has the hydroxybutyrate) (PHB) and poly(L-lactide-co-
potential of inducing an immune response m-caprolactone) (PCL). These materials can also
and, therefore, gelatin, a hydrolyzed version be blended with other natural biopolymers such
of collagen, has found application in nerve as collagen, gelatin, and chitosan to enhance cell
conduit design either in its native form or in attachment and migration [82–85]. In spite of
combination with other polymeric biomaterials the various benefits of using biopolymer-based
such as polycaprolactone (PCL) [57]. In addition nerve conduits, they are yet unable to match the
to collagen hydrogel fillers, other biologically performance of autologous nerve grafts in longer
derived materials such as fibrin [58–60], agarose nerve gaps (>10 mm) and possibly need to be
[61–64], chitosan [65–67], alginate [68], and supplemented with trophic agents and growth
hyaluronic acid [69, 70] have also been used as factors to achieve this outcome.
structural substrates in PNS regeneration. Although nerve conduits made from biodegrad-
Synthetic biodegradable polymers can also able polymers do avoid problems of donor site
be used to fabricate conduits with several nano- morbidity and immune rejection, they still do
and microscale structural features that can be not present regenerating nerves with the native
Fig. 22.3 Nerve regeneration through aligned construct Scale bar: 200 μm. (f) Representative NF160 (a marker
and presence of both migrated Schwann cells and en- for axons) immunostained aligned construct (transverse
dogenously deposited laminin protein. Representative im- cross section, 5 mm from proximal nerve stump). Scale
munostained images of (a) nerve regeneration and (b) bar: 500 μm. (g) Regenerated axons (red) encircled by
migrated Schwann cells from both proximal and distal laminin+ pocket structures (green) from box in (f). Ar-
nerve stump. Polysulfone nerve conduit. Magnified and rowheads indicate blood vessels, and inset shows mag-
immunostained images of (c) endogenously deposited nified axons and laminin+ pocket structures. Arrows in-
laminin and (d) regenerating axons, from box in (a). dicate laminin+ pocket structures without regrown axon.
(e) Combined image of laminin (green), axon (red), and (Reprinted from Kim et al. [73] Copyright (2008) with
aligned fiber films (blue). Arrows indicate fiber films. permission from Elsevier)
topography and cellular adhesion sites presented nerve graft (AxoGen, Inc., Alachua, Florida). In
by native ECM. Acellular nerves are typically vivo studies comparing autografts to Avance®
cadaver or donor nerve tissue that is extensively acellular nerves and Integra NeuraGen® type-
decellularized by a process that involves 1 collagen conduits in 14 mm and 28 mm
detergent washing, enzymatic degradation of rat sciatic nerve defects revealed that nerve
CSPGs, and gamma sterilization. The resulting regeneration in autografts was better than both
decellularized product retains the structural, Avance® and Integra NeuraGen® allografts. The
topographical, and biochemical features of Avance® allograft, however, showed better nerve
native nerve tissue while avoiding problems of regeneration and innervation of distal ends than
immune rejection. The most commonly used Integra NeuraGen® collagen conduits [86]. In the
acellular nerve product being used clinically for long term, better decellularization techniques that
peripheral nerve repair is Avance® processed preserve the basal lamina, along with delivery of
cultured Schwann cells and nerve growth factors, improved functional recovery [95]. Hyaluronic
can lead to much improved acellular nerve acid when combined with poly-D-Lysine and
grafts that could possibly challenge autograft implanted as a solid sponge was found to
performance. encourage migration of neurons and astrocytes in
a model for TBI [96]. Agarose, a polysaccharide
derived from red algae, is known for its
22.2.1.2 Implantable Hydrogels for biocompatibility and has been used with great
CNS Regeneration success for neurite regeneration [87, 97–99].
In CNS injuries such as TBI and SCI, secondary Thermo-reversible agarose hydrogels containing
injury contributes to progressive necrosis and lipid microtubes allowed the sustained release
cavitation of the lesion site leading to lack of of a growth factor and facilitated 3D neurite
parenchymal continuity and hence the absence extension in vivo while inducing a minimum
of any significant nerve regeneration. While inflammatory response (Fig. 22.4) [100]. This
the role of excitotoxic necrosis has been well effect was further potentiated when BDNF was
established in the past, the triggering of apoptotic released in combination with constitutively active
pathways post-CNS injury is also implicated as forms of the Rho GTPases Cdc42 and Rac1 (Fig.
an important contributor to traumatic ischemic 22.5) [101].
damage of the CNS [40]. In these instances, the Self-assembling nanofiber peptide hydrogel
use of hydrogel-based scaffolds to provide nerve scaffolds are one of the more promising delivery
growth-enhancing substrates while providing systems being employed for CNS repair. “Self-
structural stability is an attractive means of assembly” of these scaffolds is mediated by
promoting endogenous nerve and tissue repair. non-covalent bonds between hydrophilic and
Hydrogels are water-insoluble polymers that hydrophobic peptides to form nanofibers that are
swell in water and exhibit tissue-like properties. on the order of 10–20 nm. These scaffolds further
Their macroporous properties enable attachment organize to form hydrogels that can be used to
of cells and promote ion and nutrient exchange, deliver a variety of proteins and small molecules
eventually forming the path to tissue regrowth. safely and efficaciously [102, 103]. Since these
From a tissue engineering standpoint, hydrogels scaffolds are devoid of chemical crosslinkers
need to be biocompatible and non/minimally and are wholly composed of peptides, their
immunogenic, biodegradable or bioresorbable degradation products are nontoxic and can be
to circumvent the need for later surgical used by cells for growth and repair. Second, their
removal, compatible for in situ gelling to enable nanostructure is closer to that of native ECM
conformation to defect, should enable robust and can therefore provide cells with their natural
ECM and cell attachment, and promote outgrowth microenvironment [104, 105]. In vitro studies
of axonal processes [87, 88]. using self-assembled, laminin-derived isoleucin-
A variety of naturally occurring polymers lysine-valine-alanine-valine (IKVAV) hydrogels
have been used in hydrogels for CNS tissue to encapsulate neural progenitor cells were found
engineering. Collagen, when used in vitro, to selectively enhance growth and differentiation
has shown cell attachment and neuritogenesis of neurons over the length of the study (22 days)
[89, 90]; however, the same was not observed [106]. Previously, an in vivo study using hamster
when collagen gels were implanted in a spinal optic tract transection followed by transplantation
cord defect [91]. In comparison, collagen gels of a self-assembling nanofiber scaffold was found
supplemented with growth factors induced to promote significant repair and regeneration as
neuritogenesis and reduced cavitation of the CNS evidenced by histology and functional recovery
defect [92–94]. Methylcellulose and hyaluronic [107]. Self-assembling nanofiber scaffolds that
acid copolymer have been used in spinal cord were pre-cultured with neural progenitor cells and
repair to create a fast gelling hydrogel that Schwann cells exhibited good tissue integration
displays reduced inflammatory response and and robust cell migration and blood vessel
Fig. 22.4 Schematic of spinal cord hemisection injury top of the lesion and covered with stiffer 0.7% SeaKem
model and delivery of enzyme to the lesion site. The 1% agarose gel to keep the scaffold in place. (Reprinted from
SeaPrep agarose gel-microtube scaffold is implanted on Lee et al. [100])
formation when transplanted into a spinal cord low viscosity polymer at <20 ◦ C and rapid gelling
dorsal column transection [108]. IKVAV self- at physiological temperatures. Additionally,
assembling nanofibers have also been shown to copolymerization with other naturally occurring
reduce astrogliosis and cell death, and promote and synthetic polymers can yield a copolymer
regeneration of sensory and motor fibers in a with a variety of desirable characteristics [115].
mouse model of spinal cord compression injury PNIPAAm–polyethylene glycol copolymers
[109]. were found to match the compressive modulus
In addition to hydrogels made from naturally of native neuronal tissue and also facilitated the
occurring polymers, synthetic polymers have also sustained release of neurotrophic factors over a
found extensive application in CNS repair as they period of 4 weeks [116].
can be mechanically tailored to suit neural tissue
and provide long-term structural support when
needed. Poly(2-hydroxyethyl methacrylate) 22.2.2 Neurotrophic Factors
(pHEMA) sponge-laden guidance channels
showed axonal elongation when implanted in Neurotrophic growth factors play important
an SCI [110]; however, these channels were also roles in regulating axonal growth, survival,
found to collapse after implantation and needed neurotransmission, and plasticity, and positively
to be reinforced to be functional [111]. When influence nerve regeneration after injury to the
pHEMA and a similar but more biocompatible nervous system [117]. An important family of
polymer, poly(hydroxypropyl methacrylate) neurotrophic factors are neurotrophins, a class
(pHPMA), were implanted unmodified into a of neuron-supporting molecules including nerve
cortical lesion, migration of axons and astrocytes growth factor (NGF), brain-derived neurotrophic
was observed in both cases although better factor (BDNF), neurotrophin-3 (NT-3), and
connective tissue formation was observed in neurotrophin 4/5 (NT-4/5); ciliary neurotrophic
the case of pHPMA [112]. Further modification factor (CNTF); and glial cell line-derived
of pHPMA with cell-adhesive peptides such growth factor (GDNF) [117, 118]. While a
as the fibronectin-derived arginylglycylaspartic majority of these neurotrophic factors elicit
acid (RGD) or IKVAV peptides also resulted in favorable response on axonal growth, the effect
better axonal outgrowth and favorable wound of individual growth factors on axonal elongation
repair [113, 114]. Poly(N-isopropyl acrylamide) after injury depends heavily on the severity of
(PNIPAAm) is another nonbiodegradable the injury, neuronal cell type, and population of
polymer that is being extensively applied for growth factor receptors. Neurotrophic factors
biomedical purposes due to its thermo-reversible induce axonal elongation in a chemotactic
characteristics that enable phase transition to a manner and hence need to be present in high
Fig. 22.5 Characterization of CST+ axons extending distal interface of the inhibitory region shown with a black
through CS-56+ regions proximal to lesion site. (a) dotted line as indicated with a white arrow. (f) Percentage
Schematic of the inhibitory regions quantified is depicted of axons in CS-56+ inhibitory regions. A significantly
with the inhibitory region magnified in a schematic to higher percentage of axons crossed the distal interface
the right. For axon quantification in CS-56+ inhibitory of the inhibitory region in the BDNF, CA-Cdc42, and
regions, axons were placed into three categories: axons CA-Rac1 compared to the untreated and agarose controls.
that stopped before the proximal interface of the inhibitory Also, the axons in the controls stopped at the proximal
region, axons that stopped within the inhibitory region, interface of the inhibitory region at a significantly higher
and axons that passed the distal interface of the inhibitory percentage than in the spinal cords treated with CA-Rac1.
region. (b) Image of BDA (green) and CS-56 (red) at the (g) In the spinal cords treated with CA-Rac1, the axons ex-
proximal side of the lesion site. The white box is magnified tended a significantly further distance within the inhibitory
at the bottom right corner demonstrating how the BDA+ region than the untreated and agarose controls. One-way
axons stall at the CS-56+ regions. (c–e) Images of BDA+ ANOVA and Tukey’s test were used to statistically analyze
axons and CS-56+ inhibitory regions proximal to the the data. [(a) and (c) p < 0.05 compared to untreated
lesion site. The white box represents the image overlaid control, and (b) and (d) p < 0.05 compared to agarose
with BDA. (c) and (d) The axons stopped in the middle of control]. The data represent mean ± SEM. (Reprinted
the CS-56+ regions for the controls, whereas in (e), it can from Jain et al. [101])
be seen that with CA-Rac1 treatment, the axons cross the
enough concentrations at the lesion site in order NGF is a potent initiator of sensory axon
to facilitate nerve regeneration. growth and is upregulated at the lesion site after
both CNS and PNS injuries [119, 120]. NGF, of sensory and autonomous neurons [141], and
when continuously infused over a period of induce survival and outgrowth of motor neurons
14 days into the dorsal root entry zone of a spinal after injuries to the CNS and PNS [142].
cord inflicted with a peripheral dorsal root crush, While it is evident that neurotrophic factors
showed a 37% increase in sensory axons entering are critical for the survival and regeneration of
the dorsal root entry zone when compared to neurons after injury to the CNS and PNS, their
before infusion [121]. The continuous delivery of efficacy can be improved delivering locally, and
NGF using polymer nerve conduits and PLG at precise concentrations that are beneficial for re-
microspheres was shown to increase nerve generating neurons. This can be achieved through
regeneration in the CNS [121–123] and in the the tailoring of the delivery substrate materials
PNS when compared to nerve conduits filled to achieve temporal and spatial control, or, as
with NGF [122, 124, 125]. Excessive dosing of discussed later in this chapter, via gene therapy
NGF, however, has its problems and most often approaches.
leads to unwanted plasticity and sprouting of
uninjured sensory nerves [126, 127].
BDNF, NT-3, and NT-4/5 are also known to 22.2.3 Cellular Engineering
promote motor and sensory neuron outgrowth Approaches
and survival [128, 129]. NT-3-secreting nerve
conduits implanted in a mid-thoracic spinal cord Cells are an excellent delivery vehicle that can po-
lesion showed an increase in axonal length of tentially be used in combination with biomaterial-
corticospinal axons distal to the lesion [130], based repair strategies to deliver extracellular ma-
whereas BDNF and NT-3 infusions have been trix (ECM), trophic factors, and cell adhesion
shown to prevent death of corticospinal neurons molecules. Glial cells such as Schwann cells,
after axotomy [131], indicating possibly that NT- astrocytes, and oligodendrocytes as well as neural
3 when administered alone promotes regenera- precursor cells and other stem cell types are being
tion while NT-3 and BDNF together prevent cell investigated for their transplantation potential to
death. NT-4/5 has been reported to promote nerve support nerve repair.
regeneration in both the CNS and PNS [128, 132].
CNTF is found in large amounts in adult ro- 22.2.3.1 Stem Cells
dent sciatic nerves and within the cytoplasm of The potential for nerve regeneration after injury
myelinating Schwann cells and astrocytes [133], to the nervous system was traditionally thought
and is also found to be upregulated after injury to to be limited due to the absence of significant
the CNS [134]. Studies conducting CNTF infu- neurogenesis. Using thymidine as a label for mi-
sions reportedly promote axon regeneration after tosis, Joseph Altman demonstrated that cell di-
injury to the spinal cord [135] and the PNS [136]. vision does occur in the hippocampus [143] and
In PNS injuries, the occurrence of cell death post- olfactory bulb [144], and that the potential for
injury is found to coincide with the low expres- neurogenesis does indeed exist. However, it was
sion of levels of CNTF, and wherein local admin- only when neuronal precursor cells isolated from
istration of CNTF was found to prevent cell death the forebrain were shown to differentiate into
and degeneration of neuronal cell bodies post- neurons [145, 146] that the potential of stem cell
injury [137]. Overexposure to CNTF, however, is therapy for nerve regeneration was fully realized
known to induce reactive gliosis, a phenomenon [147].
that induces secretion of nerve inhibitory CSPGs, A number of stem cell types are currently
thereby creating an inhibitory environment for being investigated to treat CNS and PNS injuries.
regenerating nerves [138, 139]. Stem cells are of interest in tissue engineering and
GDNF is known to induce neuronal outgrowth repair as they have the propensity to replace the
of sensory neurons through the PNS–CNS tran- lost or injured cells, have immunomodulatory
sition zone [123, 127, 140], promote survival effects, and can induce neuroprotection and
remyelination cascades [31]. Transplantation of system cells through a process known as direct
embryonic stem cell (ESC)-derived neural pro- reprogramming. Vierbuchen et al. showed that by
genitor cells into injured spinal cords reportedly using a combination of three transcription factors,
differentiated into astrocytes, oligodendrocytes, adult mouse fibroblasts could be reprogrammed
and neurons, and ESC-treated animals also directly into functional neurons in vitro [154].
showed better weight support and hind limb More recent studies have since followed, show-
coordination when compared to sham-treated ing that this technique can produce a variety of
animals [148]. While ESCs are advantageous specific, therapy-relevant neuronal cells and glial
given their capacity to differentiate into multiple from both fibroblasts and astrocytes, in vitro and
cell types, these cells carry ethical concerns and in vivo [155–158].
have the potential—when differentiation goes NSCs are clearly useful as a means to obtain
awry—to be tumorigenic [149]. Other stem otherwise difficult-to-obtain adult cell types.
cell types such as hematopoietic stem cells, For example, Schwann cells play an important
mesenchymal stem cells, and umbilical cord role in nerve repair and regeneration after CNS
blood stem cells can be made to differentiate and PNS injuries; however, obtaining a mature
into neuronal cells, but must undergo laborious population of Schwann cells for nerve repair
protocols for transdifferentiation into neural cells. involves creating a neurological defect. In these
Alternatively, neural stem cells (NSCs) are situations, NSCs can be used to play the role of
adult stem cells that have the potential to in- Schwann cells due to their ability to differentiate
tegrate into the cellular milieu of the nervous into neurons and myelinating glial cells [159].
system and therefore are a rational choice when Polysialylated neural cell adhesion molecule
considering stem cell therapy for CNS and PNS (NCAM)-positive NSCs when transplanted into a
injuries. The use of NSCs in nerve regeneration focal demyelinating spinal cord lesion were found
has been investigated in the recent past owing to generate both oligodendrocytes and Schwann
to their potential for differentiation into multi- cells, and completely remyelinated axons in the
ple cell types such as neurons, astrocytes, and CNS lesions [160]. Human NSCs transplanted
oligodendrocytes; self-renewal and production of into the lumbar cord of injured adult nude rats
homologous new cells after injury; capacity for showed extensive differentiation into neurons
ex vivo genetic modification; and transplantabil- and synapse formation with host motor neurons
ity. Neural stem cells have been isolated from [161]. In a combinatorial approach, NSCs were
the spinal cord, brain [150], muscle [151], bone transplanted using a PLGA scaffold constructed
marrow, and umbilical cord blood [152]. The iso- from a 50:50 blend of PLGA and PLGA + poly-
lation of NSCs from muscle and peripheral blood L-Lysine into an adult rat spinal cord hemisection
has nevertheless become controversial since the model. Open-field locomotion results indicated
discovery that NSCs have been found to populate that animals implanted with scaffolds containing
the hematopoietic system and muscles after in- NSCs induced a significantly greater functional
jury. Studies have shown that NSCs committed to recovery in the sub-chronic phase that extended
becoming neurons and glia changed to expressing into the chronic phase (70 days post-injury) when
endothelial markers when cocultured with human compared to animal groups implanted with NSCs
endothelial cells by cell fusion-mediated acquisi- alone or lesion controls [162]. These results
tion of lineage-specific determinants [105]. Fur- should nevertheless be interpreted with caution
thermore, it was found that myogenic conver- since functional recovery may have resulted from
sion of NSCs required their direct contact with tissue preservation rather than regeneration.
muscle cells, suggesting that spontaneous cell Besides their use as therapies for cell
fusion between inducing and induced cells is replacement, neuroprotection, regeneration, and
required for myogenic conversion [153]. There remyelination, human mesenchymal stem cells
is also the potential to directly convert a pa- (hMSCs) and NSCs are also known to play an
tient’s nonneural cells toward functional nervous important role in modulating the immune system
by limiting the immune response following factors, leading to inconsistent results and adverse
an injury to the CNS. In mice treated with effects [126, 127, 138, 139]. Additionally, it is
hMSCs after a transient common carotid artery believed that temporal control over neurotrophic
occlusion, a marked reduction in neuronal factor concentration is required to facilitate ax-
death was associated with the upregulation of onal regeneration into target organs [166], a chal-
neuroprotective genes and downregulation of lenge that is not easily addressed by the afore-
ischemia-induced inflammatory genes [163]. mentioned delivery methods. A more targeted and
Adult neural progenitor cells when delivered controlled approach to deliver these factors and
into the cerebrospinal fluid (CSF) along with a that can be tuned to temporally regulate their
myelin-derived peptide resulted in functional expression is the use of ex vivo delivery of cells
recovery after SCI, possibly by modulating engineered to express them.
the endogenous T-cell and microglial immune Fibroblasts have been extensively engineered
response [106]. as a model cell type for the delivery of
The work done with the previously described NGF, NT-3, BDNF, and GDNF. Genetically
implantable hydrogels systems can also be reap- engineered cells expressing different growth
propriated to simultaneously engineer and im- factors are reported to induce different patterns of
prove the transplantation of stem cell therapies regeneration. NGF-expressing fibroblasts when
as well as provide a direct biochemical benefit introduced into acutely and chronically injured
to neuroprotection. In a controlled cortical impact spinal cord lesions have been shown to induce
model of TBI, Betancur et al. showed that NSCs growth of sensory and noradrenergic axons
encapsulated in chondroitin sulfate glycosamino- into the fibroblast graft [167–169]. Fibroblasts
glycan (CS-GAGs) hydrogels into the lesion site expressing NT-3, however, seemed to induce
provided superior neuroprotection and reparative growth of corticospinal axons around the graft
benefit when compared to NSCs alone (Fig. 22.6) and into the spared gray matter [130]. Behavioral
[164]. CS-GAG matrices with and without NSCs analysis of animals grafted with NT-3-expressing,
were demonstrated to have a beneficial effect on using tests where animals are trained to cross a
injured CNS. The CS-GAG matrix also had the horizontal ladder toward a food source, revealed
added benefit of being able to maintain the un- that animals that received NT-3-expressing grafts
differentiated state of the NSCs, thus prolonging had fewer foot slips when compared to control
the period of effective “bystander effect,” and (reporter gene expressing) grafts. These effects
leading to a significant enhancement in neural were found to persist in graft-implanted animals
tissue protection after injury. for 6 weeks and 3 months post-injury [130].
In summary, NSC therapy for treatment of Fibroblasts engineered to express BDNF,
CNS and PNS injuries is potentially a safe, non- when implanted in dorsal-over hemisection
tumorigenic method of providing trophic and cel- lesions, have been shown to induce outgrowth
lular replacement after injury. Additionally, their of sensory, noradrenergic, and motor axons,
beneficial effects can be further enhanced by ex and prevent the death of corticospinal neurons
vivo gene therapy and by combinatorial cell ther- [170]. BDNF- and NT-3-releasing fibroblast
apy to achieve long-term functional outcomes. grafts, when implanted in a contusion injury,
induced far greater outgrowth and myelination
22.2.3.2 Genetic Engineering of cholinergic and sensory axons than the
Neurotrophic growth factors are traditionally de- reporter gene expressing control [171, 172]. In
livered via continuous infusion [131, 135] and other studies, Schwann cells expressing BDNF,
more recently by slow-releasing biodegradable when implanted into a completely transected
polymers [165]. Although these systems success- spinal cord, were able to elicit growth of
fully deliver prolonged doses of growth factors, cerulospinal and raphespinal axons [173]—in
lack of control over concentrations can result in contrast, fibroblasts expressing BDNF were
the delivery of extremely high doses of these unable to induce the same in T-8 contusion
Fig. 22.6 Impact of

chondroitin sulfate
glycosaminoglycans
(CS-GAGs) and neural
stem cells (NSCs) on
extent of neuronal presence
after TBI. (a–d) Nissl
staining on brain sections
obtained 4 weeks post-TBI.
(e) Quantification of Nissl
staining indicates that the
combination of CS-GAG
and NSC approached
neuronal densities in
uninjured sham controls
(∗ p < 0.05; ns: not
significant). (Reprinted
with permission from
Betancur et al. [164]
Copyright (2017)
American Chemical
Society)
injuries [172]. The reasons for these conflicting points rather than chronic time points, when
results are unknown and can at least be attributed the need for growth factor presence is the most
to differences in the injury models used. While and the chances of secondary lesions caused by
a high rate of cell survival has been reported for graft placement are minimal. For example, when
transplanted genetically engineered fibroblasts engineered fibroblasts secreting BDNF and NT-
[174–176], studies conducted to date suggest 3 were transplanted into C3/C4 hemisectioned
that the best chances of behavioral recovery after adult rats at a chronic time point 6 weeks after
SCI occur when grafts are placed at acute time injury, partial sprouting and neuroprotection
were observed although regeneration was modest and the number of cholinergic neurons was
[177]. These results indicate that although partial similar to those found in the reporter gene
recovery can be observed in sensory-motor expressing controls [181]. In a more recent
tasks in transplanted animals, overall behavioral study, tetracycline-inducible expression of BDNF
recovery is modest, possibly due to additional by genetically modified fibroblasts was also
damage caused by graft placement at chronic achieved, showing that transient growth factor
time points. At 14 weeks after transplantation of expression was sufficient to sustain regenerating
BDNF- and NT-3-expressing fibroblasts, it was axons for prolonged periods of time in spinal
found that NT-3- and BDNF-expressing grafts cord lesions [182].
provided greater neuroprotection and modest
growth of regenerating axons when compared to 22.2.3.3 Gene Therapy
control grafts. NT-3-secreting grafts significantly Nerve regeneration in CNS lesions and over long
reduced retrograde degeneration and atrophy of gaps in PNS defects requires sustained availabil-
the red nucleus, and both NT-3- and BDNF- ity of neurotrophic growth factors. Injection or in-
secreting grafts promoted modest growth of fusion of recombinant proteins and neurotrophic
rubrospinal axons, and also partially rescued growth factors is inefficient due to low diffusivity
and protected axotomized supraspinal neurons and consequently requires the administration of
from atrophy [176]. high treatment doses. Gene therapy is one tool
GDNF-expressing fibroblast grafts, when by which in situ production of these molecules
implanted in mid-thoracic lesions, induced the can be achieved. The first stage of this process
outgrowth of hypoglossal motor and sensory involves identifying and isolating the target genes
axons and induced an increase in expression of interest. The isolated genes are then cloned into
of the growth-related protein calcitonin gene- a construct that contains information for stable
related peptide. This was also accompanied by expression of the gene in the target organism.
an increase in numbers of myelinating and non- The gene of interest is then carried and stably
myelinating Schwann cells [166, 178, 179]. incorporated into a random location in the tar-
Although cellular grafts expressing neu- get organism’s genome by a vector of choice.
rotrophic factors induce axonal migration into The process of gene therapy in the context of
grafts, continuous expression of these factors treating CNS/PNS injuries involves transduction
may prevent regenerating axons from leaving the and transfection of therapeutic genes via viral or
grafts to reinnervate target tissue. Controlling nonviral means, and subsequent gene expression
gene expression such that neurotrophic factor in a subset of cells that can then be used to make
expression is turned “on” in order to encourage the therapeutic proteins of interest.
axonal ingress into the graft and turned “off” Nonviral means of gene delivery involve direct
to facilitate axonal egress out of the graft may injection of naked plasmid DNA, lipid-mediated
ultimately enable patterning of axonal growth. transfection techniques, direct delivery of DNA
This process of controlling gene expression using gene guns, and more recently targeted de-
involves placing the gene of interest in an livery of DNA encapsulated biodegradable ma-
autoregulatory retroviral cassette that enables trices using surface targeting of cell-specific lig-
reversible transgene expression in response to ands [118]. Direct injection of naked DNA re-
tetracycline [180]. When animals inflicted with sults in variable gene expression possibly due
injuries to the fimbria–fornix were injected in the to nonspecific uptake and cleavage by resident
medial septum with retroviral NGF-expressing endonucleases. DNA delivery via gene guns is
fibroblasts, a significant outgrowth of cholinergic a more efficient method of naked DNA delivery
neurons was observed 2 weeks post-injury. In that typically results in high transduction rates but
contrast, when animals implanted with NGF- also results in tissue damage due to the process of
expressing grafts were fed water containing forcibly inducing gene delivery via high voltage
doxycycline, NGF expression was turned off, and pressure [118]. Cationic lipid transfection
agents are commonly used to transfect cells in favor certain cells and tissues. Murlidharan et al.
vitro; however, variable transfection efficiencies showed this with an AAV chimera designed to tar-
and cell toxicity issues plague the use of these get neurons preferentially over glia [195]. Using
agents, making their use for gene therapy applica- this AAV construct, they were able to deliver an
tions challenging [183]. More advanced nonviral in vivo gene-editing payload (CRISPR/Cas9) that
gene delivery systems using cell-specific recep- could further increase the specificity of therapy
tor binding peptides to deliver genes to target through targeting knockout of a schizophrenia
cells [184] and gene-activated matrices (GAMs) risk gene. These viral technologies, along with
to present genes of interest immobilized within advances in gene editing payloads, are promising
a biodegradable matrix allow for more targeted platforms for tissue engineering especially when
gene delivery and sustained protein production there is a clear underlying genetic etiology.
within CNS lesions [185].
Although nonviral vectors have a number of
advantages, their variable delivery efficiencies 22.2.4 Immunomodulation
and inability to transport genes of interest across
an intact nuclear membrane make the use of vi- Another tissue engineering strategy is to target
ral gene delivery methods an attractive alterna- the endogenous immune system as a means
tive. To date, several modes of viral gene deliv- of diminishing or managing the immune-
ery using retrovirus (RV), adenovirus (AV), and mediated effects on tissue degeneration. In
adeno-associated virus (AAV) vectors have been particular, the inflammation-relevant plasticity of
tested in various tissue and animal types, with monocytes/macrophages makes them especially
adenovirus [186–188] and herpes simplex virus attractive as targets for phenotype modulation. A
(HSV) [189, 190] being mostly used for gene short-term (3 week) study using a 15 mm nerve
transfer to the nervous system. Individual viral gap model demonstrated that delivery of the
vectors have both advantages and disadvantages immunomodulatory cytokine interleukin 4 (IL-
that need to be assessed before being consid- 4) could polarize injury-local macrophages to
ered for any particular application. RV has the express more “pro-healing” phenotypes [196].
ability to stably integrate genes of interest into Pro-healing modulation of macrophages led
the genome, but stable transduction is wholly to an increase in Schwann cell recruitment
dependent upon the ability of the infected cell to and axonal growth as compared to scaffold
undergo mitotic cell division. AV is the vector controls without chemical intervention and
of choice for gene transfer into nonmitotic cells faired dramatically better than in cases where
such as neurons; however, infected cells induce an the pro-inflammatory cytokine interferon γ
immune response that limits their application in was introduced. The premise of the study was
vivo. HSV is capable of delivering a large payload that local immunomodulation led to a bias of
of DNA; however, lack of stable transduction the local immune microenvironment triggering
and cytotoxicity are some of the disadvantages a regenerative biochemical cascade involving
of these vectors [191]. To circumvent drawbacks Schwann cells and neuronal processes, eventually
of traditional viral vectors and to provide a safe leading to improved repair.
and efficient alternative for use in humans, hy- In a later study, Mokarram et al. further
brid/chimeric viral vectors that combine desirable showed that recruitment rather than direct mod-
characteristics of conventional viral vectors have ulation of the constituent immune system could
been developed for gene therapy applications in have a dramatic impact on PNS repair [197]. In
the nervous system [192–194]. Viruses are also a this study, fractalkine was delivered from a nerve
favorable substrate for engineering tissue speci- repair conduit in a long-gap nerve transection
ficity, as different serotypes have different base- model and was shown to preferentially recruit
line invasion preferences, and can be further engi- reparative monocytes, leading to a significant
neered or mutated to discover viral chimeras that improvement in bridging, over gold-standard
A 15 mm gap; 4 weeks
Nerve Guidance Channel (Polysulfone)

Nerve Nerve
Distal Proximal
End End
Agarose Hydrogel IL-4
Cytokines or
Fractalkine (FKN)
4 weeks
Proximal
Distal
B C
3000
*
15 mm ####
Number of Axons
##
2000
1000
0
IL-4 treated FKN treated Autograft
Fig. 22.7 The effect of fractalkine (FKN) and interleukin (blue, DAPI) at the distal nerve stump. (c) The number
4 (IL-4) on axonal growth. (a) After nerve axotomy, auto- of regenerated axons indicated a significant improvement
graft or scaffolds containing IL-4 or FKN were sutured using FKN over IL-4 (p < 0.01) though autograft led to the
to the injured ends of the nerve using the critical gap most regenerated axons (p < 0.0001 vs. IL-4; p < 0.05 vs
size of 15 mm. After 4 weeks (b) immunohistochemical FKN). (Reprinted from Mokarram et al. [197])
staining was performed for axons (NF160, red) and nuclei
cytokine modulation strategies, and reaching very recovery and rehabilitation after injury [198,
close to the measures of regeneration found using 199]. Electrical activity is known to stabilize
autografts (including axonal count, percentage of the synapse, induce gene expression changes,
myelinated axons, and improved neuromuscular and facilitate neurogenesis. The mechanism
responses measured using electrophysiology) by which electrical activity induces these
(Fig. 22.7). outcomes, however, is largely unclear. It is
believed that increased electrical activity of the
neurons induces the influx and accumulation
22.2.5 Electrical Stimulation for of intracellular calcium which then triggers a
Repair and Regeneration secondary cascade–mediated synthesis of “early
genes.” Protein products of these early genes are
Electrical stimulation of the nervous system has then responsible for activation of “late genes” that
been shown to accelerate plasticity and promote control neurogenesis [200]. Therefore, a genetic
neuroprosthesis can be developed by controlling femoral nerves were stimulated at 20 Hz fre-

the electrical activity of the neuron [201, 202]. quency proximal to repair site over multiple time
Direct current (DC) is generally applied for periods. Results from these studies showed that
electrical stimulation of nerves due to the endoge- electrical stimulation over a period of 1 h led to a
nous presence of DC voltage gradients within significant increase in DRG neurons regenerating
tissues [203]. In early studies, the application of into cutaneous and muscle branches, and that
DC electric fields (approx. 100 mV/mm) from an these neurons were associated with a significant
electrode-containing, silicone conduit onto the increase in expression of growth-associated pro-
dorsal half of guinea pig spinal cords showed tein 43 (GAP-43) mRNA and BDNF 2 days post-
robust regeneration of axons into the conduit injury when compared to sham-treated and other
when compared to control guidance channels stimulation time points [198]. In more recent
[204, 205]. The DC currents used for electrical studies, functional electrical stimulation (FES)
stimulation in these studies, however, was known has been used in rats that were inflicted with a
to damage both electrode and associated tissue complete SCI at T8 and T9. After implantation
[201]. Since these early studies, the efficacy of FES probes to measure hind limb movement
of DC stimulation parameters has been widely 3 weeks post-injury, dividing cells were labeled
investigated with several reports demonstrating with bromodeoxyuridine (BrdU). Half the ani-
DC stimulation-induced improvements in mals that received BrdU were termed the “cell-
network plasticity and repair [206–208]. birth” group and sacrificed immediately after ad-
Alternating current (AC) has also been used to ministration, while the other half, termed the “cell
electrically stimulate peripheral nerve regenera- survival” group, were sacrificed after 7 days. Re-
tion. Early studies have shown that low-frequency sults from these studies showed that FES induced
electrical stimulation using AC was effective to an 82–86% increase in cell birth, which is sus-
help enable reinnervation of de-innervated mus- tained in the “cell survival” group. Results from
cles [209, 210]. Animals that were electrically these cells showed the expression of stem cell and
stimulated at frequencies of 1–4 Hz for 15–30 min glial cell markers such as nestin, NG2, GFAP, and
after nerve crush injury reportedly regained toe APC in FES rats [214]. Other recent studies also
spread function and reinnervation of the flexor show that short periods of electrical stimulation
muscles within 6 days after injury, and it was lead to a significant increase in expression of
more effective if stimulation regimes were initi- nerve growth-associated genes such as GAP-43,
ated immediately after injury [209, 211]. These along with enhanced protein expression of growth
effects, however, could also be due to more ef- factors such as BDNF and insulin-like growth
fective sprouting and reinnervation rather than factor-1 [198, 215, 216]. Goldberg et al. showed
regeneration [212]. that the rather slow outgrowth of retinal ganglion
To test whether these effects were also true for cell axons mediated by peptide growth factors is
transected nerves, femoral nerves were transected markedly increased when the cells were electri-
and sutured, and animals were subjected to 20 Hz cally stimulated at physiological levels, proving
of electrical stimulation along with retrograde that electrically stimulated neurons utilize peptide
labeling of neurons to count the number of regen- growth factors for survival and axonal outgrowth
erating axons [213]. In these studies, it was noted better than unstimulated neurons [217].
that although the motor axons did not reinnervate
into motor pathways immediately after transec-
tion and suture, the axons did indeed progres- 22.3 Conclusion
sively regenerate over time and eventually rein-
nervated the target muscle 4 weeks post-injury. In The factors and circumstances that influence
a subsequent study conducted to assess the effects nerve regeneration after injury to the CNS and
of time of stimulation, transected and sutured PNS are complex. This chapter provides a review
of currently available technologies to engineer Lathia, C. Rimorin, M. Tieu, R. Larsen, T. Casper,

nervous tissue regeneration. In the future, CN- E. Barkan, M. Kroll, S. Parry, N.V. Shapovalova,
D. Hirschstein, J. Pendergraft, H.A. Sullivan, T.K.
S/PNS regeneration strategies should continue to
Kim, A. Szafer, N. Dee, P. Groblewski, I. Wicker-
evolve an interdisciplinary approach that includes sham, A. Cetin, J.A. Harris, B.P. Levi, S.M. Sunkin,
a sound understanding of the underlying molec- L. Madisen, T.L. Daigle, L. Looger, A. Bernard,
ular and cellular environment post-injury, and J. Phillips, E. Lein, M. Hawrylycz, K. Svoboda,
A.R. Jones, C. Koch, H. Zeng, Shared and distinct
an applied knowledge of the rapidly progressing
transcriptomic cell types across neocortical areas.
fields of biomaterials, gene therapy, stem cell Nature 563, 72–78 (2018)
biology, and neural prostheses. A combination of 2. C.H. Tischbirek, T. Noda, M. Tohmi, A. Birkner, I.
these fields and technologies can be applied to Nelken, A. Konnerth, In vivo functional mapping of
a cortical column at single-neuron resolution. Cell
design multipronged strategies along the path to Rep. 27(5), 1319–1326 (2019)
enabling neural regeneration, with the ultimate 3. N.J. Abbott, L. Ronnback, E. Hansson, Astrocyte-
goal of restoring functional recovery. endothelial interactions at the blood-brain barrier.
Nat. Rev. Neurosci. 7(1), 41–53 (2006)
4. A. Aspelund, S. Antila, S.T. Proulx, T.V. Karlsen,
Homework S. Karaman, M. Detmar, H. Wiig, K. Alitalo, A
dural lymphatic vascular system that drains brain
interstitial fluid and macromolecules. J. Exp. Med.
1. Outline the components of the two major 212(7), 991–999 (2015)
subdivisions of the nervous system. 5. J.J. Iliff, M. Wang, Y. Liao, B.A. Plogg, W. Peng,
2. What are the main functions of the blood– G.A. Gundersen, H. Benveniste, G.E. Vates, R.
Deane, S.A. Goldman, E.A. Nagelhus, M. Neder-
brain and the blood–spinal cord barriers? gaard, A paravascular pathway facilitates CSF flow
3. Why are astrocytes important to consider as through the brain parenchyma and the clearance of
a cellular target for neural tissue repair? interstitial solutes, including amyloid β. Sci. Transl.
4. State two unique attributes of neural Med. 4(147), 147ra111 (2012)
6. A. Louveau, I. Smirnov, T.J. Keyes, J.D. Eccles, S.J.
tissue. Rouhani, J.D. Peske, N.C. Derecki, D. Castle, J.W.
5. What are the long-term consequences of CNS Mandell, K.S. Lee, T.H. Harris, J. Kipnis, Structural
injuries? and functional features of central nervous system
6. What are glial cells present in the PNS lymphatic vessels. Nature 523(7560), 337 (2015)
7. M.V. Sofroniew, Reactive astrocytes in neural re-
called? pair and protection. Neuroscientist 11(5), 400–407
7. Name one application of implantable neural (2005)
devices. 8. M. Eddleston, L. Mucke, Molecular profile of reac-
8. How do polymeric nerve conduits containing tive astrocytes–implications for their role in neuro-
logic disease. Neuroscience 54(1), 15–36 (1993)
internal scaffolding facilitate nerve regenera- 9. J. Silver, J.H. Miller, Regeneration beyond the glial
tion? scar. Nat. Rev. Neurosci. 5(2), 146–156 (2004)
9. Why would neural stem cells be preferential 10. J.M. Brown, J. Xia, B. Zhuang, K.S. Cho, C.J.
to other stem cell types in repairing CN- Rogers, C.I. Gama, M. Rawat, S.E. Tully, N. Ue-
tani, D.E. Mason, M.L. Tremblay, E.C. Peters, O.
S/PNS tissues? Habuchi, D.F. Chen, L.C. Hsieh-Wilson, A sulfated
10. What are the desired characteristics of an carbohydrate epitope inhibits axon regeneration af-
implantable material? ter injury. Proc. Natl. Acad. Sci. 109(13), 4768–4773
11. When would gene therapy be favorable to the (2012)
11. L. Karumbaiah, S. Anand, R. Thazhath, Y. Zhong,
use of genetically engineered cells? R.J. McKeon, R.V. Bellamkonda, Targeted down-
regulation of N-acetylgalactosamine 4-sulfate 6-
O-sulfotransferase significantly mitigates chon-
References droitin sulfate proteoglycan-mediated inhibition.
Glia 59(6), 981–996 (2011)
1. B. Tasic, Z. Yao, L.T. Graybuck, K.A. Smith, T.N. 12. S.E. Tully, R. Mabon, C.I. Gama, S.M. Tsai, X.
Nguyen, D. Bertagnolli, J. Goldy, E. Garren, M.N. Liu, L.C. Hsieh-Wilson, A chondroitin sulfate small
Economo, S. Viswanathan, O. Penn, T. Bakken, molecule that stimulates neuronal growth. J. Am.
V. Menon, J. Miller, O. Fong, K.E. Hirokawa, K. Chem. Soc. 126(25), 7736–7737 (2004)
13. A. Nimmerjahn, F. Kirchhoff, F. Helmchen, Rest- 2007 and 2013. MMWR Surveill. Summ. 66(9), 1
ing microglial cells are highly dynamic surveillants (2017)
of brain parenchyma in vivo. Science 308(5726), 30. C. Werner, K. Engelhard, Pathophysiology of trau-
1314–1318 (2005) matic brain injury. Br. J. Anaesth. 99(1), 4–9 (2007)
14. F. Aloisi, Immune function of microglia. Glia 36(2), 31. J.W. Rowland, G.W. Hawryluk, B. Kwon, M.G.
165–179 (2001) Fehlings, Current status of acute spinal cord injury
15. D. Giulian, Ameboid microglia as effectors of in- pathophysiology and emerging therapies: Promise
flammation in the central nervous system. J. Neu- on the horizon. Neurosurg. Focus. 25(5), E2 (2008)
rosci. Res. 18(1), 155–171 (1987). 132–153 32. M.T. Filbin, Myelin-associated inhibitors of axonal
16. W.J. Streit, M.B. Graeber, G.W. Kreutzberg, Func- regeneration in the adult mammalian CNS. Nat. Rev.
tional plasticity of microglia: A review. Glia 1(5), Neurosci. 4(9), 703–713 (2003)
301–307 (1988) 33. Z. He, V. Koprivica, The Nogo signaling pathway for
17. R.P. Bunge, M.B. Bunge, Rish, Electron micro- regeneration block. Annu. Rev. Neurosci. 27, 341–
scopic study of demyelination in an experimentally 368 (2004)
induced lesion in adult cat spinal cord. J. Biophys. 34. M.A. Anderson, J.E. Burda, Y. Ren, Y. Ao, T.M.
Biochem. Cytol. 7, 685–696 (1960) O’Shea, R. Kawaguchi, G. Coppola, B.S. Khakh,
18. M. Bradl, H. Lassmann, Oligodendrocytes: Biology T.J. Deming, M.V. Sofroniew, Astrocyte scar forma-
and pathology. Acta Neuropathol. 119(1), 37–53 tion aids central nervous system axon regeneration.
(2010) Nature 532(7598), 195 (2016)
19. C. French-Constant, J. Camara, Lessons from oligo- 35. S. Ramón y Cajal, Degeneration and Regeneration
dendrocyte biology on promoting repair in multiple of the Nervous System (Oxford University Press,
sclerosis. J. Neurol. 254, 15–22 (2007) London, 1928)
20. I. Glezer, A. Lapointe, S. Rivest, Innate immunity 36. S. David, A.J. Aguayo, Axonal elongation into pe-
triggers oligodendrocyte progenitor reactivity and ripheral nervous system “bridges” after central ner-
confines damages to brain injuries. FASEB J. 20(6), vous system injury in adult rats. Science 214(4523),
750–752 (2006) 931–933 (1981)
21. V. Bartanusz, D. Jezova, B. Alajajian, M. Digicayli- 37. V.J. Tom, M.P. Steinmetz, J.H. Miller, C.M. Doller,
oglu, The blood–spinal cord barrier: Morphology J. Silver, Studies on the development and behavior
and clinical implications. Ann. Neurol. 70(2), 194– of the dystrophic growth cone, the hallmark of re-
206 (2011) generation failure, in an in vitro model of the glial
22. M.M. Rich, J.W. Lichtman, In vivo visualization of scar and after spinal cord injury. J. Neurosci. 24(29),
pre- and postsynaptic changes during synapse elim- 6531–6539 (2004)
ination in reinnervated mouse muscle. J. Neurosci. 38. J. Noble, C.A. Munro, V.S. Prasad, R. Midha, Anal-
9(5), 1781–1805 (1989) ysis of upper and lower extremity peripheral nerve
23. C. Birchmeier, K.A. Nave, Neuregulin-1, a key ax- injuries in a population of patients with multiple
onal signal that drives Schwann cell growth and injuries. J. Trauma 45(1), 116–122 (1998)
differentiation. Glia 56(14), 1491–1497 (2008) 39. M.G. Burnett, E.L. Zager, Pathophysiology of pe-
24. M. Abercrombie, M.L. Johnson, The outwander- ripheral nerve injury: A brief review. Neurosurg.
ing of cells in tissue cultures of nerves undergoing Focus. 16(5), E1 (2004)
Wallerian degeneration. J. Exp. Biol. 19, 266–283 40. J.A. Kouyoumdjian, Peripheral nerve injuries: A ret-
(1942) rospective survey of 456 cases. Muscle Nerve 34(6),
25. Y.J. Son, W.J. Thompson, Schwann cell processes 785–788 (2006)
guide regeneration of peripheral axons. Neuron 41. B. Malik, M. Stillman, Chemotherapy-induced pe-
14(1), 125–132 (1995) ripheral neuropathy. Curr. Pain Headache Rep.
26. M.J. Politis, K. Ederle, P.S. Spencer, Tropism in 12(3), 165–174 (2008)
nerve regeneration in vivo. Attraction of regenerat- 42. A. Hartemann, N. Attal, D. Bouhassira, I. Dumont,
ing axons by diffusible factors derived from cells in H. Gin, S. Jeanne, G. Said, J.L. Richard, Painful
distal nerve stumps of transected peripheral nerves. diabetic neuropathy: Diagnosis and management.
Brain Res. 253(1–2), 1–12 (1982) Diabetes Metab. 37(5), 377–388 (2011)
27. C. Ide, K. Tohyama, R. Yokota, T. Nitatori, S. On- 43. H.J. Seddon, Three types of brain injury. Brain 66,
odera, Schwann cell basal lamina and nerve regen- 237–288 (1943)
eration. Brain Res. 288(1–2), 61–75 (1983) 44. V. Mukhatyar, L. Karumbaiah, J. Yeh, R. Bel-
28. A.B. Peterson, L. Xu, J. Daugherty, M.J. Breiding, lamkonda, Tissue engineering strategies designed to
Surveillance report of traumatic brain injury-related facilitate the endogenous regenerative potential of
emergency department visits, hospitalizations, and peripheral nerves. Adv. Mater. 21(46), 4670–4679
deaths, United States, 2014 (2019) (2009)
29. C.A. Taylor, J.M. Bell, M.J. Breiding, L. Xu, Trau- 45. R.P. Bunge, Expanding roles for the Schwann
matic brain injury–related emergency department cell: Ensheathment, myelination, trophism and re-
visits, hospitalizations, and deaths—United States, generation. Curr. Opin. Neurobiol. 3(5), 805–809
(1993)
46. J.L. Salzer, R.P. Bunge, Studies of Schwann cell pro- cell matrix in poly-epsilon-caprolactone conduits
liferation. I. an analysis in tissue culture of prolifer- enhances guided nerve regeneration. Int. J. Artif.
ation during development, Wallerian degeneration, Organs 27(2), 127–136 (2004)
and direct injury. J. Cell Biol. 84(3), 739–752 (1980) 59. D.F. Kalbermatten, P.J. Kingham, D. Mahay, C.
47. F.J. Vingerhoets, J.G. Villemure, P. Temperli, C. Mantovani, J. Pettersson, W. Raffoul, H. Balcin, G.
Pollo, E. Pralong, J. Ghika, Subthalamic DBS re- Pierer, G. Terenghi, Fibrin matrix for suspension
places levodopa in Parkinson’s disease: Two-year of regenerative cells in an artificial nerve conduit.
follow-up. Neurology 58(3), 396–401 (2002) J. Plast. Reconstr. Aesthet. Surg. 61(6), 669–675
48. T.D. Kozai, A.S. Jaquins-Gerstl, A.L. Vazquez, A.C. (2008)
Michael, X.T. Cui, Brain tissue responses to neural 60. D.F. Kalbermatten, J. Pettersson, P.J. Kingham, G.
implants impact signal sensitivity and intervention Pierer, M. Wiberg, G. Terenghi, New fibrin conduit
strategies. ACS Chem. Neurosci. 6(1), 48–67 (2015) for peripheral nerve repair. J. Reconstr. Microsurg.
49. T. Saxena, L. Karumbaiah, E.A. Gaupp, R. Patkar, 25(1), 27–33 (2009)
K. Patil, M. Betancur, G.B. Stanley, R.V. Bel- 61. A.P. Balgude, X. Yu, A. Szymanski, R.V. Bel-
lamkonda, The impact of chronic blood–brain bar- lamkonda, Agarose gel stiffness determines rate of
rier breach on intracortical electrode function. Bio- DRG neurite extension in 3D cultures. Biomaterials
materials 34(20), 4703–4713 (2013) 22(10), 1077–1084 (2001)
50. W. Shen, L. Karumbaiah, X. Liu, T. Saxena, S. 62. G.P. Dillon, X. Yu, A. Sridharan, J.P. Ranieri, R.V.
Chen, R. Patkar, R.V. Bellamkonda, M.G. Allen, Bellamkonda, The influence of physical structure
Extracellular matrix-based intracortical microelec- and charge on neurite extension in a 3D hydrogel
trodes: Toward a microfabricated neural interface scaffold. J. Biomater. Sci. Polym. Ed. 9(10), 1049–
based on natural materials. Microsyst. Nanoeng. 1, 1069 (1998)
15010 (2015) 63. M.C. Dodla, R.V. Bellamkonda, Differences be-
51. G. Lundborg, R.H. Gelberman, F.M. Longo, H.C. tween the effect of anisotropic and isotropic laminin
Powell, S. Varon, In vivo regeneration of cut nerves and nerve growth factor presenting scaffolds on
encased in silicone tubes: Growth across a six- nerve regeneration across long peripheral nerve
millimeter gap. J. Neuropathol. Exp. Neurol. 41(4), gaps. Biomaterials 29(1), 33–46 (2008)
412–422 (1982) 64. R.O. Labrador, M. Buti, X. Navarro, Peripheral
52. Z.L. Shen, F. Lassner, M. Becker, G.F. Walter, A. nerve repair: Role of agarose matrix density on
Bader, A. Berger, Viability of cultured nerve grafts: functional recovery. Neuroreport 6(15), 2022–2026
An assessment of proliferation of Schwann cells and (1995)
fibroblasts. Microsurgery 19(8), 356–363 (1999) 65. S. Amado, M.J. Simoes, P.A. Armada da Silva, A.L.
53. L.J. Chamberlain, I.V. Yannas, A. Arrizabalaga, H.P. Luis, Y. Shirosaki, M.A. Lopes, J.D. Santos, F. Freg-
Hsu, T.V. Norregaard, M. Spector, Early peripheral nan, G. Gambarotta, S. Raimondo, M. Fornaro, A.P.
nerve healing in collagen and silicone tube implants: Veloso, A.S. Varejao, A.C. Mauricio, S. Geuna, Use
Myofibroblasts and the cellular response. Biomate- of hybrid chitosan membranes and N1E-115 cells for
rials 19(15), 1393–1403 (1998a) promoting nerve regeneration in an axonotmesis rat
54. L.J. Chamberlain, I.V. Yannas, H.P. Hsu, G. model. Biomaterials 29(33), 4409–4419 (2008)
Strichartz, M. Spector, Collagen-GAG substrate en- 66. M.E. Chavez-Delgado, J. Mora-Galindo, U. Gomez-
hances the quality of nerve regeneration through Pinedo, A. Feria-Velasco, S. Castro-Castaneda, F.A.
collagen tubes up to level of autograft. Exp. Neurol. Lopez-Dellamary Toral, S. Luquin-De Anda, L.M.
154(2), 315–329 (1998b) Garcia-Segura, J. Garcia-Estrada, Facial nerve re-
55. S.T. Li, S.J. Archibald, C. Krarup, R.D. Madison, generation through progesterone-loaded chitosan
Peripheral nerve repair with collagen conduits. Clin. prosthesis. A preliminary report. J Biomed Mater
Mater. 9(3–4), 195–200 (1992) Res B Appl Biomater 67(2), 702–711 (2003)
56. A. Bozkurt, R. Deumens, C. Beckmann, L. Olde 67. M. Patel, L. Mao, B. Wu, P. VandeVord, GDNF
Damink, F. Schugner, I. Heschel, B. Sellhaus, J. blended chitosan nerve guides: An in vivo study. J.
Weis, W. Jahnen-Dechent, G.A. Brook, N. Pallua, Biomed. Mater. Res. A 90(1), 154–165 (2009)
In vitro cell alignment obtained with a Schwann cell 68. A. Mosahebi, M. Simon, M. Wiberg, G. Terenghi,
enriched microstructured nerve guide with longitu- A novel use of alginate hydrogel as Schwann cell
dinal guidance channels. Biomaterials 30(2), 169– matrix. Tissue Eng. 7(5), 525–534 (2001)
179 (2009) 69. Y. Sakai, Y. Matsuyama, K. Takahashi, T. Sato,
57. L. Ghasemi-Mobarakeh, M.P. Prabhakaran, M. Mor- T. Hattori, S. Nakashima, N. Ishiguro, New arti-
shed, M.H. Nasr-Esfahani, S. Ramakrishna, Elec- ficial nerve conduits made with photocrosslinked
trospun poly(epsilon-caprolactone)/gelatin nanofi- hyaluronic acid for peripheral nerve regeneration.
brous scaffolds for nerve tissue engineering. Bioma- Biomed. Mater. Eng. 17(3), 191–197 (2007)
terials 29(34), 4532–4539 (2008) 70. B.R. Seckel, D. Jones, K.J. Hekimian, K.K. Wang,
58. T.J. Galla, S.V. Vedecnik, J. Halbgewachs, S. Stein- D.P. Chakalis, P.D. Costas, Hyaluronic acid through
mann, C. Friedrich, G.B. Stark, Fibrin/Schwann a new injectable nerve guide delivery system en-
hances peripheral nerve regeneration in the rat. J. cellularization of hybrid constructs. Biomacro-
Neurosci. Res. 40(3), 318–324 (1995) molecules 9(8), 2097–2103 (2008)
71. P. Aebischer, V. Guenard, S.R. Winn, R.F. Valentini, 83. S. Panseri, C. Cunha, J. Lowery, U. Del Carro, F.
P.M. Galletti, Blind-ended semipermeable guidance Taraballi, S. Amadio, A. Vescovi, F. Gelain, Elec-
channels support peripheral nerve regeneration in trospun micro- and nanofiber tubes for functional
the absence of a distal nerve stump. Brain Res. nervous regeneration in sciatic nerve transections.
454(1–2), 179–187 (1988) BMC Biotechnol. 8, 39 (2008)
72. S. Atzet, S. Curtin, P. Trinh, S. Bryant, B. Ratner, 84. M.P. Prabhakaran, J.R. Venugopal, T.T. Chyan, L.B.
Degradable poly(2-hydroxyethyl methacrylate)-co- Hai, C.K. Chan, A.Y. Lim, S. Ramakrishna, Electro-
polycaprolactone hydrogels for tissue engineering spun biocomposite nanofibrous scaffolds for neural
scaffolds. Biomacromolecules 9(12), 3370–3377 tissue engineering. Tissue Eng. Part A 14(11), 1787–
(2008) 1797 (2008)
73. Y.T. Kim, V.K. Haftel, S. Kumar, R.V. Bellamkonda, 85. C.M. Valmikinathan, J. Tian, J. Wang, X. Yu, Novel
The role of aligned polymer fiber-based constructs nanofibrous spiral scaffolds for neural tissue engi-
in the bridging of long peripheral nerve gaps. Bio- neering. J. Neural Eng. 5(4), 422–432 (2008)
materials 29(21), 3117–3127 (2008) 86. E.L. Whitlock, S.H. Tuffaha, J.P. Luciano, Y. Yan,
74. I.P. Clements, Y.T. Kim, A.W. English, X. Lu, D.A. Hunter, C.K. Magill, A.M. Moore, A.Y. Tong,
A. Chung, R.V. Bellamkonda, Thin-film enhanced S.E. Mackinnon, G.H. Borschel, Processed allo-
nerve guidance channels for peripheral nerve repair. grafts and type I collagen conduits for repair of
Biomaterials 30(23–24), 3834–3846 (2009) peripheral nerve gaps. Muscle Nerve 39(6), 787–799
75. D.M. Thompson, H.M. Buettner, Oriented Schwann (2009)
cell monolayers for directed neurite outgrowth. Ann. 87. A. Jain, Y.T. Kim, R.J. McKeon, R.V. Bellamkonda,
Biomed. Eng. 32(8), 1120–1130 (2004) In situ gelling hydrogels for conformal repair of
76. D.J. Bryan, J.B. Tang, A.H. Holway, K.M. Rieger- spinal cord defects, and local delivery of BDNF
Christ, D.J. Trantolo, D.L. Wise, I.C. Summerhayes, after spinal cord injury. Biomaterials 27(3), 497–504
Enhanced peripheral nerve regeneration elicited by (2006)
cell-mediated events delivered via a bioresorbable 88. Y. Zhong, R.V. Bellamkonda, Biomaterials for the
PLGA guide. J. Reconstr. Microsurg. 19(2), 125– central nervous system. J. R. Soc. Interface 5(26),
134 (2003) 957–975 (2008)
77. W.F. Den Dunnen, B. Van der Lei, J.M. Schaken- 89. S. Carbonetto, M.M. Gruver, D.C. Turner, Nerve
raad, E.H. Blaauw, I. Stokroos, A.J. Pennings, P.H. fiber growth in culture on fibronectin, collagen, and
Robinson, Long-term evaluation of nerve regenera- glycosaminoglycan substrates. J. Neurosci. 3(11),
tion in a biodegradable nerve guide. Microsurgery 2324–2335 (1983)
14(8), 508–515 (1993) 90. J.M. Thompson, D.J. Pelto, Attachment, survival
78. G.R. Evans, K. Brandt, M.S. Widmer, L. Lu, R.K. and neurite extension of chick embryo retinal neu-
Meszlenyi, P.K. Gupta, A.G. Mikos, J. Hodges, J. rons on various culture substrates. Dev. Neurosci.
Williams, A. Gurlek, A. Nabawi, R. Lohman, C.W. 5(5–6), 447–457 (1982)
Patrick Jr., In vivo evaluation of poly (L-lactic acid) 91. E.A. Joosten, P.R. Bar, W.H. Gispen, Collagen im-
porous conduits for peripheral nerve regeneration. plants and cortico-spinal axonal growth after mid-
Biomaterials 20(12), 1109–1115 (1999) thoracic spinal cord lesion in the adult rat. J. Neu-
79. H. Molander, Y. Olsson, O. Engkvist, S. Bowald, I. rosci. Res. 41(4), 481–490 (1995)
Eriksson, Regeneration of peripheral nerve through 92. D.A. Houweling, A.J. Lankhorst, W.H. Gispen,
a polyglactin tube. Muscle Nerve 5(1), 54–57 P.R. Bar, E.A. Joosten, Collagen containing
(1982) neurotrophin-3 (NT-3) attracts regrowing injured
80. A. Valero-Cabre, K. Tsironis, E. Skouras, G. Perego, corticospinal axons in the adult rat spinal cord and
X. Navarro, W.F. Neiss, Superior muscle reinnerva- promotes partial functional recovery. Exp. Neurol.
tion after autologous nerve graft or poly-L-lactide- 153(1), 49–59 (1998a)
epsilon-caprolactone (PLC) tube implantation in 93. D.A. Houweling, J.T. van Asseldonk, A.J.
comparison to silicone tube repair. J. Neurosci. Res. Lankhorst, F.P. Hamers, D. Martin, P.R. Bar, E.A.
63(2), 214–223 (2001) Joosten, Local application of collagen containing
81. D. Pankajakshan, L.P. Philipose, M. Palakkal, K. brain-derived neurotrophic factor decreases the loss
Krishnan, L.K. Krishnan, Development of a fibrin of function after spinal cord injury in the adult rat.
composite-coated poly(epsilon-caprolactone) scaf- Neurosci. Lett. 251(3), 193–196 (1998b)
fold for potential vascular tissue engineering appli- 94. M.C. Jimenez Hamann, C.H. Tator, M.S. Shoichet,
cations. J Biomed Mater Res B Appl Biomater 87(2), Injectable intrathecal delivery system for localized
570–579 (2008) administration of EGF and FGF-2 to the injured rat
82. A.K. Ekaputra, G.D. Prestwich, S.M. Cool, D.W. spinal cord. Exp. Neurol. 194(1), 106–119 (2005)
Hutmacher, Combining electrospun scaffolds with 95. D. Gupta, C.H. Tator, M.S. Shoichet, Fast-gelling in-
electrosprayed hydrogels leads to three-dimensional jectable blend of hyaluronan and methylcellulose for
intrathecal, localized delivery to the injured spinal nanofiber scaffold. Nanomedicine 3(4), 311–321
cord. Biomaterials 27(11), 2370–2379 (2006) (2007)
96. W.M. Tian, S.P. Hou, J. Ma, C.L. Zhang, Q.Y. Xu, 109. V.M. Tysseling-Mattiace, V. Sahni, K.L. Niece, D.
I.S. Lee, H.D. Li, M. Spector, F.Z. Cui, Hyaluronic Birch, C. Czeisler, M.G. Fehlings, S.I. Stupp, J.A.
acid-poly-D-lysine-based three-dimensional hydro- Kessler, Self-assembling nanofibers inhibit glial scar
gel for traumatic brain injury. Tissue Eng. 11(3–4), formation and promote axon elongation after spinal
513–525 (2005) cord injury. J. Neurosci. 28(14), 3814–3823 (2008)
97. R. Bellamkonda, J.P. Ranieri, P. Aebischer, Laminin 110. E.C. Tsai, P.D. Dalton, M.S. Shoichet, C.H. Tator,
oligopeptide derivatized agarose gels allow three- Synthetic hydrogel guidance channels facilitate re-
dimensional neurite extension in vitro. J. Neurosci. generation of adult rat brainstem motor axons after
Res. 41(4), 501–509 (1995a) complete spinal cord transection. J. Neurotrauma
98. R. Bellamkonda, J.P. Ranieri, N. Bouche, P. Aebis- 21(6), 789–804 (2004)
cher, Hydrogel-based three-dimensional matrix for 111. Y. Katayama, R. Montenegro, T. Freier, R. Midha,
neural cells. J. Biomed. Mater. Res. 29(5), 663–671 J.S. Belkas, M.S. Shoichet, Coil-reinforced hydrogel
(1995b) tubes promote nerve regeneration equivalent to that
99. X. Yu, G.P. Dillon, R.B. Bellamkonda, A laminin of nerve autografts. Biomaterials 27(3), 505–518
and nerve growth factor-laden three-dimensional (2006)
scaffold for enhanced neurite extension. Tissue Eng. 112. S. Woerly, E. Pinet, L. De Robertis, M. Bousmina,
5(4), 291–304 (1999) G. Laroche, T. Roitback, L. Vargova, E. Sykova,
100. H. Lee, R.J. McKeon, R.V. Bellamkonda, Sustained Heterogeneous PHPMA hydrogels for tissue repair
delivery of thermostabilized chABC enhances ax- and axonal regeneration in the injured spinal cord. J.
onal sprouting and functional recovery after spinal Biomater. Sci. Polym. Ed. 9(7), 681–711 (1998)
cord injury. Proc. Natl. Acad. Sci. U. S. A. 107(8), 113. F.Z. Cui, W.M. Tian, Y.W. Fan, S.P. Hou, Q.Y.
3340–3345 (2010) Xu, I.S. Lee, Cerebrum repair with PHPMA hydro-
101. A. Jain, R.J. McKeon, S.M. Brady-Kalnay, R.V. Bel- gel immobilized with neurite-promoting peptides in
lamkonda, Sustained delivery of activated rho GT- traumatic brain injury of adult rat model. J. Bioact.
Pases and BDNF promotes axon growth in CSPG- Compat. Polym. 18, 413–432 (2003)
rich regions following spinal cord injury. PLoS One 114. S. Woerly, E. Pinet, L. de Robertis, D. Van Diep, M.
6(1), e16135 (2011) Bousmina, Spinal cord repair with PHPMA hydro-
102. F.H. Hausheer, R.L. Schilsky, S. Bain, E.J. gel containing RGD peptides (NeuroGel). Biomate-
Berghorn, F. Lieberman, Diagnosis, management, rials 22(10), 1095–1111 (2001)
and evaluation of chemotherapy-induced peripheral 115. D. Mortisen, M. Peroglio, M. Alini, D. Eglin, Tai-
neuropathy. Semin. Oncol. 33(1), 15–49 (2006) loring thermoreversible hyaluronan hydrogels by
103. S. Koutsopoulos, L.D. Unsworth, Y. Nagai, S. “click” chemistry and RAFT polymerization for cell
Zhang, Controlled release of functional proteins and drug therapy. Biomacromolecules 11(5), 1261–
through designer self-assembling peptide nanofiber 1272 (2010)
hydrogel scaffold. Proc. Natl. Acad. Sci. U. S. A. 116. N. Comolli, B. Neuhuber, I. Fischer, A. Lowman, In
106(12), 4623–4628 (2009) vitro analysis of PNIPAAm-PEG, a novel, injectable
104. H. Cao, T. Liu, S.Y. Chew, The application of nanofi- scaffold for spinal cord repair. Acta Biomater. 5(4),
brous scaffolds in neural tissue engineering. Adv. 1046–1055 (2009)
Drug Deliv. Rev. 61(12), 1055–1064 (2009) 117. L.L. Jones, M. Oudega, M.B. Bunge, M.H. Tuszyn-
105. A.E. Wurmser, K. Nakashima, R.G. Summers, N. ski, Neurotrophic factors, cellular bridges and gene
Toni, K.A. D’Amour, D.C. Lie, F.H. Gage, Cell therapy for spinal cord injury. J. Physiol. 533(Pt 1),
fusion-independent differentiation of neural stem 83–89 (2001)
cells to the endothelial lineage. Nature 430(6997), 118. C.E. Schmidt, J.B. Leach, Neural tissue engineering:
350–356 (2004) Strategies for repair and regeneration. Annu. Rev.
106. Y. Ziv, H. Avidan, S. Pluchino, G. Martino, M. Biomed. Eng. 5, 293–347 (2003)
Schwartz, Synergy between immune cells and adult 119. R. Heumann, D. Lindholm, C. Bandtlow, M. Meyer,
neural stem/progenitor cells promotes functional re- M.J. Radeke, T.P. Misko, E. Shooter, H. Thoenen,
covery from spinal cord injury. Proc. Natl. Acad. Sci. Differential regulation of mRNA encoding nerve
U. S. A. 103(35), 13174–13179 (2006) growth factor and its receptor in rat sciatic nerve dur-
107. R.G. Ellis-Behnke, Y.X. Liang, S.W. You, D.K. Tay, ing development, degeneration, and regeneration:
S. Zhang, K.F. So, G.E. Schneider, Nano neuro knit- Role of macrophages. Proc. Natl. Acad. Sci. U. S.
ting: Peptide nanofiber scaffold for brain repair and A. 84(23), 8735–8739 (1987)
axon regeneration with functional return of vision. 120. Y. Murakami, S. Furukawa, A. Nitta, Y. Furukawa,
Proc. Natl. Acad. Sci. 103(13), 5054–5059 (2006) Accumulation of nerve growth factor protein at both
108. J. Guo, H. Su, Y. Zeng, Y.X. Liang, W.M. Wong, rostral and caudal stumps in the transected rat spinal
R.G. Ellis-Behnke, K.F. So, W. Wu, Reknitting cord. J. Neurol. Sci. 198(1–2), 63–69 (2002)
the injured spinal cord by self-assembling peptide
121. M. Oudega, T. Hagg, Nerve growth factor promotes 135. J.H. Ye, J.D. Houle, Treatment of the chronically
regeneration of sensory axons into adult rat spinal injured spinal cord with neurotrophic factors can
cord. Exp. Neurol. 140(2), 218–229 (1996) promote axonal regeneration from supraspinal neu-
122. J. Bloch, E.G. Fine, N. Bouche, A.D. Zurn, P. Ae- rons. Exp. Neurol. 143(1), 70–81 (1997)
bischer, Nerve growth factor- and neurotrophin-3- 136. J.P. Newman, A.N. Verity, S. Hawatmeh, W.E. Fee
releasing guidance channels promote regeneration Jr., D.J. Terris, Ciliary neurotrophic factors enhances
of the transected rat dorsal root. Exp. Neurol. 172(2), peripheral nerve regeneration. Arch. Otolaryngol.
425–432 (2001) Head Neck Surg. 122(4), 399–403 (1996)
123. M.S. Ramer, J.V. Priestley, S.B. McMahon, Func- 137. M. Sendtner, G.W. Kreutzberg, H. Thoenen, Ciliary
tional regeneration of sensory axons into the adult neurotrophic factor prevents the degeneration of mo-
spinal cord. Nature 403(6767), 312–316 (2000) tor neurons after axotomy. Nature 345(6274), 440–
124. E.G. Fine, I. Decosterd, M. Papaloizos, A.D. Zurn, 441 (1990)
P. Aebischer, GDNF and NGF released by synthetic 138. S.W. Levison, M.H. Ducceschi, G.M. Young, T.L.
guidance channels support sciatic nerve regenera- Wood, Acute exposure to CNTF in vivo induces
tion across a long gap. Eur. J. Neurosci. 15(4), 589– multiple components of reactive gliosis. Exp. Neu-
601 (2002) rol. 141(2), 256–268 (1996)
125. I.H. Whitworth, R.A. Brown, C.J. Dore, P. Anand, 139. C.G. Winter, Y. Saotome, S.W. Levison, D. Hirsh, A
C.J. Green, G. Terenghi, Nerve growth factor role for ciliary neurotrophic factor as an inducer of
enhances nerve regeneration through fibronectin reactive gliosis, the glial response to central nervous
grafts. J. Hand Surg. (Br.) 21(4), 514–522 (1996) system injury. Proc. Natl. Acad. Sci. U. S. A. 92(13),
126. N.R. Krenz, S.O. Meakin, A.V. Krassioukov, L.C. 5865–5869 (1995)
Weaver, Neutralizing intraspinal nerve growth factor 140. J.V. Priestley, M.S. Ramer, V.R. King, S.B. McMa-
blocks autonomic dysreflexia caused by spinal cord hon, R.A. Brown, Stimulating regeneration in the
injury. J. Neurosci. 19(17), 7405–7414 (1999) damaged spinal cord. J. Physiol. Paris 96(1–2), 123–
127. M.I. Romero, N. Rangappa, M.G. Garry, G.M. 133 (2002)
Smith, Functional regeneration of chronically in- 141. A. Buj-Bello, V.L. Buchman, A. Horton, A. Rosen-
jured sensory afferents into adult spinal cord af- thal, A.M. Davies, GDNF is an age-specific survival
ter neurotrophin gene therapy. J. Neurosci. 21(21), factor for sensory and autonomic neurons. Neuron
8408–8416 (2001) 15(4), 821–828 (1995)
128. S. Braun, B. Croizat, M.C. Lagrange, J.M. Warter, 142. C.E. Henderson, H.S. Phillips, R.A. Pollock, A.M.
P. Poindron, Neurotrophins increase motoneurons’ Davies, C. Lemeulle, M. Armanini, L. Simmons, B.
ability to innervate skeletal muscle fibers in rat Moffet, R.A. Vandlen, L.C. Simpson, et al., GDNF:
spinal cord–human muscle cocultures. J. Neurol. A potent survival factor for motoneurons present
Sci. 136(1–2), 17–23 (1996) in peripheral nerve and muscle. Science 266(5187),
129. C.E. Henderson, W. Camu, C. Mettling, A. Gouin, 1062–1064 (1994)
K. Poulsen, M. Karihaloo, J. Rullamas, T. Evans, 143. J. Altman, G.D. Das, Autoradiographic and histo-
S.B. McMahon, M.P. Armanini, et al., Neu- logical evidence of postnatal hippocampal neuro-
rotrophins promote motor neuron survival and are genesis in rats. J. Comp. Neurol. 124(3), 319–335
present in embryonic limb bud. Nature 363(6426), (1965)
266–270 (1993) 144. J. Altman, Autoradiographic and histological stud-
130. R. Grill, K. Murai, A. Blesch, F.H. Gage, M.H. ies of postnatal neurogenesis. IV. Cell proliferation
Tuszynski, Cellular delivery of neurotrophin-3 pro- and migration in the anterior forebrain, with special
motes corticospinal axonal growth and partial func- reference to persisting neurogenesis in the olfactory
tional recovery after spinal cord injury. J. Neurosci. bulb. J. Comp. Neurol. 137(4), 433–457 (1969)
17(14), 5560–5572 (1997a) 145. B.A. Reynolds, S. Weiss, Generation of neurons
131. K.M. Giehl, W. Tetzlaff, BDNF and NT-3, but not and astrocytes from isolated cells of the adult mam-
NGF, prevent axotomy-induced death of rat corti- malian central nervous system. Science 255(5052),
cospinal neurons in vivo. Eur. J. Neurosci. 8(6), 1707–1710 (1992)
1167–1175 (1996) 146. L.J. Richards, T.J. Kilpatrick, P.F. Bartlett, De novo
132. Q. Yin, G.J. Kemp, L.G. Yu, S.C. Wagstaff, S.P. Fro- generation of neuronal cells from the adult mouse
stick, Neurotrophin-4 delivered by fibrin glue pro- brain. Proc. Natl. Acad. Sci. U. S. A. 89(18), 8591–
motes peripheral nerve regeneration. Muscle Nerve 8595 (1992)
24(3), 345–351 (2001) 147. J.G. Emsley, B.D. Mitchell, S.S. Magavi, P. Arlotta,
133. M. Sendtner, P. Carroll, B. Holtmann, R.A. Hughes, J.D. Macklis, The repair of complex neuronal cir-
H. Thoenen, Ciliary neurotrophic factor. J. Neuro- cuitry by transplanted and endogenous precursors.
biol. 25(11), 1436–1453 (1994) NeuroRx 1(4), 452–471 (2004)
134. N.M. Oyesiku, J.N. Wilcox, D.J. Wigston, Changes 148. J.W. McDonald, X.Z. Liu, Y. Qu, S. Liu, S.K.
in expression of ciliary neurotrophic factor (CNTF) Mickey, D. Turetsky, D.I. Gottlieb, D.W. Choi,
and CNTF-receptor alpha after spinal cord injury. J. Transplanted embryonic stem cells survive, differen-
Neurobiol. 32(3), 251–261 (1997)
tiate and promote recovery in injured rat spinal cord. 162. Y.D. Teng, E.B. Lavik, X. Qu, K.I. Park, J. Ourednik,
Nat. Med. 5(12), 1410–1412 (1999) D. Zurakowski, R. Langer, E.Y. Snyder, Functional
149. F. Erdo, C. Buhrle, J. Blunk, M. Hoehn, Y. Xia, recovery following traumatic spinal cord injury me-
B. Fleischmann, M. Focking, E. Kustermann, E. diated by a unique polymer scaffold seeded with
Kolossov, J. Hescheler, K.A. Hossmann, T. Trapp, neural stem cells. Proc. Natl. Acad. Sci. U. S. A.
Host-dependent tumorigenesis of embryonic stem 99(5), 3024–3029 (2002)
cell transplantation in experimental stroke. J. Cereb. 163. H. Ohtaki, J.H. Ylostalo, J.E. Foraker, A.P. Robin-
Blood Flow Metab. 23(7), 780–785 (2003) son, R.L. Reger, S. Shioda, D.J. Prockop, Stem/pro-
150. Q. Cao, R.L. Benton, S.R. Whittemore, Stem cell genitor cells from bone marrow decrease neuronal
repair of central nervous system injury. J. Neurosci. death in global ischemia by modulation of inflam-
Res. 68(5), 501–510 (2002) matory/immune responses. Proc. Natl. Acad. Sci. U.
151. M. Romero-Ramos, P. Vourc’h, H.E. Young, P.A. S. A. 105(38), 14638–14643 (2008)
Lucas, Y. Wu, O. Chivatakarn, R. Zaman, N. 164. M.I. Betancur, H.D. Mason, M. Alvarado-Velez, P.V.
Dunkelman, M.A. el-Kalay, M.F. Chesselet, Neu- Holmes, R.V. Bellamkonda, L. Karumbaiah, Chon-
ronal differentiation of stem cells isolated from adult droitin sulfate glycosaminoglycan matrices promote
muscle. J. Neurosci. Res. 69(6), 894–907 (2002) neural stem cell maintenance and neuroprotection
152. J.R. Sanchez-Ramos, Neural cells derived from post-traumatic brain injury. ACS Biomater Sci. Eng.
adult bone marrow and umbilical cord blood. J. 3(3), 420–430 (2017)
Neurosci. Res. 69(6), 880–893 (2002) 165. Y.C. Wang, Y.T. Wu, H.Y. Huang, H.I. Lin, L.W. Lo,
153. R. Galli, A. Gritti, L. Bonfanti, A.L. Vescovi, Neural S.F. Tzeng, C.S. Yang, Sustained intraspinal delivery
stem cells: An overview. Circ. Res. 92(6), 598–608 of neurotrophic factor encapsulated in biodegrad-
(2003) able nanoparticles following contusive spinal cord
154. T. Vierbuchen, A. Ostermeier, Z.P. Pang, Y. Kokubu, injury. Biomaterials 29(34), 4546–4553 (2008)
T.C. Südhof, M. Wernig, Direct conversion of fi- 166. A. Blesch, P. Lu, M.H. Tuszynski, Neurotrophic
broblasts to functional neurons by defined factors. factors, gene therapy, and neural stem cells for
Nature 463(7284), 1035 (2010) spinal cord repair. Brain Res. Bull. 57(6), 833–838
155. Z. Guo, L. Zhang, Z. Wu, Y. Chen, F. Wang, G. Chen, (2002)
In vivo direct reprogramming of reactive glial cells 167. R.J. Grill, A. Blesch, M.H. Tuszynski, Robust
into functional neurons after brain injury and in an growth of chronically injured spinal cord axons
Alzheimer’s disease model. Cell Stem Cell 14(2), induced by grafts of genetically modified NGF-
188–202 (2014) secreting cells. Exp. Neurol. 148(2), 444–452
156. W. Niu, T. Zang, Y. Zou, S. Fang, D.K. Smith, (1997b)
R. Bachoo, C.L. Zhang, In vivo reprogramming of 168. M.H. Tuszynski, K. Gabriel, F.H. Gage, S. Suhr, S.
astrocytes to neuroblasts in the adult brain. Nat. Cell Meyer, A. Rosetti, Nerve growth factor delivery by
Biol. 15(10), 1164 (2013) gene transfer induces differential outgrowth of sen-
157. U. Pfisterer, A. Kirkeby, O. Torper, J. Wood, J. sory, motor, and noradrenergic neurites after adult
Nelander, A. Dufour, A. Björklunder, O. Lindvall, spinal cord injury. Exp. Neurol. 137(1), 157–173
J. Jakobsson, M. Parmar, Direct conversion of hu- (1996)
man fibroblasts to dopaminergic neurons. Proc. Natl. 169. M.H. Tuszynski, D.A. Peterson, J. Ray, A. Baird, Y.
Acad. Sci. 108(25), 10343–10348 (2011) Nakahara, F.H. Gage, Fibroblasts genetically mod-
158. O. Torper, U. Pfisterer, D.A. Wolf, M. Pereira, S. ified to produce nerve growth factor induce robust
Lau, J. Jakobsson, A. Björklund, S. Grealish, M. neuritic ingrowth after grafting to the spinal cord.
Parmar, Generation of induced neurons via direct Exp. Neurol. 126(1), 1–14 (1994)
conversion in vivo. Proc. Natl. Acad. Sci. 110(17), 170. P. Lu, A. Blesch, M.H. Tuszynski, Neurotrophism
7038–7043 (2013) without neurotropism: BDNF promotes survival but
159. M.M. Dong, T.H. Yi, Stem cell and peripheral nerve not growth of lesioned corticospinal neurons. J.
injury and repair. Facial Plast. Surg. 26(5), 421–427 Comp. Neurol. 436(4), 456–470 (2001)
(2010) 171. L.B. Jakeman, P. Wei, Z. Guan, B.T. Stokes, Brain-
160. H.S. Keirstead, T. Ben-Hur, B. Rogister, M.T. derived neurotrophic factor stimulates hindlimb
O’Leary, M. Dubois-Dalcq, W.F. Blakemore, stepping and sprouting of cholinergic fibers after
Polysialylated neural cell adhesion molecule- spinal cord injury. Exp. Neurol. 154(1), 170–184
positive CNS precursors generate both (1998)
oligodendrocytes and Schwann cells to remyelinate 172. D.M. McTigue, P.J. Horner, B.T. Stokes, F.H. Gage,
the CNS after transplantation. J. Neurosci. 19(17), Neurotrophin-3 and brain-derived neurotrophic fac-
7529–7536 (1999) tor induce oligodendrocyte proliferation and myeli-
161. J. Yan, L. Xu, A.M. Welsh, G. Hatfield, T. Hazel, K. nation of regenerating axons in the contused adult rat
Johe, V.E. Koliatsos, Extensive neuronal differenti- spinal cord. J. Neurosci. 18(14), 5354–5365 (1998)
ation of human neural stem cell grafts in adult rat 173. P. Menei, C. Montero-Menei, S.R. Whittemore,
spinal cord. PLoS Med. 4(2), e39 (2007) R.P. Bunge, M.B. Bunge, Schwann cells genetically
modified to secrete human BDNF promote enhanced Logan, A. Baird, Sustained effects of gene-activated
axonal regrowth across transected adult rat spinal matrices after CNS injury. Mol. Cell. Neurosci.
cord. Eur. J. Neurosci. 10(2), 607–621 (1998) 17(4), 706–716 (2001b)
174. D.M. Frim, T.A. Uhler, W.R. Galpern, M.F. Beal, 186. P.A. Dijkhuizen, W.T. Hermens, M.A. Teunis, J. Ver-
X.O. Breakefield, O. Isacson, Implanted fibrob- haagen, Adenoviral vector-directed expression of
lasts genetically engineered to produce brain- neurotrophin-3 in rat dorsal root ganglion explants
derived neurotrophic factor prevent 1-methyl-4- results in a robust neurite outgrowth response. J.
phenylpyridinium toxicity to dopaminergic neurons Neurobiol. 33(2), 172–184 (1997)
in the rat. Proc. Natl. Acad. Sci. U. S. A. 91(11), 187. C. Gravel, R. Gotz, A. Lorrain, M. Sendtner, Ade-
5104–5108 (1994a) noviral gene transfer of ciliary neurotrophic factor
175. D.M. Frim, U. Wullner, M.F. Beal, O. Isacson, Im- and brain-derived neurotrophic factor leads to long-
planted NGF-producing fibroblasts induce catalase term survival of axotomized motor neurons. Nat.
and modify ATP levels but do not affect gluta- Med. 3(7), 765–770 (1997)
mate receptor binding or NMDA receptor expression 188. W.T. Hermens, R.J. Giger, A.J. Holtmaat, P.A. Di-
in the rat striatum. Exp. Neurol. 128(2), 172–180 jkhuizen, D.A. Houweling, J. Verhaagen, Transient
(1994b) gene transfer to neurons and glia: Analysis of ade-
176. C.A. Tobias, J.S. Shumsky, M. Shibata, M.H. noviral vector performance in the CNS and PNS. J.
Tuszynski, I. Fischer, A. Tessler, M. Murray, De- Neurosci. Methods 71(1), 85–98 (1997)
layed grafting of BDNF and NT-3 producing fi- 189. H.J. Federoff, M.D. Geschwind, A.I. Geller, J.A.
broblasts into the injured spinal cord stimulates Kessler, Expression of nerve growth factor in vivo
sprouting, partially rescues axotomized red nucleus from a defective herpes simplex virus 1 vector pre-
neurons from loss and atrophy, and provides limited vents effects of axotomy on sympathetic ganglia.
regeneration. Exp. Neurol. 184(1), 97–113 (2003) Proc. Natl. Acad. Sci. U. S. A. 89(5), 1636–1640
177. J.S. Shumsky, C.A. Tobias, M. Tumolo, W.D. Long, (1992)
S.F. Giszter, M. Murray, Delayed transplantation of 190. M. Yamada, A. Natsume, M. Mata, T. Oligino, J.
fibroblasts genetically modified to secrete BDNF Goss, J. Glorioso, D.J. Fink, Herpes simplex virus
and NT-3 into a spinal cord injury site is associ- vector-mediated expression of Bcl-2 protects spinal
ated with limited recovery of function. Exp. Neurol. motor neurons from degeneration following root
184(1), 114–130 (2003) avulsion. Exp. Neurol. 168(2), 225–230 (2001)
178. A. Blesch, J.M. Conner, M.H. Tuszynski, Modula- 191. P.D. Robbins, S.C. Ghivizzani, Viral vectors for
tion of neuronal survival and axonal growth in vivo gene therapy. Pharmacol. Ther. 80(1), 35–47 (1998)
by tetracycline-regulated neurotrophin expression. 192. J.H. Kordower, J. Bloch, S.Y. Ma, Y. Chu, S. Palfi,
Gene Ther. 8(12), 954–960 (2001) B.Z. Roitberg, M. Emborg, P. Hantraye, N. Deglon,
179. A. Blesch, M.H. Tuszynski, Cellular GDNF deliv- P. Aebischer, Lentiviral gene transfer to the non-
ery promotes growth of motor and dorsal column human primate brain. Exp. Neurol. 160(1), 1–16
sensory axons after partial and complete spinal cord (1999)
transections and induces remyelination. J. Comp. 193. L. Naldini, U. Blomer, F.H. Gage, D. Trono, I.M.
Neurol. 467(3), 403–417 (2003) Verma, Efficient transfer, integration, and sustained
180. A. Hofmann, G.P. Nolan, H.M. Blau, Rapid retro- long-term expression of the transgene in adult rat
viral delivery of tetracycline-inducible genes in a brains injected with a lentiviral vector. Proc. Natl.
single autoregulatory cassette. Proc. Natl. Acad. Sci. Acad. Sci. U. S. A. 93(21), 11382–11388 (1996)
U. S. A. 93(11), 5185–5190 (1996) 194. M. Shimamura, R. Morishita, M. Endoh, K. Os-
181. A. Blesch, M.H. Tuszynski, GDNF gene delivery hima, M. Aoki, S. Waguri, Y. Uchiyama, Y. Kaneda,
to injured adult CNS motor neurons promotes ax- HVJ-envelope vector for gene transfer into central
onal growth, expression of the trophic neuropeptide nervous system. Biochem. Biophys. Res. Commun.
CGRP, and cellular protection. J. Comp. Neurol. 300(2), 464–471 (2003)
436(4), 399–410 (2001) 195. G. Murlidharan, K. Sakamoto, L. Rao, T. Corriher,
182. A. Blesch, M.H. Tuszynski, Transient growth factor D. Wang, G. Gao, P. Sullivan, A. Asokan, CNS-
delivery sustains regenerated axons after spinal cord restricted transduction and CRISPR/Cas9-mediated
injury. J. Neurosci. 27(39), 10535–10545 (2007) gene deletion with an engineered AAV vector. Mol.
183. M. Berry, L. Barrett, L. Seymour, A. Baird, A. Lo- Ther.–Nucleic Acids 5, e338 (2016)
gan, Gene therapy for central nervous system repair. 196. N. Mokarram, A. Merchant, V. Mukhatyar, G.
Curr. Opin. Mol. Ther. 3(4), 338–349 (2001a) Patel, R.V. Bellamkonda, Effect of modulating
184. M.A. Barry, W.J. Dower, S.A. Johnston, Toward macrophage phenotype on peripheral nerve repair.
cell-targeting gene therapy vectors: Selection of cell- Biomaterials 33, 8793–8801 (2012)
binding peptides from random peptide-presenting 197. N. Mokarram, K. Dymanus, A. Srinivasan, J.G.
phage libraries. Nat. Med. 2(3), 299–305 (1996) Lyon, J. Tipton, J. Chu, A.W. English, R.V. Bel-
185. M. Berry, A.M. Gonzalez, W. Clarke, L. Greenlees, lamkonda, Immunoengineering nerve repair. Proc.
L. Barrett, W. Tsang, L. Seymour, J. Bonadio, A. Natl. Acad. Sci. 114(26), E5077–E5084 (2017)
198. N.M. Geremia, T. Gordon, T.M. Brushart, A.A. Al- 208. C.F.V. Latchoumane, L. Jackson, M.S.E. Sendi, K.F.
Majed, V.M. Verge, Electrical stimulation promotes Tehrani, L.J. Mortensen, S.L. Stice, M. Ghovanloo,
sensory neuron regeneration and growth-associated L. Karumbaiah, Chronic electrical stimulation pro-
gene expression. Exp. Neurol. 205(2), 347–359 motes the excitability and plasticity of ESC-derived
(2007) neurons following glutamate-induced inhibition in
199. T. Gordon, E. Udina, V.M. Verge, E.I. de Chaves, vitro. Sci. Rep. 8(1), 10957 (2018)
Brief electrical stimulation accelerates axon regen- 209. W.A. Nix, H.C. Hopf, Electrical stimulation of re-
eration in the peripheral nervous system and pro- generating nerve and its effect on motor recovery.
motes sensory axon regeneration in the central ner- Brain Res. 272(1), 21–25 (1983)
vous system. Mot. Control. 13(4), 412–441 (2009) 210. G.C. Roman, H.K. Strahlendorf, P.W. Coates, B.A.
200. O. Steward, E.M. Schuman, Protein synthesis at Rowley, Stimulation of sciatic nerve regeneration in
synaptic sites on dendrites. Annu. Rev. Neurosci. 24, the adult rat by low-intensity electric current. Exp.
299–325 (2001) Neurol. 98(2), 222–232 (1987)
201. W.M. Grill, M.D. Craggs, R.D. Foreman, C.L. Lud- 211. S. Pockett, R.M. Gavin, Acceleration of peripheral
low, J.L. Buller, Emerging clinical applications of nerve regeneration after crush injury in rat. Neu-
electrical stimulation: Opportunities for restoration rosci. Lett. 59(2), 221–224 (1985)
of function. J. Rehabil. Res. Dev. 38(6), 641–653 212. T. Gordon, T.M. Brushart, K.M. Chan, Augmenting
(2001a) nerve regeneration with electrical stimulation. Neu-
202. W.M. Grill, J.W. McDonald, P.H. Peckham, W. rol. Res. 30(10), 1012–1022 (2008)
Heetderks, J. Kocsis, M. Weinrich, At the interface: 213. T.M. Brushart, Motor axons preferentially reinner-
Convergence of neural regeneration and neural pros- vate motor pathways. J. Neurosci. 13(6), 2730–2738
theses for restoration of function. J. Rehabil. Res. (1993)
Dev. 38(6), 633–639 (2001b) 214. D. Becker, D.S. Gary, E.S. Rosenzweig, W.M. Grill,
203. C.D. McCaig, A.M. Rajnicek, B. Song, M. Zhao, J.W. McDonald, Functional electrical stimulation
Controlling cell behavior electrically: Current views helps replenish progenitor cells in the injured spinal
and future potential. Physiol. Rev. 85(3), 943–978 cord of adult rats. Exp. Neurol. 222(2), 211–218
(2005) (2010)
204. W.F. Agnew, D.B. McCreery, Neural Prostheses: 215. T. Sato, T. Fujikado, T.S. Lee, Y. Tano, Direct ef-
Fundamental Studies (Prentice-Hall, Inc., Engle- fect of electrical stimulation on induction of brain-
wood Cliffs, 1990) derived neurotrophic factor from cultured retinal
205. R.B. Borgens, Electrically mediated regeneration Muller cells. Investig. Ophthalmol. Vis. Sci. 49(10),
and guidance of adult mammalian spinal axons into 4641–4646 (2008a)
polymeric channels. Neuroscience 91(1), 251–264 216. T. Sato, T. Fujikado, T. Morimoto, K. Matsushita,
(1999) T. Harada, Y. Tano, Effect of electrical stimulation
206. B. Fritsch, J. Reis, K. Martinowich, H.M. Schambra, on IGF-1 transcription by L-type calcium channels
Y. Ji, L.G. Cohen, B. Lu, Direct current stimula- in cultured retinal Muller cells. Jpn. J. Ophthalmol.
tion promotes BDNF-dependent synaptic plasticity: 52(3), 217–223 (2008b)
Potential implications for motor learning. Neuron 217. J.L. Goldberg, J.S. Espinosa, Y. Xu, N. Davidson,
66(2), 198–204 (2010) G.T. Kovacs, B.A. Barres, Retinal ganglion cells
207. D.J. Guggenmos, M. Azin, S. Barbay, J.D. do not extend axons by default: Promotion by neu-
Mahnken, C. Dunham, P. Mohseni, R.J. Nudo, rotrophic signaling and electrical activity. Neuron
Restoration of function after brain damage using a 33(5), 689–702 (2002)
neural prosthesis. Proc. Natl. Acad. Sci. 110(52),
21177–21182 (2013)
Index
A Alternating current (AC) stimulation, 272, 279

AAV, see Adeno associated viruses (AAV) Alzheimer’s disease (AD), 442
Ablations, 223, 224 Alzheimer’s Disease Neuroimaging Initiative 1
ACC, see Anterior cingulate cortex (ACC) (ADNI-1), 442, 457
Acetylcholine, 34, 44, 47 AMD/ARMD, see Age-related macular degeneration
Acoustic-resolution PAM (AR-PAM), 362, 363 (AMD/ARMD)
Action potentials, 475 2-Amino-3-(5-methyl-3-oxo-1,2-oxazol-4-yl)propanoic
discharge rates, 34, 39, 52 acid) (AMPA), 488
EPSP, 10 Ampere’s circuital law, 252
generation, 26 Amyotrophic lateral sclerosis (ALS), 132, 141, 153,
modality of pain, 15 163–165, 189
in nerve cells, 2–3 Analog/digital filtering theory, 111
neuronal, 133, 138 Analog front-ends (AFE)
one-way propagation, 3, 4 bandwidth, 79
and short-wave ripples, 84 biopotential signals, 72
waveform, 2, 3 characteristics, 72
Active grounding, 83–84 cross talk, 80
Active motor threshold, 263 IA, 74–77
Activity-dependent plasticity, 136 IDR, 79–80
AD, see Alzheimer’s disease (AD) interference and common-mode rejection, 82–84
ADC, see Analog-to-digital converter (ADC) opamps, 72–74
Adeno associated viruses (AAV), 298, 299, 656 power consumption, 79
ADNI-1, see Alzheimer’s Disease Neuroimaging Analog-to-digital converter (ADC), 67, 84
Initiative 1 (ADNI-1) Anesthesia monitoring, 104, 105
ADPs, see After-depolarizations (ADPs) ANGN, see Artificial neuron-glia network (ANGN)
Adrenal medulla, 34, 35 Anisotropy, 388
Adrenergic synapses, 34 ANN, see Artificial neural networks (ANN)
AE, see Auto-encoder (AE) Anode electrode, 274
AEDs, see Automated external defibrillators (AEDs) ANS, see Autonomic nervous system (ANS)
AEPs, see Auditory evoked potentials (AEPs) ANSYS Maxwell 3D software, 324
AFE, see Analog front-ends (AFE) Anterior cingulate cortex (ACC), 235
Afferents (sensory inputs), 5 Antiepileptic drugs (AEDs), 536, 556
into motor systems Aphasia, 27
CNS, 15 Area under the curve (AUC), 256, 541
GTOs, 17 ARMA, see Autoregressive moving average (ARMA)
muscle spindles, 16–17 AR-PAM, see Acoustic-resolution PAM (AR-PAM)
After-depolarizations (ADPs), 480 Artifact/noise removal, 148–149
After-hyperpolarizations (AHPs), 480 Artificial neural networks (ANN), 434
Age-related macular degeneration (AMD/ARMD), 572, CNN, 549
592–593 definition, 547
AHPs, see After-hyperpolarizations (AHPs) ELM, 547
Airborne sound, 58 LSTM, 549
Alpha desynchronization, 99 MLP/feedforward network, 547
Alpha motor neurons, 208 ReLU, 547
Alpha waves, 44, 45, 99, 100 RNN, 549
ALS, see Amyotrophic lateral sclerosis (ALS) Artificial neuron-glia network (ANGN), 559

B. He (ed.), Neural Engineering, https://doi.org/10.1007/978-3-030-43395-6
670 Index
Ascending reticular activating systems and cerebral cortex, 41

description, 41 Parkinson’s disease, 31
intralaminar midline, 41 symptoms, 31
and limbic, 41–46 and thalamocortical interactions, 234
subthalamus, 41 and thalamus, 23, 24
Astigmatism, 49 Basal gangliathalamo-cortical motor network, 227
Astrocytes, 3, 4 Basic life support (BLS), 116
Astroglia, 3 Bayesian algorithm (BSFE), 321
Asynergia/dyssynergia, 30 Bayesian classifiers, 151
Attention-based BCIs, 165–166 Bayesian neural network (BNN), 559
Attitudinal reflexes, 27 Bayesian theory, 396–397
AUC, see Area under the curve (AUC) BBB, see Blood–brain barrier (BBB)
Auditory evoked potentials (AEPs), 163–165 BCI applications
Auditory sensations, 57–58 neurofeedback, 156
Augmented reality (AR), 156 replacing lost communication, 153
Autocorrelation function, 416 replacing lost motor function and neuroplasticity, 154
Autocorrelation matrix, 109–110 supplementing normal function, 154–156
Auto-encoder (AE) VR and AR, 156
brain image classification, 450–451 BCI performance assessment and training
brain image registration, 451 communication systems, 168
decoder, 449 intra-system and inter-system performance
encoder, 449 comparisons, 168
SAEs, 449, 450 signal acquisition protocols, 167
SCAEs, 450 system performance assessment, 168–170
structure, 449 user performance assessment, 168
unsupervised models, 448, 449 BCIs, see Brain–computer interfaces (BCIs)
Automated external defibrillators (AEDs), 116 BDNF, see Brain-derived neurotrophic factor (BDNF)
Autonomic ganglion, 32, 33 Beamformer techniques, 398
Autonomic nervous system (ANS), 37 Benchtop experimental setups, 66
adrenal medulla, 34, 35 Beta band, 99, 100, 106, 145, 147
autonomic ganglion, 32, 33 Beta oscillations, 232
central organization, 34–35 Beta waves, 44, 45, 99
homeostasis, 32 BF, see Biceps femoris (BF)
neurotransmitters, 32, 34 B-field energy, 249
parasympathetic system, 32, 33 Biceps femoris (BF), 320
sympathetic system, 32, 33 Bidirectional BMI, 215
Autoregression (AR), 98, 149, 418, 512, 544 Biofeedback/neurofeedback machines, EEG
advantageous, 107 absence status epilepticus, 102, 103
characterizing, 107 applications, 103
diagnostic power, 109 brain states and disease conditions, 103
dominant frequency analysis, 107 finger skin resistance/temperature, 102
dominant frequency peaks, 108–109 frequency-following response, 102
FFT, 107, 108 frequency/spectral domain methods, 103
mathematical formulation, 107–108 joint-time frequency methods, 104
modified spectrum, 107, 108 μ-rhythm suppression, 102–104
NS, 109 signal analysis methods, 103
parameters, 108 time-domain measurements, 103
power spectrum, 108–109 Bioinstrumentation design and neuromodulation, 225
Autoregressive feedforward models, 559 Biological computational paradigm, 293
Autoregressive moving average (ARMA), 107 Biological sources, light, 304
Biophysical models, 392, 500
Biopotentials
B anatomical structures, 65–66
Backpropagation, 436, 437 BCI, 65
Band-pass filter, 72, 149–151 cellular-scale resolution, 66
Bandwidth, 66, 69, 73, 76, 79, 81, 82, 85, 98, 107, 111, characteristics, 72
114, 140 ECoG, 66
Basal ganglia EEG, 66
brain stem nuclei, 21 electrode arrays, 66
Index 671
and electrodes (see Electrodes) label-free PACT, 364, 365

EMG, 66 nervous system activity, 8
MEAs, 66 PAM
μECoG, 66 brain vasculature, single-cell resolution, 364
neural instrument, 67 label-free histology, 364
spatial resolution, 66 populations of glial cells, 4
spatiotemporal characteristics, 66, 67 skull, 8
Biot–Savart law, 253 spectral PAT, 366
Biphasic active recharge patterns, 229 and spinal cord, 7–9
Bipolar derivation, 148 three-dimensional spatial resolution, 8
Bipolar/tripolar concentric electrodes, 386 Brain center, 35
Bispectrum, 105 Brain–computer interfaces (BCIs), 65, 86–88, 529
Bistability, 480–481 adaptive controllers and user interface, 137–138
Blindness, 308 applications (see BCI applications)
Blood–brain barrier (BBB), 640 bidirectional, 133
Blood oxygenation level-dependent (BOLD), 140, 143, brain-to-device control, 138
401, 404 clinical and practical validation, 170–171
blood and tissue susceptibility, 334 CNS activity, 133
deoxygenated hemoglobin, 334 components, 135–136
oxygenated hemoglobin, 333 definition, 132–133
synchronization, 334 dependent and independent, 134–135
tissue contrast, 333 EEG, 102, 131–132
BLS, see Basic life support (BLS) efficiency and inefficiency, 172–173
BM, see Boltzmann machine (BM) expectations, 170
BMIs, see Brain-machine interfaces (BMIs) goal, 132
BNN, see Bayesian neural network (BNN) hybrid, 135
BOLD, see Blood oxygenation level-dependent (BOLD) neuromuscular disorders, 131, 132
BOLD response, neural activity output, 133–134
biophysical basis, 334 peer-reviewed, 174
CBF, 335 performance assessment and training (see BCI
CBV, 335 performance assessment and training)
CMRO2 , 334 reciprocal learning, 173–174
elevation, 334 research and development, 136–137
feasibility, 335 signal acquisition (see Signal acquisition)
hemodynamic changes, 335 signal processing (see Signal processing)
BOLD time series analysis signals, 138–139
block-design paradigm, 337 training, 171–172
boxcar function, 339 Brain-controlled FES
correlation-based method, 337 closed-loop iBMI-FES system, 211
experimental paradigm, 337 EMG, 210
least-squares estimation, 339 firing rate, 210
linear time-invariant system, 337 flexible and dexterous movements, 211
mapping brain activations, 337 motor cortex, 210
multivariate pattern analysis, 340 neural activity, 210
neural responses, 339, 340 nonlinear function, 210
neurovascular coupling, 338, 339 paralyzed people, 211
regression parameters, 338 point-to-point movements, 212
stimulation condition, 338 SCI, 212
subject-level statistic, 340 virtual reality, 212
sustained neural response, 340 Brain-controlled robot’s reaching and grasping, 187
visual thalamus, 338 Brain-derived neurotrophic factor (BDNF), 649
voxel index, 337 Brain electric source model, 387
voxel time series, 337 Brain image analysis
Boltzmann constant, 76 cGAN, 455, 456
Boltzmann machine (BM), 559 CycleGAN, 454, 455
Boundary element method, 388, 389 dEa-GAN, 455, 457
Bradykinesia scores, 220 Ea-GANs, 455, 457
Brain GANs, 454–456
ability, 58, 60 gEa-GAN, 455, 457
672 Index
Brain image analysis (cont.) Capsular hemiplegia, 32

PET, 454, 455 Cardiac arrest (CA)
Pix2Pix, 455 AEDs, 116
Brain image augmentation, 457–459 BLS, 116
Brain image classification, 442, 450–451 clinical trials, 117
Brain image pre-processing, 434–435 CPR, 116
Brain image registration, 451 detecting and counting bursts, 118–119
Brain image segmentation, 443–445 EEG and entropy
Brain–machine interfaces (BMIs), 86 burst counts, 119
communication, 189 CPR and ROSC, 119, 120
decoders, 191 framework, 120
description, 186 IQ, 120, 121
iBMI (see Intracortical BMI (iBMI)) qEEG analysis, 120, 121
M1, 193 spiky bursts, 119
reaching and grasping, 188 temporal and spectral indicators, 119
robotic arms and motorized wheelchair, 191 hypoxic-asphyxic, 117–119
sensory feedback, 191 ICDs, 116
speed and accuracy, 191 and infant asphyxic episodes, 117
See also Brain–computer interfaces (BCIs) multi-institutional groups, 116–117
Brain stem neurological complications, 116
anatomical locations, 23 neurological injury, 116
cerebellum and spinal cord, 30, 31 NS, 117–119
classification, 22 orexin, 121–123
decerebrate rigidity, 30 survivors, 116
equilibrium, 22 Cardiopulmonary resuscitation (CPR), 116, 119, 120
lateral vestibular nucleus, 23 CARTOOL, 404
midbrain, 30 Cataracts, 49
muscle/limb movements, 22 Catecholamines, 34
nuclei and functional regions, 22 Causal connectivity
red nucleus, 23 BSMART software package, 427
reticular formation, 23 eConnectome, 427, 428
Brain stimulation applications, 282 Granger causality, 415, 427
Brain stimulation technology, 294 MATLAB-based toolbox, 427
Brain-to-brain communications, 166–167 Causal mapping, 237
Brain waves CBF, see Cerebral blood flow (CBF)
alpha, 44 CBV, see Cerebral blood volume (CBV)
beta, 44 CCD, see Cortical current density (CCD)
characteristics, 43 CDS, see Correlated double sampling (CDS)
delta, 44 Cell membranes, 465, 468
EEG, 43–45 Cellular neuroscience, 466
electrical recordings, 43 Central auditory system, 58–60
theta, 44 Central nervous system (CNS), 132, 173, 640
Broca’s speech region, 27, 28 components/organelles, 1
BS, see Burst suppression (BS) control center, 7
BSMART software package, 427 convergence, 5–6
Burg algorithm, 108 and dendrites, 4
Bursting, 478 divergence, 5–6
Burst suppression (BS) IPSPs, 7
anesthetic-induced, 121 memory and learning, 4
and bursting, 123 nerve cells, 1, 2
detection, 106 sensory axis, 7, 10
EEG, 105 somatovisceral sensibility, 17–18
pattern, 101 synaptic plasticity, 4
synaptogenesis, 4
temporal and spatial facilitation, 6
C Central thermoregulation
CA, see Cardiac arrest (CA) active evaporation, 41
Caenorhabditis elegans, 299, 510, 511 conduction, 41
Canonical HRF, 335 conserve heat, 40
Index 673
convection, 41 ITR, 200

effector responses, 40 joint probability, 199
heat loss, 41 K discrete classes, 196
heat production, 40 MLE, 199
passive evaporation, 41 neural activity, 198, 199
radiation, 41 neuron’s spiking activity, 196
Central vestibular system, 26–27 performance, 200
Centromedian-parafascicular (Cm-Pf), 233 Poisson and conditionally independent, 200
Cerebellum, 186 probability distributions, 197
adiadochokinesia, 30 probability model, 197
afferent connections, 23 single-neuron, 198
anatomical locations, 23 speed accuracy trade-off, 200
asynergia/dyssynergia, 30 spike counts, 197
ataxia, 30 training data, 196
cerebellar cortex, 23 training phase, 196
dysmetria, 30 unambiguous, 197
intention tremors, 30 CLEAR-ECoG, see Ultra-flexible carbon-layered
nystagmus, 30 electrode array ECoG (CLEAR-ECoG)
vertigo, 30 Clinical programming, 225, 234, 238
Cerebral blood flow (CBF), 306, 335, 364 Clonus/tremor reflex, 12
Cerebral blood volume (CBV), 335 Closed-loop DBS, 80
Cerebral metabolic rate of oxygen (CMRO2 ), 334 Closed-loop DBS systems development
Cerebrospinal fluid (CSF), 640 analog-to-digital conversion, 233
CES, see Cranial electrotherapy stimulation (CES) beta oscillations, 232
CFC, see Cross-frequency coupling (CFC) biomarkers characterization, 232
CGH, see Computer-generated holography (CGH) clinical programming, 234
Charge balance, 74, 82, 118, 226 Cm-Pf, 233
Charge transfer resistance, 69 cortical electrodes, 234
Chlamydomonas reinhardtii, 295 dyskinesias, 234
Cholinergic synapses, 34 ECoG, 233
Chondroitin sulfate proteoglycans (CSPGs), 640 EMG, 233
Chronically implanted devices, 224 front-end filtering, 233
Chronic brain recordings, 239 general characteristics, 234
Chronic nerve recordings GPi peak theta activity, 232
ENG recordings, 322 medication refractory movement disorders, 231
fascicles control plantar, 322 movement disorders, 234
GN and TA, 322 neural signals, 234
information transfer rate, 323 NeuroPace device, 233
ITR, 323 next-generation neurostimulators, 232
post-implantation, 323 Parkinsonian beta band, 232
RMS, 322 Parkinson’s disease, 232
SNR ranged, 323 RMS, 234
Ciliary muscle, 49, 50 smarter implantable neurostimulators, 235
Ciliary neurotrophic factor (GDNF), 649 smart system, 231
Circular E-field pattern, 250 stimulation amplitude, 233
Circular/ring array-based PACT (RA-PACT), 357, 358 Closed-loop neurofeedback, 67, 80
Clamping circuits Closed-loop neuroprosthesis, 67
Ag/AgCl filament, 77 Closed-loop retina prostheses, 80
current clamp, 78 Clustered regularly interspaced short palindromic repeats
dynamic clamp, 78–79 (CRISPR), 308
membrane voltage, 77 Cm-Pf, see Centromedian-parafascicular (Cm-Pf)
patch clamp electrode, 77 CMRR, see Common-mode rejection ratio (CMRR)
voltage clamp, 77–78 CNN, see Convolutional neural network (CNN)
Classification decoders CNS, see Central nervous system (CNS)
Bayes’ rule, 198 Cochlea, 56, 57
class prediction, 198 Code-modulated VEP (c-VEP), 163
conditional probability, 198 Cognitive control, 511
decision boundaries, 199 Cognitive tasks, 171–172
Gaussian distribution, 197, 199 Coherence function, 417
674 Index
Coil forces, 251–252 Conventional sinusoidal biphasic stimulator, 250

Coil heating, 251 Convergence, 5–6, 18, 52
Cold and warmth, 39, 40 ConvNet/CNN, 549
Common-mode rejection ratio (CMRR) Convolutional kernels, 438
active grounding, 83–84 Convolutional layers, 438, 440
differential sensing circuit, 82 Convolutional neural network (CNN)
input impedance-boosting techniques, 82–83 brain image
noise and interference, 74 classification, 442
Common spatial pattern analysis (CSP), 149 segmentation, 443–445
Complex intraocular fluid flow mechanism, 568 combination, 441
Complex-valued transformation, 113 convolutional layers, 438
Computational models, 229, 231, 283 deep learning models, 437
Computer-generated holography (CGH), 303, 306 dropout, 437
Conditional GANs (cGANs), 453, 455 FCNs, 439–441
Conductance-based models, 464 fully connected layers, 439
Cone photoreceptors, 51, 52 MICNN, 442
Conserve heat, 40 MLPs, 437, 438
Constant-Q/quality factor analysis, 111 pooling layers, 438
Contemporary DBS therapy, 224 receptive field, 437
Continuous decoders intracortical control residual learning, 439–440
Kalman (see Kalman filter) sparse connectivity, 437, 439
OLE, 203 U-Net, 440–441
PVA, 201–203 weight sharing, 437
trajectory, 200 Core temperature, 39, 46
Continuous wavelet transform (CWT), 110, 538 Correlated double sampling (CDS), 81
analog/digital filtering theory, 111 Cortical control signal, 210
analytic function, 114 Cortical current density (CCD)
bandpass filter functions, 111 Bayesian theory, 396–397
complex-valued transformation, 113 CPI, 393
definition, 110 dipole source models, 393
Fourier transform, 113 DSL, 392
frequency domain, 113 inverse problem, 426
Gaussian function, 111, 113 linear inverse filters, 393–395
Hilbert transforms, 114 regularization parameters, 395–396
larger-scale factor generates, 111 Cortical imaging technique, 405
Morlet wavelet, 111–113 Cortical potential imaging (CPI), 392
mother/basic wavelet, 111 Corticorubral and corticoreticular tracts, 24
notion of scale, 111 Corticospinal and corticobulbar tracts, 24
RID, 114 Cost-sensitive SVMs (CSVMs), 544
smaller-scale function, 111 Cough reflex, 12
smoothed WVD, 114 Coulomb gauge, 253, 254
STFT, 114 Coupled autoregressive models, 419
T-F reciprocity, 112 CPI, see Cortical potential imaging (CPI)
TFRs, 114 CPR, see Cardiopulmonary resuscitation (CPR)
voices, 113 Cranial electrotherapy stimulation (CES), 274, 277, 280
wide frequency range, 113 Craniosacral system, 32, 33
WVD, 114 CRISPR, see Clustered regularly interspaced short
Contralateral hippocampal, 238 palindromic repeats (CRISPR)
Control energy, 509, 510 Crossed extensor reflex, 12–13
Control input, 507 Cross-frequency coupling (CFC), 539–540
Controllability, 507–508, 512, 515 Cross talk, 80
Control muscles, 186 Crystallography, 298
Control neural activity CSF, see Cerebrospinal fluid (CSF)
human brain, 511 CSP, see Common spatial pattern analysis (CSP)
neuronal control, model organisms, 510–511 CSPGs, see Chondroitin sulfate proteoglycans (CSPGs)
Control response, 506–507 CSVMs, see Cost-sensitive SVMs (CSVMs)
Control theory, see Linear systems theory Cuff electrodes, 316
Conventional monophasic stimulator, 250 Cuff transfer function, 326
Conventional sinusoidal biphasic rTMS devices, 250 Current balancing IA, 76–77
Index 675
Current clamp, 78 functional imaging studies, 226

Current control, 281 LFP, 227
Current flow modeling macroscale effect, 226
brain current flow, 283 macroscale electrodes, 226
computational models, 283 Parkinson’s disease, 226
DOF, 284 reported inhibited activity, 226
electrode size and position, 283 stimulation-induced regular pattern, 227
FEM, 284 DC, see Direct current (DC)
field/current density streamlines, 284 DCS, see Direct cortical stimulation (DCS)
focal transcranial stimulation, 284 DCT, see Doppler coherence tomography (DCT)
HD electrodes, 284 dEa-GAN, see Discriminator-induced Ea-GAN
home-based system, 284 (dEa-GAN)
montages and electrode designs, 283 Decerebrate rigidity, 30
MRI, 284 Decoder calibration, 204
neurophysiological and behavioral changes, 283 Deconvolutional layers, 440
volume conduction models, 283 Deep brain stimulation (DBS), 132, 493
Current transfer capacity, 69 applications, 224
Cutaneous receptors, 22 clinical outcomes, 225
Cutaneous thermoreception, 39–40 closed-loop systems (see Closed-loop DBS systems
c-VEP, see Code-modulated VEP (c-VEP) development)
CWT, see Continuous wavelet transform (CWT) contemporary therapy, 224
Cycle GAN (CycleGAN), 454, 455, 457, 458, 460 definition, 223
electrical stimulation, 224
electrode design/innovation, 229–230
D electrophysiological recordings, 225
DBS, see Deep brain stimulation (DBS) emerging indications, 235–236
DBS electrode design/innovation imaging and computational tools, 230–231
computational modeling, 229 implantable components, 225
novel lead designs, 229 instrumentation technology (see DBS instrumentation)
VTA, 229 neurosurgical treatment option, 223
DBS emerging neurological indications nonsurgical approaches, 236–238
ACC, 235 novel temporal patterns, 227–229
Alzheimer’s disease, 235 Parkinson’s disease treatment, 224
associated burdens, 235 scientific and technical advance treatment, 224
food consumption, 236 therapeutic effects, 224
large-scale clinical trials, 236 therapeutic mechanisms, 226–227
OCD, 235 Deep learning
Parkinson’s disease, 235 AE (see Auto-encoder (AE))
potential targets, 235 ANN, 434
PPN and MS, 235 brain image analysis, 434, 459
PTSD, 236 brain image pre-processing, 434–435
surgical target, 235 CNN (see Convolutional neural network (CNN))
TBI, 235 GANs (see Generative adversarial networks (GANs))
DBS imaging and computational tools GPUs, 459
anatomical structures, 230 neural network models, 435–437
cathodic and anodic stimulation, 231 neuroimages, 434
computational models, 231 neuroimaging-based applications, 459
diffusion tensor imaging, 231 RNNs (see Recurrent neural networks (RNNs))
dynamic visualization, 231 semi-supervised, 434
MRI, 231 supervised, 434
DBS instrumentation unsupervised, 434
implantable biomedical, 225 See also Deep neural network
monopolar stimulation, 226 DeepMedic, 443–445
novel waveform patterns, 226 Deep neural network, 149
rechargeable IPGs, 226 Degree of freedom (DoF), 284, 318
regulated voltage stimulation, 226 Delta waves, 44, 45, 99, 107
stimulation waveform, 226 Dendrites, 1, 2, 4
DBS therapeutic mechanisms Dendritic channel expression, 481–482
basal gangliathalamo-cortical motor network, 227 Dendritic excitability, 482
676 Index
Dendritic spines, 4 Dropout, 437

Depolarization, 3, 7, 14, 26, 145, 146 Dry electrode, 276
Depolarizes, 3, 78 DSL, see Dipole source localization (DSL)
Desynchronization, 100, 102, 121 DSP, see Digital signal processing (DSP)
Detecting and counting bursts, 118–119 DTF, see Directed transfer function (DTF)
Detrended fluctuation analysis (DFA), 104–105 DWT, see Discrete wavelet transform (DWT)
Device output commands, 136 Dyadic scheme, 115
Device safety, 252 Dynamical brain networks
Device-to-brain technologies, 134, 135 exogenous control, 501–502
Device voltage compliance, 282 Hodgkin-Huxley model, 499–501
DFA, see Detrended fluctuation analysis (DFA) linear systems, 501
dFC, see Dynamic functional connectivity (dFC) model linearization, 503–505
Diabetic retinopathy, 593–594 neurons, 499
Difference amplifier, 74, 75 realism vs. tractability, 500
Differential-difference amplifier, 77 spatial consideration, 500–501
Digital micro-mirror device (DMD), 303 temporal consideration, 500–501
Digital optical phase conjugation (DOPC), 303, 304 Wilson-Cowan model, 499, 500
Digital signal processing (DSP), 67, 72, 79, 80 Dynamic clamp, 78–79
Dipolar current source, 70 Dynamic functional connectivity (dFC), 448
Dipole models, 380, 387 Dynamic impedance, 282
Dipole source localization (DSL) Dynamic motor signals, 328
brain electric source model, 390
ECD, 390
ECG/MEG, 390 E
vs. electric potential, 391 Ea-GANs, see Edge-aware conditional GAN models
iterative procedure, 391 (Ea-GANs)
LCMV, 392 Ear-EEG, 85
magnetic fields, 391 ECD, see Equivalent dipole model (ECD)
MUSIC, 392 ECG, see Electrocardiography (ECG)
PCA, 391 ECM, see Extracellular matrix (ECM)
spatiotemporal source localization, 391 ECoG, see Electrocorticography/electrocorticogram
spurious dipoles, 391 (ECoG)
statistical parametric maps, 392 μECoG, see Microelectrocortigography (μECoG)
volume conductor properties, 391 ECT, see Electroconvulsive therapy (ECT)
Direct cortical stimulation (DCS), 133 EDA, see Electrodermal activity (EDA)
Direct current (DC), 274 Edge-aware conditional GAN models (Ea-GANs), 455
Directed transfer function (DTF), 423 EEG, see Electroencephalogram (EEG)
Discrete TFRs, 115 EEG applications
Discrete wavelet transform (DWT), 115, 116 BCI, 102
Discriminator-induced Ea-GAN (dEa-GAN), 455, 457 biofeedback machines, 102–104
Disk approximation, 71 CA, 116–123
“Displaced” amacrine cells, 570, 585 epilepsy monitoring, 101, 102
Displacement current, 252 sleep studies, 101–102
Distribution model, 387 EEG bands
Divergence, 5–6 alpha, 99, 100
DLPFC, see Dorsolateral prefrontal cortex (DLPFC) autoregressive frequency, 98
DMD, see Digital micro-mirror device (DMD) beta, 99, 100
DoF, see Degree of freedom (DoF) delta, 99
Dominant frequency, 108–109 gamma, 99–100
Dominant frequency analysis, 107 nonparametric and parametric, 98–99
DOPC, see Digital optical phase conjugation (DOPC) theta, 99
Doppler coherence tomography (DCT), 306 EEG-based BCI systems
Dorsal cochlear nucleus, 58 AEPs, 163–165
Dorsal premotor cortex (PMd), 194 attention-based BCIs, 165–166
Dorsolateral prefrontal cortex (DLPFC), 283 brain-to-brain communications and interactions,
Double-cone coils, 250 166–167
Driven right leg (DRL), 83–84 categorization, 157
DRL, see Driven right leg (DRL) external/exogenous, 157
Index 677
general-purpose software platforms squid giant axon, 477

BCI2000, 157, 158 STOs, 478–480
OpenViBE, 157–158 Electrical impedance tomography (EIT), 84
hybrid, 165 Electrical recordings, 43
internal/endogenous, 157 Electrical synapses, 489
offline analyses, 157 Electric field models, 231
P300, 160, 161 boundary conditions, 255
quadcopter, 159 boundary element, 255
robotic arm in humans, 160 cerebrospinal fluid, 256
SMRs, 158–160 conductivity values, 255
VEPs, 160–163 fundamental limitations, 256–257
EEGLAB, 404 MEG, 256
EEG/MEG MRI, 255
anatomic connectivity measurements, 426 Ohm’s law, 256
brain electrical activity, 383 uniform and isotropic conductivity, 255
causal connectivity, 427 Electric potentials
CCD (see Cortical current density (CCD)) chemical potential, 467
cortical network activity, 426 Nernst-Planck equation, 468
CPI, 392 Nernst potential, 468
deeper sources, 385 passive membrane, 467
dipole source, 383 potassium ions, 467
DSL, 390–392 reversal potential, 468
DTF, 424, 426 Electrocardiography (ECG), 83
ECoG-based connectivity analysis, 426 Electrochemical electrode, 275
eConnectome, 425, 426 Electrochemical reactions, 275
electrical potential, 389 Electroconvulsive therapy (ECT), 272, 274, 276, 279, 285
electrodes, 383 Electrocorticography/electrocorticogram (ECoG), 136
electromagnetic activity, 385 BCI, 86–88
forward solutions, 388–389 biomarkers, 234
imaging modalities, 427 causal connectivity, 425
magnetic field, 389 causal source activity, 424
magnetic flux, 384 cerebral cortex, 66
multichannel data acquisition system, 390 chronically implantable neural interfaces, 86
network connectivity, 425 definition, 392
nonlinear interpolations, 383 DTF, 423, 424
pseudo-colors, 383 ENIAC, 86, 88
sEEG, 385 epidural, 71, 84
source models, 387–389 functional connectivity, 423
spatial resolution, 386 IDTF, 424
temporal components, 383 implantable high-density, 89
VCD (see Volume current density (VCD)) mECoG, 86
visualization tools, 383 minimally invasive, 86, 87
volume conductor models, 383, 387, 388 motor cortex, 233
volume source scanning method, 426 next-generation neural interfaces and applications, 86,
Efferent connections 88
motor cortex, 24 opportunities, 305
visual cortex and subcortical structure, 53 recordings, 234, 392
Efferent neurons, 5 spatiotemporal resolution, 86
Eigenanalysis, 110 surrogate data method, 423, 424
Eigenvalue decomposition (EVD), 541 3D source spacing, 399
EIT, see Electrical impedance tomography (EIT) time series, 423
Electrical activity transparent, 305
ADPs, 480 Electrode arrays, 294
AHPs, 480 Electrode-based stimulation system, 294
bistability, 480–481 Electrode design, 272
bursting, 478 Electrode-electrolyte double-layer interface, 68
electrophysiological characteristics, 478 Electrode impedances, 282
post-inhibitory rebound spiking, 481 Electrode interfaces, 316
Spike-frequency adaptation, 480 Electrode preparation technique, 275
678 Index
Electrode resistance Electroneurogram (ENG), 315, 319, 322

body resistance, 281 Electrophysiological and imaging methods
electrode-skin resistance, 281 EEG, 262
quality electrode design, 281 EMG, 261
reproducibility and tolerability, 281 epidural recordings, 262
resistance measurement circuit, 281 fMRI and PET, 262–263
skin irritation, 281 fNIRS, 262
Electrodermal activity (EDA), 85 Electrophysiological connectome (eConnectome), 404,
Electrodes, 380 425–428
characterization, 68 Electrophysiological measurements
component, 68 directed cortical interactions, 422–423
current transfer capacity, 69 ECoG, 424
electrode-electrolyte double-layer interface, 68 E/MEG, 424–427
half-cell potential, 69 Electrophysiological source imaging (ESI), 149
impedance anatomical constraint, 405
non-polarizable, Faradaic electrodes, 68, 69 brain electric activity, 404
polarizable, non-Faradaic electrodes, 68, 69 CARTOOL, 404
and instrumentation cortical imaging technique, 405
in-ear placement, 85–86 DSL, 405
scale and invasiveness, 84, 85 EEG (see Electroencephalogram (EEG))
temporal, spatial and spectral resolution, 84 EEGLAB, 404
noise, 69 EEG/MEG mapping (see EEG/MEG)
penetrating, 86, 88 functional constraint, 405
properties, 68 linear inverse filter, 405
signal degradation, 68 MEG (see Magnetoencephalography (MEG))
volume conduction, 70–72 OpenMEEG, 404
water window, 69 regularization technique, 405
Electrode–tissue interface, 226 SL, 386
Electroencephalogram (EEG), 131, 141–143 subject-specific head models, 404
acquired as evoked, 98 temporal constraint, 405
advantages, 98 3D brain tomographic imaging, 405
applications, 101–104 2D cortical imaging, 405
bands, 98–100 Electrophysiology, 69, 83, 85, 133, 138, 143, 144, 149,
brain electrical activity, 380, 382, 383 159, 304, 511
brain waves, 43–45 Electroretinogram (ERG), 595
clinical uses, 98 Element-element interactions, 498
dipole models, 380 ELM, see Extreme learning machines (ELM)
electric potentials, 380 Embedded algorithms, 150
electrodes, 98, 380 EMD, see Empirical mode decomposition (EMD)
electromagnetic fields, 380, 381, 383 EMG, see Electromyography (EMG)
ERPs/ERFs, 382 EMG interference and rejection
event-related potentials, 98 benchtop testing, 319
frequency domain methods, 106–116 BF and ST, 320
HFOs, 381 contamination, 319
IFCN, 97 external shielding, 319
magnetic fluxes, 380 high-pass filter, 320
MEG, 380 interference, 320
montage of electrodes, 98 near-zero correlation coefficients, 320
MVPA, 386, 387 phase digital filter, 320
neuronal synchrony, 380 SNR increasing, open bandwidth, 321
paroxysmal discharges, 101, 102 Empirical mode decomposition (EMD), 544, 545
recordings, 97–98 Encapsulated neural interfacing acquisition chip
signal processing methods, 98 (ENIAC), 86, 88
spatial sampling, 382 Encapsulation, 215
SQUID, 381, 382 Energy efficiency and repetitive TMS, 249–250
time-domain method, 104–106 Energy spectral density, 107
Electrolyte interface, 275 ENG, see Electroneurogram (ENG)
Electromagnetic radiation, 49 ENIAC, see Encapsulated neural interfacing acquisition
Electromyography (EMG), 66, 136, 210, 211, 233, 261 chip (ENIAC)
Index 679
Entropy normalized units, 466

CA, 119–121 passive membrane, 466
IQ, 115–116 lipid membrane, 465
Envelope-extraction method, 149, 151 membrane properties, 465
Epidural ECoG, 84 Excitation states
Epidural recordings, 262 definition, 2, 3
Epilepsy Excitatory post-synaptic potentials (EPSPs), 6, 7, 10
AEDs, 536 Extension leads/connectors, 225
antiepileptic therapies, 536 External stimuli/events, 382
brain Extracellular matrix (ECM), 640
arithmetic progression, 536 Extrafascicular approaches, 316
CWT, 538 Extra-lemniscal system, 18
function, 536 Extreme learning machines (ELM), 558
LFOs, 537 Eye and retina
Scalp EEG, 536 amacrine cells, 569, 570
wavelet function, 538 ARGUS I, 574
CFC, 539–540 ARGUS II, 574
definition, 535 complex intraocular fluid flow mechanism, 568
EEG-based machine learning approaches, 536 cones, 569
machine intelligence, 536 eye disease, 572
model performance, 540–541 fovea, 571
monitoring, 101, 102 glial cells, 570
seizure detection/forecasting (see Seizure ILM, 571
detection/forecasting) Mueller cell end plates, 570
seizures, 536 photoreceptors, 569, 570
WPC, 538–539 retina Implants, 574
EPSPs, see Excitatory post-synaptic potentials (EPSPs) retina lines, 569
Equilibrium retinal prosthesis
central vestibular system, 26–27 artificial vision approaches, 572
macular organs, 25–26 optic nerve prosthesis, 573
secondary sensory cells, 25 retinal implants, 573
semicircular canals, 26 subretinal approach, 573
vestibular organ, 25, 26 signal transmission, 570
vestibular reflexes, 27 subretinal implant system, 575
Equivalent dipole model (ECD), 387, 388, 390 system OFF, 575
ERG, see Electroretinogram (ERG) uvea, 569
ERPs, see Event-related potentials (ERPs) Eye movements, 54, 55
ESI, see Electrophysiological source imaging (ESI)
Essential tremor, 229
Eustachian (auditory) tubes, 56
EVD, see Eigenvalue decomposition (EVD) F
Event-related block-design paradigm, 336 FA, see Factor analysis (FA)
Event-related design, 336 Factor analysis (FA), 526
Event-related desynchronization (ERD), 140, 145, 146 False prediction rate (FPR), 544
Event-related potentials (ERPs), 102, 132, 145–147, 160, Faraday’s electrical constant, 2
165 Faraday’s law, 252
Event-related synchronization (ERS), 140, 145 Fast computational modeling, 229
Evoked potentials/(magnetic) fields/event-related Fast Fourier transform (FFT), 98–99, 107, 108
potentials and/fields (ERPs/ERFs), 382 Fatigue-resistant muscle fibers, 214
Excitability FCNs, see Fully convolutional networks (FCNs)
electric potentials, 467–468 FDA, see Food and Drug Administration (FDA)
Hodgkin-Huxley model (see Hodgkin-Huxley model) FDA-cleared protocols, 264
resting potential, 468–470 Feature extraction, 135
voltage-gated conductances, 470 artifact/noise removal, 148–149
Excitable membranes feature selection and dimensionality reduction,
electrical circuits 150–151
membrane capacitance, 465 methods, 149–150
membrane conductance, 465–466 signal enhancement, 148–149
680 Index
Feature translation parametric, 107–109

algorithm, 136 wavelets, 110–116
algorithms, 151 Frequency-following response, 102
continuous, 152–153 Frequency-modulated VEP (f-VEP), 161
discrete, 153 Frontal eye fields (FEF), 521
Feedback-based neuromodulation system, 316 Front-end amplifier noise model, 80–81
Feed-forward networks, 435, 492 FSL, see FMRIB Software Library (FSL)
Feed-forward programmed movement, 29 FT, see Fourier transform (FT)
FEF, see Frontal eye fields (FEF) Fully connected bidirectional LSTM (Full-BiLSTM)
FEM, see Finite element method (FEM) model, 448
FES, see Functional electrical stimulation (FES) Fully connected layers, 439
FES electrodes, 209 Fully convolutional networks (FCNs), 439, 445
FFT, see Fast Fourier transform (FFT) Fully differential capacitively coupled amplifier, 75, 76
Fiber numerical aperture, 302 Fully differential operational amplifier, 74
Fight/flight reaction, 34 Functional connectivity
Filter algorithms, 150 coupled autoregressive models, 419
FINE, see Flat interface nerve electrode (FINE) Granger causality (see Granger causality)
Fingerprint, 192 stochastic processes (see Stochastic processes)
Finite element method (FEM), 72, 284, 325, 326, 388, variable model, 419
390, 392 Functional electrical stimulation (FES), 154, 188
Finite element model, 324 alpha motor neurons, 208
First-order low-pass filter, 259 biphasic/charge-balanced pulses, 209
Fisher discriminants, 151 brain-controlled (see Brain-controlled FES)
Flaccid paralysis, 32 challenges, 212–214
Flat interface nerve electrode (FINE), 317–319 cortical control signals, 209, 210
Flexible polymer-based multielectrode arrays, 193 electrodes, 209
Flexor hyperactivity, 29–30 motor unit, 209
Flexor reflex, 12 myoelectric control, 209
Flicker noise, 81 neuromuscular junction, 209
fMRI, see Functional magnetic resonance imaging neuromuscular stimulation, 208
(fMRI) requirements, 208
FMRIB Software Library (FSL), 435 stimulation amplitude, 209
FMRI naturalistic paradigm surface stimulation, 209
auditory stimulation, 344 waveform patterns, 209
challenges and opportunities, 345 Functional imaging, 226, 307, 415, 416, 433, 457
inter-subject functional connectivity analysis, 344 Functional magnetic resonance imaging (fMRI), 8, 138,
model-based analysis, 344 143–144, 262, 263, 401–404
naturalistic stimuli, 344 emphasis, 332
natural visual paradigm, 344 functional mapping, 341–342
reproducibility, 344 HRF (see Hemodynamic response function (HRF))
stimulus-response relationship, 344 human psychological and cognitive studies, 332
yielding inter-subject functional connectivity, 344 naturalistic paradigm (see fMRI naturalistic paradigm)
fNIRS, see Functional near-infrared spectroscopy resting state, 342–343
(fNIRS) time series analysis (see BOLD time series analysis)
1/f noise, 81 Functional motor and sensory connections, 316
Focal underdetermined system solution (FOCUSS), 400 Functional near-infrared spectroscopy (fNIRS), 138, 144,
Focused transducer, 361 262
Focused ultrasound, 236–237 f-VEP, see Frequency-modulated VEP (f-VEP)
FOCUSS, see Focal underdetermined system solution
(FOCUSS)
Food and Drug Administration (FDA), 226 G
Forward problem, 388–389 Galvanostat, 78
Fourier transform (FT), 113 Gamma/alpha fibers, 326
autocorrelation function, 106 Gamma band, 99–100
FFT, 98–99, 107, 108 Ganglion cell models
FPR, see False prediction rate (FPR) complex, 609–610
Free electrolyte, 276 Gaussian center-surround model, 608–609
Frequency domain methods Gaussians model vs. receptive field, 605–608
nonparametric spectral methods, 106–107 multielectrode recordings, 610–613
Index 681
systems analysis, 603–604 functional connectivity, 415

types, 613–614 lesion techniques, 421
X/Y cells in cat, 604–605 LFP, 421
GANs, see Generative adversarial networks (GANs) neural transmission, 415
Gastrocnemius (GN), 322 neuronal ensembles, 418
Gated recurrent unit (GRU), 447, 550 neuronal interactions, 415
Gating, 2, 5 5-node oscillatory network, 420
Gaussian center-surround model, 608–609 pairwise analysis, 420
Gaussian distribution, 197 spectral measure, 419
Gaussian function, 111, 113 spectral power, 419
Gaussian noise, 204, 326 spectral representations, 419
Gaussian process factor analysis (GPFA), 526 stationary time series, 418
Gauss’s law, 254 stochastic processes, 415
GDNF, see Glial cell line-derived growth factor (GDNF) time series, 419
Gene c-fos–a marker, 237 variable model, 419–421
Gene delivery mechanism, 298–299 Green function, 352
Generalized cross-validation (GCV) method, 396 GRU, see Gated recurrent unit (GRU)
Generalized linear model (GLM), 485, 487, 526 GTOs, see Golgi tendon organs (GTOs)
biophysically based models, 48 Gyromagnetic ratio, 332
LNP, 487
neuronal spiking activity, 486
neurophysiological data, 486 H
Poisson process, 486 Hairy skin, 20
processing stages, 486 Half-cell potential, 69
simple models, 485 Halobacterium salinarum, 295
spike history dynamics, 486–487 Hand-muscle MEP, 263
Generative adversarial networks (GANs) HDE, see Humanitarian device exemption (HDE)
brain image Heating function, 352
analysis, 451, 454–457 Heat production, 40
augmentation, 457–459 Helmholtz layer, 69
cGANs, 453, 455 Hemodynamic response function (HRF)
CNN-based deep learning models, 452 block-design paradigm, 336
CycleGAN, 454 BOLD response prediction, 336
discriminative models, 452 BOLD signal, 335
Pix2Pix, 453–454 boxcar function, 336
unsupervised learning, 452 event-related design, 336
GENESIS, 493 event-unrelated signal, 336
Genetically encoded photoactuators., 298 MATLAB-based SPM software, 335
Genetic engineering methods, 298 M-sequence, 336
GFAP, see Glial fibrillary acidic protein (GFAP) neurovascular coupling, 335
Glabrous (hairless), 20 overlapping responses, 336
Glaucoma, 50, 593 statistical parametric mapping, 336
Glia cells, 3 Hemoglobin, 356
Glial cell line-derived growth factor (GDNF), 649 Henneman’s size principle, 212
Glial cells, 4 H-FCN, see Hierarchically fully convolutional network
Glial fibrillary acidic protein (GFAP), 640 (H-FCN)
GLM, see Generalized linear model (GLM) HFOs, see High-frequency oscillations (HFOs)
Globus pallidus interna (GPi), 223 HI, see Hypoxic ischemic (HI)
Glycosaminoglycans (GAGs), 640 Hidden Markov model (HMM), 151, 553
GN, see Gastrocnemius (GN) Hierarchically fully convolutional network (H-FCN),
Goldman-Hodgkin-Katz (GHK) equation, 468 442, 443
Golgi tendon organs (GTOs), 17, 22 HIFU, see High-intensity focused ultrasound (HIFU)
GPFA, see Gaussian process factor analysis (GPFA) High-definition (HD) electrodes
GPi, see Globus pallidus interna (GPi) EEG electrodes, 281
Granger causality electrolyte spread, 281
alpha oscillations, 421, 422 interior dimensions, 280
autoregressive models, 418 montage, 280
electrophysiological measurements (see skin contact, 280
Electrophysiological measurements) tDCS waveforms, 280
682 Index
High-density microelectrodes, 84 optogenetic neural probes, 304–305

High-frequency oscillations (HFOs), 381 two-photon optogenetic stimulation, 305–306
High-intensity focused ultrasound (HIFU), 236, 237 Hyperopia, 49
High-pass filter, 72 Hyperpolarization, 474, 475, 478
High SNR amplification, neural signals Hyperthermia, 39
chronic recording, 318 Hypokinetic/hyperkinetic symptoms, 234
electrode arrays, 319 Hypothalamo-pituitary system, 36
FINE, 318 Hypothalamus
hybrid bionic systems, 318 afferent and efferent pathways/connections, 35, 36
mainstream neural amplifiers, 318 cardiovascular responses, 37
novel design implementation, 318 functional organization, 36, 38
OTAs, 319 inferior brain, 35
ultralow-noise neural amplifier, 319 medial regions, 35
High-speed sensorimotor interactions, 173 neuroendocrine interface, 36
Hilbert transforms, 114 spinobulboreticular pathways, 35, 36
Hippocampal neurons, 237 upper brain stem and limbic system, 35
Hjorth descriptors, 105, 106 Hypothermia, 39
HMM, see Hidden Markov model (HMM) Hypoxic-asphyxic cardiac arrest, 117–119
Hodgkin-Huxley model Hypoxic ischemic (HI), 109
action potential, 475
conductance-based models, 464
I
depolarization, 474
IA, see Instrumentation amplifiers (IA)
hyperpolarization, 474, 475
iBMI, see Intracortical BMI (iBMI)
ionic conductances, 471, 475
iBMI-controlled FES, 188
ionic currents, 475
iBMI-FES system, 211
K+
ICA, see Independent component analysis (ICA)
conductance, 471–472
ICDs, see Implantable cardioverter defibrillators (ICDs)
current, 472–473
ICs, see Integrated circuits (ICs)
membrane capacitance, 473
IDR, see Input dynamic range (IDR)
Na+
IDTF, see Integrated DTF (IDTF)
conductance, 471–472
IEDs, see Interictal epileptiform discharges (IEDs)
current, 472–473
iEEG, see Intracranial EEG (iEEG)
neuronal models, 470
IFCN, see International Federation of Clinical
normalized units, 473–474
Neurophysiology (IFCN)
propagating activity, 475–477
ILM, see Internal limiting membrane (ILM)
refractory period, 475
Image reconstruction, 353–354
space clamp, 470–471
Imaging-based measurements, 433
squid giant axon, 470, 475
Imaging methods
threshold, 475
MRI, 98
time evolution, 473
PET, 98
voltage clamp, 470–471
IMFs, see Intrinsic mode functions (IMFs)
Hodgkin–Huxley-type equations, 260
Impedance, 281
Homeostasis
Implantable cardioverter defibrillators (ICDs), 116
cardiovascular responses, 36, 37
Implantable pulse generator (IPG), 225, 226
definition, 35
Implanted electrode array, 225
hormone system, 36
Implanted electrodes, 232
multilevel/multi-hormone feedback mechanisms, 36
Impulse response, 505–506
structures, 37
Incremental membrane polarization, 285
HRF, see Hemodynamic impulse response function
Independent component analysis (ICA), 149, 343
(HRF)
Indium tin oxide (ITO), 305
Human cerebral cortex, 7
Induced E-field waveform, 251
Humanitarian device exemption (HDE), 235
Induced electric field
Human nervous system, 1, 2
boundary condition, 255
anatomy and functioning, 8
charge accumulation, 254
Hybrid imaging technique, 350
closed-form solution, 254
Hybrid platforms
conductivity boundaries, 254
OCT, 306–307
Coulomb gauge, 253
ofMRI, 307–308
Gauss’s law, 254
optical/nonoptical recording, 304
homogeneous medium, 253
Index 683
Laplace’s equation, 254 BrainGate, 187

Lenz’s law, 254 building better electrodes, 215–216
normal component, 255 components, 189–191
Ohm’s law, 254 electrode implementation, 186
symmetric geometry, 255 extrinsic motor control, 187
In-ear placement, 85–86 feasibility, 189
“Informational lesioning” hypothesis, 227 FES (see Functional electrical stimulation (FES))
Information quantity (IQ), 115–116, 120, 121 goal, 188
Information transfer rate (ITR), 161, 163, 168, 169, 200, inputs, 191
323 kinematic control signals, 207
Infrared (IR), 305 neural activity, 188
Inhibitory postsynaptic potentials (IPSPs), 7 paralysis, 188
Inner product, 509, 510 rat motor cortex, 187
Input dynamic range (IDR), 79–80 recordings (see Neural signal recordings)
Input impedance-boosting techniques, 82–83 restoring somatosensory feedback, 215
Instrumentation amplifiers (IA) Intracortical spelling devices, 196
architectures, 74, 75 Intracranial EEG (iEEG), 544
CMRR, 74 Intrafascicular interface, 316
current balancing, 75–77 Intraocular pressure (IOP), 593
difference amplifier, 74, 75 Intrinsic and extrinsic motor control variables,
differential-difference amplifier, 77 187
fully differential capacitively coupled amplifier, 75, Intrinsic mode functions (IMFs), 545
76 Intrinsic time-scale decomposition, 546
high input impedance, 74 Invasive brain stimulation techniques, 285
3-opamp, 74–76 Invasive techniques
switched-capacitor, 75, 76 complementary imaging techniques, 140
Integrate-and-fire models cortical surface, 140–141
conductances, 482 intracortical, 140
LIF, 482–484 Inverse AR filtering, 107
neuron, 482 Inverse estimation techniques
passive membrane, 482 beamformer techniques, 398
QIF, 484 ECoG, 399
RIF, 484 LWMN, 398
simple models, 484–485 MN, 397
Integrate-and-fire neural model, 259, 260 MUSIC algorithm, 398
Integrated circuits (ICs), 84 sEEG, 399
Integrated DTF (IDTF), 424 SOZs, 398
Integrating electromagnetic and hemodynamic imaging 3D source imaging techniques, 398
adaptive Wiener filter, 403 WMN, 397
CCD, 403 Inverse problem, 386, 389, 392, 393, 395,
EEG/MEG source imaging, 401 399, 402
fMRI, 401–404 Inverse reconstruction methods, 354, 357
neuronal activity, 401 Inverting and non-inverting amplifier, 74
Interference, CMRR, 82–84 Inverting operational amplifiers, 73–74
Interictal epileptiform discharges (IEDs), 537 In vitro applications, 298
Internal limiting membrane (ILM), 571 Ionic conductances, 471
International Federation of Clinical Neurophysiology Ionotropic synaptic transmission, 488
(IFCN), 97 IOP, see Intraocular pressure (IOP)
International 10/20 system, 141, 142 IPSPs, see Inhibitory postsynaptic potentials (IPSPs)
Interneurons, 5 IQ, see Information quantity (IQ)
Inter-pulse interval, 263 IQ values, 122
Inter-subject functional connectivity analysis, 344 IR, see Infrared (IR)
Interventional procedures, 294, 295 Iris, 49
Interventional psychiatry, 293 ITO, see Indium tin oxide (ITO)
Intracellular recording and clamping circuits, ITR, see Information transfer rate (ITR)
77–79
Intracortical BMI (iBMI)
ALS, 189 J
basic science, 216 Joint receptors, 22
684 Index
K LIFE, see Longitudinal intrafascicular electrode (LIFE)

Kalman filter Ligand-gated ion channels, 2
additive Gaussian noise, 204 Light delivery mechanism
arm moving, 203 biological sources, 304
constraints, 203 light-guiding systems, 300–302
covariance matrix, 204 light-tissue interaction, 299–300
decoder calibration, 204, 207 spatial light modulators, 302–304
explicit assumptions, 206 Light-guiding systems
Gaussian random variables, 205 fiber numerical aperture, 302
high-performance closed loop iBMI, 206 glass-made capillary, 302
intuitions, 204 Kubelka-Munk model, 301
Kalman gain, 206 LED/SLED, 300
linear Gaussian relationships, 204, 205 MEMS, 302
movement kinematics, 205 no-absorption assumption, 301
movement velocity, 205 optical fibers, 300
multivariate linear regression, 205 practical formulation, 300
Newton’s law, 204 spectral sensitivity, 302
observation model, 205 VoA, 302
one-step prediction, 205, 206 Light-sensitive proteins, 294
position and acceleration, 204 Light-tissue interaction
state model, 204, 205 absorption coefficient, 300
KCL, see Kirchhoff’s current law (KCL) brain tissue optical properties, 300
Kinematic signals, 195 closed-loop control procedures, 300
Kirchhoff’s current law (KCL), 73, 466 DOT, 300
Kronecker delta, 110 electromagnetic theory, 300
Kubelka-Munk model, 301 parameters, 300
protocol/feedback, 299
RTE, 300
L
statistical methods, 300
Label-Free Histology-Like PAM, 364
target area, 299
LA-PACT, see Linear array-based PACT (LA-PACT)
Limbic reticular activating systems, 41–46
Laplacian weighted minimum-norm (LWMN), 397, 398
Limbic systems
Larmor frequency, 332
anatomy, 41
Laser diodes (LD)/super-luminescence light-emitting
brain areas, 41
diodes (SLED), 300
emotional behaviors, 41
Laser pulses, 308
gray matter, 41
Latent factor analysis via dynamical systems (LFADS),
learning and memory, 41, 43
526
monoaminergic system, 41
Lateral geniculate, 53
Limited voltage stimulation, 282
Lateral hypothalamus (LH), 236
Linear array-based PACT (LA-PACT), 357
Lateral intraparietal area (LIP), 528
Linear classifiers, 151
Lateral vestibular nucleus, 23
Linear inverse filters
LCMV, see Linearly constrained minimum variance
EEG/MEG measurements, 393
(LCMV)
electrodes/sensors, 394
Leaky integrate-and-fire (LIF), 483–484
general inverse, 394
Leaky integratormodel, 259
TIK, 395
Leaky ReLu, 444
Linearly constrained minimum variance (LCMV), 392
Learning algorithms, 343
Linear-nonlinear-Poisson (LNP), 486
Learning vector quantization (LVQ), 158
Linear systems theory
Lemniscal system, 18
brain network dynamics, 513
Lens and cornea, 49, 50
cognitive function, 513
Lesion techniques, 421
connectivity, 505, 513
Levinson recursive solution to Yule-Walker equations,
controllability, 507–508, 513
108
controlling dynamical neural systems, 511
LFADS, see Latent factor analysis via dynamical systems
control response, 506–507
(LFADS)
control strategies, 513
LFOs, see Low-frequency oscillations (LFOs)
control theory, 513
LFP, see Local field potential (LFP)
convolution vs. control input, 507
LH, see Lateral hypothalamus (LH)
dimensionality, 512
LIF, see Leaky integrate-and-fire (LIF)
Index 685
disease, 514 HFOs, 381

impulse response, 505–506 MVPA, 386, 387
intervention, 514 neurons, 380
limitation, 512 sensitive magnetic flux detector, 381
minimum energy control, 509–510 SQUID, 382
model validation, 512 Mahalanobis distance-based classifiers, 151
neural activity, 505 Major depressive disorder (MDD), 283
nonlinear dynamical systems, 512 MATLAB-based toolbox, 427
numerical stability, 512 Maximum likelihood estimation (MLE), 199
therapeutic intervention, 505 Maxwell’s equations, 70
LIP, see Lateral intraparietal area (LIP) MCI, see Mild cognitive impairment (MCI)
LLF, see Low-level features (LLF) MDD, see Major depressive disorder (MDD)
L1-norm methods, 399 MEAs, see Multielectrode arrays (MEAs)
LNP, see Linear-nonlinear-Poisson (LNP) Mechanoreception
Local field potential (LFP), 86, 147, 421, 536 definition, 19
band-pass filtering, 193 glabrous (hairless) and hairy skin, 20
membrane potentials, 191 Meissner corpuscles, 20
neural signal, 192 Merkel’s disks, 20
recordings, 227 Pacinian corpuscles, 20
Long-interval intracortical inhibition, 263 properties, 19
Longitudinal intrafascicular electrode (LIFE), 316 Ruffini corpuscles, 20
Longitudinal relaxation, 333 touch points, 19, 20
Long short-term memory (LSTM) model, 446–447, 549 mECoG, see Modular-ECoG (mECoG)
Low-frequency oscillations (LFOs), 537 Medical-grade tES devices, 286
Low-level features (LLF), 450, 451 MEG, see Magnetoencephalography (MEG)
Low-pass filter, 72 Meissner corpuscles, 20
Luciferin, 304 Membrane capacitance, 465
LVQ, see Learning vector quantization (LVQ) Membrane conductance, 465–466
LWMN, see Laplacian weighted minimum-norm Membrane depolarization, 258
(LWMN) Membrane polarization, 285
Membrane voltage, 77
MEMS, see Micro-electromechanical systems (MEMS)
M MEMS-based galvanometers, 303
Machine intelligence, applications MEPs, see Motor-evoked potentials (MEPs)
AED treatments, 556, 557 Merkel’s disks, 20
EEG signals, 554 Metabotropic receptors, 490
SOZ electrodes, 555 Method of Images (MoI), 71, 72
SVM classifier, 554, 555 MICNN, see Multi-instance CNN (MICNN)
Machine learning algorithms, 137–138, 557 Microbial halorhodopsin NpHR, 298
Macular organs, 25–26 Microbial rhodopsin, 295
Magnetic field, 253 Microelectrocortigography (μECoG), 66, 86
Magnetic resonance imaging (MRI), 98, 255 Micro-electromechanical systems (MEMS), 302, 303
bulk magnetization, 333 Microglia, 3
gyromagnetic ratio, 332 Microwires, 193
hydrogen protons, 332 Midbrain, 30
Larmor frequency, 332 Middle temporal (MT) area, 528
longitudinal relaxation, 333 Mild cognitive impairment (MCI), 450
magnetic field, 332 Mindful meditation, 172
NMR, 332 Minimum energy control, 509–510
on resonance, 333 Minimum-norm (MN), 397
RF excitation, 332 Minimum-norm least-squares (MNLS), 394, 395
RF signals, 333 Mitigate flicker noise, 81
spinning protons, 332 Mixture-of-experts model, 543
transverse plane, 332 MLE, see Maximum likelihood estimation (MLE)
transverse relaxation, 333 MLP, see Multilayer perception (MLP)
Magnetoencephalography (MEG), 84, 143, 555 MN, see Minimum-norm (MN)
brain electrical activity, 380, 382 MNLS, see Minimum-norm least-squares (MNLS)
dipole models, 380 Model-based analysis, 344
electrophysiological principles, 381 Model-based time series analysis, 341
external stimuli/events, 382
686 Index
ModelDB, 494 forces and muscle activity, 195

Model linearization M1, 193, 195
controlled neural system, 503 muscle-like input signal, 196
linear system, 503, 504 PMd and PPC, 193, 196
non-linear system, 503 PMv and SMA, 194, 196
Modern DBS therapy, 225 S1, 194
Modular-ECoG (mECoG), 86 velocity/ acceleration, 195
MoI, see Method of Images (MoI) Motor task group, 28
Monoaminergic system, 41 Motor threshold, 263
Monophasic E-field pulses, 257 Mouse thymus cell antigen 1 (Thy1) promoter, 299
Monophasic waveform, 272 Movement intent recovery
Monopolar stimulation, 226 commercial interface, 317
Monopole model, 387 DoF, 318
Monosynaptic stretch reflex, 10–12 EMG-controlled/myoelectric device, 317
Morlet wavelet, 111–113 fascicle activity, 318
MOS transistor, 81 FINE, 317
Motor aphasia, 27 ideal motor interface, 318
Motor control, 186 limb injuries, 317
afferents (sensory inputs) prosthetic control, 317
CNS, 15 SNR, 318
GTOs, 17 wrapping electrodes, 318
muscle spindles, 16–17 MRI, see Magnetic resonance imaging (MRI)
basal ganglia, 24, 25 MS, see Multiple sclerosis (MS)
brain stem, 22–23 M-sequence, 336
cerebellum, 23 mu-band, 99
motor cortex, 23–24 Multielectrode arrays
motor neuron recruitment, 28–29 flexible polymer-based, 193
pathophysiology, 29–32 microwires, 193
relative hierarchical/functional organization, 21–22 recording technology, 193
speech, 27–28 silicon-based arrays, 193
spinal cord, 22 Multielectrode arrays (MEAs), 66
thalamus, 24, 25 Multi-instance CNN (MICNN), 442
voluntary movement, 21–22 Multilayer perception (MLP), 435, 436
Motor cortex, 186, 193 Multimodal imaging, 402
anatomical locations, 23 Multiple sclerosis (MS), 235
capsular hemiplegia, 32 Multiple signal classification (MUSIC) algorithm,
corticorubral and corticoreticular tracts, 24 109–110, 392, 398
corticospinal and corticobulbar tracts, 24 Multiple signaling pathways, 307
definition, 23 Multiscale imaging, 356
efferent connections, 24 Multiunit activity, 191, 192
motor abilities, 31 Multivariate autoregressive (MVAR), 420, 423
premotor areas, 24 Multivariate pattern analysis (MVPA), 386, 387
primary, 24 μ-rhythm suppression, 102–104
stroke, 32 Muscarinic, 34
supplementary motor cortex area, 24 Muscle-based skills, 173, 174
Motor-evoked potentials (MEPs), 261–264, 267 Muscle spindles, 16–17, 22
Motor imagery, 171, 173 MVAR, see Multivariate autoregressive (MVAR)
Motor neuron recruitment MVPA, see Multivariate pattern analysis (MVPA)
development of motor skill, 28–29 Myelin, 3
feed-forward programmed movement, 29 Myoelectric control, 209
inverse function, 28 Myopia, 49
motor task group, 28
muscle/limb forces, 28
sensory feedback, 29 N
size principle, 28 NA, see Numerical aperture (NA)
Motor neurophysiology NAcc, see Nucleus accumbens (NAcc)
BMI output device, 195 National Heart, Lung, and Blood Institute (NHLBI),
controlled stimulation, 195 116–117
firing rate, 195 National Institute of Neurological Disorders and Stroke
(NINDS), 117
Index 687
National Institutes of Health (NIH), 116 excitable membranes (see Excitable membranes)
Natural recruitment order, 213 generalized linear model, 485–486
Natural sensory and motor fibers, 318 GENESIS, 493
NDS, see Neurological deficit score (NDS) Hodgkin-Huxley model (see Hodgkin-Huxley model)
Near-infrared (NIR), 357 integrate-and-fire models (see Integrate-and-fire
Negative feedback control system, 37, 38, 41, 42 models)
Neocortical pyramidal neurons, 482 laudable approach, 487
Nernst equation, 2 medicine, 493
Nernst-Planck equation, 468 ModelDB, 494
Nernst/reversal potential, 468, 471, 473, 488 NEST, 493
Nerve-electrode interface, 317 NEURON, 493
Nerve growth factor (NGF), 649 neurons, 487
Net noise contributions, 81–82 nonlinear dendritic process
Neural activation models dendritic channel expression, 481–482
biphasic stimulation, 260 dendritic excitability, 482
cell geometry, 260 short-term synaptic plasticity, 490
E-field waveform, 260 single neurons (see Single neurons)
Hodgkin–Huxley-type equations, 260 synapse models, 488–490
membrane depolarization, 259, 260 Neural network (NN), 149, 151, 296
predictions, 259 activation function, 436
rheobase, 259 backpropagation, 436, 437
second-order partial differential equation, 260 feed-forward neural network, 435
Neural activities, PAT mini-batch size, 437
imaging brain diseases, whole brain level, 366, 367 MLP, 435, 436
imaging brain glucose metabolism, 367–368 neurons, 435
large-scale networks, 366 SGD, 437
monitoring brain hemodynamic responses, 366 Neural recording and stimulation, 70–72
PACT, 370, 371 Neural recording properties, FINE
TAT, 371 EMG contamination, 323
voltage-/calcium-sensitive indicators, 368–370 fascicles, 323
Neural depolarization, 249 finite element model, 324
Neural engineering motor neural activity, 323
EEG, 86, 88 NEURON model, 324–325
electrodes and instrumentation, 84–86 open bandwidth neural, 323
experiment and biological model, 85 representative spectra, 326–327
penetrating electrodes, 86, 88 single fiber action potential, 325–326
Neuralink, 216 Neural signal recordings
Neural instrument LFP, 191, 192
AFE, 67 multiunit activity, 191, 192
closed-loop neuroprosthesis, 67 single-unit activity, 191, 192
DSP, 67 Neural signals, 190, 195
neurofeedback capabilities, 67 ERPs, 146–147
wireless closed-loop, 67 P300 ERP, 145–147
Neural interfaces SCP, 145, 146
ADC, 67 SMRs, 144–145
AFE, 72–77 spikes and local field potentials, 147
BCIs (see Brain–computer interfaces (BCIs)) Neural systems, 500
DSP, 67 Neuroendocrine interface, 36
intracellular recording and clamping circuits, Neurofeedback, 135, 156
77–79 Neuroimaging, PAT
Neural modeling brain (see Brain)
BRIAN, 495 neurological diseases, 363
CellML, 496 non-optical imaging modalities, 364
computational models, 463, 464 Neurological deficit score (NDS), 117, 118
databases, 495, 496 Neurological diseases, 363
dynamical systems models, 489 Neurological injury/disorders, 186
electrical activity (see Electrical activity) Neuromodulation, 134, 135, 235, 280, 282, 295
electrophysiological behaviors, 487 Neuromuscular junctions, 10, 12, 208
excitability (see Excitability) Neuronal action potentials, 138
688 Index
Neuronal circuits homeostasis, 35–37

antagonist inhibition, 8–9, 11 hypothalamus, 35–37
feedback inhibition, 9, 11 maintenance of posture (see Posture)
forward inhibition, 9, 11 motor system (see Motor control)
lateral inhibition, 9, 11 neuronal circuits (see Neuronal circuits)
positive feedback, 9 neurons (see Neurons)
reflexes (see Reflexes) pain, 46–49
synaptic depression, 9 reticular activating systems, 41–46
synaptic potentiation, 9 sensory systems (see Sensory systems)
Neuronal control somatovisceral sensibility, 17–20
C. elegans, 510, 511 sound and hearing, 54–58
sensory receptor neurons, 511 synapses (see Synapses)
simple model, 510 thermoregulation, 37–41
structural controllability, 510 vision, 49–55
Neuronal membrane depolarization Neuropixels, 88
assumptions, 258 Neuroplasticity, 154
cable equation, 258 Neuroprosthesis, 154
cell membranes, 258 Neuroscience, 366
charge accumulation, 258 Neurostimulators, 225, 226
E-field direction, 259 Neurotransmitter acetylcholine, 208
first-order low-pass filter, 259 Neurotransmitters
leaky integratormodel, 259 ANS, 32, 34
membrane depolarization, 258 CNS, 133
pyramidal neurons, 259 dopamine, 31
Neuronal population code presynaptic, 5
BCIs, 529 Neurovascular coupling, 335
correlation, pair of neurons Neutral (comfort) zone, 39
encoding, 522 Next-generation neurostimulators, 232
measures, 524 NGF, see Nerve growth factor (NGF)
noise correlation, 523 NHLBI, see National Heart, Lung, and Blood Institute
sensory neurons, 521 (NHLBI)
tuning curves, 523 Nicotinic, 34
decoding frameworks, 530 NIH, see National Institutes of Health (NIH)
single neurons, 520–521 NINDS, see National Institute of Neurological Disorders
“spikes”, 519 and Stroke (NINDS)
synchrony, pair of neurons NIR, see Near-infrared (NIR)
CCG, 525, 526 NMDA, see N-Methyl-D-aspartic acid (NMDA)
measures, 524 N-Methyl-D-aspartic acid (NMDA), 482, 488, 489, 493
shuffled correlation, 525 NMR, see Nuclear magnetic resonance (NMR)
spiking activity, 525 NN, see Neural network (NN)
Neurons, 435 Nociceptors, 47, 48
action potential, 324 Node analysis technique, 73
afferents, 5 Noise, 55
CNS, 3, 5 AFE, 80
cytoarchitectural organization, 24 in biopotential recording, 80
definition, 2 electrodes, 69
efferents, 5 front-end amplifier noise model, 80–81
interneurons, 5 input referred noise, 80
ion selective channels, 2 net noise contributions, 81–82
membrane currents, 325 SNR, 80
model MRG axon, 325 Noninvasive EEG-based BCI technology, 159
plasma membrane, 1–2 Noninvasive methodology, 238
Ranvier/active membrane, 325 Noninvasive recording technology, 172
receptive fields, 15 Noninvasive techniques
NeuroPace device, 233 EEG, 141–143
Neurophysiology fMRI, 143–144
ANS (see Autonomic nervous system (ANS)) fNIRS, 144
CNS (see Central nervous system (CNS)) MEG, 143
equilibrium (see Equilibrium) Non-inverting operational amplifier, 73
Index 689
Nonlinear decoder, 212 node analysis technique, 73

Nonlinear dendritic process non-inverting, 73
dendritic channel expression, 481–482 Operation transconductance amplifiers (OTAs), 83, 319
dendritic excitability, 482 Opsins
Nonlinear interpolations, 383 allmicrobial rhodopsins, 295
Nonlinear inverse techniques bioprospecting and searching, 296
L1-norm methods, 399, 401 Ca2+, 298
L2-norm methods, 399, 401 computer-controlled laser module, 296
MEG/EEG measurements, 399–401 crystallography, 298
nonparametric algorithm, 400 freshwater algae, 295
SCEA, 400 high-energy blue wavelengths, 296
3D EEG/MEG, 399 implanting light guides, 296
weighted norm minimization, 400 light sensitivity, 298
Nonparametric spectral methods, 106–107 microbial halorhodopsin, 298
Non-polarizable, Faradaic electrodes, 68, 69 microbial rhodopsin, 295
Nonrapid eye movement (NREM) sleep, 102 photon interaction, 295
Non-REM sleep, 45 photosensitive optogenetic proteins, 295
Nonspecific sensory pathways, 18, 19 SFO, 298
Noradrenaline, 34 single-celled microorganisms, 295
Normalized separation (NS), 109, 117–119 spectral sensitivity, 296
Novel temporal patterns, stimulation in vivo applications, 297
biphasic pulse patterns, 229 Volvox ChR, 296
bradykinesia scores, 228 Optical-acoustic detectors, 353
computational models, 229 Optical coherence tomography (OCT), 582
essential tremor, 227 angiography, 306
impact, 227 DBF, 306
non-regular patterns, 229 DCT, 306
Parkinsonian basal ganglia, 229 monitor vascular dynamics, 307
Parkinson’s diseases, 228 multiple signaling pathways, 307
rechargeable neurostimulators, 229 neurovascular coupling, 306
regular pulse train, 227 optical imaging method, 306
NS, see Normalized separation (NS) Optical contrasts, 350, 357, 364
Nuclear magnetic resonance (NMR), 332 Optical-resolution PAM (OR-PAM), 362, 363, 366–369,
Nucleus accumbens (NAcc), 235, 236 373
Numerical aperture (NA), 300, 301 Optical sensors, 353
Nyquist sampling theorem, 360 Optical stimulation paradigms, 298
Optimal linear estimator (OLE)
arm trajectories, 203
O
assumptions, 203
OAT, see Optoacoustic tomography (OAT)
Bayes’ rules, 203
Obsessive compulsive disorder (OCD), 235
cosine tuning model, 203
OCD, see Obsessive compulsive disorder (OCD)
encoding model, 203
OCT, see Optical coherence tomography (OCT)
firing rates, 203
Oddball paradigm, 145, 163
preferred directions, 203
ODEs, see Ordinary differential equations (ODEs)
Optoacoustic tomography (OAT), 349
ofMRI, see Optogenetic functional magnetic resonance
Optogenetic functional magnetic resonance imaging
imaging (ofMRI)
(ofMRI)
OLE, see Optimal linear estimator (OLE)
BOLD signal, 307, 308
Oligodendroglia, 3
coherence tomography, 307
One-dimensional control signals, 209
fMRI imaging, 307
One-step prediction, 205, 206
laser pulses, 308
3-opamp IA, 74–76
light absorption and heat generation, 308
Open-loop transconductance amplifier, 76
neuroscience research, 307
OpenMEEG, 404
OCT, 307
Operational amplifiers (opamps)
thalamus/hippocampus, 307
configurations, 73
Optogenetic neural probes
filtering, 72
CLEAR-ECoG, 305
fully differential amplifier, 74
ECoG, 305
inverting amplifier, 73–74
electrode recording site, 304
KCL, 73
690 Index
Optogenetic neural probes (cont.) neurostimulation, 49

electrophysiology, 304 neurosurgical, 49
fiber electrode assemblies, 305 nociceptors, 47, 48
graphene electrodes, 305 pharmacological, 48
ITO, 305 physical, 48
light-guiding structures, 304 points, 46, 47
NIR and IR, 305 projected pain, 47, 48
signal-to-noise ratio, 305 referred pain, 47, 48
STN, 304 somatosensory modalities, 47
Optogenetics tissue-damaging/noxious, 46
electrode-based brain interface, 295 Paired-pulse paradigms, 263–264
hybrid brain interface platforms, 295 Paired-pulse protocol, 259
optical recording techniques, 295 Pairwise correlations
tools (see Optogenetics tools) dimensionality reduction, 526
Optogenetic stimulation, 296 FA, 526
blindness, 308 machine learning, 526
clinical applications, 308 neurons to behavior, relating
CRISPR, 308 binocular rivalry paradigm, 527
electrode-based brain interface platforms, 308 choice probability, 528
human trials, 308 FA, 527
neurological and psychiatric disorders, 309 GPFA, 529
photodetectors, 309 single-neuron activity, 527
photoreceptor cells, 308 PCA, 526
spatial-temporal stimulation pattern, 308 recording technologies, 527
therapeutic procedures, 308 single-neuron metrics, 526
troubling feature, 308 PAM, see Photoacoustic microscopy (PAM)
Optogenetics tools PA-PACT, see Planar array-based PACT (PA-PACT)
gene delivery mechanism, 298–299 Papez memory circuit, 235
opsins (see Opsins) Parametric (modeling) methods
target species, 299 AR, 107–109
Ordinary differential equations (ODEs), 473, 477, 484, ARMA, 107
548 inverse AR filtering, 107
Orexin MUSIC algorithm, 109–110
administration, 122 standard values, 107
deficiency, 121 Paraplegia, 29
intraventricular injection, 121–122 Parasympathetic postganglionic synapses, 34
IQ values, 122 Parasympathetic system, 32, 33
NDS, 122, 123 Parkinsonian basal ganglia, 229
ORXA, 122, 123 Parkinsonian beta band, 232
post-ROSC, 122 Parkinson’s disease, 31, 223, 227, 228
OR-PAM, see Optical-resolution PAM (OR-PAM) Parkinson’s/electrode arrays, 294
ORXA, 122, 123 Paroxysmal discharges
Oscillations, 417 nonperiodic, 101
Ossicles, 56, 57 nonsinusoidal and periodic, 101, 102
OTAs, see Operation transconductance amplifiers (OTAs) shape characteristics, 101
Oval window, 56 sinusoidal, 101
symmetry of alpha activity, 101
Paroxysmal disorders, 234
P
Parseval’s theorem, 107
Pacemaker, 225
Partial directed coherence (PDC), 423
Pacinian corpuscles, 20
Passive membrane, 466
PACT, see Photoacoustic computed tomography (PACT)
PAT, see Photoacoustic tomography (PAT)
Pain
Patch clamp electrode, 77
classification, 46, 47
Pathophysiology, motor control
description, 46
basal ganglia, 31
detection, 46
brain stem, 30, 31
intensity
cerebellum, 30
effective reactions, 46
deep brain stimulation electrodes, 29
mechanical/thermal/chemical stimuli, 46–47
motor cortex, 31–32
neuralgia, 47, 48
Index 691
spinal cord, 29–30 non-ionizing laser pulses, 349

PA waves, 353 OAT, 349
PCA, see Principal component analysis (PCA) optical excitation
PDC, see Partial directed coherence (PDC) stress relaxation time, 351
Pearson product-moment correlation coefficient thermal relaxation time, 351
(PPMCC), 168 optical imaging, 350
Pedunculopontine nucleus (PPN), 235 PACT (see Photoacoustic computed tomography
Penetrating electrodes, 86, 88 (PACT))
Penetration depth problem, 297 PAM (see Photoacoustic microscopy (PAM))
Performance quality, 79 PA wave, 350
Period-amplitude model, 105 photoacoustic detection, 353
Periodic auto-zeroing, 81 photoacoustic equation, 352
Peripheral nervous system (PNS), 132, 640 scale spatial resolutions, 355
Peripheral paralysis, 30 versatile contrasts, 356
Peristimulus time histograms (PSTHs), 588 Photocurrent responses, 600
Persistent activity Photoreceptors, 584–585
biological system, 494 cone, 51, 52
neurons, 493 rhodopsin, 51
recurrent excitation, 492 rod, 50–52
spike-frequency adaptation, 493 transduction process
PET, see Positron emission tomography (PET) CNS, 53
P300 event-related potential (ERP), 145–147 convergence and divergence, 52
Phase locking value (PLV), 554 definition, 52
Phase-modulated VEP (p-VEP), 163 human brain, 53
Phase-modulating SLMs, 303 lateral geniculate, 53
Photoacoustic computed tomography (PACT) monocularly vs. binocularly vision, 54
array-based PACT system, 361 subcortical structure, 53
inverse reconstruction methods, 357 superior colliculi, 53
LA-PACT, 357 visual cortex, 53
Nyquist sampling theorem, 360 Photosensitive proteins, 296
RA-PACT, 357, 358 Phototransduction, 570
reconstructed images, 361 Piezoelectric-based detectors, 353
SA-PACT, 358, 359 Pink noise, 81
spatial resolution, 360 Pix2Pix GANs, 453–454
spatial sampling frequency, 360, 361 Place theory, 58
Photoacoustic detection, 353 Planar array-based PACT (PA-PACT), 359,
Photoacoustic (PA) effect, 349 360
Photoacoustic microscopy (PAM) Plasma membrane, 1–2
acoustic detection, 361 Plasticity, 4, 490
AR-PAM, 362, 363 PMOS transistor, 81
focused transducer, 361 PNN, see Probabilistic neural network (PNN)
OR-PAM, 362, 363, 373 PNS, see Peripheral nervous system (PNS)
Photoacoustic tomography (PAT) Poikilotherms, 37
biomedical imaging modalities, 350 Point spread function (PSF), 360
classification, 354, 355 Poisson process, 486
computed tomography, 357 Polarizable, non-Faradaic electrodes, 68, 69
endogenous contrasts, 356, 357 Polysynaptic reflexes, 12
exogenous contrast, 357 Pooling layers, 438
forward solution, 352–353 Population vector algorithm (PVA)
hemoglobin, 356 biologically inspired, 202
hybrid imaging technique, 350 classification decoders, 202
image reconstruction, 353–354 cosine-tuned neuron, 201
implementations, 354 description, 201
molecular imaging, 350 neuron’s activity, 201
multiscale imaging, 356 neuron’s firing rate, 201
2-NBDG, 372 neuron’s tuning parameters, 201
neural activities (see Neural activities, PAT) preferred direction, 201
neural imaging (see Neural imaging, PAT) uniform distribution, 202
NIR, 357 weighted vector sum, 201
692 Index
Positron emission tomography (PET), 98, 235, 262, 367, R

454, 455, 457, 458 Radial basis function (RBF), 543, 548
Posterior parietal cortex (PPC), 143, 193, 194 Radiation, 41
Post-inhibitory rebound spiking, 481 Rapid eye movement (REM) sleep, 45–46, 102
Post-resuscitative and initial Utility in Life Saving Efforts RBF, see Radial basis function (RBF)
(PULSE), 116 RBM, see Restricted Boltzmann machines (RBM)
Post-traumatic stress disorder (PTSD), 235 Real-time fMRI (rtfMRI), 144
Postural reflexes, 27 Rear-infrared (NIR), 305
Posture Recalibrated feedback intention-trained Kalman filter
afferent inputs/pathways, 25 (ReFIT-KF), 207
body positions, 24 Receiver operating characteristic (ROC) curve, 528, 541
efferent pathways, 25 Receptive fields, 15
goal-directed movements, 24 Rechargeable neurostimulators, 229
maintenance, 25 Reciprocal inhibition, 9
Potassium conductance, 471–472 Rectified linear unit (ReLU), 436, 547
Potassium ions, 467 Recurrent excitation, 492
Potentiostat, 77 Recurrent neural networks (RNNs), 445, 549
Power consumption, 79, 81 LSTM, 446–447
Power spectral density, 107 non-linear units, 445
Power spectrum, 106–109 time series data, 447–448
PPC, see Posterior parietal cortex (PPC) types of layers, 445, 446
PPMCC, see Pearson product-moment correlation Red nucleus, 23
coefficient (PPMCC) Reduced inference distribution (RID),
PPN, see Pedunculopontine nucleus (PPN) 114
Preferred direction, 201 ReFIT-KF, see Recalibrated feedback intention-trained
Premotor cortex (PMd), 193, 196 Kalman filter (ReFIT-KF)
Pre-pulse, 214 Reflexes
Presbyopia, 49 clonus/tremor reflex, 12
Primary motor cortex (M1), 193, 196 CNS, 10, 11
Principal component analysis (PCA), 149, 392, 526 cough reflex, 12
Principal eigenvectors, 110 crossed extensor reflex, 12–13
Probabilistic neural network (PNN), 547, 558 flexor reflex, 12
Propagation monosynaptic stretch reflex, 10–12
action potential, 477 neuromuscular junctions, 10, 12
Hodgkin-Huxley model, 475 peripheral effects, 10, 11
intracellular medium and membrane, 477 peripheral receptor, 9
membrane potential, 476 polysynaptic reflexes, 12
myelin sheath, 477 reflex facilitation, 10
nonlinear partial differential equation, 477 suckling reflex, 12
physiological parameters, 477 Reflex times, 13
Pseudo-resistors, 76 Refractory period, 475
PSF, see Point spread function (PSF) Regularization parameters, 395–396
PSTHs, see Peristimulus time histograms (PSTHs) ReLU, see Rectified linear unit (ReLU)
PULSE, see Post-resuscitative and initial Utility in Life Repetitive TMS (rTMS), 264
Saving Efforts (PULSE) Residual connection, 440
Pulse trains, 264–265 Residual entropy, 119
Pupil, 49 Residual learning, 439–440
PVA, see Population vector algorithm (PVA) Resonate-and-fire (RIF) model, 484
p-VEP, see Phase-modulated VEP (p-VEP) Resting membrane potential, 1, 2
Resting motor threshold, 263
Resting potential, 468–470
Resting state fMRI
Q
functional connectivity, 342
qEEG analysis, 120, 121
functional network pattern, 343
Quadratic integrate-and-fire (QIF) model, 484
ICA, 343
Quadriplegia, 29
InstaCorr implemented, 342
Quasi-static, 252
learning algorithms, 343
Quasi-static approximation, 258
mapping, 343
Index 693
networks, 343 RP, 592

predefined experimental paradigm, 342 vascular occlusive disease, 594
seed-based correlation, 342 Retinal eccentricity, 589–591
statistical parametric mapping, 342 Retinal fields, 586–589
temporal correlation, 342 Retinal ganglion cells (RGCs), 582
Resting-state fMRI (RS-fMRI), 448 Retinal molecule, 295
Resting state networks, 343 Retinal pigment epithelium (RPE), 591
Restricted Boltzmann machines (RBM), 559 Retinal prostheses
Resuscitation Outcomes Consortium (ROC), 116 blindness, 567, 572
Reticular activating systems camera, 575–576
arousal/inhibition, 42 definition, 567
brain regions and neural pathways, 42, 44 eye and retina (see Eye and retina)
brain waves, 43–45 image processing, 576–577
definition, 42 retinal degeneration, 571
limbic and ascending, 41–46 retinal stimulating electrode arrays, 577–578
modulation, 42 Retinitis pigmentosa (RP), 592
sleep, 45–46 Return of spontaneous circulation (ROSC), 119, 120
stimulations, 42 Reversal potential, 468
thalamocortical system, 43 RF excitation, 332
wakefulness, 43 RGCs, see Retinal ganglion cells (RGCs)
Reticular formation, 23 Rheobase, 259
Retina, 49–51, 582 Rhodopsin, 51
acuity, 589–591 RID, see Reduced inference distribution (RID)
bioengineering opportunities, 623–624 Riemannian geometry–based method, 152
choriocapillaris vs. retinal capillaries, 592 RMS, see Root-mean-square (RMS)
circuits, 585–586 RNNs, see Recurrent neural networks (RNNs)
disease (see Retinal disease) Robotic limb, 215, 216
eccentricity, 589–591 Robust control mechanism, 317
engineering approaches ROC, see Resuscitation Outcomes Consortium (ROC)
extracellular volume, 622–623 Rod photoreceptors, 50–52
hydrogen ion, 621–622 Root-mean-square (RMS), 234, 322
ion distribution, 620–621 ROSC, see Return of spontaneous circulation (ROSC)
net changes, 623 RPE, see Retinal pigment epithelium (RPE)
oxygen, 618–620 RS-fMRI, see Resting-state fMRI (RS-fMRI)
fields, 586–589 rtfMRI, see Real-time fMRI (rtfMRI)
function, 583–584 rTMS, see Repetitive TMS (rTMS)
ganglion (see Ganglion cell models) Ruffini corpuscles, 20
mammals, 584
photoreceptors, 584–585
S
physiology/pathophysiology, 594–595
Saccades, 54
biochemically based analysis, 598–599
Sacculi, 25, 26
diagnostic value of a-wave, 600
SAEs, see Stacked auto-encoders (SAEs)
input-output analysis, 597–598
SA-PACT, see Spherical array-based PACT (SA-PACT)
photoreceptor model, 595–597
SCAEs, see Stacked convolutional auto-encoders
response to light, 599–600
(SCAEs)
post-receptor ERG analyses
Scala media, 56
B-wave analyses, 600–602
Scala tympani, 56
specific retinal areas, 602–603
Scala vestibuli, 56
structure, 583
Scanning laser ophthalmoscopy (SLO), 582
vasculature, 591
SCEA, see Self-coherence enhancement algorithm
Retinal acuity, 589–591
(SCEA)
Retinal circuits, 585–586
Schwann cells, 3
Retinal connectivity models, 614–617
SCI, see Spinal cord injury (SCI)
Retinal detachment, 594
SCPs, see Slow cortical potentials (SCPs)
Retinal disease
Secondary sensory cells, 25
AMD, 592–593
Second-order partial differential equation, 260
diabetic retinopathy, 593–594
Second-order polynomial, 117
glaucoma, 593
Second-order Volterra operations, 105
retinal detachment, 594
Seed-based correlation, 342, 343
694 Index
sEEG, see Stereo-EEG (sEEG) and effectors, 13

Segmental reflexes, 34 modalities, 13, 14
Segmentation analysis, EEG, 105 perception, 14
Segregated functional circuit model, 224 quality, 13–14
Seizure detection/forecasting quantity, 14
deep neural networks (see Artificial neural networks receptors, 14–15
(ANN)) threshold stimulus, 14
EEG signals, 541 Separation algorithm derivation
linear methods, 542 BSFE, 321
binary classification, 542 chronic dog preparation, 322
CSVMs, 544 EEG, 321
FPR, 544 elements, 321
iEEG, 544 extrafascicular electrodes, 321
logistic regression model, 542 lead field matrix, 321
regularization, 544 proportional motor activity, 321
SVMs, 543, 544 sLORETA and FOCUSS, 321
machine learning algorithms, 542 SFO, see Step-function opsin (SFO)
model performance, improving SGD, see Stochastic gradient descent (SGD)
AUC, 551, 552 Shannon entropy, 115
EEG signal, 550, 554 Short-interval intracortical inhibition, 263
EEG state classifier, 551 Short-term synaptic plasticity, 490
ensemble models, 552 Short-time Fourier transform (STFT), 114, 538
HMM, 553 Short-wave ripples, 84
MLP, 552 Signal acquisition
state classification, 548 brain signals, 135
unsupervised learning algorithms, 553 goal of, 139
tree-based methods, 544 invasive techniques, 139–141
Seizure detection methods, 105 neural signals, 144–147
Seizure-onset zones (SOZs), 398, 537 noninvasive techniques, 141–144
Self-adhesive electrodes and processing, 170
AC stimulation, 279 Signal correlation/rsignal, 522
cranial stimulation devices, 280 Signal degradation, 215
current flow-suited stimulation, 280 Signal matrix, 110
DC stimulation, 279 Signal processing
female pin connection, 279 feature extraction, 148–151
head stimulation, 279 feature translation, 151–153
noninvasive neuromodulation, 280 Signal-to-noise ratio (SNR), 79–81, 261, 305
skin surface, 278 Silicon-based arrays, 193
use, 278 Silicon-controlled rectifiers, 249
Self-coherence enhancement algorithm (SCEA), 400 Simple models, 484–485, 498, 510
Semicircular canals, 25, 26 SiNAPS multi-shank probe, 88
Semimembranosus (SM), 3207 Single fiber action potential, FINE simulation
Semitendinosus (ST), 320 NEURON simulation, 325
Sensitive magnetic flux detector, 381 parameters, 325
Sensorimotor control, 216 recorded signal, 325
Sensorimotor rhythms (SMRs), 132, 144–145, 158–160 transfer function, 325
Sensory aphasia, 27, 28 Single hamstring EMG, 320
Sensory areas, 193 Single neurons
Sensory receptors feed-forward networks, 491
cutaneous afferents, 15, 16 network models, 491
membrane potential, 14 persistent activity, 492–493
physical stimulus, 14 Single pulses, 263
receptive fields, 15 Single-unit activity, 191, 192
relative densities, 15 Skull stripping, 435
skeletal muscles, 15 SL, see Surface Laplacian (SL)
spinal dermatomes, 15 SLD, see Square law detector (SLD)
Sensory systems Sleep
accommodation, 14 alert wakefulness, 45
afferents, 13, 15–17 definition, 45
Index 695
EEG, 101–102 Source models, 387–388

mechanisms, 46 SOZs, see Seizure-onset zones (SOZs)
quiet wakefulness, 45 Sparsity-enforcing regularizations, 397
REM, 45–46 Spastic hemiplegia, 32
slow wave, 45 Spatial facilitation, 6, 7
types, 45 Spatial light modulators
SLM, see Spatial light modulators (SLM) CGH, 303
SLO, see Scanning laser ophthalmoscopy (SLO) DMD, 303
Slope detector methods, 106 DOPC, 303, 304
Slow cortical potentials (SCPs), 132, 145, 146 large-scaled networks, 302
Slow eye movements, 54 LED arrays, 303
Slow wave sleep, 45 MEMS-based galvanometers, 303
SM, see Semimembranosus (SM) SLM, 303
SMA, see Supplementary motor area (SMA) stimulation patterns, 303
Smell (olfactory), 58–60 Spatial light modulators (SLM), 299, 303
Smoothed WVD, 114 Spatial resolution, 84
SMRs, see Sensorimotor rhythms (SMRs) Spatiotemporal source localization, 391
SNR, see Signal-to-noise ratio (SNR) Specific sensory pathways, 18, 19
Sodium conductance, 471–472 Spectral analysis, 106–107
Somatosensation, 215 Spectral resolution, 84
Somatosensory feedback, 215 Spectral sensitivity, 297
Somatovisceral sensibility Speech, 27–28
associative system, 18 Speech synthesizer, 196
CNS, 17–18 Speech transcription program, 196
extra-lemniscal system, 18 Speed of sound (SOS), 354
lemniscal system, 18 Spherical array-based PACT (SA-PACT), 358, 359
limbic system, 18 Spike-frequency adaptation, 480, 493
mechanoreception, 19–20 Spike history dynamics, 486
motor system, 18 Spikes, 147
nonspecific sensory pathways, 18, 19 Spike sorting, 192
signaling pathways, 17, 18 Spinal cord
somatosensory projection areas in cortex, 19 and brain, 7–9
specific sensory pathways, 18, 19 flexor hyperactivity, 29–30
vegetative system, 18 motor control, 22
SOS, see Speed of sound (SOS) paraplegia, 29
Sound and hearing peripheral paralysis, 30
auditory sensations, 57–58 quadriplegia, 29
central auditory system, 58–60 reflex flexion, 30
definition, 54–55 spinal shock, 29
functional anatomy Spinal cord injury (SCI), 35, 132, 134, 154, 7186
cochlea, 56 Spinal shock, 29
ear canal, 56 Spiral ganglion, 57
Eustachian (auditory) tubes, 56 SPM, see Statistical parametric mapping (SPM)
malleus, 56 Sponge-based electrode
middle and inner ear sensory structures, 56, 57 CES devices, 277
ossicles, 56, 57 conductive rubber, 278
receptors, 57 conventional paradigm, 278
scala media, 56 effective electrode size, 276
scala tympani, 56 headgear, 276
scala vestibuli, 56 NeuroConn, 277
spiral ganglion, 57 nonsalinized water, 278
stria vascularis, 57 pre-saturated, 278
musical quality, 55 protocols, 277
noise, 55 reusable conductive rubber electrode, 277
oscillations, 55 Sponge stimulation, 286
place theory, 58 Square law detector (SLD), 119
sound pressure, 55, 56 SQUID, see Superconducting quantum interference
tone, 55, 56 device (SQUID)
Sound pressure, 55, 56 SSVEP, see Steady-state visual evoked potential (SSVEP)
696 Index
SSVEP-based BCI speller, 163, 164 AMPA, 488

ST, see Semitendinosus (ST) anatomy, 3, 5
Stacked auto-encoders (SAEs), 449, 450 electrical, 489
Stacked convolutional auto-encoders (SCAEs), 450, 451 ionotropic synaptic transmission, 488
Standard stimulation waveform, 226 membrane potential, 489
State model, 204, 205 metabotropic receptors, 490
State-of-the-art communication BMIs, 200 nerve cells, 5
State-of-the-art source models, 387 neurons, 4, 488
Static reflexes, 27 NMDA, 488, 489
Static temperature sensations, 39 postsynaptic effect, 488
Stationary stochastic processes, 416, 417 voltage waveforms, 489
Statistical parametric mapping (SPM), 336, 435 Synaptic facilitation, 6, 7
Statokinetic reflexes, 27 Synaptic plasticity, 4
Statolith organs, 25 Synaptogenesis, 4
Steady-state visual evoked potential (SSVEP), 147, 161 Systemic nervous system, 5
Step-function opsin (SFO), 298, 306
Stereo-EEG (sEEG), 385
T
Stereotactic surgery, 224
TA, see Tibialis anterior (TA)
STFT, see Short-time Fourier transform (STFT)
tACS, see Transcranial alternating current stimulation
Stimulus sequence design, 161, 162
(tACS)
STN, see Subthalamic nucleus (STN)
Target species, optogenetic experiments
Stochastic gradient descent (SGD), 437
C. elegans, 299
Stochastic processes
cortical tissue, 299
autocorrelation function, 416
Drosophila, 299
autocovariance function, 416
FMRI, 299
coherence function, 417
Zebrafish, 299
cross-correlation function, 417
Task-based fMRI, functional mapping
functional connectivity, 417
brain research, 342
neural time series, 416, 417
clinical applications, 341
oscillations, 417
localization, 341
periodogram method, 417
model-based time series analysis, 341
stationary process, 416, 417
neuromodulation techniques, 342
STOs, see Subthreshold oscillations (STOs)
neuropathological diagnosis, 341
Stroke, 32
neuroscientific impact, 342
Structural controllability, 510
preclinical animal models, 342
Subcortical regions, 186
psychological mechanism, 342
Subject-specific head models, 404
somatotopic organization, 341
Subthalamic nucleus (STN), 224, 304
Taste (gustatory), 58–60
Subthreshold oscillations (STOs), 478–480
TAT, see Thermoacoustic tomography (TAT)
Suckling reflex, 12
TBIs, see Traumatic brain injuries (TBIs)
Sudden unexpected death in epilepsy (SUDEP), 536
TC, see Threshold crossings (TC)
SUDEP, Sudden unexpected death in epilepsy (SUDEP)
tDCS, see Transcranial direct-current stimulation (tDCS)
Superconducting quantum interference device (SQUID),
Teager energy operator (TEO), 105–106, 119
143, 381, 382
Teager-Kaiser energy algorithm, 105–106
Superior colliculi, 53
Temporal-domain processing, 148–149
Supplementary motor area (SMA), 194, 196
Temporal facilitation, 6, 7
Support vector machine (SVM), 450, 541, 543–546,
Temporal interference, 237–238
548–558
Temporal resolution, 84
Support vector machines (SVM), 151
TEO, see Teager energy operator (TEO)
Surface Laplacian (SL), 148, 386
tES, see Transcranial electrical stimulation (tES)
Surrogate data method, 423, 424
tES biophysics/mechanisms, 285
SVM, see Support vector machine (SVM)
tES devices and dose
Switched-capacitor IA, 75, 76
AC types, 272
Symmetric geometry, 255
anode electrode, 274
Sympathetic nervous system, 34
brain stimulation, 272
Sympathetic system, 32, 33
DC types, 274
Synapses
ECT, 272
activation variable, 488
electrode design, 272
alpha function, 488
electrodes, 274
Index 697
powered current-controlled stimulator, 274 anesthesia monitoring, 104, 105

pulses, 272 bispectrum, 105
ramp up and ramp down, 274 DFA, 104–105
subclasses, 272 Hjorth descriptors, 105
waveform, 271, 274 period-amplitude model, 105
tES devices tolerability, 285–286 period/interval analysis, 104
tES electrodes segmentation analysis, 105
cardinal functions, 275 Teager-Kaiser energy algorithm, 105–106
clinical trials, 274 Time-frequency (T-F) distribution, 113
design, 274 Time-frequency (T-F) reciprocity, 112
electrochemical changes, 275 Time-frequency representations (TFRs), 114, 149, 152
electrochemistry, 275 Time-modulated VEP (t-VEP), 161
functions, 275 Time series data, RNN, 447–448
head anatomical landmark., 275 Tissue engineering
interface, 275 cellular approaches
limited-intensity tES techniques, 274 gene therapy, 655–656
selection and preparation, 274 genetic, 653–655
self-adhesive (see Self-adhesive electrodes) stem cells, 651–653
separation mechanism, 275 electrical stimulation, 657–658
skin irritation risk, 275 immunomodulation, 656–657
sophisticated placement techniques, 275 nervous system
sponge electrode (see Sponge-based electrode) cells/tissues of brain, 640–641
sponge/paste, 275 PNS, 641
types, 276 spinal cord, 641
tES use targets
clinical and behavioral interventions, 283 degenerative disease, 643–644
direct cortical modulation, 283 nervous system injuries, 642–643
electrode montage, 283 neural device integration, 644–645
MDD, 283 technologies
TFRs, see Time-frequency representations (TFRs) CNS injuries, 648–649
tFUS, see Transcranial focused ultrasound (tFUS) material approach, 645–646
Thalamocortical (TC) system, 43, 478 nerve conduits, 646–648
Thalamus, 24, 25, 223, 224, 231 TMS, see Transcranial magnetic stimulation (TMS)
Thermal noise, 80–81 TMS biophysics
Thermoacoustic tomography (TAT), 371 charge balanced, 258
Thermoregulation electrical stimulation, 258
central thermoregulation, 40–41 monophasic electrical, 258
core temperature, 39 neural activation models, 259–260
cutaneous thermoreception, 39–40 neuronal membrane depolarization, 258–259
heat energy, 37 TMS circuit topology
homeothermic, 37 circuit series resistance, 247
hypothalamus, 37 coil current, 247
negative feedback control system, 37, 38, conventional biphasic stimulator, 249
41, 42 conventional monophasic stimulator, 247
optimal temperature, 37 damping resistor R, 249
poikilothermic, 37 independent control, 249
Theta waves, 44, 99 monophasic pulse, 249
Thoracolumbar system, 32, 33 monophasic stimulator, 249
Three-dimensional subcortical structures, 231 resonant circuit, 247
3D source imaging, 397 silicon-controlled rectifiers, 247, 249
Threshold crossings (TC), 192 underdamped sinusoidal current, 249
Threshold stimulus, 14 TMS coils
Tibialis anterior (TA), 322 coil forces, 250–252
TIK, see Tikhonov regularization (TIK) coil heating, 250
Tikhonov regularization (TIK), 395 double-cone coil, 250
TIME, see Transverse intrafascicular multichannel E-field waveform, 250
electrode (TIME) ferromagnetic core, 250
Time-domain method magnetic field B proportional, 250
amplitude distribution, 104 single circular winding, 250
698 Index
TMS devices Traumatic brain injuries (TBIs), 235, 642

coils, 250–252 Tree-based methods
pulse generators, 247–250 decision tree, 544
safety, 252 k-nearest neighbors, 545
TMS–EEG signal, 262 logistic model trees, 546
TMS-induced E-field, 256 random forest, 546
TMS physics recursive algorithm, 545
approximations, 252 Tremors, 30
B-field, 253 Trial and error method, 217
conductive biological tissues, 253 tRNS, see Transcranial random noise stimulation (tRNS)
displacement current, 252 Truncated singular value decomposition (TSVD),
E-field, 252 394–396
electric field models (see Electric field models) TSVD, see Truncated singular value decomposition
Faraday’s law, 252 (TSVD)
induced electric field, 253–255 t-VEP, see Time-modulated VEP (t-VEP)
magnetic field, 253 Twisted pair, 82
quasi-static, 252 2D Fabry-Perot (FP), 359
TMS pulse generators Two-photon optogenetic stimulation
circuit topology, 247–249 ChR2-positive neurons, 306
energy efficiency and repetitive TMS, 249–250 depolarization, 306
TMS stimulation paradigms electron-photon interaction, 306
clinical applications, 264–265 opsins, 306
paired pulses, 263–264 phototoxicity effects, 306
pulse train, 264 SFO, 306
research applications, 265 TPM, 305
single pulses, 263
TMS stimulation responses measuring
behavioral changes, 260 U
behavior cognition characterization, 261 UBP, se Universal back–projection (UBP)
brain blood oxygenation, 260 Ultra-flexible carbon-layered electrode array ECoG
electrophysiological and imaging methods, 261–263 (CLEAR-ECoG), 305
Tone, 55, 56 Ultralow-noise amplifier, 328
Tourette syndrome, 234 Ultrasonic detectors, 353, 354
tPCS, see Transcranial pulsed current stimulation (tPCS) Unconscious internal modalities, 13
Traditional moving-average method, 149, 150 U-Net, 440–441, 443, 445
Transcranial, 280 Universal back-projection (UBP), 353
Transcranial alternating current stimulation (tACS), 272, Unsupervised learning, 448, 449
283 USEA, see Utah Slanted Electrode Array (USEA)
Transcranial direct-current stimulation (tDCS), 274, 283, US Food and Drug Administration (FDA), 193, 264
285 Utah array, 86, 88, 193, 194
Transcranial electrical stimulation (tES), 132, 271 Utah Slanted Electrode Array (USEA), 316
Transcranial focused ultrasound (tFUS), 132 Utriculi, 25, 26
Transcranial magnetic stimulation (TMS), 132, 166, 167
brain stimulation, 206 V
description, 246 Variational mode decomposition (VMID), 547
devices (see TMS devices) Vascular endothelial growth factor (VEGF), 583, 594
E-field (see TMS electric fields) Vascular occlusive disease, 594
electrical stimulation, 246 VCD, see Volume current density (VCD)
imaging modalities, 265 VC/VS, see Ventral capsule and ventral striatum (VC/VS)
magnetic nerve stimulation, 246 VEGF, see Vascular endothelial growth factor (VEGF)
phosphenes, 246 Ventral capsule and ventral striatum (VC/VS), 235
TES, 246 Ventral cochlear nucleus, 58
tolerability advantage, 246 Ventral oralis (VO), 235
Transcranial pulsed current stimulation (tPCS), 272, 283 Ventral premotor cortex (PMv), 194, 196
Transcranial random noise stimulation (tRNS), 274 Ventromedial hypothalamus (VMH), 236
Transverse intrafascicular multichannel electrode VEP, see Visual evoked potentials (VEPs)
(TIME), 316 Vestibular organ, 25, 26
Transverse plane, 332 Vestibular reflexes, 27
Transverse relaxation, 333 Vim couples, 234
Index 699
Virtual reality (VR), 156 homogeneous and isotropic, 71

Vision Maxwell’s equations, 70
astigmatism, 49 medium, 72
bright/dark contrasts, 49, 50 MoI, 71, 72
cataracts, 49 stimulation, 70–72
electromagnetic radiation, 49 transmembrane currents, 70
eye movements, 54, 55 Volume conductor models, 255, 387, 388
functional anatomy, 49, 50 Volume current density (VCD)
glaucoma, 50 integrating electromagnetic and hemodynamic,
hyperopia, 49 401–404
luminescence/luminous intensity, 49 inverse estimation techniques, 397–399
myopia, 49 nonlinear inverse techniques, 399–401
presbyopia, 49 3D source imaging, 397
receptor transduction process, 52–54 Volume of activation (VoA), 302
retina, 49, 50 Volume of tissue activation (VTA), 229, 230
visual focusing system, 49–50 Volume source scanning method, 426
visual receptor cells, 50–52 Volvox carteri, 296
Visual cortex, 53 Volvox ChR, 296
Visual evoked potentials (VEPs), 131, 147, 160–163 VR, see Virtual reality (VR)
Visual receptor cells, 50–52 VTA, see Volume of tissue activation (VTA)
VMH, see Ventromedial hypothalamus (VMH)
VMID, see Variational mode decomposition (VMID)
VO, see Ventral oralis (VO) W
VoA, see Volume of activation (VoA) Warburg impedance, 69
Voices, 113 Water window, 69
Voltage-based threshold, 192 Waveform patterns, 209
Voltage-/calcium-sensitive indicators, 368–370 Wavelet phase coherence (WPC), 538–539
Voltage clamp, 77–78 Wavelets
Voltage-controlled stimulation, 282 CWT, 110–114
Voltage-gated conductances, 470 DWT, 115
Voltage-gated ion channels, 2 and entropy, 115–116
Voltage limits, 281 Wavelet transform (WT), 115
Volterra filter, 105 Wearable sensor-based DBS, 234
Volume conduction models, 283 Weighted minimum-norm (WMN), 397
Volume conduction theory Whole brain imaging, 366, 367
anisotropy, 72 Wigner-Ville distribution (WVD), 114
bipolar recording, 70 Wilson-Cowan model, 499, 500
cable equation, 70 WMN, see Weighted minimum-norm (WMN)
dipolar current source, 70 WPC, see Wavelet phase coherence (WPC)
disk approximation, 71 Wrapper algorithms, 151
electrical potential, 70, 71 WT, see Wavelet transform (WT)
electrical recording, 70–72 WVD, see Wigner-Ville distribution (WVD)

Neural Engineering

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Neural Engineering

Uploaded by

Copyright:

Available Formats

Bin He Editor

ISBN 978-3-030-43394-9 ISBN 978-3-030-43395-6 (eBook)

© Springer Nature Switzerland AG 2013, 2020

There has been tremendous progress in the engineering of tools designed to

Brain-computer interfaces (BCIs) have emerged as a novel technology

tES devices are applied to conditions spanning neurological and psychiatric

neuroscientific applications. Existing challenges are also discussed in trans-

of neurotrophic factors and stem cells, genetic approaches to tissue engineer-

Pittsburgh, PA, USA Bin He

13 Electrophysiological Mapping and Source Imaging . . . . . . . . 379

Abraham Akinin Department of Bioengineering, UC San Diego, La Jolla,

Lei Ding Stephenson School of Biomedical Engineering, University of Ok-

Lari M. Koponen Department of Psychiatry & Behavioral Sciences, Duke

Matthew A. Smith Department of Biomedical Engineering and Neuro-

Abstract Hypothalamus · Homeostasis ·

© Springer Nature Switzerland AG 2020 1

If the concentration gradient for any ion is

Eion = 61/Z · log (Cout /Cin )

where Eion is the equilibrium potential for a given

Fig. 1.2 Shown here is a

Fig. 1.5 A schematic representation of the general

In general, neurons can be divided into three

Stimulus Stimulus Stimulus

Fig. 1.13 Shown here is

1.2.3 The Relative Distributions The relative densities of a receptor popula-

Fig. 1.14 Shown here is a

each such neuronal Field Receptor

Neuron A Neuron B Neuron C

fibers—and the numbers of each can also

Fig. 1.18 Shown here is a

1.3.3 Somatosensory Projection 1.3.4 Mechanoreception

Fig. 1.20 The schematic

Meissner’s Merkel’s Pacinian Hair-follicle Tactile Ruffini

1.4 General Anatomic Simplistically, the primary function of this

Association Coretx &

S keletal Muscle, Tendon, Joint,

Fig. 1.22 Shown here are

centers, or formations. Listed below are several

• The red nucleus lies primarily within the

cortex includes the primary motor cortex in the

Corticospinal and corticobulbar tracts (the ma-

Fig. 1.28 A schematic diagram of the left horizontal

1.5.1.2 Semicircular Canals

1.6.2 Motor Neuron Recruitment of motor neurons that becomes activated in an

One example is the condition known as capsular 1.8.1 Sympathetic System

Fig. 1.31 Provided here is

parasympathetic system predominates during

1.8.4 The Adrenal Medulla

The adrenal medulla lies within the inner cores

1.10 Regulation of Body

In this section, regulation of body temperature

Cold Exposure and HYPER THERMIA

1.10.1 Core Temperature nonthermal stimuli. Like other receptors, each of

Skin surface Interthreshold

attention on a topic. The state of wakefulness 1.11.2 Brain Waves

Fig. 1.45 Recorded surface potentials from an electroen-

Fig. 1.47 Rapid transition in the EEG waveforms due to

Fig. 1.49 Generalized sleep pattern that may occur dur-

As an individual passes from a state of wake-

• Alert wakefulness: Typically beta waves are

placed in contact with the skin can induce pain re-

Fig. 1.53 Schematic

Fig. 1.55 Relative

Fig. 1.56 Shown here are