Endogenous and exogenous hemodynamic signals in primary visual cortex of alert non-human primates

Mariana M. B. Cardoso Dissertation presented to obtain the Ph.D degree in Biology | Neuroscience Instituto de Tecnologia Química e Biológica António Xavier | Universidade Nova de Lisboa Oeiras, May, 2016 Insert here an image with rounded corners Endogenous and exogenous hemodynamic signals in primary visual cortex of alert non-human primates Mariana Marcelino Belchior Cardoso Dissertation presented to obtain the Ph.D degree in Biology | Neuroscience Instituto de Tecnologia Química e Biológica António Xavier | Universidade Nova de Lisboa Oeiras, May, 2016 Endogenous and exogenous hemodynamic signals in primary visual cortex of alert non-human primates Research work coordinated by: ii Financial Support Financial support for this thesis was provided by Fundação para a Ciência e Tecnologia and Fundo Social Europeu through the Quadro Comunitário de Apoio, doctoral fellowship: SFRH/BD/33276/2007. iii Acknowledgements Thank you to the super-nine PGCN/INDP-2007: Íris, Isabel, José, Margarida, Maria, Patrícia, Patrício, Pedro, Rodrigo: colleagues and friends, over the past several years (better not to name how many) were of impressive support in different forms and moments. Thank you in particular to Margarida without whom the “chapter thesis” would have been hard to imagine possible, and cleaning my tears over my poor writing. Thank you to Íris for her unconditional positivism. Thank you, Zé, for letting me learn to appreciate your wonderful imagination. Thank you to my advisor, Aniruddha Das. Who kept his door open at all times. That hosted me in his laboratory for a very, very long time; and was always patient with my slow progress. Thank you for the opportunity of working in your laboratory and access to such rich setups and data. Thank you especially to Bruss Lima and Maria (Masha) Bezlepkina. Bruss was the most patient teacher I could have wished for. With you, Bruss, I got to learn about electrophysiology and I wish I could say I had not learnt the sound of dying neurons. Masha kept me sane when that was far from a given, thank you for that and the friendship. Thank you to the other members in the lab, Yevgeniy, I have learned a lot by overlapping with you in the lab. Elena as well as Yana were wonderful ‘mothers’ when I arrived, thank you. Thank you as well to the wonderful veterinary team, in particular thank you to Dr. Girma Asfaw, Dr. Rodolfo Ricart and Dr. Amy Cassano. In the thesis will not be included early work done with rodents, but there were a few critical inspiring people that should still be acknowledged here: Mark Andermann, Daniel W. Wesson, Tomas Hromadka, Dinu Florin Albeanu, Ashesh K. Dhawale. Also Marc Bucklin from Aniruddha Das’s lab, that was always ready for a challenge. Thank you to wonderful Dorothée, and the awesome husband Arnaud, whose time and good hearing skills helped me keep focused on what my iv priorities might be. Thank you, Angelie, with whom all is possible. Thank you Franco, if I had an old brother, I wish he would have been like you. Thank you to the early friends with whom I learned about the Big city: Gil, Joe, Yan and Renata: the first host. Thank to the friends that keep checking on me from afar, Mafalda, Fátima, Inês, Mónica, Pedro, Joana, Miguel, Lena. Thank you, Zach, Rui, Marta, for creating this wonderful program and scientific environment and giving me the opportunity to be part of it. Thank you to my thesis committee, Christian and Michael, who were always supportive, positive and encouraging. Thank you to the hosting institution, Columbia University, and specifically to a few people that handled my process efficiently and with a smile, Fred Loweff and Alla Kerzhner. Thank you also to the Champalimaud structure and organization: for the great flexibility and ease of working with. Thank you, Stephan, for support and patience through the least fun times. Thank you to my family, Adelino, Isabel, Marta and Samuel, for the unconditional support (even financial), help, energy and trust! Finally, thank you to the animals: not on the class of human primates, and not volunteer on this process, but without whom this project would have not been possible. v Resumo O uso de técnicas de imagiologia cerebral (neuroimagem) em particular aquelas adequadas ao uso em humanos tem experienciado rápida disseminação. E tem havido grande crescimento nas áreas de investigação que estudam a correlação entre actividade neuronal e processos cognitivos. No entanto os fundamentos neuronais dos sinais resultantes de neuroimagem não estão ainda totalmente caracterizados; esta é ainda uma área de intensa investigação. E no contexto de tarefas comportamentais/cognitivas, a interpretação de neuroimagem é ainda mais complexa. Em sistemas sensoriais primários, há a perspectiva de que as respostas de neuroimagem reflectem sobretudo informação sobre estímulos sensoriais externos. Há vasta investigação que se foca no mapeamento de estímulos e classes de estímulos no córtex. No entanto, mesmo sistemas sensoriais primários podem conter informação sobre comportamento, como por exemplo estádios de comportamento. Por exemplo, embora o córtex visual primário responda principalmente a estímulos visuais, também é modulado por respostas como atenção. A forma como estas respostas estão codificadas é fundamental para a interpretação de neuroimagem. Igualmente, compreender como diferentes aspectos das respostas de neuroimagem estão relacionados com a actividade neuronal subjacente é crucial para a interpretação de neuroimagem. No laboratório tinha sido identificada uma resposta hemodinâmica relacionada com execução de tarefas (task-related), no córtex visual primário de primatas executando uma tarefa de fixação visual periódica. Este sinal foi observado na ausência de estimulação visual directa (não havendo portanto estímulo visual ou resposta visual a estímulos). Não se observaram alterações na quantidade de potenciais de acção dos neurónios na região de onde se reportou a actividade hemodinâmica. No entanto, nestas tarefas a resposta hemodinâmica relacionada com a tarefa é robusta e esta vi resposta ajusta-se à duração da tarefa. Esta resposta confirma a presença de uma resposta endógena numa área sensorial primária. A ausência de correlação com a activitade neuronal (medida como alterações na frequência de potenciais de acção) reinforça a importância de compreender os mecanismos que estão na base da resposta hemodinâmica. O trabalho apresentado nesta tese tem como objectivo auxiliar a interpretação de neuroimagem, em particular tenta distinguir contribuições endógenas e exógenas. Seguimos dois caminhos experimentais: um que tenta caracterizar as respostas hemodinâmicas relacionas com a apresentação de estímulos, e outra cujo objectivo é de compreender contribuições endógenas para o sinal. Em ambas as circunstâncias, para além de neuroimagem, foi medida actividade neuronal (na forma de potenciais de acção locais, LFP, ou frequência de potenciais de acção neuronais locais). Gravámos simultaneamente actividade de neuroimagem bem como de eletrofisiologia em primatas a executar tarefas periódicas. A técnica de neuroimagem utilizada, imagiologia óptica de sinais intrínsecos, baseia-se na absorção preferencial de luz visível pela hemoglobina presente em tecidos; trata-se portanto uma medida indirecta de metabolismo. Recolhemos dados no córtex visual primário, uma região cortical bem conhecida, com respostas neuronais a estímulos bem caracterizada e que anatomicamente se localiza na superfície do cérebro, portanto ajustada ao uso de imagem óptica intrínseca. Dada a montagem experimental disponível no laboratório, há a possibilidade de comparar dados de neuroimagem e electrofisiologia adquiridos simultaneamante (não típico aquando o uso de outras técnicas de neuroimagem) o que nos coloca numa posição priveligiada para avaliar a relação entre neuroimagem e actividade neuronal. Utilizámos uma tarefa periódica com a apresentação de estímulos visuais com uma relação bem estabelecida entre propriedades do estímulo e actividade neuronal: usámos contraste do estímulo para testar uma gama de intensidades de actividade neuronal. Contraste e actividade electrofisiológica têm uma relação monotó- vii nica: aumentos no contraste estão associados a aumentos na frequência de potenciais de acção (obedecendo uma função hiperbólica). Ao utilizármos contraste para induzir alterações previsíveis em actividade neuronal, observámos que numa tarefa periódica a resposta de neuroimagem reflecte a soma linear de uma componente exógena (relacionada com o estímulo) e outra endógena (relacionada com a tarefa). A componente relacionada com o estímulo tem uma relação linear com a actividade neuronal local (medida como alterações na frequência de potenciais de acção). Finalmente, queriamos também avaliar potenciais contribuições para a factores da resposta endógena (independente do estímulo) que possam contribuir para o sinal de neuroimagem. Usámos a mesma tarefa já antes utilizada do laboratório: fixação na ausência de estimulação visual (para além do ponto de fixação), observámos períodos em que o animal não estava envolvido na tarefa estavam associados com alterações lentas na resposta hemodinâmica. Para além da já mencionada resposta hemodinâmica relacionada com execução de tarefas (task-related), propomos que o envolvimento (engagement) numa tarefa tem uma contribuição significativa para alterações lentas no nosso sinal de neuroimagem. Observámos aumentos na resposta hemodinâmica acompanhados de diminuição do batimento cardíaco e ligeira diminuição na frequência dos potenciais de acção, no córtex visual primário. As alterações na resposta hemodinâmica são significativas em magnitude. O trabalho apresentado nesta tese avança assim a nossa compreensão da natureza da neuroimagem. viii Abstract The advent of neuroimaging techniques in particular the ones suitable for studies in alert humans has disseminated fast. Research in fields involving neuro-correlates of cognitive processes has flourished. Still the neural underpinnings of the neuroimaging signals remain to be fully characterized; this field is an active topic of research. In the context of behavior/cognition, the interpretation of neuroimaging signals is even more intricate. In early sensory systems, neuroimaging signals are thought to primarily carry information about sensory inputs; there is significant research focused on mapping evermore specific stimuli and stimulus classes to cortical regions. Nevertheless even early sensory systems can relay information on behavior, like brain states. For example, it is known that visual cortex even though primarily responding to visual stimuli, is also sensitive to such signals as attention. The way these signals are differentially encoded is critical in trying to interpret neuroimaging signals. Also how the different aspects of neuroimaging signals related to underlying neuronal activity is critical for neuroimaging interpretation. Earlier the lab showed the existence of a hemodynamic task-related signal, in primary visual cortex of alert non-human primates performing a periodic fixation task. This signal was observed in the absence of direct visual stimulation (there was no stimulus present or a stimulus response). In the vicinity of the recorded hemodynamic response, there were no changes in the spiking rates of neurons, which could predict the homodynamic taskrelated signal. Nevertheless there was a robust task-related hemodynamic response, which entrains to trial schedule. It confirms the presence of an endogenous response in an early sensory area. The lack of a local neuronal correlate (as changes in firing rate) further emphasizes the importance of understanding the mechanisms underlying the hemodynamic signals. xv 2.3.10 Cross-validation of HRFSTIM kernels across sessions .............. 20 2.3.11 Deconvolution ............................................................................. 20 2.3.12 Checking the stability of our primary findings against variability in electrode recordings .............................................................................. 21 2.4 Results................................................................................................. 22 2.4.1 Spikes poorly predict hemodynamics in periodic task ............. 23 2.4.2 Modified linear model with two signal components.................. 25 2.4.3 Trial-related signal consistent in stimulus and dark room ....... 31 2.4.4 Spikes poorly predict blank-trial and dark-room signals .......... 36 2.4.5 Blank subtraction required to estimate spikes from imaging... 40 2.5 Discussion ........................................................................................... 44 2.6 Supplementary information ................................................................ 48 2.6.1 Appendix: Homogeneous Linear (‘Null’) and Modified Linear Model (MLM) of Spike-predicted Hemodynamics Appendix: Homogeneous Linear (‘Null’) and Modified Linear Model (MLM) of Spike-predicted Hemodynamics ............................................................... 48 2.6.2 Supplementary Figures .............................................................. 52 2.7 Acknowledgements ............................................................................ 59 3 The hemodynamic task-related signal in primary visual cortex of alert non-human primates performing simple tasks ................................................ 61 3.1 Introduction ......................................................................................... 61 3.2 Methods ............................................................................................... 63 3.3 Variable Trial Durations ..................................................................... 64 3.4 Variable Fixation Durations ............................................................... 73 3.5 Variable Reward Amount ................................................................... 79 3.6 Task-related Hemodynamic Response in Visual Cortex to an Auditory-Motor Task ...................................................................................... 87 3.7 Discussion ........................................................................................... 94 4 Slow drifts in brain blood volume are associated with behavioral changes in primate V1....................................................................................... 99 xvi 4.1 Abstract ............................................................................................... 99 4.2 Introduction ....................................................................................... 100 4.3 Methods ............................................................................................. 104 4.3.1 Summary ................................................................................... 104 4.3.2 Surgery, recording chambers and artificial dura .................... 105 4.3.3 Behavior..................................................................................... 105 4.3.4 Imaging ...................................................................................... 106 4.3.5 Electrophysiology ...................................................................... 107 4.3.6 Arousal Related Index (ARI) .................................................... 108 4.3.7 Alertness metrics: brain movement and eyes open/closed ... 109 4.3.8 Interneuronal correlations: electrode pair-wise correlations . 109 4.3.9 Correlations between performance and different responses 109 4.3.10 Correlations between heart rate and hemodynamics ............ 110 4.3.11 Cross-validation (k-fold cross-validation) ................................ 110 4.4 Results............................................................................................... 111 4.4.1 Hemodynamic mean trial response correlates inversely with engagement .............................................................................................. 111 4.4.2 Complex interplay between hemodynamics and heart rate .. 116 4.4.3 Different neural metrics and their correlation to engagement 119 4.5 Discussion ......................................................................................... 123 4.6 Acknowledgments ............................................................................ 127 5 General Discussion .................................................................................. 129 5.1 Exogenous responses in V1 ............................................................ 129 5.2 Endogenous responses in V1 ......................................................... 130 5.3 Future directions ............................................................................... 133 5.3.1 Locus coeruleus – noradrenergic system ............................... 133 5.3.2 Neural basis of neuroimaging .................................................. 134 References ....................................................................................................... 136 1 1. Introduction 1.1 Hemodynamic responses in the brain Understanding the functioning of the brain has captured our fascination for a long time. In the quest to understand the inner workings of the brain, several techniques have been developed. These range immensely in application and scope. In this thesis we used a combination of neuroimaging and electrophysiology to probe questions pertaining to the behaving brain. The space of questions that can be answered using neuroimaging in alert subjects is extremely vast. The questions addressed here have as seed the results found previously in the lab (Sirotin & Das 2009), where a task-related hemodynamic signal was described in alert non-human primates, when animals were engaged in a periodic fixation task. This hemodynamic taskrelated signal was observed in the absence of concurrent changes in local neuronal activity. In this thesis we aimed at understanding, firstly if the hemodynamic responses to external stimuli have a comprehensive relationship to underlying changes in spiking activity. Furthermore, we wanted to understand the correlations, if any, between the hemodynamic response and the behavior of the subjects; thus exploring also endogenous responses and not only responses to external sensory stimuli. To address these questions, we studied alert non-human primates (as in the original study, Sirotin & Das 2009). Interestingly, such signals were also observed in humans (Jack et al. 2006, Sylvester et al. 2007, Donner et al. 2008). It is worth noting, these recordings in human subjects comprised only neuroimaging. The recording of electrophysiologycal activity in human subjects tends to rely on techniques like EEG; those were not used in mentioned human studies. Moreover, EEG recordings might not reflect unubiquitous spiking activity (Saltzberg et al. 1971). Electrophysiological 2 techniques aiming to record spiking activity tend to be still intrusive techniques not routinely performed in human subjects. We recorded activity from the primary visual cortex (V1), which is the entry point of visual information into cortex. This is a structure on the cortical surface, which has been very well characterized in terms of its neural responses. Known properties of V1 date back to the seminal work by Hubel and Wiesel (Hubel & Wiesel 1968). V1 has a retinotopic organization, and not only is position relative to the fovea encoded in its responses, there are also several known properties of visual stimuli that can be differentially encoded in V1, as contrast, direction, orientation, spatial or temporal frequency. By combining optical imaging of intrinsic signals (OIIS) with simultaneous multi-unit electrophysiological activity (MUA) in V1, it is possible to address questions pertaining to the neural underpinnings of the neuroimaging response, as well as to improve our understanding of the signals encoded in visual cortex. Finally, V1 is a cortical region dedicated to vision, but that receives modulatory input from multiple structures in the brain. It is known to be influenced by endogenous variables such as attention (Bashinski & Bacharach 1980) or the timing of reward delivery (Shuler & Bear 2006, this is work with rodents). V1 is therefore particularly well positioned to study the contributions of endogenous and exogenous signals into the hemodynamic responses and their neural correlates. 1.2 Neuroimaging: intrinsic signal optical imaging Intrinsic signal optical imaging techniques were developed in the 80's by Amiram Grinvald's group (Grinvald et al. 1986). Optical imaging of intrinsic signals (OIIS) techniques measure changes in the intensity of light reflected off cortical tissue. If one illuminates cortex with visible light the largest absorber of light in tissue is hemoglobin. Thus this form of imaging allows the monitoring of changes in hemoglobin tissue concentration; analogous to 3 how one can identify vasculature in tissue, i.e., distinguishing arteries and veins below the skin, by visual inspection. At different wavelengths, hemoglobin absorbs light differently. Depending on hemoglobin’s redox state light is also absorbed differently (e. g. Zijlstra & Buursma 1997), hence it allows to distinguish changes in oxy-hemoglobin (HbO), deoxy-hemoglobin (HbR) and total blood volume (HbT; the sum of oxyand deoxy-hemoglobin). OIIS at wavelengths equally absorbed by oxyand deoxy-hemoglobin (isosbestic points, wavelengths of overlap of the absorption spectra) provides information about HbT. 530 nm is an isosbestic point. Another relevant wavelength is around 605 nm; at this wavelength HbR absorbs light around 5 fold more strongly than HbO. The combined imaging at the two wavelengths allows us to decompose the net response into HbO and HbR components (Sirotin et al. 2009); or, into blood volume and blood oxygenation components. Blood oxygenation is what is typically measured in functional magnetic resonance imaging (fMRI), blood-oxygen level dependent (BOLD) (Ogawa et al. 1990). Parenthetically, simultaneous recording of fMRI and OIIS signals show a good agreement between the two techniques, e.g. Jezzard et al. 1994, Fukuda et al. 2006 (in this latter study they used an isosbestic point, wavelength of 570 nm). The data presented in this thesis was acquired at 530 nm (isosbestic point; green wavelength). This wavelength provides a larger percent signal change than say the 605 nm (“deoxy-hemoglobin signal”), and is a signal with a known relationship between hemoglobin species (Sirotin et al. 2009). At this wavelength the hemodynamic signal is a proxy for changes in total blood volume. When the cortical surface is illuminated with visible light, light is preferentially absorbed by the vasculature, as mentioned above. Acquired images are darker where vasculature lies. Depending on image quality and magnification one can identify blood vessels. Capillaries too small to be resolved in the images comprise the capillary bed and parenchyma and they also contribute to changes in the imaging signal. There are a number of other 4 aspects that can contribute to the global OIIS, as light scattering. In the intact brain, heart rate or breathing should also be taken into account, as there is evidence of their influence in the BOLD signal (Chang et al. 2009, Birn et al. 2008). By using two different wavelengths in the same experiment (530, and 605 nm), we can distinguish arteries and veins. Arteries have low contrast when imaged at 605 nm, but veins have high contrast. At 530 nm, both arteries and veins should have similar contrast; hence comparing images acquired at these wavelengths permits distinguishing arteries from veins. Even though the results included on this thesis reflect changes measured with the 530 nm illumination wavelength, some measures were made using both 530 and 605 nm. The intensity value resulting from the OIIS (light intensity from a CCD camera) is in arbitrary units; its absolute value is not informative, but differential analysis can inform about changes in the hemodynamic species’ concentration. Bonhoeffer and Grinvald (Bonhoeffer & Grinvald 1991) presented subtraction and division methods for analysis of this type of data; they found similar results with either approach. In the experiments included in this thesis we used fractional changes of the light intensity reflected off the cortical surface, relative to the mean (light reflected), as a proxy for changes in concentration of total blood volume. In Chapter 2 we used fractional changes in the intensity of light reflected off the cortical surface, here increases in blood volume are reflected as decreases in the fractional changes, but Chapters 3 and 4 use the negative of this change, in this case increases in blood volume are reflected as increases in the signal’s fractional change (this difference reflects publication history). The experimental setup was composed of a CCD camera, an ensemble of lenses focusing the imaging plane in the cortical surface, taking pictures of the brain surface. Attached to the recording chamber is the illuminator (a ring like structure holding optical fibers; each fiber is coupled to a LED) as 5 well as the electrode holder (another ring like structure with the capacity to hold one or two electrodes connected to electrode micro drivers). The imaging region is kept under sterile conditions and separated from contact with outside air (a titanium chamber encloses a clear glass, sealing the chamber). To ensure optical clarity the imaging chamber it is filled with a clear solution of agarose. A schematic of the setup can be found in Figure 1.1. In Figure 1.1a, one can see the edge of the imaging lenses, as well as the electrode holder (metal piece attached to recording chamber). The illumination system is composed of optical fibers coupled to LEDs, it produces a uniform illumination profile as in Figure 1.1a. In Figure 1.1b is a closeup picture of the recording chamber, where can be noticed an aperture (through a silicon-coated glass, sealing the chamber) for one recording electrode. Figure 1.1 – The experimental setup. (a) Picture of the setup illustrating: imaging arrangement, with LED illumination of 530nm. (b). Picture of the imaging region with single electrode. Images were acquired at 15 Hz (or 7.5 Hz, if two wavelengths were used). Below is a typical image (Figure 1.2a, b) from one of our experiments. Here it can be noticed the artery that comes on gray on Figure 1.2a and is dark in Figure 1.2b, has a different profile from veins (visible and dark on both images). When looking at the time series, at this imaging frequency, one can clearly notice the effect of the brain’s pulsation (heart beat driven a b 6 palpitation) (Figure 1.2d). Pulsation can be removed by subtracting a highpass filtered version of the imaging signal (Figure 1.2c). From this pulsation one can estimate heart rate. The use of neuroimaging techniques ultimately aims at estimating neural activity, but neither OIIS nor BOLD fMRI, are direct measures of neural activity (review on BOLD interpretation, e.g. Logothetis & Wandell 2004). Figure 1.2 Intrinsic signal optical imaging in V1. (a) Reflectance of cortical surface with incident light of 605 nm. (b) Similar to a, but incident light of 530 nm. The letters in red: a – artery, v – vein. (c) Average reflectance for the first 30 s of imaging (wavelength: 530 nm), in gray is raw signal and in green is the signal without the high frequency pulsation (low-pass filtered version of the raw signal). (d) Pulsation signal observed at 530 nm on the first 30 s of imaging. a b c d 010 20 30 2.9 3 3.1x104 time (sec) reflectance (a.u.) 010 20 30 -400 0 400 time (sec) reflectance (a.u.) a vv a 7 1.3 Neural basis of intrinsic signal optical imaging In this section, I aim to bring attention to the complexity of the neuroimaging signals and their respective interpretations. Spiking activity is an energetically demanding process for the brain, with several components of neural activity weighing in (Laughlin et al. 1998, Attwell & Laughlin 2001). Metabolic demand in the brain is mostly replenished by the blood supply, as neurons do not have significant energy reserves. Therefore, if the relationship between neuronal activity and metabolic demand, and between metabolic demand and blood supply was linear, neuroimaging would have a clear relationship with underlying neuronal activity. However, there is ample evidence of a more complex relationship. Thus, understanding the neuro-vascular coupling, the coupling of spiking activity to metabolism and from metabolism to blood supply, will help interpret neuroimaging. The main metabolic substrates of the brain are glucose and oxygen; both reach the brain through blood circulation. There is considerable research into the metabolic cost of spiking (for estimations of metabolic neural cost in the rodent brain see: Attwell & Laughlin 2001). The development of autoradiography allowed measuring local glucose consumption in alert and anesthetized rodents using a radioactive analogue of glucose, which permits monitoring of metabolic rate (Sokoloff et al. 1977). The development of positron emission tomography permitted a better temporal and spatial mapping of radioactive species (a broad perspective of the field is presented in Raichle 1979). More relevant to the work presented in this thesis: metabolic measures have been used in primary visual cortex of alert humans undergoing physiological visual stimulation. These confirmed an increased metabolic rate with visual simulation, using nuclear magnetic resonance spectroscopy (Prichard et al. 1991). Finally when considering oxygen consumption, there have been reported increases in oxygen rate of 8 consumption in response to electric pulses (in rabbits’ isolated neurons, Ritchie 1967). Both glucose and oxygen consumption increase in response to neuronal activation, therefore measures of metabolism can be appropriate proxies for monitoring changes in neural activity. Ultimately, if hemodynamic changes reflect metabolism, then the hemodynamic response can be an attractive proxy for changes in neural activity. Given the advent of techniques, such as fMRI that allows monitoring large portions of brain (virtually non-intrusively) make those increasingly relevant. It should be noted that OIIS requires the imaged surface to be exposed, making it an intrusive technique. The use of neuroimaging techniques should however be combined with research efforts on its interpretation. Intrinsic signal optical imaging also reflects light scattering from neuropil activation. Early work in isolated nerve cells showed that action potentials in unmyelinated isolated fibers cause birefringence changes as well as light scattering, in the visible light spectrum (Cohen et al. 1968). There is also evidence from work done in hippocampal slices, where transmitted and reflected light profiles were observed to be, on average, symmetric and opposite in sign. This suggests a bigger role for light scattering than light absorption with intrinsic signal optical imaging (Aitken et al. 1999). Work with intrinsic signal optical imaging in the olfactory bulb of anesthetized rodents used multiple imaging wavelengths to address whether the response to stimulating odors in olfactory glomeruli results more from light scattering in tissue with neuronal activation or from changes in HbO/HbR concentration in blood. In their case, they concluded that light scattering is in the origin of the imaging response (Meister & Bonhoeffer 2001). Neuronal activity is responsible for other changes that can be accessed by looking at hemodynamic responses, other than direct light scattering changes Functional hyperemia – increased blood flow in response to neuronal activity – also contributes to hemodynamic changes. Astrocytes that have a privileged relationships with blood vessels are known to play an important role in mediating hyperemia (for a review see Iadecola & 15 In our earlier work (Sirotin & Das 2009), we only compared brain signals at the two extremes of visual drive. To measure stimulus-evoked signals, we used near-maximal stimulus intensities at which the visual input dominated; meanwhile, we characterized the trial-related signal only in essentially complete darkness. A question not explored in the earlier work was how these signals would interact when presented together in different proportions in routine visual tasks involving stimuli of varied intensities, and how this admixture of signals would affect the interpretation of brain images. We addressed these questions using our technique of simultaneous optical imaging and electrode recording in alert, task-engaged macaques. Here, however, we presented visual stimuli over the full contrast range (0% to 100%); for some experiments, we also included trials in complete darkness. This allowed us to test whether the net imaging signal could be separated into stimulus-evoked (that is, correlated with stimulus contrast and evoked neural spiking) and trial-related components (dependent on task structure, but not stimulation or local spiking) over a full range of V1 spiking and hemodynamics. Furthermore, as the primary use of neuroimaging is to estimate local neural activity (often done implicitly, but also quantitatively by deconvolving the imaging signal using an HRF (Glover 1999)), we examined the accuracy of this estimate with and without correcting for the trial-related signal. 2.3 Methods 2.3.1 Summary Simultaneous intrinsic-signal optical imaging and electrophysiology were acquired from alert macaques engaged in passive fixation tasks (n = 34 sites, 5 hemispheres in 3 monkeys, plus 14 additional experiments) using methods developed previously (Bonhoeffer & Grinvald 1996, Sirotin & Das 2009; Shtoyerman et al. 2000). All experimental procedures were performed 16 in accordance with the US National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the Institutional Animal Care and Use Committees of Columbia University and the New York State Psychiatric Institute. 2.3.2 Behavior and stimuli Animals held fixation periodically for juice reward, cued by the color of a fixation spot (fixation window, 1.0–3.5 degrees in diameter; monitor distance, 133 cm; fixation duration, 3–4 s; trial duration, 10–20 s). For experiments with visual stimulation, stimuli consisted of sine-wave gratings (contrasts, 0% (blank), 6.25%, 12.5%, 25%, 50% and 100%; mean luminance = background luminance = 46 cd m−2; spatial frequency, 2 cycles per degree; drift speed, 4 degrees per s; diameter, 2–4 degrees; orientation optimized for the electrode recording site). Trials typically comprised single fixations, with stimulus presented during fixation. For some experiments (Figure 2.5), trials comprised sequences of two or three fixations with the stimulus presented only on the first fixation. Stimuli were block randomized, that is, presented in blocks each containing a single full set of contrasts in random order. In the block, stimuli were repeated following errors (incorrect fixation) until the animal had a correct trial for each stimulus in a block (for multi-fixation trials, all fixations had to be correct for a trial to be correct). Some experiments included 3.125% contrast for finer resolution at low contrasts; some others used a reduced set of contrasts to increase the number of trials per condition. Dark-room experiments were performed in a completely dark room with the monitor covered and the fixation point behind a pinhole (as described in ref. Sirotin & Das 2009). Eye fixation and pupil diameter were recorded using an infrared eye tracker (Matsuda et al. 2000). 17 2.3.3 Surgery, recording chambers and artificial dura After the monkeys were trained on visual fixation tasks, craniotomies were performed over the animals’ V1 and glass-windowed stainless steel recording chambers were implanted, under surgical anesthesia, using standard sterile procedures (Shtoyerman et al. 2000), to image a ~ 79mm2 area of V1 covering visual eccentricities from ~1 to 5°. The exposed dura was resected and replaced with a soft, clear silicone artificial dura. After the animals had recovered from surgery, their V1 was optically imaged, routinely, while they engaged in the fixation task. Recording chambers and artificial dura were fabricated in our laboratory using published methods (Arieli et al. 2002). 2.3.4 Hardware Camera, Dalsa 1M30P (binned to 256 × 256 pixels, 7.5 or 15 frames per s); frame grabber, Optical PCI Bus Digital (Coreco Imaging). Software was developed in our laboratory based on a previously described system (Kalatsky & Stryker 2003). Illumination, high-intensity LEDs (Agilent Technologies, Purdy Technologies) with emission wavelength centered at 530 nm (green, equally absorbed in oxyand deoxyhemoglobin). Lens, macroscope of back-to-back camera lenses focused on the cortical surface. Imaging, trial data (trial onset, stimulus onset, identity and duration, etc.) and behavioral data (eye position, pupil size, timing of fixation breaks, fixation acquisitions, trial outcome) were acquired continuously. Data analyses were performed offline using custom software in MATLAB (MathWorks). 2.3.5 Image pre-processing Prior to analysis, acquired images were (if necessary) motion corrected by aligning each frame to the first frame by shifting and rotating the images using the blood vessels as a reference (Lucas & Kanade 1981). Slow temporal drifts (>30 s) were removed with high-pass filtering, and cortical pulsations with low-pass filtering using the Chronux MATLAB Toolbox 18 function runline.m (typical heart rates were ~2–3 Hz, much faster than the typical hemodynamic response frequencies of ~<0.5 Hz). 2.3.6 Electrophysiology Electrode recordings were made simultaneously with optical imaging. Recording electrodes (FHC, AlphaOmega; typical impedances were ~600– 1,000 kΩ) were advanced into the recording chamber through a siliconecovered hole in the external glass window, using a custom-made low-profile microdrive. Recording sites were mostly, but not exclusively, confined to upper layers. Signals were recorded and amplified using a Plexon recording system. The electrode signal was split into spiking (100 Hz to 8 kHz bandpass) and LFP (0.7–170 Hz); LFP data not shown. No attempt was made at isolating single units and all measured spiking was MUA (defined as each negative-going crossing of a threshold = ~4× the r.m.s. of the baseline obtained while the animal looked at a grey screen (Sirotin & Das 2009)). The MUA signals were then high-pass filtered to remove slow drifts (>30 s), down sampled to the imaging frame rate (7.5 or 15 samples per s) and aligned offline with the images. 2.3.7 HRF kernel fitting Each HRF was modeled as a gamma-variate function kernel of the form where  = (T/W)2 *8.0*log(2.0),  =W2/(T*8.0*log(2.0)), A is the amplitude, T is the time to peak and W is the full width at half maximum (Cohen 1997; Sirotin & Das 2009; Madsen 1992). This functional form allows for parametrically varying kernel amplitude, latency and width. For fitting, we used a downhill simplex algorithm (fminsearch, MATLAB) minimizing the sum square difference between measured and predicted hemodynamics. All fits used periodic functions (30 repetitions) constructed from means of the 19 relevant signals across contrasts, aligned to trial onsets, for correct trials alone. Thus, HRFSTIM was obtained by fitting a periodic pattern of the mean SSTIM to the mean HSTIM, the HRFNULL by fitting the mean S to the mean H (over correct trials alone), the HRFBLANK by fitting the mean SBLANK to the mean HBLANK, and the HRFDARK by fitting the mean SDARK to the mean HDARK. 2.3.8 Goodness of fit of predicted hemodynamics Fit was quantified as R2 = 1 − (variance of the residual error)/(variance of measured hemodynamics) (Supplementary Note, equation (10)), expressed either separately for each contrast or as mean R2, that is, calculated for the mean signals averaged across all contrasts. For all fits other than of the blank signal, predictions (and residual errors) were calculated by convolving the full raw measured spike trace with the relevant HRF and then separating later into correct trials by contrast, or averaging across contrasts. This is more reliable than convolving synthetic periodic functions constructed from mean signals because with periodic functions there is a risk of getting a match, not with the true signal, but with a signal phase-shifted by a fraction of a trial period (Das & Sirotin 2011). Such mismatches are highly unlikely in the measured signal with its random sequence of stimulus intensities and corresponding evoked hemodynamics (Das & Sirotin 2011). The blank signal fit using HRFBLANK was tested using periodic functions, as in this case we were testing the fit using a kernel that specifically did not fit the full stimulated spike sequence. 2.3.9 Bootstrapping to get confidence limits on R2 For each experiment, 200 bootstrap data sets were constructed, each with the same number of trials as the original, using random resampling with replacement (Supplementary Figure 2.10). The resampled hemodynamic and spike trials were then fitted against each other separately for both models (MLM and null) and R2 values were calculated as before (Supple- 20 mentary Note, equation (10)). The 95% confidence limits were obtained by taking the 2.5th to the 97.5th percentiles; similarly, 80% confidence limits by taking the 10th to the 90th percentile. Random reselection was done separately by contrast to have the same number of trials per contrast. However, each contrast used the same random number set to maintain stimulus blocks and reduce variability resulting from long-term drifts in physiology or recording stability. This was particularly necessary for the MLM, which involves subtracting the mean blank signals HBLANK and SBLANK from all other contrasts; if blocks are not maintained, this subtraction leads to a number of noisy outliers in the bootstrap estimate when a set of blanks trials dominated by one epoch of a session (for example, high signal) is subtracted from nonblank trials dominated by a different epoch (for example, low signal). 2.3.10 Cross-validation of HRFSTIM kernels across sessions For each session, we created a leave-one-out mean HRFSTIM kernel by averaging the two timing parameters (peak latency and width) across all kernels excluding the given one. Kernel amplitude was obtained by fitting, using this mean kernel to fit the given session’s data (HRFSTIM amplitude depends on an arbitrary scale factor in electrode recording; Supplementary Figure 2.11). This leave-one-out mean kernel with the best fitted amplitude was then used to obtain the cross-validation prediction and corresponding R2. Cross-validation was performed either across all animals or restricting the leave-one-out averaging to other kernels for the given animal. 2.3.11 Deconvolution The spike trace estimated by deconvolution was defined as where H is the relevant hemodynamic signal, HRF is the corresponding optimal HRF kernel and F and F−1 indicate forward and inverse (fast) Fourier 21 transforms, respectively. Given that F(HRF) has low power at high frequencies, reflecting the slow hemodynamic response, we filtered using a Hamming window with a 0.5-Hz cutoff in frequency space. This avoided high-frequency noise in the hemodynamic signal from being amplified during deconvolution. The same filter was used to discount high frequencies in the measured spike rate before correlating with the deconvolved estimate. 2.3.12 Checking the stability of our primary findings against variability in electrode recordings If measured spiking S is a veridical scaled sample of the true spiking s of our models despite measurement variability across experiments (different electrodes, different thresholds for spike detection for MUA), then the amplitude of the fitted HRF should simply scale inversely with measured spiking for a given experiment (Supplementary Figure 2.11 and Supplementary Note, equation (7)) The scale factor (between the measured S and the true s) will cancel out in all equations for a given experiment, leaving model features unchanged (that is, kernel shape, trial-related signal T and R2). We tested for this in two ways. First, we tested the effect of varying spike detection thresholds. In five experiments, we recorded the electrode signal at a low threshold and then rethresholded off-line to generate multiple sets of spiking data S for the same imaging data (for example, peak spike rates from about 300 s−1 to about 10 s−1 for progressively higher thresholds; Supplementary Figure 2.11a,b). These rethresholded spike data were then fitted separately against the common imaging signal (Supplementary Figure 2.11c–g). In a second test, we checked the linearity of the relation linking HRFSTIM amplitude against the inverse of the SSTIM amplitude over our full data set (integration window for mean SSTIM coextensive with stimulus duration as in Figure 2.2a; Supplementary Figure 2.11h). 22 2.4 Results For these experiments, we used three rhesus macaques (monkeys Y, T and S; n = 34 recording sites across five hemispheres; monkey S was also used previously (Sirotin & Das 2009)). The animals’ task, which was cued by the color of a fixation spot, involved fixating and relaxing (that is, free viewing) periodically for a juice reward. This task is known to evoke robust trialrelated signals in V1 (ref. Sirotin & Das 2009). A trial typically comprised a single fixation, with fixed trial periodicity of 10–20 s. For one set of experiments, trials consisted of sequences of two or three fixations, each rewarded for correct fixation. Visual stimuli comprised drifting sine-wave gratings that were presented passively while the animal fixated. The grating contrast was typically varied in five log2 steps plus a blank, presented in randomized order; the contrasts varied in some experiments and grating orientation was optimized for each electrode recording site. In addition, to compare with our earlier results (Sirotin & Das 2009), we performed a set of experiments in darkness (see Online Methods). We recorded concurrent MUA and hemodynamics from V1. For hemodynamics, we used intrinsic-signal optical imaging, a high-resolution optical analog of fMRI that deduces cortical hemodynamics by measuring fractional changes in the intensity of light reflected off the cortical surface at wavelengths absorbed by hemoglobin (Bonhoeffer & Grinvald 1996 Sirotin & Das 2009; Sirotin et al. 2009). We specifically used the blood volume signal imaged at 530 nm (green), as it directly measures changes in total local tissue hemoglobin concentration, and thus in local blood volume (Sirotin & Das 2009; Devor et al. 2003; Sheth et al. 2004; Nemoto et al. 2004). Furthermore, it matches corresponding fMRI signals (Fukuda et al. 2006). A particular advantage of this imaging signal is that its impulse response to a brief sensory stimulus is monophasic, with an increase in absorption followed by a monotonic return to baseline, presumably reflecting the stimulustriggered increase and subsequent decline in local blood volume (Sirotin et 23 al. 2009; Devor et al. 2003; Sheth et al. 2004; Nemoto et al. 2004). The monophasic stimulus-triggered response makes the imaging signal easy to interpret and to model mathematically (see Supplementary Note). 2.4.1 Spikes poorly predict hemodynamics in periodic task Our recordings showed, as expected, stimulus-driven spiking and hemodynamic responses with amplitudes monotonically reflecting stimulus contrast trial by trial (Figure 2.1). In addition, many recordings revealed a robust spiking signal locked to trial onset that was common to all of the spike traces and was most evident for blank trials (SBLANK; Figure 2.1b). Additional evidence suggests, however, that this blank-trial spiking, maintained low during fixation and high in between fixations, is also visual (Supplementary Figure 2.8). The time course of this signal matched that of the animal’s eye traces (Supplementary Figure 2.8a), and it was extinguished in the dark, even when the animal’s eye trace patterns remained unchanged (Supplementary Figure 2.8b). This signal was therefore likely a result of the animal looking around the dimly lit room and then at the gray monitor, periodically, in each trial. 24 Figure 2.1 – The full hemodynamic signal is poorly predicted by local multiunit spiking. (a) Top, a section of the full recorded spiking signal (S) for a representative experimental session (black trace). Bottom, corresponding measured hemodynamic signal (H, black) and the best prediction obtained from spiking (orange, S, indicates convolution; Supplementary Note, equation (3)). Inset, best fitting kernel HRFNULL (amplitude normalized) obtained by fitting H to S. Mean R2 = 0.49, as calculated using mean signals averaged across contrasts (n = 261 trials total, roughly 43 per contrast). Red line segments indicate stimulus application and vertical dotted lines indicate fixation trial onset. Red and black arrows below traces indicate typical responses to high-contrast (100% contrast) and blank (0% contrast) stimuli, respectively; for hemodynamics, increasing negative amplitudes, that is, increasing absorption of light by cortex, equals increasing blood volume. Note the poor match between the observed and predicted traces leading to a large residual and, consequently, low mean R2. (b) Trial-aligned averages of spiking (S) for each contrast. The trial structure is indicated by the color bars (gray, fixate; red, stimulus; no bar, relax). Note the prominent blank-trial spiking signal SBLANK. (c) Data presented as in b for hemodynamics (H). (d) Data are presented as in b for corresponding predicted hemodynamics (solid lines, top) and residuals (H- , dotted lines, bottom; separated vertically for visibility). Individual R2, calculated separately per contrast, are shown alongside each prediction. Data were obtained from monkey S. Error bars represent s.e.m. b. Time (sec) 0 2 4 6 8 10 c. dR/R 0.02 -0.04 0 -0.02 Time (sec) 0 2 4 6 8 10 Blank (0.0) 6.25 12.50 25.00 50.00 100.00 Spike rate (Hz) 0 100 200 300 Contrast (%) Spikes , all correct trials SHemo , all correct trials H trial sequence: grey: 'fixate', red: ‘stim’; rest of trial: ‘relax; d. 0 Time (sec) 0 2 4 6 8 10 dR/R 0 -0.02 a. 800 850 900 950 1000 -0.04 0 0.04 0 200 400 Spike rate (Hz) dR/R Time (sec) 0 -1 0 15 R2 = 0.49 HRFN ULL Predicted Residual -4.21 -0.12 0.26 0.46 0.59 0.58 SBLANK HBLANK 31 responses, averaged over the window as in Figure 2.2b. (c) The spiking and hemodynamic responses shown in a and b, plotted against each other. The gray lines are regression lines for each session and the red line is the average of the regression lines. Expressions show regression and R2, both for the experiments in Figure 2.1 and Figure 2.2a–c (exp) and the population (avg). Population averages were calculated from session values, weighted by number of trials within a session (n = 34 sessions, 3 monkeys). 2.4.3 Trial-related signal consistent in stimulus and dark room With the stimulus-evoked portion of the signal well characterized by our MLM model, we next estimated the posited spikeand stimulus-independent trial-related signal T (Figure 2.4and Supplementary Note, equation (2,9)). According to the MLM, this is the signal that remains after subtracting away, from the full measured hemodynamics H, all components that can be predicted from spikes. To estimate spike-predicted components, we used our simplifying assumption that the HRFSTIM kernel can be applied uniformly to all spiking, whether stimulus evoked or uncontrolled (blank trial). The HRFSTIM was obtained, as before (Supplementary Note, equation (7)), by fitting stimulus-evoked spiking (SSTIM; Figure 2.4a) to hemodynamics (HSTIM; Figure 2.4b). The hemodynamics predicted from full spiking S using this kernel (Figure 2.4c) were clearly different from the full measured hemodynamics (Figure 2.4b). Qualitatively, however, the latter appear to be a sum of the prediction riding on top of a large contrast-independent response. Indeed, subtracting the predicted from the measured hemodynamics left large remaining signals T that matched each other closely across contrasts. Note, moreover, their substantial strength, which was 1.5-fold greater than that of the maximal HSTIM (compare Figure 2.4b with Figure 2.4d). It is important to emphasize that, in our framework, these unpredicted hemodynamic signals are not the results of nonspecific spiking (for example, SBLANK); they comprise the components that, according to the MLM, remain after using HRFSTIM to account for the entirety of the spiking-related 32 hemodynamics, both stimulus evoked and nonspecific (Supplementary Note, equation (9)). Figure 2.4 – Estimated trial-related signal T is consistent across contrasts, across experiments, and between stimulated and dark-room trials. (a,b) Spiking (a) and hemodynamics (b) from another representative session. Insets, corresponding SSTIM and HSTIM. The trial structure is indicated by the a. b. Spike Rate (Hz) 0 200 400 600 0.02 -0.03 -0.02 -0.01 0.01 dR/R 0 0.02 -0.03 -0.02 -0.01 0.01 dR/R 0 0.02 -0.02 -0.01 0.01 dR/R 0 0.02 -0.03 -0.02 -0.01 0.01 dR/R 0 Time (sec) 0 5 10 15 Blank 3.125 6.25 12.5 25 50 100 -10 0 15 R2 = HRFST IM f. c. d. e. Spike 0 200 T: Stimulus T: Dark 0.02 -0.03 -0.02 -0.01 0.01 dR/R 0 Time (sec) 0 5 10 15 g. -1 -0.5 0 0.5 1 3 6 0 T: Stimulus T: Dark h. 600 0 015 Corr. r 0 -0.02 015 Population: Median Corr. r 0 0.25 0.5 0.75 1 0 15 30 0.02 -0.03 -0.02 -0.01 0.01 dR/R 0 0.90 Measured: mean Pred from spikes Spikes SDARK Trial-related signals : T Hemo: H Spikes: SSTIM trial sequence: grey: 'fixate' Stimulated trials Dark-room and comparison with stimulated trials Spikes: S Hemo: HSTIM Hemo HDARK Measured: individual Preds X HRFSTIM S Pairwise correlation T: Dark vs. Stimulus Population T: Dark vs. Stimulus: T: Dark vs. Stimulus Corr. r= 0.94 # sessions # sessions 33 color bars (gray, fixate; red, stimulus; no bar, relax). (c) Prediction convolving optimal kernel HRFSTIM (inset) with full spiking S (that is, HRFSTIM ⊗S). (d) Estimated trial-related signals ( ; Supplementary Note, equation (9)) shown individually by contrast; red indicates mean across contrasts. Note the high amplitude of mean T (s.d. = 0.0088, compared with 0.0058 for HSTIM using 100% contrast; a 1.5-fold difference). Note the marked similarity of signals T across contrasts (correlation (Pearson’s r) with leave-one-out means: 0.99, 0.99, 0.99, 0.99, 0.99, 0.99 and 0.96 for contrasts 0–100% in sequence; n = 175 trials, 25 per contrast, 7 contrasts, median r = 0.99). Inset, population histogram of median r (mean (s.e.m.) = 0.94 (0.01), n = 34). (e) Dark-room trials for the sessions shown in a–d. Top, hemodynamic traces, HDARK. Gray lines are individual traces, all correct trials (n = 45), the green line is the mean of correct trials, and the red line is the prediction, convolving HRFSTIM with dark-room spiking SDARK (bottom trace, black). Trial structure indicated on time axis (periodic fixations in darkness). (f) Mean trial-related signals T from stimulus-driven (black) and dark-room trials (green), same session (Pearson’s r = 0.94). (g) Signals T as in f for full population (n = 19 pairs, monkey T). (h) Pairwise correlations between stimulated and corresponding dark-room T over population (mean (s.e.m.), pairwise Pearson’s r = 0.61 (0.08), n = 19 pairs). We quantified the similarity of the trial-related signals T to each other, at different contrasts in an experiment, by correlating the signal T at each contrast (including contrast = 0, blank) with the leave-one-out mean of the signals T calculated at all the other contrasts. All of the resultant correlation (Pearson’s r) values were very close to 1.0 (Figure 2.4d). This pattern was repeated over our population of 34 experiments giving, in each case, a median r close to 1.0 (Figure 2.4d). We wanted to test how well the trial-related signals thus calculated matched each other across experiments and how similar they were to the trial-related signals observed in dark-room fixation tasks (Sirotin & Das 2009). For monkey T, we were successful in getting sets of both dark room and visually stimulated trials in 19 experiments (5 of the current 34, and an additional 14 from a separate project using the same fixation task). Over this population, we found a close match of each residual with the dark-room signal at the 34 same recording site, as well as a marked similarity of these signals across experiments (Figure 2.4e–h). As in our earlier published data (Sirotin & Das 2009), the dark-room trials evoked high-amplitude stereotyped signals of clockwork-like periodicity despite weak spiking (Figure 2.5e). However, the dark-room trial-related signal T (that is, after subtracting the, albeit very small, spike-related prediction obtained by convolving with HRFSTIM; Figure 2.4e) closely matched the mean trial-related signal T from the visually evoked trials (Figure 2.4f). A similar pattern was seen for each experiment. The sets of all dark-room and visually stimulated trial-related signals were markedly similar (Figure 2.4g) and matched each other well when correlated pairwise for each recording site (Figure 2.4h). This provides compelling evidence for our MLM, that is, that the full hemodynamic signal evoked in an alert task-engaged subject is the linear sum of a spike-related component and a distinct trial-related component that is independent of local spiking or visual stimulation (Supplementary Note, equation (2)). As an additional test of our premise that the trial-related signal is determined by trial structure independent of stimulus or evoked spiking, we designed a set of experiments in which we varied trial structure while keeping the stimulation parameters unchanged (n = 10 experiments in 2 animals, monkeys S and T; Figure 2.5). Both series consisted of 30-s trials with identical stimulation (gratings, with contrasts: 0% (blank), 12.5% or 100% in block-randomized order, shown once per trial). The trials had different fine structure, however. For one set, the monkey made two fixations at 15-s intervals in each 30-s trial (Figure 2.5a–c), whereas in the other set, the monkey made three fixations at 10-s intervals per 30-s trial (Figure 2.5d–f). The stimulus was presented only during the first fixation, whereas subsequent fixations were on to the blank monitor. Notably, we only considered those trials in which the animal performed sequences of correct fixations extending over the full 30-s trial to be correct trials. Blank trials, correspondingly, consisted of two (Figure 2.5a–c) or three (Figure 2.5d–f) successive correct blankmonitor fixations starting with the blank (0% contrast) stimulus. 35 Figure 2.5 – Estimated trial-related signal T reflects trial timing independent of stimulus timing or contrast. (a,b) Spiking (a) and hemodynamics (b) for trials consisting of fixation sequences in which the animal fixated with 15-s periodicity, but the stimulus (three contrasts, including 0%, blank) was shown at 30-s intervals, that is, only at the first fixation of each pair. Insets, corresponding SSTIM and HSTIM, calculated by subtracting away blank-trial signals consisting of responses to the pair of fixations on to the blank monitor starting with the blank stimulus (indicated by blue curves). Note monophasic stimulus-evoked HSTIM with no evidence of oscillatory rebounds during the blank epoch. The trial structure is indicated by the color bars (gray, fixate; red, stimulus; no bar, relax). (c) T per contrast. Inset, optimal HRFSTIM calculated over 30-s trials. Note that the signal T is close to exactly periodic at the 15-s fixation periodicity, with identical amplitudes for the first and second fixation periods independent of stimulus strength or evoked spikes in the first fixation (correlation of calculated T across stimulus contrasts: 0.98 (median of pairwise correlations between each T and the leave-one-out mean of the other two), n = 74 trials total, roughly 25 per contrast). (d–f) Data are presented as in a–c, with the same stimuli, presented at the same 30-s intervals, but with the monkey fixating every 10 d. Spike rate (Hz) Time (sec) 0 100 200 300 400 a. Blank 12.50% 100.00% Spike rate (Hz) Time (sec) 0 110 220 330 440 b. dR/R Time (sec) -0.04 -0.03 0 0.02 -0.02 -0.01 0.01 e. Time (sec) dR/R -0.02 -0.15 0 0.01 -0.01 -0.005 0.005 c. Time (sec) f. Time (sec) -1010 0 20 R2 = 0.96 HRFST IM 0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30 -1010 0 20 R2 = 0.97 HRFST IM Trial-related signals : T Two 15-sec fixations in one 30-sec trial Spikes: S Three 10-sec fixations in one 30-sec trial Trial-related signals : T Correlation r = 0.98 (median) Correlation r = 0.97 (median) Spikes: SHemo: H 010 20 30 0 -200 200 400 Spikes: SSTIM 3020100 0 -0.03 -0.015 0.015 Hemo: HSTIM 010 20 30 -100 100 0 200 300 Spikes: SSTIM 010 20 30 -0.03 -0.02 -0.01 0 0.01 Hemo: HS TIM Hemo: H 36 s. The stimulus was shown only on the first fixation of each triplet (n = 83 trials total, roughly 28 per contrast). All error bars indicate s.e.m. Even though the recorded spiking S, for both trial structures, was dominated by stimulation at the 30-s trial periodicity (Figure 2.5a,d), the recorded hemodynamics H showed additional powerful modulations, even for the blank trials (Figure 2.5b,e). Notably, this modulation matched the fixation schedule and was thus distinct for the two-fixation versus the three-fixation trials. The signal amplitudes in the second and third intervals were not proportional to the amplitude of the signal in the first fixation interval, as they would have been if they were a result of ringing following the initial stimulation. On subtracting away the relevant blank-trial signals, the stimulusevoked hemodynamics HSTIM in each case showed a monophasic decline to baseline, with comparable time courses (Figure 2.5b,e), as would be expected for a monophasic blood-volume response to the stimulus Sirotin et al. 2009. Finally, the trial-related signals T were, in each case, periodic at the trial fine structure with no apparent modulation by the stimulus (T obtained as above by subtracting away from each measured signal H components predicted from full spiking using the relevant HRFSTIM; Figure 2.5c,f). 2.4.4 Spikes poorly predict blank-trial and dark-room signals As a counterproposal to our MLM, it could be argued that there is no need to invoke any special spike-independent trial-related signal T. Although we have provided evidence that blank subtraction leads to a markedly improved linear fit between the stimulus-evoked portions of the signal, it does not follow that the blank-trial hemodynamics necessarily contain signal components independent of spiking as proposed in the MLM. Instead, it could be that blank-trial hemodynamics are related linearly to the sometimes substantial blank-trial spiking (Figure 2.1b) through a distinct HRF kernel appropriate for low spiking levels that is very different from the kernel linking the stimulus-evoked signals. This, one could argue, is the reason for the mismatch 37 when trying to predict the full hemodynamics from the full spiking (Figure 2.1). As we show below, however, any such distinct HRF kernels appear arbitrarily variable and unreliable, making this counterproposal highly nonparsimonious and thus implausible. We first tested whether the blank-trial hemodynamics could be predicted linearly from spiking alone (Figure 2.6a–d and Supplementary Note, equation (11)), as opposed to being modeled by a sum of spike-predicted and trial-related components (that is, MLM; Supplementary Note, equation (4)). Indeed, we could make a reasonable prediction (R2 = 0.46; Figure 2.6a) by using the optimal blank-fitted kernel HRFBLANK obtained by fitting the blanktrial hemodynamics to blank-trial spiking. This was better than the value of R2 = −0.19 for the prediction using the same session’s HRFSTIM (Figure 2.6a). This, however, is not surprising. By definition, the fitting process discovers a kernel that maximally accounts for the variance in the fitted signal. However, the fit here was likely fortuitous. The blank-fitted kernel was fivefold larger in amplitude and opposite in sign to the HRFSTIM, giving absurd predictions for the stimulus-evoked signal when convolved with the same session’s stimulus-evoked spiking SSTIM (Figure 2.6b). Over the population, these blank-fitted kernels were highly variable in amplitude relative to the corresponding HRFSTIM (Figure 2.6c) and, moreover, showed a wide scatter in peak latency and width (Figure 2.6c,d). Notably, the presence of both positive and negative amplitudes made it meaningless to even perform a cross-validation test to see how well the kernel from one day can be used to predict the blank signals from other days, in sharp contrast to the reliable predictions obtained through cross-validation for the HRFSTIM (Figure 2.2f). 38 Figure 2.6 – Blank-trial and dark-room hemodynamic responses are poorly fitted to spiking. (a) Blank-trial responses (same experiment as shown in dR/R HRFDAR K 010 -45 0 515 R2 = 0 200 0 0.02 -0.03 -0.02 -0.01 0.01 Hz e. g. HRFBLANK HRFDARK HRFDARK Dark-room-fitted vs. blank-fitted kernels Time (sec) 0 5 10 15 0 15 Time (sec) 0 15 Time (sec) 015 Peak latency Peak width Blank-fitted vs. stimulus-fitted kernels Time (sec) 0.58 Dark-room: Spikes: SDARK Dark-room: Hemo Measured: HDARK Pred: using HRFDARK Time (sec) a. Hz 0 -0.02 -0.01 0.01 dR/R 0 5 10 15 0 200 HRFB LA NK 0010 5 515 R2 = 0.46 Measured: HBLANK Pred: using HRFBLANK Pred: using HRFSTIM Blank: Spikes: SBLANK Blank: Hemo Peak latency Peak width HRFBLANK HRFSTIM HRFSTIM 015 0 15 Time (sec) 015 d. Time (sec) Time (sec) b. Time (sec) 0 5 10 15 0.04 -0.01 0 0.01 0.03 dR/R 0.02 0.05 0.06 0 -0.02 015 Stim: Hemo pred using HRFBL ANK HSTIM -20 -10 0 10 HRFBLANK HRFDAR K normalized by HRFBLANK 200 5 10 15 -1 0 1 f. Time (sec) 0 3 -20 -10 0 10 Amp. Blank/Stim # sessions 20 -10 0 10 200 5 10 15 HRF HRF B LA NK STIM normalized by c. 0-10-20 10 Amp. Blank/Stim 0 2 6 4 # sessions Time (sec) HRFBLANK 015 Time (sec) HRFBLANK Time (sec) HRFBLANK 015 Time (sec) HRFBLANK 015 Time (sec) HRFBLANK Time (sec) 39 Figure 2.4). Top, SBLANK. Bottom, corresponding HBLANK comparing the measured value (blue) with two alternative predictions: one from SBLANK using session’s stimulus-fitted kernel (red, HRFSTIM SBLANK, R2 = −0.19, n = 25 trials) and the other using the optimal blank-fitted kernel (brown, HRFBLANK SBLANK, R2 = 0.46). Inset, the blank-fitted kernel HRFBLANK (amplitude normalized to HRFSTIM). Gray bar indicates fixation. (b) Predictions (HRFBLANK SSTIM) of stimulus-evoked hemodynamics using blank-fitted kernel. Compare with measured HSTIM (inset) (mean R2 = −11.9). (c) Population of HRFBLANK kernels, each normalized by amplitude of corresponding HRFSTIM kernel. Inset, histogram of HRFBLANK amplitudes normalized by corresponding HRFSTIM (Amp. Blank/Stim., n = 34). (d) Peak latencies (left) and widths (right) of HRFBLANK versus HRFSTIM. Note the high variability (s.e.m.) in both parameters for the HRFBLANK. Population average latency (s.e.m): blank, 4.6 (0.6) s; stimulus, 3.1 (0.2) s; population average width (s.e.m.): blank, 5.8 (1.0) s; stimulus, 3.3 (0.2) s; n = 33; ignoring one outlier with width = 1.25 × 108 s for the blank). (e) Data are presented as in a for the dark-room task (data from Figure 2.4e). Top, SDARK. Bottom, corresponding HDARK comparing measured trace (green) with prediction using the optimal dark-fitted kernel (magenta, HRFDARK SDARK, R2 = 0.58). Inset, dark-fitted kernel HRFDARK (amplitude normalized to HRFSTIM). (f) Top, HRFBLANK kernels normalized by absolute values of their own amplitudes (note variability in time courses). Bottom, HRFDARK kernels normalized by corresponding HRFBLANK amplitudes. Inset, histogram of HRFDARK amplitudes normalized by HRFBLANK (Amp. Dark/Stim., n = 19 sessions). (g) Scatter plots of peak latencies (left, Pearson’s r = 0.04) and widths (right, r =0.07) comparing HRFDARK and corresponding HRFBLANK (n = 18; outlier ignored as in d). The presence of both dark-room and visually stimulated trials for 19 recording sites allowed for additional tests of the counterproposal to the MLM. If there exist valid low-spiking-level kernels linearly linking blank-trial spiking to hemodynamics, then such kernels should also be reasonable for linking dark-room spiking to hemodynamics. We tested this possibility by calculating the optimal dark-fitted kernels for each of these sessions (Supplementary Note, equation (12)). These kernels, again provided, by definition, good fits for the given dark-room signals; but they were arbitrarily different from the corresponding blank-fitted kernels. Thus, for the particular 40 example session, the dark-fitted kernel was opposite in sign and much larger in amplitude (40-fold versus fivefold larger than the amplitude of the session’s HRFSTIM; Figure 2.6e), reflecting the smaller dark-room spiking amplitude compared with blank-trial spiking. Over the population, we found similarly poor correspondence in amplitude, latency and width between darkand blank-fitted kernels (Figure 2.6f,g). The apparently arbitrary shapes and sizes of these kernels, when combined with our earlier evidence for lawful and stereotyped trial-related signals when fitting the MLM (Figure 2.4 and Figure 2.5), strongly suggest that darkand blank-fitted kernels reflect only accidental matches linking weak residual spikes to hemodynamics that are actually dominated by spike-independent trial-related signals. 2.4.5 Blank subtraction required to estimate spikes from imaging The primary use of neuroimaging is as a proxy for local neural activity. We wanted to quantify the importance of blank subtraction when equating the imaging signal with neural response. To this end, we compared the validity with which we could deduce measured spiking from full versus blanksubtracted hemodynamics by deconvolving (Glover 1999) with the relevant optimal HRF. We expected that deconvolving a given blank-subtracted (that is, stimulus evoked) signal HSTIM using its optimal kernel HRFSTIM would trivially return a valid estimate of the corresponding blank-subtracted (that is, stimulus evoked) spiking SSTIM, as these signals were well fitted to each other. What we wanted to assess, for comparison, was the reliability with which the full measured spiking S could be obtained from the full hemodynamics H using a similar deconvolution with its optimal kernel HRFNULL. Given that convolution is equivalent to the product of Fourier transforms in frequency space, we deconvolved by dividing the Fourier transform of the hemodynamic signal by the Fourier transform of the relevant HRF kernel. As HRFs have very little power at high temporal frequencies, reflecting the slow time course of the hemodynamic response Sirotin et al. 2009, we restricted 47 proposed earlier on empirical grounds, already do so effectively; these include designs that linearly subtract either blank (Pestilli et al. 2011; Shtoyerman et al. 2000; Larsson et al. 2006) or nonspecific global from local signals (Donner et al. 2008; Fox et al. 2006), designs that contrast one sensory stimulus against another in a common task structure (Meng et al. 2005; Cheng et al. 2001), and designs that regress the hemodynamic signal against a range of stimulus intensities (Engel et al. 1997; Boynton et al. 1999). Notably, for such subtraction to properly reveal stimulus-related signals, the subtracted trial presumably needs to be identical to stimulated trials in all respects (timing, reward, evoked anticipation, etc.), differing only in not containing the stimulus of interest. At another level, however, the robust link between stimulus-evoked hemodynamics and spiking throws into sharper relief the lack of such a link for the trial-related signal and suggests that other non-sensory signals may be similarly poorly related to local spiking (Jack et al. 2006; Donner et al. 2008). For example, there is a well-known discrepancy between robust fMRI evidence for attentional modulation in human V1 (ref. Ress et al. 2000) and the lack of such modulation in electrode recordings from macaques (Luck et al. 1997). Our findings raise the possibility that this entire class of non-sensory signals could have neural underpinnings distinct from sensory-driven spiking activity. The trial-related signal described here is also distinct from the coherent ongoing V1 activity imaged in anesthetized animals (Arieli et al. 1995), as the latter was closely correlated with local spiking and LFP (Arieli et al. 1995). The trial-related signal may involve neuromodulatory (Logothetis 2008) input from some brain stem center that tracks behavioral timing or it may reflect feedback from some higher cortical center. Furthermore, it could act preferentially on cells other than pyramidal neurons, such as interneurons or astrocytes. There is also the possibility of direct neuromodulatory control of blood vessels giving rise to the hemodynamic signal (Krimer et al. 1998). Finally, when interpreting recorded hemodynamics as a measure of local neural activity, it remains to be established whether the 48 strength of the trial-related signal can be equated with that of stimulusevoked signals on any common measure, such as a metabolic one of energy consumption. A number of additional questions remain. We have demonstrated the MLM for area V1. It would be important to see how well it generalizes over other brain regions. Furthermore, we controlled spike rate by varying the contrast of large, uniform gratings, which changes activity monotonically over the population of activated neurons. We could get valuable insights by controlling spike rates in a manner that differentially targets different neuronal populations, for example, by having punctate stimuli, thereby possibly getting different rates of spatial drop-off in different signals (LFP, spiking, hemodynamics). We noted that the goodness of fit between sensory hemodynamics and gamma-band LFP was much more variable than with spiking (data not shown), in contrast with earlier reports Logothetis et al. 2001. Finding answers to these questions would be critical for interpreting functional brain imaging. 2.6 Supplementary information 2.6.1 Appendix: Homogeneous Linear (‘Null’) and Modified Linear Model (MLM) of Spike-predicted Hemodynamics Appendix: Homogeneous Linear (‘Null’) and Modified Linear Model (MLM) of Spike-predicted Hemodynamics Let us define the full measured hemodynamic signal as H(t) and the net local spiking as s(t). The homogeneous linear model (our ‘Null’ model) proposes that the two should be related through: H = hrf s + n (1) where ( ) denotes convolution, hrf(t) is an appropriate hemodynamic response kernel and n(t) is noise (the explicit ‘(t)’ dependence is dropped in this and subsequent equations to avoid clutter). By contrast, the Modified Linear Model (‘MLM’) proposes that H and s are related through: 49 H = hrf s + T + n (2) where T is the anticipatory trial-related hemodynamic signal Sirotin & Das 2009. is modeled here as a spike-independent signal at trial periodicity, aligned to trial onsets and present in all correctly completed trials Sirotin et al. 2012. We assume that the shape of T is determined only by trial timing, independent of whether the trial has a visual stimulus or a blank, or even involves dark-room fixation. Note that these equations are expressions of the functional forms of the respective models; for any data set, the optimal hrf satisfying Eq 1 is likely to be different from that satisfying Eq 2. To test these models against data we assumed that the hrf kernels – here, ‘impulse functions’ quantifying the monophasic spike-triggered increase in local blood volume – can be modeled as standard monophasic gammavariate functions (Cohen 1997). Optimal kernels were calculated for each model by fitting to data through a standard least-squares routine – specifically, fitting the appropriate mean measured hemodynamic response, aligned to trial onsets and averaged over all contrasts and all correct trials, to the corresponding mean spiking response. (See Methods) For the ‘null’ model this process generates the kernel defined as ‘HRFNULL’ (best fitting the equation H = HRFNULL S) where S is the measured spiking. The optimal predicted hemodynamics, using this model, then becomes: (3) (A note on nomenclature: upper case ‘S’, ‘HRF’ etc will be used throughout to distinguish the measured spiking and fitted kernels from their ‘idealized’ equivalents (in lower case) as in Eq 1,2. This is to formally recognize that while the H in Eq 1 can be equated with measured hemodynamics, the measured multi-unit is S only a sample of the ‘true’ net local spiking ‘s’. The sample on any given session depends on the electrode tip, MUA spiking threshold etc. With later control experiments we show that the sampling is reliable across days, with the primary variability being restricted to an 50 irrelevant scale factor. (see last section of Methods and Supplementary Figure 2.11).) For fitting the MLM to data it is necessary to first separate the trial-related T from spike-predicted hemodynamic components. We do so by subtracting blank-trial responses from other trials, based on our assumption that T is stereotyped and present in all correct trials. In blank trials, hemodynamics HBLANK and spiking sBLANK should be related by: HBLANK = hrf sBLANK + T + n (4) The sBLANK is the trial-related blank-trial spiking noted earlier (Figure 2.1b), presumably due to uncontrolled visual input associated with the animal’s trial-locked eye fixations. Note that the MLM explicitly relates this spiking to its hemodynamic correlate with a common hrf like any other visually evoked spiking. On taking means across all correct trials, aligned to trial onsets, (denoted by the symbol ) we get: (5) which, on subtracting from Eq 2 gives: (Note: blank-trial subtraction has been used empirically earlier in image analysis (Shtoyerman et al. 2000; Larsson et al. 2006)): (6) Let us define these blank-subtracted hemodynamics and spiking as the corresponding ‘Stimulus-evoked’ quantities HSTIM = H - HBLANK and sSTIM = s - sBLANK . Eq 6 can then be rewritten: HSTIM = hrf sSTIM + n (7) The hrf in this model (identical in Eqs 2 and 4 through 7) can now be estimated by fitting the measured sSTIM = S - SBLANK to the measured HSTIM = H - HBLANK using Eq 7, to give an optimal gamma-variate kernel HRFSTIM. The corresponding predicted ‘Stimulus evoked’ hemodynamics is then: (8) This HRFSTIM can be used to obtain the presumed trial-related signal T. We can define T as the mean signal that remains after subtracting, from the full 51 measured H, all signal components predictable from measured spiking whether stimulus evoked or uncontrolled (Eq 2). With our assumption that all visually driven hemodynamics is related to corresponding spiking through a common HRFSTIM, we get: (9) Finally, to evaluate the goodness of fit of each model we compared model predictions with their experimentally obtained correlates. The comparisons were quantified using the measure R2, defined: (10) where and are, respectively, the variance of the measured hemodynamics, and of the residual error. This latter quantity is defined, in general, as the difference between measured and predicted signals. R2 always has a value <= 1; the closer it is to 1, the smaller the residual relative to the measured signal, and the better the fit. In cases where the prediction is made using the optimal kernel for the given measured signal, R2 defines the fraction of signal variance explained by the prediction, and it lies, by definition, between 0 and 1. For the null model this is the case, for example, when comparing the mean averaged across all contrasts, against the corresponding mean measured H since these quantities are optimally linked via HRFNULL (see Eq 3. The residual here is defined as ). However, R2 is also a useful measure in other comparisons, e.g. when quantifying how well the optimal kernel for the overall mean signal (averaged across contrasts) predicts the signal for particular contrasts. For the subset of trials at a particular contrast the error can be much larger than the measured signal, leading to large negative R2 values. We used the same formalism to test two controls that we contrasted against the MLM (Results, Section 6). As the first control we checked whether the blank-trial hemodynamic signal HBLANK could be predicted linearly from blank-trial spiking alone: 52 HBLANK = hrf sBLANK + n (11) This equation can be fitted using the measured SBLANK to formally define an optimal kernel ‘HRFBLANK’, and corresponding prediction . The goodness of this fit could be quantified using a measure R2 defined as above, with the residual being and the measured signal HBLANK. The same formalism can be used to test the control model that dark-room hemodynamics can be predicted using dark-room spiking alone: HDARK = hrf sDARK + n (12) This equation can be fit similarly using the measured SDARK to define an optimal kernel ‘HRFDARK’, corresponding prediction , residual and goodness of fit R2. 2.6.2 Supplementary Figures Figure 2.8 – Blank-trial spike trace SBLANK is likely visually driven, entrained to eye fixation schedule. (a) Upper panel: Same as Figure 2.1b (full spiking, aligned to trial onsets and averaged by contrast). Lower panel: Corresponding eye tracker records, also averaged by stimulus contrast, showing the monkey’s eye position relative to the fixation point. Note the Time (sec) 0 2 4 6 8 10 0 25 Deg. Eye Dist. Blank (0.0) 6.25 12.50 25.00 50.00 100.00 Contrast (%) Spike Rate (Hz) 0 100 200 300 a. Spike Rate (Hz) b. 0 20 40 60 Stimulus Blank Dark Room Time (sec) Deg. 0 5 10 15 0 14 Eye Dist. SBLANK 53 striking similarity between the eye position traces (for all stimulus contrasts) and the blank-trial spike trace SBLANK: high spiking before the animal fixates (i.e. before t=0 s, trial onset); lower spiking while the animal fixates on the grey monitor (t= 0 to 5.5 s) and high again when the animal looks away at the end of the fixation period. The same spiking pattern forms the common baseline for all spiking responses, independent of stimulus contrast. (b) Blank-trial spiking pattern is extinguished in darkness, even though the eye fixation schedule is unchanged. Upper panel: Spiking records from a day with both visually stimulated trials (one 25% contrast stimulus and a blank) and dark-room trials in the same experimental session. (All trials involved a common pattern of two fixations, as in Figure 2.5a-c; 9-s fixations, 18-s trials). Both the ‘stimulus’ and the ‘blank’ trials show the same common baseline of reduced spiking as the animal fixates and increased spiking in between fixations. No such fixation-linked fluctuations in spiking can be seen in the dark-room trials. Lower panel: Corresponding eye-position traces, essentially identical regardless of trial type, (stimulated or darkroom). Periods of fixation and stimulation indicated on the time line in color key as in all other figures. Figure 2.9 – Schematic of Modified Linear Model (MLM). (a) Stimulusevoked spiking, shown as responses to stimuli of different contrasts, linearly predicts corresponding stimulus-evoked hemodynamics (b) However, since the animal is engaged in a periodic task, on every (correct) trial there is an additional stereotyped Trial-Related Signal (c) that adds on linearly to give the net hemodynamic signal (d). Importantly, because the Trial-Related Signal adds on to every trial, it can also be linearly removed to reveal the ‘pure’ stimulus-evoked hemodynamics (panel b) by subtracting the blanktrial response (‘HBLANK’, panel d) from responses at other contrasts (other traces, panel d). The blank-trial spiking ‘SBLANK’ is shown here as a flat line in panel a. If in addition to the specific stimulus-evoked spiking there had also been a trial-related spiking response – presumably due to the animal moving his eyes periodically away from and back towards the monitor (Figure 2.1b, Supplementary Figure 2.8) – this spiking signal (not shown 02 4 6 8 10 Time (sec) 0.01 -0.06 -0.05 -0.02 0 -0.04 -0.03 -0.01 Blank 12.50 25.00 50.00 100.00 dR/R Spike Rate (Hz) 0 2 4 6 8 10 Time (sec) -100 0 100 200 300 Stim-evoked Spiking Stim-evoked Hemo 02 4 6 8 10 Time (sec) 0.04 -0.03 -0.02 0.01 0.03 -0.01 0 0.02 dR/R + Trial-Related Signal =02 4 6 8 10 Time (sec) 0.02 -0.05 -0.04 -0.01 0.01 -0.03 -0.02 0 Net Hemo Signal a. b. c. d. HBLANK SBLANK 54 here, to avoid clutter) would just have added uniformly to each trace in panel a. The corresponding hemodynamic response – predicted, according to the MLM, using the same kernel as the stimulus-evoked hemodynamics – would have added uniformly to each trace in panels b and d including, in particular, HBLANK. On subtracting the net blank-trial response HBLANK these other trialrelated signals would also get linearly subtracted away revealing just the stimulus-evoked signals (Figure 2.2a,b). Trial timing is indicated as usual in all panels, with periodic fixation, stimulus presentation and end of trial (i.e. start of next trial). Stimulation period is not indicated in panel c to emphasize that the trial-related signal is driven by the task structure alone. 55 a. HRFST IM : Modified Linear Model HRFNULL : Null Model e. -1 -0.5 0 0.5 1 1 0 0.5 -0.5 -1 Null Model Modified Linear Model (MLM) R2 mean d.1 0 0.5 -0.5 -1 -1.5 -2.5 -2 -3-1 -0.5 0 0.5 1 Null Model Modified Linear Model (MLM) Average of R2 per contrast Contrast: 6.25% 60 0 10 20 30 40 50 0 0.2 0.4 0.6 0.8 1-1.2 -1.0 -0.8 -0.6 -0.4 -0.2-1.4 b. Average of per contrast R2 0 0.2 0.4 0.6 0.8 1 R2 15 30 45 60 75 90 0 c. 200 0 0.2 0.4 0.6 0.8 1 R2 40 80 120 160 0 R 2 mean R2 0 0.2 0.4 0.6 0.8 1 0 50 40 30 20 10 Contrast: 12.5% 0 0.2 0.4 0.6 0.8 1 0 40 30 20 10 Contrast: 25% 0 0.2 0.4 0.6 0.8 1 0 80 60 40 20 Contrast: 50% 0 0.2 0.4 0.6 0.8 1 0 120 90 60 30 Contrast: 100% Modified Linear Model Null Model Original Median 95% confdc Original Median 95% confdc 6.25% 0.94 0.89 [0.73; 0.96] -0.12 -0.23 [-0.81; 0.18] 12.5% 0.94 0.91 [0.73; 0.97] 0.26 0.26 [0.06; 0.41] 25% 0.87 0.88 [0.74; 0.96] 0.46 0.47 [0.37; 0.56] 50% 0.96 0.94 [0.87; 0.98] 0.58 0.57 [0.48; 0.66] 100% 0.95 0.94 [0.90; 0.97] 0.59 0.61 [0.55; 0.66] Avg Contr 0.93 0.91 [0.86; 0.94] 0.35 0.34 [0.22; 0.43] Mean 0.99 0.99 [0.86; 0.94] 0.49 0.49 [0.43; 0.54] R2 56 Figure 2.10 – Comparing confidence intervals for the goodness of fit R2, for the Modified Linear Model (MLM) vs. the Null Model, bootstrap results. (Table and a-c: for same example data set as in Figure 2.1, Figure 2.2; d,e: population). (a) Superimposed histograms of R2 distributions, for the Modified Linear Model (MLM), in red, and the Null Model in orange (200 bootstrap runs, using sets of trials chosen randomly with replacement, each set processed using both MLM, i.e. fitting HRFSTIM, and Null Model, i.e. fitting HRFNULL, and calculating corresponding R2. See Methods for details). R2 calculated separately per contrast, as in the values labeling individual traces in Figure 2.2c. Note that while the histograms for the different contrasts have the same X scale, the 6.25% contrast panel has different X limits because at this contrast the R2s extend into negative values. N = 261 trials total, roughly 43 per contrast. Table summarizing the bootstrapping results for the example data set. Original refers to the R2 obtained for the full (original) dataset. Median designates the median of the bootstrapping procedure, and ‘95% confdc’ the 95% confidence interval calculated for the bootstrapping. Numbers in left-most column refer to stimulus contrasts; ‘Avg Contr’ refers to the average across all contrast, i.e. as in panel b, and ‘Mean’ refers to the mean R2 i.e. as in panel c. (b) R2 values from panel a, averaged across contrasts. Same calculation as in Figure 2.2d. (c) Mean R2 for the same bootstrap trial selections as in panels a,b but calculated after first averaging the trials across contrasts (calculations as in Figure 2.2c inset). (d) Comparing bootstrap-derived estimates of R2 for Null vs. MLM over population of experiments. Calculated separately by contrast and then averaged, as in panel b. We used 80% confidence limits in the population data, to accommodate some data sets with long tails in bootstrap distributions. Data points show the medians of bootstrap estimates while error bars show corresponding 80% confidence limits. Same data as shown in Figure 2.2e. Note, however, that the data points in Figure 2.2e indicate R2 for the experimentally obtained set of trials and are thus consistently slightly higher than the medians of the corresponding bootstrap estimates here. (e) Same, but for ‘Mean R2’ calculated as in panel c. Same data as shown in Figure 2.2d (again, data points slightly different from Figure 2.2d since Figure 2.2d shows R2 for actual trial set). 63 The data presented here concurs with the view that the hemodynamic response integrates activity from a variety of sources. Considering local neural activity as the sole predictor of hemodynamic activity does not give an accurate picture of the resulting response. 3.2 Methods Methods used here were the same as described in Sirotin & Das 2009. Simultaneous intrinsic signal optical imaging and electrophysiology were acquired from alert macaques engaged in different tasks. All experiments presented here were performed in darkness or quasi-total darkness. All experimental procedures were performed in accordance with the US National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the Institutional Animal Care and Use Committees of Columbia University and the New York State Psychiatric Institute. Detailed methods description of the experimental methodology used in the lab can be found elsewhere (e. g. Sirotin & Das 2009, Cardoso et al. 2012, Lima et al. 2014). Here are only referenced the methods that are particularly relevant for the data presented below. The data included in this chapter reflects a combination of different experiments, hence specific experimental details or analysis are included when appropriate throughout the text. Imaging data was acquired at either 7.5 or 15 Hz, at this rate pulsation is observable in the imaging response. Cortical pulsations were removed by low-pass filtering, using the Chronux MATLAB Toolbox function runline.m (typical heart rates were ~2–3 Hz, much faster than the typical hemodynamic response frequencies of ~<0.5 Hz). Heart rate was estimated from the pulsation. Each pulsation event has a peak and trough; the distance between peaks and troughs was used to identify lawful pulsation events. Heart rate was estimated as the frequency of the pulsation events. Slow temporal drifts (>30 s) were removed with high-pass filtering (using the 64 Chronux MATLAB Toolbox function runline.m), except: when calculating the power spectrum of the hemodynamic response on the variable inter-trialinterval manipulation and when looking at the slow changes on the hemodynamic response as the animals progress into blocks of high or low reward, in the experiments of manipulation of reward amount. In these exceptions, a linear trend (regression on the full time series) was subtracted from the response. Exceptions described in the text. In the section where we manipulated fixation duration, one of the fixation duration schedules that we presented obeyed the rule of a ‘memoryless’ process. By memoryless process it is meant that the probability of an event happening in the future given it has not yet happened is independent of the time that has passed since the beginning of that process. More formally: p(x > ta + tb | x> ta) = p(x> tb). Using the rules of conditional probabilities: p(x > ta + tb | x> ta) = p(x> ta + tb  x> ta) / p(x> ta), and as time is continuous: p(x> ta + tb  x> ta) = p(x> ta + tb), therefore it comes that p(x > ta + tb | x> ta) = p(x> tb) = p(x> ta + tb)/ p(x> ta), thus p(x> ta + tb) = p(x > ta) * p(x > tb). Not demonstrated here, but mathematically this universally will be valid if the p(x> ti) will be given by an exponential function: p(x>ti) = (the probability will be between 0-1, we will use a random number between 0-1 to generate samples for t, fixation duration). We sampled fixation durations within fixed time intervals (there was a minimum fixation time, tmin and a maximum one, tmax) t = tmin –  *log(rand[0;1]) and t  tmax (if t > tmax, then a new t is calculated, until t  tmax). 3.3 Variable Trial Durations Aiming at characterizing different aspects of the task-related response, we present data collected similarly to data in Sirotin & Das 2009, with small variations mentioned below. Subjects performed a fixation task in quasi total darkness with the monitor having only a small fixation point exposed 65 (otherwise all light from monitor was blocked, as in the original study by Sirotin & Das 2009). In this manipulation of the task, the subject was asked to perform the fixation task in two different schedules. Either the schedule was of short trials, with a shorter inter-trial interval, ITI, (from 6 to10 s, only one short ITI used in each session), or long trials, with a longer ITI (12 to 20 s, only one long ITI used in each session). The task the animal performed was a fixation task. The color of the fixation point cues the subject of when to aim for fixation. Once the animal achieves fixation the color of the fixation point changes and once the fixation period ends the fixation point turns off and the animal receives a liquid reward for holding fixation. Finally, there is a time delay (ITI) between trials. Trials in which the animal did not hold fixation for the requested period were not rewarded. Trials were presented in different schedules, where ITI was changed. The different schedules were presented in blocks. In general, we requested the subjects to perform 20 complete trials until the schedule changed (starting a new block). The animal was not cued for the change in ITI schedule. Sessions could start with a short or long ITI (the order of the first ITI, if long or short, did not affect the results). Once again, the subject’s task was to hold fixation for the period the fixation point had a particular color (the subject was requested to hold fixation for 2-4 s, varying between sessions). We recorded the hemodynamic response using intrinsic signal optical imaging, already described, and simultaneous electrical recordings in the imaging region. Furthermore, from the imaging response we extracted the heart rate response (see methods). Based on what we know about the task-related response from prior work done in our laboratory (Sirotin & Das 2009), we hypothesized that, within the total trial lengths that we have used here, the task-related response should, on average, entrain to the trial duration. Still several aspects of this entrainment are not understood. One of the questions we had about the task-related response was: what are the events that trigger the entrainment of the hemodynamic response? 66 With this systematic alternation of blocks of short and long trials, we wanted to evaluate if there were systematic changes in the time course of the responses. In particular we wanted to know if there are changes early in a short vs. a long trial, up until the duration of the shorter ITI. Behaviorally, trials should be similar (until the duration of the shorter ITI) as the task the animal has to do is the same: fixation. Up until when the duration of the ITI does not exceed that on a short ITI trial, all the trials (short or long ITI) are indistinguishable. The aim was to understand what aspects of trials (e.g., effort of fixation, reward) could influence the hemodynamic entrainment, i.e., what behavioral events could be involved in eliciting or triggering the hemodynamic task-related response. Simultaneous to the recording of the hemodynamic response, we recorded MUA activity of one or two electrodes in the same region where we recorded the neuroimaging data. We also recoded the heart rate response; reconstructed from the brain pulsation observed in the neuroimaging response. We looked for concomitant changes in either of these additional metrics with the hemodynamic response. Figure 3.1 shows a typical session (11 sessions were recorded for one subject, monkey T). In Figure 3.1a is the mean hemodynamic response to short (top) or long (middle) trials. It can be noticed that the responses to short or long trials are different, but the profiles up until the duration of the short trial are similar (Figure 3.1a bottom panel). For the session shown, the mean trial duration for a short trial was 15.92  0.02 s and for a long trial was 23.95  0.03 s (changes in trial durations reflect small changes in the time the animal takes to achieve fixation on each trial). Looking at the spectral profile of the hemodynamic signal (Figure 3.1d) it can be noticed that there are peaks in this spectrum that closely match the two schedules of completed trials (15.2 and 21.4 s, for short and long trials, respectively). There are other peaks in the spectrum not directly related to the schedule of completed trials, though. It should be mentioned that incomplete trials had shorter durations because the subject did not hold 67 fixation during all the requested period and no reward is administered, These time windows were not discounted from the duration of the inter-trial interval. Incomplete trials have less stereotypical durations as the animals interrupt trials at different moments. For all sessions there were more complete than incomplete trials. Still, the presence of peaks at higher frequencies than either short or long trial frequencies could be related to incomplete trials. The profile for the heart rate response in Figure 3.1b was also similar for the two trial schedules, and different from the hemodynamic response. It is worth noting that the heart rate spectrum for this session did not show clear peaks at the different trial schedule frequencies (Figure 3.1e). Finally, the spiking rates of a multi-unit electrode located the same region as the imaging did not show a significant pattern associated with this task (Figure 3.1c), as expected from Sirotin & Das 2009. For this session, the MUA spectrum had multiple peaks, but not tightly matching the trial schedules (Figure 3.1f). 68 Figure 3.1 – Single session showing that different trial schedules have associated different hemodynamic, heart rate and MUA responses. (a) Hemodynamic responses to short trials (top), long trials (middle) and overlay of average short trials (black) and long trials (gray) (bottom). Error 0 0.1 0.2 -0.2 0 0.2 0.4 0.6 Frequency (Hz) Log Power Spectrum 8.5 5.85.1 11.3 18.1 30.8 0 0.05 0.1 0.15 0.2 -8 -6 -4 -2 Frequency (Hz) Log Power Spectrum 14.7 11.9 8.0 0 0.05 0.1 0.15 0.2 -12 -10 -8 -6 Frequency (Hz) Log Power Spectrum 21.4 15.2 12.5 010 20 0 50 100 150 Spiking (Hz) time (sec) relative to trial start 010 20 1.4 1.6 1.8 2 Heart rate (Hz) time (sec) relative to trial start 010 20 -0.01 -0.005 0 0.005 0.01 -dR/R time (sec) relative to trial start 010 20 0 50 100 150 Spiking (Hz) 010 20 1.4 1.6 1.8 2 Heart rate (Hz) 010 20 -0.01 -0.005 0 0.005 0.01 -dR/R 010 20 0 50 100 150 Spiking (Hz) 010 20 1.4 1.6 1.8 2 Heart rate (Hz) 010 20 -0.01 -0.005 0 0.005 0.01 -dR/R fed cba 69 bars calculated as s.e.m.. The red horizontal lines indicate the period the animal was requested to hold fixation. (b) Similar to a, but for the heart rate response. (c) Similar to a, but for the multi-unit spiking activity. (d) Logpower spectrum of the hemodynamic response. (Power spectra were calculated on the data without discounting of slow drifts that might occur throughout the imaging session, that were in general always used, unless otherwise stated, as here.) The red crosses indicate peaks around the trials’ periodicities; the numbers indicate the corresponding period. (e) Log-power spectrum of the heart rate; similar to d. (f) Log-power spectrum of the MUA activity, similar do d. From observing the single case presented in Figure 3.1 it is noticeable the apparent similarity of the hemodynamic response during some initial period in which short and long trials are indistinguishable. We therefore went on to quantify the similarity in the shape of the hemodynamic profile within the first seconds of completed short or long trials. We looked at the initial 10 s of all completed trials, either for trials coming from blocks of short or long trials (n = 11 sessions). We selected this interval because it was smaller than the length of a short trial (for all considered sessions). For this quantification we first computed the Pearson’s correlation coefficient for the initial 10 s of all pairs of short trials; and the Pearson’s correlation coefficient for the initial 10 s of all pairs of long trials. We established the distribution of correlation coefficient profiles for short-short and long-long trials. We then calculated the Pearson’s correlation coefficient for the initial 10 s of all the pairs of short and long trials. We used the previous correlation coefficients for the pairs of short-short and long-long trials to have a comparison baseline for the correlation coefficients of short-long trials. Our null hypothesis was that the distribution of correlation coefficients for short-long trials is indistinguishable from the distribution for short-short (or long-long) trials. In Figure 3.2a, it can be noticed that the distribution of Pearson’s correlation coefficients for shortshort trials or long-long trials are similar. In this session the hemodynamic task-related response is fairly stereotypical: the correlation profile is skewed towards 1. More importantly this distribution was also similar for compari- 70 sons between short-long trials (lighter gray, Figure 3.2a). Independently of the particular shapes of the task-related response on each session, or how stereotypical this response was within a session, we observed that the distribution of our similarity measure (the Pearson’s correlation) was preserved (within a session). The similarity between short and long trials was independent of the variability of the task-related response within each session. For each session we tested how similar the distributions were for short-short trials versus long-long trials (Figure 3.2b); no session reached significance. Then we used a non-parametric Wilcoxon signed-rank test to compare the distribution profiles of correlation coefficients short-short vs. long-long vs. short-long trials. Once again the comparison of the shapes of the distribution of correlation coefficients for short-long trials against either short-short or long-long trials did not reach significance for any session (Figure 3.2c). This indicates that the three distributions were not statistically different. It supports the hypothesis that for the initial duration of short or long trials the hemodynamic responses are indistinguishable (within the trial schedules that we have tested). Figure 3.2 – The initial few seconds (10 s) are indistinguishable for short or long trials. (a) For the same session as in Figure 3.1, the computed correlation between the initial 10 s of all completed trials, either long or short. In black is the correlation profile between the initial seconds of short trials (trials share a significant similarity, therefore correlation tends to be high: close to 1). In dark gray is the correlation profile between the initial seconds a b c 0 0.5 1 0 1 2 3 p-value number of sessions 0 0.5 1 p-value number of sessions 0 2 4 -1 0 1 Correlation between trial pairs Portion of trials Short-Short Long-Long Short-Long 0 0.1 0.2 71 between long trials. In light gray is the correlation profile between the first few seconds of long and short trials. Each correlation profile was normalized by the total number of pairs compared. (b) Population data summary for 11 sessions. p-value of the comparison between the correlation shapes for short or long trials. Note: red line markes significance level (significance value, , corrected for multiple comparisons; correlation profiles were binned in 21 intervals: Bonferroni correction, resulting  = 0.05/21); no session is below the significance level. (c) Similar to b, but for the combination between the correlation shapes of short-long trials relative to either short of long trial only correlation distribution (total of p-values: 22, two for each of the 11 sessions). Note: no session reaches significance (same significance level as in b). We observed that the hemodynamic response tended to entrain to the trial schedule, but the initial response to short or long trials (which contains the initial increase in blood volume) was indistinguishable for the two conditions. We cannot disregard that for different trial schedules (much shorter, much longer or larger difference between short and long trials) the hemodynamic response to short and long trials could be dissimilar. Lastly, for these set of experiments we wanted to understand if some of the observed peaks in the power spectrum could be related to the periodicity of short and long trials. Furthermore, we wanted to know if either of the activity measures (hemodynamic, heart rate or MUA) had spectral peaks that better matched the trial schedule, than the other metrics. In this attempt we generously looked for 20 peaks in the power spectrum, below 0.5 Hz. The number of peaks found per metric (hemodynamic, heart rate or MUA) per session was variable. Some sessions did not present clear peaks below 0.5 Hz (this is the reason for the different number of data points in Figure 3.3bd). Next, for each session, we computed the minimal distance between the spectral peaks and the frequencies corresponding to short and long trials (Figure 3.3a, in seconds). In this sample of sessions there was no clear superiority of the hemodynamic response in having peaks that matched the short and long ITIs. This metric is rather crude; the spectra did not always 72 have clear peaks. It suggests that spectra might be noisy or the entrainment of the different measures with trial periodicity, even though weak, happens to all measures (for these trial durations). In Figure 3.3b-d we plotted the peak positions against the period of the trials (short or long). The peak positions were chosen to be the closest to the actual trials durations (short or long). The different panels (In Figure 3.3b-d ) reflect the different metrics, hemodynamics, heart rate and MUA. If the spectral peaks would fully reflect the trial duration, then the distance between the closest spectral peak and the trial period should be zero; there is however large dispersion. The peaks’ positions tend somewhat to follow one another on the different metrics. Heart rate had the lowest number of detectable peaks (below 0.5 Hz) and its relationship with the other metrics was the least uniform. Heart rate seems to entrain less with trial duration, when compared with the hemodynamic response or multi-unit activity. Figure 3.3 – Power spectrum peaks only partially reflect the difference in trial lengths. (a) Histogram of distance (in seconds) between the lengths for b c d 10 15 20 25 30 0 10 20 30 HEMO HR 010 20 30 0 10 20 30 HR SPKS 10 15 20 25 30 HEMO 0 10 20 30 SPKS -2 0 2 4 6 8 10 12 14 16 0 10 20 a time distance (s) counts SPKS HEMO HR 79 intervals, that explains why there are more segments than experimental sessions). In black are the segments that had a longer peak for longer fixations, in red the sessions where the hemodynamic response for the longer fixation peaked before the one for the shorter fixation. Note that most lines are blue. Dotted segments were used for monkey T, solid ones for monkey S and dotted-interrupted ones for monkey E. (b) The distribution of the amplitudes associated with the peaks in a. The peak amplitudes were zscored so they could be represented on the same plot. Same color code and line style as in a. (c) Similar to a, but for the heart rate peak response (here we considered the maximum peak, independently of how many peaks the average response exhibited). (d) Similar to b, but for the heart rate peak amplitude. (e) Scatter plot of the peak positions of the hemodynamic versus heart rate responses showing, for each experimental session, the value for the longer fixation relative to that for the shorter (set to zero). Lines join short-long pairs for each session. Note: responses tend to be predominantly in the first quadrant. Dashed, solid and dotted-interrupted segments used for monkeys T, S and E, respectively. (f) Similarly to what was done to the peak positions in e, here for the peak amplitudes. The data presented comes from several variations on the fixation task, with only a small number of sessions for each manipulation. Nevertheless it opens interesting questions pertaining the task-related response: could the effort of fixation or anticipation of reward, for example, play a role in triggering this hemodynamic task-related response? 3.5 Variable Reward Amount Previously we manipulated aspects of the timing of the trials; but wanted to further explore behavioral correlates of the mean hemodynamic task-response. We set up a slightly different task with the goal of altering the reward level that the animals experienced, and thus explore potential hemodynamic task-related response changes. We manipulated the reward amount that the subject received in each trial, in a block wise manner (no additional cues). Otherwise the animals performed the same fixation task. 80 As in the previous manipulations, the fixation task was presented in quasitotal darkness. Reward was manipulated in blocks; blocks of either of low or high reward. There was at least a twofold difference between the reward levels. The block structure we imposed required the animal to complete 10 trials before the reward level changed. Behaviorally, the animals tended to be sensitive to the level of reward; breaking fixation or engaging in trials less frequently when the reward was lower, as shown in Figure 3.6. In Figure 3.6, the probability of completing trials of low reward, in red, was in general lower than for high reward trials, in blue. It should be mentioned that the ideal behavior is to complete trials independent of reward level. Incomplete trials were not rewarded and incurred in a ‘punishing’ inter-trial-interval until the start of a new trial (no active punishments or additional time details were included). In 3 of the 35 sessions the performance in blocks of low reward trials was higher than for high reward ones. Figure 3.6 – Behavior of the fixation task in blocks of high and low reward. In red is the fraction of completed low reward trials and in blue the fraction of completed high reward trials. Each pair of lines (red-blue) is for one session. (3 monkeys: S: 6 sessions, T: 19 sessions and E: 10 sessions.) Once more we looked at trial average hemodynamic task-related response. In Figure 3.7a-d are illustrated responses to a single session (for monkey T). In Figure 3.7a-c are the average trial triggered hemodynamic responses for Monkey S Monkey T Monkey E 0 0.5 1 Performance 81 low and high reward trials. It could be noticed that for this session the magnitude of the hemodynamic response for high reward trials was higher than for low reward ones. This was not accompanied by concurrent spiking changes (Figure 3.7d). We wanted to know if the same trend was observed for the population; we tested the magnitude of high and low reward trials. To test for significance, we used a Wilcoxon signed-rank test (p-value=6.7x10-6) indicating the magnitude difference between high and low reward to be directional (Figure 3.7e). Magnitude of low reward trials was on average smaller than for high reward ones. a b c d e f 0 0.005 0.01 0.015 0 0.005 0.01 0.015 Std for LOW reward trials after discouting DC contribution Std for HIGH reward trials after discouting DC contribution Y = 0.96 X - 0.47x10 R = 0.927 -3 2 Time (s) relative to fixation onset 0 5 10 15 -0.02 0 0.02 Low Reward -dR/R 0 5 10 15 -0.02 0 0.02 Time (s) relative to fixation onset -dR/R Hi Reward -8 -4 0 4 8x 10-3 -dR/R 0 5 10 15 Time (s) relative to fixation onset 0 5 10 15 -20 0 10 20 30 40 Time (s) relative to fixation onset Firing rate (relative to mean) -10 Low High 0 0.01 0.02 0.03 -dR/R mean trial amplitude Reward 82 Figure 3.7 – Magnitude of the hemodynamic task-related response changes with reward amount. (a) Hemodynamic task-related response for an individual session. In gray the individual response on each completed trial of low reward; in red is the average response. (b) Similar to a, but considering all complete trials of high reward in the same session; in blue is the average response. (c) Overlay of the average responses in a and b. Error bars represent standard error of the mean. (d) Similar to c, but for the electrophysiological data. Note that the difference between the two distributions of spiking responses is only significantly different at one time point, in this session, which probably coincides with reward delivery (not consistently observed for other sessions). (e) Hemodynamic amplitude (peak to trough) of low and high reward trials. Each line represents one session, where the peak to trough amplitude of the average low reward trial is on the left and the corresponding peak to trough for high reward is on the right, for the same session. In red are the sessions where the reported amplitude for low reward trials was higher than for high reward ones. Significance test: Wilcoxon signed rank task, p-value: 6.7x10-6. (f) Noise variability measure between low and high reward trials. Scatter plot of the mean standard deviation (in time) for low versus high reward completed trials (before estimating the standard deviation of each trial, for subsequent average, to each trial the mean response on the trial was subtracted, as in panels a-c). The 90% confidence intervals were estimated from bootstrapping (100 repetitions). Red line represents a regression line through the data (parameters indicated on the plot’s inset). Note that this regression is below the equality line. We looked at the time dynamics of the task-related signal in low and high reward trials. We noticed that the trials in lower reward blocks had higher variability in time, than in higher reward blocks, (Figure 3.7f, a regression line through the data is below the equality line). Taken together, we observed changes in the hemodynamic response with reward level. Both the response magnitude and its variability in time changed with reward level: higher response magnitude and lower time variability for high reward trials and lower response magnitude and higher time variability for low reward trials. 83 Finally, we noticed that between blocks of low and high reward trials, there were slower fluctuations in the hemodynamic signal. To quantify this observation we took the initial hemodynamic signal change and removed only a linear trend from this signal and no other low frequency changes. For each trial we calculated the mean hemodynamic response to each trial (across the duration of each trial). As exemplified for the single session in Figure 3.8a,c. it shows a segment of a session; one could notice that mean hemodynamic response fluctuates with reward amount as one moves along a block of low or high reward. In this dataset, as one goes along a low reward block the mean hemodynamic level tended to increase (increased blood volume level changes), and as one moved along a high reward block, the mean hemodynamic level tended to decrease (decrease in blood volume changes). Looking in more detail at what was happening at the block level, as we illustrate in Figure 3.8c, where we looked at the same segment of data as in Figure 3.8a, but where only the mean trial response to completed trials is represented. We noticed that along a block the mean trial hemodynamic response to each trial seemed to evolve in a monotonic way. To evaluate if mean trial responses changed systematically with progression along a reward block, a regression line was fit to the mean trial response of the first 10 completed trials in each block (reward blocks with fewer completed trials were not included, and for the ones with higher number of completed trials, only the first 10 completed trials were considered). The regression lines are illustrated by the black lines on the plot Figure 3.8c. To test if there were systematic differences between regression lines of each of the reward levels (low or high), the slopes of the regression lines of all sessions were combined, segregated by reward level on each block, and we tested if the two distributions (slopes for low reward blocks and slopes for high reward blocks) were the same, using a Wilcoxon rank-sum test. In Figure 3.8d, it can be noted that the distributions are different, indicating that along a block of lower reward the hemodynamic mean trial response tended to increase and conversely for high reward blocks. 84 Figure 3.8 – Reward modulates not only the average hemodynamic response as well as the mean hemodynamic level. (a) Hemodynamic response on a segment of a session (same session at in Figure 3.7a-d). The dotted lines represent the separations between blocks of low and high reward, reward level on each block is indicated by the horizontal colored lines on the bottom; red for low reward blocks, and blue for high reward ones. (b) Distribution of the amplitude of the mean hemodynamic response for each complete trial (normalized by the standard deviation of the hemodynamic response change for the same session as in a), separated by reward amount (n = 7462 and 7315, low and high reward trials) (Additionally, in 29 of the 35 session the blood volume was lower on the average of trials for high reward trials than for low reward ones.) Wilcoxon rank-sum test on the two distributions: p-value = 4x10-89 < 0.05/(7462x7315), using the conservative correction of dividing the rate significance value by of the total possible combination of comparisons. (c) Same segment as in a, here only complete a b c d 2000 3000 4000 -0.02 0 0.02 Time (s) -dR/R 1000 -4 -2 0 2 4 6 x10-3 Low High Reward Slopes of mean signal (-dR/R) along a block 4000 -0.02 Time (s) 3000 2000 1000 -0.01 0 0.01 -dR/R -2 0 2 4 Low High Reward normalized DC amplitude (-dR/R) 85 trials are considered; the hemodynamic response to each trial is represented by the mean response on that trial (in red low reward trials, in blue to high reward ones). Dotted vertical lines (as in a) show transitions between reward blocks. Solid black lines represent individual regression lines that were fitted to each individual reward block (only blocks with 10 or more complete trials were included). Note: predominantly positive slopes in blocks of low reward trials, versus the predominantly negative ones for high reward blocks. (d) Distribution of the slopes for all the considered blocks from all sessions segregated by reward level on the block (n = 35 sessions, 487 blocks for low reward and 565 blocks for high reward). The distributions tested with Wilcoxon rank-sum test: in p-value = 5x10-63 < 0.05/(487x565), using the conservative correction of dividing the rate significance value by total possible combinations of comparisons. This manipulation of reward amount clearly showed that the hemodynamic response in visual cortex changes with reward level. The hemodynamic task-related response had higher magnitude and lower variability for high reward trials than low reward ones. These effects rode on top of the slower changes observed on the hemodynamic response, specifically, the mean hemodynamic signal tended to fluctuate slowly (we measured it at the trial period) and decreased as the subject progressed into a high reward block, and then increased as the subject progressed into low reward blocks. There is extensive research on the effects of reward in the brain. It has been shown in BOLD fMRI experiments with human subjects that increased reward probability is associated with increased stimulus BOLD fMRI activity (Serences 2008). Contrasting with these and our results, there is work using contrast-agent-enhanced fMRI techniques as well as D1-selective dopamine antagonist in alert non-human primates performing a range of visual tasks (Arsenault et al. 2013). This work was focused on changes in visual areas as the animals performed visual tasks. Their observations contradict somewhat we present here; they observed a selective decrease in fMRI activity when comparing rewarded to un-rewarded conditions, and they were able to relate these changes to D1 dopamine receptor activity (using a D1-selective antagonist they got smaller effects). There is a significant experimental 86 difference between Arsenault et al. 2013 experiments and Serences 2008 (and ours), they both used conditions with uncued rewards and not 100% contingency between cues and reward. The approach Arsenault et al. 2013 took is interesting, as they aimed at a looking at the effects of reward in visual areas independently of potentially other effects as attention. If a particular task, time or stimulus predicts an upcoming reward then a subject will naturally tend to attend to the predictive cue. In a different BOLD fMRI study, using a complex visual detection task, the effects of ‘incentive’ in the visually evoked responses, showed enhancement of those in several areas including visual cortex (Engelmann et al. 2009). Incentive was manipulated by the amount of future reward (money) for correct trials; arguably not the same as our reward manipulation. Moreover, we also did not have a stimulus, which we use look for changes in visually evoked responses. Still it is conceivable that the changes we observed in trial responses might be of the same class as the ones mentioned in Engelmann et al. 2009. The effects of attention in BOLD fMRI are well known, with associated increases of BOLD response with attention (Jack et al. 2006, Sylvester et al. 2007, Donner et al. 2008, Pestilli et al. 2011). In humans a similar reduction in BOLD fMRI responses in visual cortex for visual tasks with the presence of reward has been observed (Knapen et al. 2012). Here similarly, reward was not predicted by any specific stimuli. In the primate, Arsenault et al. 2013, observed as well that a decrease in fMRI response with reward was related to the reward amount (higher rewards caused larger decreases in the fMRI response). Using a paradigm where different stimuli in the visual field were associated (or not) with reward in human fMRI BOLD experiments, it was observed that the responses associated with reward stimuli had enhanced responses relative to unrewarded presentations of the same stimuli (Knapen and Donner, personal communication). Our experiments were performed in quasi total darkness and even though we did not image in the cortical representation of the fovea and reward had 100% probability if 87 subjects held fixation, we would not expect to find the difference in results arising from the fact we were not recording the fovea representation. These ambiguous results of the influence of reward in cortical visual areas as the subjects perform visual tasks, suggest that reward might be enhancing rewarded stimuli representation as well as reducing the signal-to- -noise ratio by means of reducing the mean signal. Further research is needed to better understand this question. 3.6 Task-related Hemodynamic Response in Visual Cortex to an Auditory-Motor Task Finally, we wanted to further explore the specificity of the task-related hemodynamic response to visual tasks. On the initial assessment of this problem in the laboratory an auditory/motor task was used, while recording in primary visual cortex. At that point results were inconclusive (it was not the focus of that study, but see supplementary material of Sirotin & Das 2009). We repeated and expanded those initial experiments. Here, we present data (15 sessions for monkey T and 1 session for monkey S) for the same task as initially used in Sirotin & Das 2009. In this task the animal initiated trials by pulling a lever, after a fixed delay (in the range of 10-15 s, in different sessions) a tone started, lasting for a fixed duration (2-4 s, in different sessions) after which, the tone changes pitch, cueing the animal that it could release the lever, the animal had a grace period (up to 2.5 s) to release the lever, if it did release it within that grace period, then a reward was delivered and the trial ends. All this was done in darkness, the monitor is off. Animals tended to entrain to this task and initiate a trial soon after the end of the previous one. It resulted that these trials had comparable durations to one another. The fixed time interval was selected so that the duration of a completed trial of this motor-auditory task was similar to the timing of the fixation task previously used. 88 We present also data from 7 sessions (for monkey E) of a modification of this task. Monkey E was initially trained on this same auditory/motor task that the other 2 animals performed, but with very poor performance. We modified the task making it closer to the fixation task, and the animal’s performance improved. The data presented here for monkey E is for this modified auditory/motor task. In this modified task, trials started with the presentation of a tone, cueing the animal to pull a lever. The animal was given a grace period to pull the lever (up to 2.5 s), failure to do so, meant an aborted trial. If the animal pulled the lever within the grace period, then the tone changed pitch and the animal was expected to hold to lever until the tone was turned off (2-4 s, in different sessions), cueing the animal that it could release the lever. Once more the animal was given a grace period (up to 3 s) to release the lever, and if it did release it within that grace period, then the animal was given reward and a fixed inter-trial interval (10-12 s, in different sessions) started until the beginning of the following trial. The data presented here (for either version of the auditory/motor task) was from recording days where at least one other session was also recorded. This other experiment was a fixation task done in quasi-total darkness. We aimed at recording trials in both sessions (fixation task or auditory/motor task) with similar durations. The recordings are always from primary visual cortex (for a subset of sessions, 12, electrophysiological data was also recorded, data not included here). We compared the trial average hemodynamic response for each session to each task type: fixation or auditory-motor task. The similarity between the task-related responses’ shapes to both task types in V1 is striking. In Figure 3.9, are two recording examples Figure 3.9a-f is for monkey T on the original auditory/motor task (compared with the fixation task) and Figure 3.9g-l is data for monkey E on the modified auditory/motor task. Both examples were compared against the fixation task. The hemodynamic taskrelated responses were similar between either of the auditory/motor tasks, when compared to the fixation task performed on the same day. 95 To explore the role that reward is playing in the hemodynamic signal, we designed another variation of the fixation task where we manipulated reward amount. If on the one hand this reward-related task does not disambiguate the aforesaid question, it unequivocally suggested that the task-related response is influenced by reward. Not only was the amplitude of the taskrelated response modulated by reward amount, but also a change in a slow, trial length, hemodynamic response reflected reward level. Finally, in the last experiment presented, where a fixation task was compared to an auditory/motor task, lead us to conclude that the hemodynamic task-related response can be a global response. At least a task that did not engage visual cortex, as the one used, still elicited a hemodynamic task-related response in primary visual cortex. Equally importantly, it was the striking finding that independent of the differences in the timing of the two tasks, in V1, the hemodynamic task-related response had on average a similar profile. Once more, this opens further questions. It points to some role of the decision of engaging on a trial, or some effort in doing or attending to trials, independent of the specific details of the task. There is something intrinsically related to the decision of taking up a trial that might be influencing the existence of the hemodynamic task-related response. On a different analysis of the same fixation task previously published in the laboratory, we observed that in trials that the animal fails to initiate did not on average present a task-related response (Sirotin et al. 2012, Figure 6). These non-initiated trials were significantly different from initiated ones, independently of the subject receiving or not a reward. In non-initiated trials the subject did not make an active effort of engaging with task on that trial. In trials the animal did not complete, but initiated, the difference to completed ones, rewarded, and the ones the animal initiated but does not complete was not as compelling as the difference between completed trials and noninitiated ones. 96 We looked back at the analysis and data included in Sirotin et al. 2012, Figure 6. We calculated a Pearson’s correlation coefficient between each trial in a session and the average completed trial for that session. For each session we have an average correlation for each trial category (noninitiated, initiated and rewarded, or completed, and initiated and not rewarded, or aborted). Figure 3.11 shows a scatter plot of the average correlation between completed trials and non-initiated or aborted ones, for each session previously used for Figure 6 in Sirotin et al. 2012. This suggested that independently of completing a trial there was a significant difference between engaging or not in a trial. Figure 3.11 – Lower correlation between non-initiated trials and completed ones, than between aborted trials and completed ones. Scatter plot of the session by session correlation between completed trials with the average completed trial (for each session) and the correlation between either aborted (black) or non-initiated trials (grey) (n = 68 sessions, 2 animals, monkey S and T). Error bars reflect s.e.m. Regression lines: grey (scatter plot aborted vs. completed trials), y = 0.19x10-3 + 0.57x, r2 = 0.21, black (scatter plot noninitiated vs. completed trials), y = -0.28x10-1 + 0.14x, r2 = 0.04. The task-related hemodynamic activity in V1 probably reflects different aspects of behavior other that just visual properties of the scene. Even though some of the experiments presented here were testing the influence of fixation on the hemodynamic task-related response, it can be argued that the effects observed there were small and as observed in Figure 3.5A average correlation (non-initiated trials) (aborted trials) 0 0.5 1 -0.5 0 0.5 1 average correlation (completed trials) 97 (distribution along the vertical axis) for each fixation duration presented there was a wide range of hemodynamic response delays. This suggested that fixation duration should not be triggering the hemodynamic response (at least as could be described by a simple hemodynamic response function). The hemodynamic response, probably reflects many aspects of behavior and physiology, we cannot exclude the possibility that there is a role for fixation in this response, but it should not be the sole driver specifically to the task-related hemodynamic response. The combined anecdotal evidence of a few experimental variations allows speculation on what mechanisms might be (or might not be) underlying this hemodynamic task-related response. Bearing in mind that we expect this response to be global (present in visual cortex in response to an auditory/ motor task, independent of the different timing of sequence of events for the auditory/motor task relative to the fixation task) and that there is a more pronounced difference between non-initiated trials to completed ones than between aborted trials and completed ones. It seems like a mechanism as attention could be very relevant when considering this signal. Potentially not attention in the sense of spatial attention, when one would expect at least some cortical dedication to the tasks performed, but a more global attention or anticipation of upcoming events. There is evidence from research in humans of global attention-like (e.g. Ress et al. 2000, Silver et al. 2007, Sylvester et al. 2007, Donner et al. 2008, Pestilli et al. 2011) and task-related changes in BOLD fMRI (Jack et al. 2006, Elkhetali et al. 2015, Griffis et al. 2015). We would be eager to see fMRI experiments that would look at other brain areas (cortical and subcortical) that could shed some light into potential neuromodulatory mechanisms. One can also entertain the idea that there is some potential for a global response related to arousal/engagement to have a role in this signal. Recent publications on the physiology of the locus ceruleus in the context of behavior, explore the role of phasic locus ceruleus firing with the decision of execute actions that are task relevant (Kalwani et al. 2014 and 98 Joshi et al. 2016). For the simple tasks that we used, it might be foreseeable that the most relevant decision that requires execution pertains engaging in the task. Without further experiments this remains in the realm of speculation. Hence, our empirical observations suggest that there is the potential that the hemodynamic response is reflecting the influence of different neuromodulatory systems. Finally, we have failed to find neural measures that reliably parallel the hemodynamic response; the closest related metric, in these observations was heart rate, but even heart rate did not fully track the observed nuances of the hemodynamic response. This reflects prior work in the laboratory where multi-linear regression models combining a set of neural regressors (MUA, various LFP bands) were used to predict the hemodynamic response (Sirotin & Das 2010a). Such models were only able to account for less than half the variance in the hemodynamic response. 99 4 Slow drifts in brain blood volume are associated with behavioral changes in primate V1 4.1 Abstract Neuroimaging signals (e.g. BOLD fMRI signals) are complex and reflect both exogenous and endogenous responses. There is extensive knowledge pertaining hemodynamic exogenous responses, while endogenous responses remain largely elusive. In the past, most studies of neuroimaging have typically focused on short-term stimulus-related hemodynamic changes on timescales of a few seconds. Here, we focus on the less investigated longterm fluctuations in local brain blood volume, and its correlation to behavioral engagement. We assessed behavioral changes through spontaneous fluctuations in the performance of monkeys engaged in a periodic fixation task. We simultaneously acquired hemodynamic signals (through intrinsic signal optical imaging), multiunit neuronal activity and local field potentials - LFP - (through intracortical microelectrode recordings), heart rate (derived from the optical imaging signal) and behavioral metrics related to quiescence. Notably, we observed robust increases in local brain blood volume during epochs in which the monkey was less engaged in the fixation task, as compared to lower blood volumes when the monkey was actively engaged or more motivated to perform the task. Accordingly, epochs of lower engagement were associated with increased power in lower LFP frequency bands, lower firing rates and lower heart rate. Therefore, epochs of low brain blood perfusion were associated with higher heart rate. Finally, we compared the state-dependent slow drift in brain blood volume with the hemodynamic task-related signal. The interplay between changes in blood volume and heart rate is complex; if on the one hand the slow drift in blood volume is negatively correlated with heart rate, the trial-related changes are positively correlated. 100 4.2 Introduction There is extensive use of neuroimaging signals for inferring neural activity. Techniques like blood-oxygen-level dependent (BOLD) functional magnetic resonance imaging (fMRI) that are minimally invasive and can measure signals from the full brain represent an advantage when compared to electrophysiology. When talking about neuroimaging signals one tends to focus on signals driven by external events. However there is significant evidence showing the presence of endogenous responses (like attentional or task-related) in neuroimaging signals (Corbetta et al. 1990, Tootell et al. 1998, Kastner et al. 1999, Ress et al. 2000, Serences et al. 2004, Jack et al. 2006, Sylvester et al. 2007, Donner et al. 2008) that are not correlated with local electrophysiology (Sirotin & Das 2009). The neuroimaging response is then the result of endogenous and exogenous contributions. Exogenous contributions reflect responses to external stimuli (like visual stimuli in visual cortex) and have a linear relationship with local neural activity (Cardoso et al. 2012, Lima et al. 2014) and endogenous neuroimaging signals do not trivially reflect underlying changes in local neural activity (Sirotin & Das 2009). Finally, in the visual system, there is ample evidence of the presence of both stimulusrelated and endogenous contributions to the hemodynamic response, in human and non-human primates (Jack et al. 2006, Donner et al. 2008, Pestilli et al. 2011, Sirotin & Das 2009). There are different activity patterns that reflect endogenous contributions to the BOLD signal; in particular work in humans shows the presence of BOLD modulation with spatial attention and task structure (Jack et al. 2006). In Jack and colleague’s study, subjects were asked to perform a near threshold visual detection task and report their decision immediately or after a fixed delay. They observed that the BOLD signal had two clearly separable components: an increase in BOLD associated with spatial attention to the near threshold stimulus, and another BOLD signal increase associated with 101 the structure of the task, and independent of the presented stimulus. The second, the task-related signal, has also been observed in our lab in nonhuman primates (Sirotin & Das 2009). An endogenous signal that has been widely reported in BOLD fMRI studies is associated with attention, there are several examples of attention in visual cortex (Kastner et al. 1999 Somers et al. 1999, Gandhi et al. 1999, Ress et al. 2000, Ress & Heeger 2003, Shulman et al. 2003, Serences et al. 2004). Previous work from our laboratory (Sirotin & Das 2009) in non-human primates performing a periodic visual fixation task compared task-related and stimulus-related hemodynamic signals with electrode recordings, in primary visual cortex. Sirotin and Das showed a task-related signal entrained to trial timing, in the absence of visual stimuli. This signal was comparable in magnitude to some stimulus related responses and could not be predicted from local multi-unit activity (MUA) or local field potentials (LFP), unlike the stimulus component. There are BOLD fMRI studies (Elkhetali et al. 2015; Griffis et al. 2015) as well that show evidence of a task-like activity in human subjects that is dependent of performance. In this study, subjects were asked to detect a change in a stimulus’s temporal frequency, in the context of visual, auditory stimulation, or bimodal auditory-visual stimulation (subjects were requested to report changes in one of the sensory modalities). In this task, there was evidence of a ‘task-initiation activity’ reflected in an increase in BOLD response in visual cortex with starting a block of trials when the subjects were asked to detect a visual stimulus, relative to performing the auditory detection task. The authors observed as well sustained changes in BOLD responses. They reported an increase in baseline activity within a block of visual detection trials relative to a block of auditory detection trials – ‘task-maintenance activity’. This last signal had higher magnitude in visual cortex for subjects performing better on the visual detection task; hence performance influenced the BOLD fMRI response. In the present study, we aimed at exploring the potential correlation between different states of engagement and the hemodynamic response. We explo- 102 red the behavioral correlates of the hemodynamic response in the alert animal preparation, using the laboratory’s established technique of simultaneous functional neuroimaging (intrinsic-signal optical imaging) and electrode recordings from primary visual cortex of non-human primates performing periodic visual fixation tasks. There is extensive literature relating changes in mental state to physiological changes. There are reported changes in heart rate and skin conductance concomitant with indices of arousal. Malmstrom and colleagues have shown changes in these metrics while subjects were watching a benign film (Malmstrom et al. 1965). Even earlier work by Angelo Mosso in cortical pulsation showed changes in pulsation rates and profiles with different mental states, for example sleep (translation of original publication: Raichle & Shepherd 2014). There are also known changes in LFP wavebands with arousal/attention, in particular the early work of Hans Berger and the development of electroencephalography. Hans Berger showed that alpha-band activity increased when subjects laid with their eyes closed relative to when they had their eyes open (for review see Millett 2001). Given the extensive evidence for different markers of endogenous activity in the brain we aimed at measuring neural and physiological signals with the goal of shedding light into the functional role of endogenous hemodynamic responses. Specifically, we recorded: MUA, pair-wise MUA correlation, LFP and heart rate. As mentioned, there are physiological and neural metrics that characterize different aspects of behavioral state. In this study we propose that there is a signature of task engagement that is reflected on the hemodynamic signal. To support our claims, we also report other behavioral markers. In the task we have used, we looked at periods in which the subjects were engaged in the fixation task, but we also looked at periods in which they were disengaged, not completing trials. This work parallels work done in a more established behavior change: arousal. We compared our observations with those known to occur with changes in arousal and alertness. 103 A common physiological marker of arousal is pupil diameter (for a review see Aston-Jones & Cohen 2005). Recent work (Reimer et al. 2014, Vinck et al. 2015) looked at different states of arousal in rodents (probing locomotion versus quiescence) observing distinctive patterns in visual cortex: increased gain of visual responses with increased arousal, decreased spiking noisecorrelations, and even recruitment of different cell classes. In terms of neuronal activity, they presented evidence for changes in spiking patterns with attention and arousal. Arousal was associated with improvement of signal to noise ratio in cortical areas with changes in ‘noise-correlations’ across electrodes, even though increased arousal was associated with lower firing rates (Livingstone & Hubel 1981). Changes in similar metrics are associated with attention; comparing correlations in the firing patterns of individual cells on a trial by trial basis (‘noise-correlations’) it has been shown that attention is associated with increases in firing rate (Moran & Desimone 1985, Luck et al. 1997), while attentional improvement was associated with decreases in interneuronal correlations (Cohen & Maunsell 2009, Cohen & Kohn 2011). We have used interneuronal correlations slightly differently in this study, computing those in time as opposed to across trial conditions, which was more appropriate for this study (visual cortex is sensitive to variations driven by eye movements, and for periods the subjects do not fixate it becomes meaningless such comparisons across engaged and disengaged trials, as the patterns of eye movements can be distinct). Finally, there is recent evidence of changes in hemodynamic signal that match changes in arousal, as measured by pupil dilation (Pisauro et al. 2016). In this study they looked at the hemodynamic response in the visual cortex of mice while viewing visual stimuli. They reported that hemodynamic response had two components; one related to the visual stimuli the other, more global and correlated with pupil diameter. The study we present here parallels Pisauro’s study and aims at further exploring the task-related component of the hemodynamic response to global changes in hemodyna- 104 mic signal, in different states of engagement in the task. In our study we did not use visual stimulation, so we could reduce influence of visual stimuli in the recording region. Moreover we include here electrophysiological data not included in Pisauro’s study. We trained subjects to perform a periodic fixation task in quasi total darkness, as reported before (Sirotin & Das 2009). Animals performing this task naturally exhibit periods of more and less engagement in the task. Our performance metric uses the segregation of initiated versus not initiated trials to estimate engagement in the task. We compared how the different measured signals relate to periods in which the subject was engaged in the fixation task versus periods in which it was not. Our data suggests that endogenous changes on the hemodynamic signal have a significant correlation with behavior; we observed increases in blood volume change with disengagement in the task. The changes in engagement were accompanied by changes in heart-rate; increased heart rate was associated with periods of higher engagement. We have observed a tendency towards an increase in lower frequency LFP power and decrease in higher frequencies in more disengaged periods. There was a slight decrease in firing rates a more robust increase in firing ‘noise correlations, across electrodes, in more disengaged periods relative to more engaged ones. This study brings attention into the importance of accounting for endogenous signals’ contribution to neuroimaging when using such techniques. 4.3 Methods 4.3.1 Summary In this chapter we used some data previously used in other publications (Sirotin & Das 2009; Sirotin et al. 2012) reanalyzed it for the questions addressed in this publication and collected a significant number of additional experiments. 111 values for each session is exemplified in Figure 4.1b. For comparisons across sessions, data was z-scored, and the average training and validation numbers were averaged, with each session contributing equally (Figure 4.1d). To further test the relationship between the values for initiated and non-initiated trials, for each pair (corresponding to the mean response on each 20% of the data for the two classes) an angle was also calculated (having initiated trials as the reference). By plotting the distribution of these angles one can observe how consistent or not is the relationship of initiated and non-initiated trials (Figure 4.1c). 4.4 Results 4.4.1 Hemodynamic mean trial response correlates inversely with engagement We wanted to understand if slow changes in the hemodynamic response were influenced by behavioral signals, similarly to previously reported (Pisauro et al. 2016) , in our setup, in primary visual cortex (V1),. We set up to record responses from non-human primates performing a fixation, in quasi total darkness, where only a fixation point was present. We simultaneously recorded intrinsic signal optical imaging, extracellular MUA and LFPs. The fixation task the subjects performed was rather simple, and with a limited number of possible outcomes. Animals could hold fixation for a required amount of time and receive reward; we call these trials completed or rewarded. Animals could initiate the fixation task, but interrupt fixation before the end of the required fixation time, not receiving reward: aborted trials. Or animals could not even initiate the fixation task: non-initiated trials. The behaviors exhibited were rather simple, but adequate to exploring correlates of task engagement in the hemodynamic signal. Therefore we focused our analysis on initiated vs. non-initiated trials. We defined a performance metric as the smoothed probability of initiating a trial (see methods). 112 The fundamental relationship between the animal’s performance in a task, hemodynamics, and different neural and physiological measurements is illustrated for a representative session in Figure 4.1. Blood volume changes increase when the subjects were less engaged in the fixation task. This can be observed in Figure 4.1 during periods of low engagement (shaded gray periods; performance below 50%) the trial mean hemodynamic signal tends to be higher. To quantify the relationship between hemodynamic signal and performance, we calculated the Spearman’s rank correlation between the mean trial hemodynamics and performance ( = -0.531, p-value < 0.01, in this session). In this session there was a negative relationship between the animal’s engagement and changes in blood volume; lower engagement was associated to higher volume. To understand if the observed changes were reliable for this session, or represented an average tendency but were not observed in shorter time periods, within each session; we used a 5-fold cross validation method. We split each session in 5 contiguous segments; one segment was kept as validation, the other 4 comprised the training set. We compared the mean hemodynamic signal to initiated versus non-initiated trials within the test and validation sets (Figure 4.1b). For this session we could segregate initiated from non-initiated trials. In particular when looking within each of the 5 individual segments (see the gray lines, which connect the average response to initiated and non-initiated trials in each segment) one could observe that in all instances the non-initiated trials (in red) had a higher hemodynamic signal than initiated ones (in blue). We quantified this directionality of initiated trials within each segment of 20% of the data having on average a smaller hemodynamic signal change than non-initiated trials. This quantification measured the angle between initiated to non-initiated trials. The measure of this angle is cartooned for one of the k-fold segments on Figure 4.1b. Angles within the first quadrant indicate higher mean blood volume changes for non-initiated than initiated trials (the reference were the initiated trials, and the same for all the other measures). 113 Performance HemodynamicsMUA Spike-Spike Corr LFP Alpha Gamma Heart Rate a 0 100 Percent Initiated -0.04 0.06 -dR/R 0 80 Firing rate (Hz) -0.2 0.5 Pearson’s r -0.4 0.3 Hz -80 80 µV 0 5x10-5 power 2x10-5 5x10-5 power Sequence of trials 100 200 300 400 500 600 ARI 0.4 1.6 power ratio Brain Mov 0 0.8 pixel Eye Prob 0 1 Prob. eyes closed 114 Figure 4.1 – How hemodynamic, heart rate and different neural and physiological measures track changes of performance in fixation task. (a) correlation  = -0.352 -0.8 0 0.8 0 10  = 0.380 -0.8 0 0.8 0 10  = -0.200 -0.8 0 0.8 0 10  = -0.250 -0.8 0 0.8 0 10  = -0.101 -0.8 0 0.8 0 10  = -0.018 -0.8 0 0.8 0 10  = -0.158 -0.8 0 0.8 0 10  = -0.101 -0.8 0 0.8 0 10  = 0.408 -0.8 0 0.8 0 10 -0.8 0 0.8 0 10  = -0.572 26 44 26 44 0.02 0.18 0.02 0.18 -0.15 0 0.15 -0.15 0 0.15 -0.015 0 0.015 -0.015 0 0.015 b -8 8 -8 8 1.6 2.8 1.6 2.8 x10-5 x10-5 2.9 3.6 2.9 3.6 x10-5 x10-5 c e f 90 270 180 0 49 -3 0 3 -3 0 3 training set: 80% of data validation set 20% of data -0.8 0 0.8 0 10 correlation # sessions  = -0.297 Hemodynamics 30 -2 0 2 -2 0 2 -0.8 0 0.8 0 10  = 0.154 MUA 16 -1 0 1 -1 0 1 -0.8 0 0.8 0 10  = -0.202 Spike-Spike Corr -1 0 1 -1 0 1 21 LFP -0.8 0 0.8 0 10  = -0.002 26 -1.5 0 1.5 -1.5 0 1.5 -0.8 0 0.8 0 10  = -0.006 Alpha 26 0-2 2 -2 0 2 -0.8 0 0.8 0 10  = -0.104 Gamma -3 0 3 -3 0 3 -0.8 0.80 0 10  = 0.331 Heart Rate 68 0.85 1.1 0.85 1.1 0-1.5 1.5 -1.5 0 1.5  = -0.054 -0.8 0 0.8 0 10 ARI 25 -0.8 0 0.8 0 10  = 0.335 Brain Mov 92 -0.8 0 0.8 0 10  = -0.557 Eye Prob 0-2 2 -2 0 2 4 4 0-2 2 -2 0 2 4 6 4 6 34 0 1.8 0 1.8 0 0.05 0 0.05 validation set training set d 115 Data from a single experimental session (for monkey T). Top: Animal’s performance (see methods) along a session as the probability of initiating a trial. Shaded grey periods reflect periods in which the animal’s performance was below 50%. Different traces: Mean hemodynamic response on each trial, for the same session as in a. Note the increases on the hemodynamic response during periods of lower performance (shaded gray areas). Similar analysis to the hemodynamic response for: heart rate, multi-unit activity electrode (‘MUA’), spiking correlations between electrodes (‘spike-spike corr’), local field potential (‘LFP’), power of alpha-band LPF (8-12Hz), arousal related index (‘ARI’), “brain movement”, and eye status. (b) Within session cross-validation for the hemodynamic response, for the same session as in a. Vertical axis: average response to 20% of the data (in successive segments) in initiated trials (blue) versus non-attempted trials (red); on the horizontal axis is the average response to the remaining 80% of the data (once more segregated by initiated and non-initiated trials, respectively). The lines connect initiated versus non-attempted trials in each 20% segment. Note, within each color category, there is less dispersion on the horizontal than vertical axis (as otherwise expected, that responses to 20% of the data are more variable between one another, than responses to 80% of the data). The angle measurement will be used in later analysis (e); the metric used considers the slope of the line that connects initiated to nonattempted trials. (c) Stacked histogram of the Spearman’s correlation between hemodynamic responses and performance for each session. Different shaded regions indicate the contribution of each of the three monkey (lighter grey: S, intermediate grey: T, darker grey: E). (d) Similar to c but for the subset of sessions that had p-value associated with Spearman’s correlations < 0.05, note that most tendencies get more evidenced. (e) Similar to panel b, for the population of data (not single session), data presented as z-score. Different animals are coded with different symbols (S: circle, T: diamond, E: square). The colored crosses indicate the mean for each condition (for clarity sake it is not included the means for each animal, but they are on average consistent with the global mean). (f) Population measure of the angular distributions of the slopes between the neural responses from initiated to non-attempted (individual angles as in panel b). Different color codes for the different animals as in panel c. The session presented is representative of the population, as noted when looking at the summary of the population data, Figure 4.1c-f. The distribution of Spearman’s rank correlation for the different sessions was on average 116 negative for all 3 animals. It indicates a clear tendency of increased blood volume during lower performance periods (Figure 4.1c, even more for the subset of sessions that had reached a significant Spearman’s correlation, Figure 4.1d); these results are reinforced by looking at the population of zscored 5-fold cross validation for all session, Figure 4.1e, and the angular directionality resulting from the same analysis, Figure 4.1f. 4.4.2 Complex interplay between hemodynamics and heart rate When looking at neuroimaging signals, one other relevant metric is heart rate. In healthy subjects, the rate at which new fresh blood arrives will be intimately linked to cardiac output. In the context of behavior there are several known heart rate modulatory effects: resting is also associated with decreased heart rate, cardiac deceleration in anticipation of behavioral events; startle responses associated with increased heart rates, bradycardia associated with attention, to name a few (for a summary for early results see e.g., Lacey & Lacey 1970, Lacey & Lacey 1978). We therefore looked at heart rate in periods of higher and lower engagement, similarly to the analysis presented for the hemodynamic response. On Figure 4.1a it can be noticed that periods of lower engagement in the fixation task were associated with decreases in heart rate ( = 0.384, pvalue < 0.01, for this single session, and it had the same trend for the population, as can be noticed in Figure 4.1c-f). This suggested a counter intuitive anti-correlated relationship between the mean trial hemodynamic signal and mean trial heart rate. One other aspect of the relationship between hemodynamics and heart rate was that the correlations tended to be more negative for sessions with lower overall engagement of the animal, as noted in Figure 4.2a. The changes in mean trial heart rate with disengagement match our expectation. Similar trends can be observed in arousal; increases in arousal are associated with increases in heart rate, conversely decreases in arousal 117 come associated with decreased heart rate. In what concerns slow changes in blood volume the observed results were more unexpected. Earlier observations from the lab (Sirotin & Das 2009) of heart rate changes during a trial showed concomitant changes with the hemodynamic response – no anti-correlations were found in that study. To confirm that the relationship between short term changes in heart rate and hemodynamics were positively correlated even when looking at different states of engagement, we looked at the average dynamic responses of both signals in a sub second scale (we looked at the signals at the imaging frequency). We looked at the average responses to initiated and non-initiated trials. Note that non-initiated trials are a construct from the way we present trials to the subject. In our paradigm, trials were presented periodically, independently of the animal engaging on them or not. Therefore, we recognize that this comparison can disproportionately average out contributions within noninitiated trials, as there is no event time structure. We observed that the average hemodynamic response to initiated trials had a peak a few seconds into the trial (Figure 4.2b). This response was overall lower for initiated than for non-initiated trials. Similarly, the average heart rate responses for initiated trials had also a peak (earlier than the peak in the hemodynamic response, Figure 4.2c). It should be noted that the average hemodynamic response change to completed trials (a subset of initiated trials; the ones that were rewarded) corresponds to the previously described task-related hemodynamic response (Sirotin & Das 2009). Moreover, these observations are consistent with our earlier observations (Sirotin & Das 2009). It is worth noting that the mean heart rate for non-initiated trials was lower than to initiated ones (as already seen in Figure 4.1). This suggests a positive correlation between changes within a trial in heart rate and hemodynamics, on top of anti-correlated changes in slower time scales, slower than the scale of individual trials, between the same metrics. To establish if the correlation between the hemodynamic and heart rate signals was in the full time series dominated by the faster time scales 118 (positive correlation) or by the slower time scales (negative correlation) we computed the cross-correlation for each session in two conditions. In the first, we simply calculated the Pearson’s correlation coefficient between the full time series for changes in hemodynamic and heart rate. This resulted in a preferentially negative correlation (Figure 4.2d). On the other hand, in the second approach we discounted the slow changes in the two metrics, by subtracting to the time course of each trial the mean response to that trial. In this case the correlation was preferentially positive (Figure 4.2e). Figure 4.2 – Relationship between the hemodynamic response and heart rate. (a) Session by session, Spearman’s correlation between the mean trial hemodynamic response and the mean trial heart rate response as a function of average performance on the session (open circles: monkey S, open diamonds or rhombus: monkey T, open squares: monkey E). (b) Trial triggered average of the magnitude normalized hemodynamic response. In blue, the average response to initiated trials, in red, the response to non-initiated trials (the different tones indicated different animals, lighter blue and lighter 0 0.5 1 -1 -0.5 0 0.5 1 Spearman's correlation Performance 0 5 10 15 -0.6 0 0.6 Hemo time (s) norm -dR/R Heart Rate 0 5 10 15 -0.6 0 0.6 time (s) norm HR a b c -0.8 0 0.8 0 10 correlation # sessions r = -0.157 20 -0.8 0 0.8 0 10 correlation # sessions r = 0.271 20 Cross correlation (mean trial subtraction) Cross correlation (no mean trial subtraction) d e 119 red indicate animal S, intermediate tones, monkey T, and darker tones, monkey E). (c) Similar to b but for heart rate changes. (d) Peak cross correlation coefficient for the full hemodynamic trace and full heart rate trace (only positive lags considered). (e) Similar to d, but where to the hemodynamic and heart rate traces the mean trial (for each trial) were subtracted. The relationship between hemodynamic signals and heart rate changes is of complex nature, and not linear. On a fast time scale (sub-trial time duration), they are correlated; the events happening with in a trial might trigger changes in heart rate that will have a reflection in the same direction in the hemodynamic response. On a slower time scale, the two metrics tended to be anti-correlated. Predicting the hemodynamic response based on heart rate is therefore not trivial: mathematically the two metrics have a nonlinearity relationship. 4.4.3 Different neural metrics and their correlation to engagement To understand the neural correlates of the hemodynamic changes we simultaneously recorded MUAs and LFPs with the imaging response (in the same cranial window). Firing rates tended to decrease in the periods the animal was less engaged in the fixation task: there was a weaker and positive correlation between firing rates and performance (Figure 4.1a,b,  = 0.171, p-value < 0.01, for MUA). We speculate that in periods the animal was less engaged in the task, the subject could take more breaks and close its eyes more often and for longer periods reducing even further the faint effects of having visual stimulation from fixation (virtually, the only source of light in the room). On the other hand, we observed the reverse pattern for the correlated activity across electrodes (see methods for our definition of the correlated activity between electrodes). In most of the sessions that we had electrophysiological recordings, we had two electrodes in visual cortex a few millimeters apart. This spike-spike correlation was on average higher for periods when the animal was less engaged in the task (Figure 4.1a,b ‘spk-spk-corr’:  = -0.366, p-value < 0.01). It should be noted that this relationship is inverse of what happens with spiking, and so the increases in 120 spike-spike correlation are not the result of overall increases in firing rates. The relationship between interneural correlations and performance could have been expected from what is known of arousal and ‘noise correlations’ (Livingstone & Hubel 1981, Cohen & Maunsell 2009, Cohen & Kohn 2011). Local field potentials (the extracellular potential in the vicinity of the electrodes) had no consistent relationship with performance on the task (Figure 4.1a,b, LPF:  = 0.013, p-value = 0.7) and the same lack of relationship held true for the population (Figure 4.1c-f). We looked at different wave-bands of the LPF signal, alpha is reported here, given its relevance in the arousal literature. Alpha power (7-12 Hz) had a negative correlation with performance (alpha:  = -0.083, p-value = 0.04, for the session in Figure 4.1a,b). Our observation that alpha power increased in periods that animal performs poorly is consistent with prevailing ideas of vigilance/attention literature (e. g. Davies & Krkovic 1965). In our experiment, presumably in less engaged periods the animal was also less attentive. Similar to the observations in alpha, gamma band power (30-90Hz) had a similar profile (gamma:  = -0.340, p-value < 0.01, for the session in Figure 4.1a,b). This was unexpected, alpha and gamma tend to be anti-correlated. Concerning trial anticipation there is evidence of gamma increases relative to conditions of low expectation (Lima et al. 2011). We therefore looked at one other metric, ‘arousal related index’ (named and described elsewhere: Moore et al. 2014, see methods for details). Summarily, this index tracks the tendency that in sleep one observes increases in low frequency power and decreases in the high frequency power. Our results indicate that this index does not fully track performance (ARI:  = -0.166, p-value < 0.01, for the session in Figure 4.1a,b). It is worth mentioning that occipital coverage as the one we had for V1 imaging experiments is not the appropriate location to identify sleep episodes. Nevertheless, we do not expect these results to reflect mostly sleep, as the population data in Figure 4.1c-f shows a significant 127 able to track some of the changes in brain states. These come accompanied by changes in intra-cortical correlations, heart rate, and quiescence. Finally it remains open what the best neural correlate for endogenous changes in hemodynamic response might be. 4.6 Acknowledgments We would like to thank Eric DeWitt for the fruitful discussions. 128 129 5 General Discussion Neuroimaging is both extremely useful and also a rather complex methodology to interpret. In this thesis the aim was to further explore neuroimaging in the context of behavior. We divided the work presented here into two broad parts. The first, aimed at addressing the relationship between stimuli- -related hemodynamic responses and underlying neural activity, in early sensory areas (Chapter 2). The second aimed at exploring the representation of behavior-related endogenous responses in the hemodynamic response, and also neurally, as well, in early sensory areas (Chapters 3 and 4). The work presented here does not close any of the lingering questions in the neuroimaging field. It nevertheless succeeds in bringing more attention to the question of how to analyze responses happening at different time scales, and coming from different sources. Is repeating trials multiple times and averaging their responses the best way to capture task relevant features? 5.1 Exogenous responses in V1 In Chapter 2 we discussed how stimulus-related and task-related components can be distinguished in the neuroimaging signal when looking at early sensory areas. We observed that in a periodic task there was a stimulus- -related contribution to the hemodynamic response and a task-related component. Moreover these two components added linearly to give the resulting hemodynamic response. This relationship though simple did not have to be like this, we could have found evidence of interaction between the stimulus and task components. Finally, the relationship between the hemodynamic response and underlying neuronal activity has also been subject of significant debate (for a review see: Logothetis & Wandell 2004). The prevailing idea has been that the hemodynamic response mostly reflects metabolic demand, and that it is better tracked by LFP activity. There is evidence in this direction (e. g., Logothetis et al. 2001, Niessing et al. 2005, Goense & 130 Logothetis 2008). The work presented here suggests also that spiking activity is well correlated with the stimulus related hemodynamic response. More recent work from the lab showed that for the same task (where were included several of the same datasets presented on Chapter 2) spiking is a better predictor of the stimulus related hemodynamic response than LFP (Lima et al. 2014). Thus we have good evidence that exogenous hemodynamic responses in V1 are linearly related to changes in local spiking activity. Therefore deconvolving the stimulus-related hemodynamic response with an HRF provides a good estimate of underlying neural activity, particularly relevant for fMRI studies. This also suggests that the hemodynamic response is tightly linked to spiking activity and presynaptic activity into a sensory area, as anyway observed in different experiments (Gurden et al. 2006, Lee et al. 2010, Scott & Murphy 2012, Kahn et al. 2013). Finally, to the neuroimaging community, this reinforces the importance of appropriate task design: aiming at estimating task non-relevant contributions (in our case, we were interested in characterizing stimulus representation) in order to separate those from stimulus-related components that will have a linear relationship to underlying local neural activity, as least in early sensory areas. The question of what the neuronal nature of endogenous hemodynamic responses is remains unanswered. 5.2 Endogenous responses in V1 In Chapter 3 we aimed at characterizing behavior aspects of the previously described task-related hemodynamic signal (Sirotin & Das 2009). In the original publication this signal had been characterized as present in V1 in a periodic fixation task, and with a time course matching that of the trial duration. We investigated this issue further by manipulating trial duration and fixation duration. The hemodynamic response tends to track the duration of the trials, but its time course points to a trial initiation response. As we saw that alternation between two schedule regimes did not produce significant 131 changes for a large portion of time into long trials, when compared to short ones. Then we looked at the effect of fixation duration. The task-related response keeps tracking the total duration of the trial, but longer fixations tend to be associated with higher task-related response amplitudes and also slower in the peaking time. Taken together, this suggests that the hemodynamic task-related response is influenced by the duration of trials. Importantly it might be triggered by initiation of a trial, but modulated by the time the animal is required to hold fixation. These are rather specific features associated with the specific fixation task we used. We used a different task, not reliant on fixation to address this question. We used a task that used auditory stimuli as cues and motor response and output. First, we observed a robust task-related response in V1 for this task, with comparable magnitudes. We then looked at the time course of such responses between the two task types and found great similarity between those. Moreover, it should be once more pointed out that we have used two slightly different versions the auditory/motor task, with similar results on both. It is worth noting that the two variations differ significantly in their event sequence. One task was designed to emulate, as much as possible, the fixation task, both in its event sequence (trials were presented automatically to the animal that could engage on them or not), and timing. The other task variation was different: the animal initiated the trials, and there was a hiatus of several seconds between trial start and the auditory cue coming on, and finally reward. It is still remarkable the similarity in the time course between this auditory/motor task and the fixation task. This suggests that there is a potential role for initiating a trial (engaging in the task) as a triggering signal for the resulting hemodynamic task-related activity. We also looked at the effects of reward/motivation to completing trials, and observed that reward amount increased the probability of initiating a trial and it increased the magnitude of the task-related hemodynamic changes. We observed as well a slower trend on the hemodynamic signal. This trend had the unexpected property of decreasing as one goes into a block of high 132 reward (and increasing when progressing into a low reward block). Apart from these changes, we have not found significant changes in the shape of the temporal profiles of high or low reward trials. Previous work in the lab (Sirotin et al. 2012) had already pointed out that the starkest difference lay between initiated or non-initiated trials. At some level this is trivial: it a task-related signal; no task, no signal. But the question remains; what in the task triggers this signal? Is it effort? Is it reward? Is it expectation? Is it something else? Understanding the behavior or such signal might help hint at what mechanisms underlie it. In the final chapter, Chapter 4, we explored further the idea of engagement or arousal and its influence on the hemodynamic response. We noticed that the slow changes in the hemodynamic response had significant amplitude and they tended to increase as engagement in the task decreases, similarly to the observations by Pisauro and colleagues (Pisauro et al. 2016). Our work here aimed at further confirming the non-local origin of these hemodynamic changes. We looked at several correlations between the different measurements, and in particular we noticed a non-linear relationship between heart-rate and hemodynamic responses; as if there were two distinct coupling modes operating in two different time scales. This observation might be particularly relevant when considering how to discount heart-rate from the fMRI signals (for some discussion about heart rate contributions to BOLD fMRI and how to account for those see Chang et al. 2009). In summary, there is a task-related signal; it is present in V1 for visual tasks as well as for an auditory/motor tasks, it is more global than a modality specific signal; it is present in initiated trials independently if trials were rewarded or not; and there is a slower (trial length) signal that tracks probably task engagement or arousal. It would be relevant to explore the role of neuromodulation, with this evidence we suggest it would be relevant to look at norepinephrine in particular. 133 5.3 Future directions 5.3.1 Locus coeruleus – noradrenergic system Norepinephrine (NA) and locus coeruleus (LC) activation are an extremely interesting system for further speculation of a neuromodulatory contribution to the hemodynamic signal related to task engagement. The work by AstonJones and colleagues showing the two modes of activation of LC, tonic and phasic, and its relationship to behavior is rather relevant (for a review see Aston-Jones & Cohen 2005). Tonic discharge associated with global changes in task disengagement and performance, and phasic changes associated to outcome and task relevant features like reward related targets (e.g. Aston-Jones et al. 1994, Usher et al. 1999). Moreover, more recent work by Gold and colleagues (Kalwani et al. 2014, Joshi et al. 2016) where they observed that LC (phasic) activation was related to ‘active’ actions differently than to withholded ones. Even though these two conditions differ in the actions the subject had to perform both lead to a rewarded outcome; suggesting that LC phasic activation encodes goal-directed actions more than solely reward. It would be extremely interesting to record activity directly from LC in both the fixation task and the auditory/motor task, and, simultaneously, observe not only LC activity changes during this task, but also how LC responses when the animal goes into low performing periods as we observed in Chapter 4. Neuroimaging of such an area with OIIS is not possible given its location in the brain. The use of fMRI techniques can be challenging given the location of LC, its small size or the fact that brainstem experiences significant pulsation. The use of appropriate coils could aid in this task, but with the disadvantage that it could be hard to resolve the whole brain. Finally, to my understanding, the exact pattern of activation/deactivation associated with the known electrophysiological responses of LC is not well characterized in neuroimaging. Moreover, MRI techniques are not ideally posed when looking at slow changes (in the scale of several seconds or 134 minutes) in response given limitation in maintaining the magnetic field in the scanner. Ideally, experiments involving LC would use electrophysiology. Alternatively, pharmacology could be used while recording the same hemodynamic responses with OIIS, in the same tasks we used before. Does inactivation of NA receptors influence the hemodynamic response? Visual cortex receives LC innervations (e.g. Kosofsky et al. 1984). As a note, initial research on NA modulation in visual cortex was associated with visual plasticity (for a brief review, see Sillito 1986). More recent work with systemic inhibition of NA transporters has been linked to changes in the ‘default mode network’ (DMN, on resting state fMRI and the DMN see Biswal et al. 1997, Raichle et al. 2001, respectively) (Minzenberg et al. 2011). In rodents, NA was shown to have a role in astroglial responses (Paukert et al. 2014). In these experiments they have looked at astrocytic Ca2+ activity in visual cortex. This activity had low baseline levels, but paring of a visual stimulus with periods of locomotion (where NA in visual cortex is elevated), consistently increased astrocytic Ca2+ activity. In the context of hemodynamic responses, these results are very interesting. In a nutshell, NA modulation into visual cortex is a promising avenue to pursue following the work presented in this thesis. 5.3.2 Neural basis of neuroimaging The work presented in this thesis sheds light on the interpretation of neuroimaging signals. Most questions pertaining to it, however, still remain. In the introduction there was an effort of bringing to light the unknown nature of the definitive mechanisms underlying the hemodynamic response. The use of intrinsic signal optical imaging allowed the presentation of results in this thesis. Moving forward there are several research questions that might be better addressed with the use of other experimental techniques. To understand and distinguish metabolic and vascular contributions to the resulting hemodynamic signal it would be relevant to use techniques where 135 those would be separated. Of particular interest are metabolic measures that can have a more direct relationship with underlying neuronal activity. We propose that a technique which shares similarities with intrinsic signal optical imaging would be relevant to advance our understanding of neuroimaging: autofluorescence imaging, specifically, flavoprotein fluorescence imaging. It is based on changes with oxidative respiration in mitochondria of molecular species that fluoresce differently in their oxidized and reduced form (Chance et al. 1979). These are even more interesting because they fluoresce in the visible spectrum. Not only have such techniques been used successfully in rodents (e.g. Shibuki et al. 2003) as they have previously been used in the lab (Sirotin & Das 2010b). This technique similarly to OIIS does not require the use of any agent like dyes or contrast to record the response. This represents an advantage, as those are often toxic and require injection/ application. It would be interesting to repeat all experiments with both OIIS and autofluorescence imaging. Of particular relevance would be to compare the relationship between the two imaging methods of exogenous and endogenous contributions to the hemodynamic response. Is the hemodynamic-metabolic coupling preserved for exogenous and endogenous signals? I have devoted little attention to the electrophysiological recordings in the collected data. We consistently collected MUA data, but its relationship with the hemodynamic response was not extremely informative, other than, reinforcing that the endogenous responses do not have their origin in V1. This information is relevant and valuable per se, in my opinion it absolutely states the importance of looking into other brain regions. However it contributes to the understanding of neurovascular coupling in a limited way. It would be relevant to do more granular analysis of the electrophysiological data, which could potentially bring to light more information. 136 References Aitken, P.G., Fayuk, D., Somjen, G.G. & Turner, D. a, 1999. Use of intrinsic optical signals to monitor physiological changes in brain tissue slices. Methods (San Diego, Calif.), 18(2), pp.91–103. Albrecht, D.G. & Hamilton, D.B., 1982. Striate cortex of monkey and cat: contrast response function. Journal of neurophysiology, 48(1), pp.217– 37. Arieli, A., Grinvald, A. & Slovin, H., 2002. Dural substitute for long-term imaging of cortical activity in behaving monkeys and its clinical implications. Journal of Neuroscience Methods, 114(2), pp.119–133. Arieli, A., Shoham, D., Hildesheim, R. & Grinvald, A., 1995. Coherent spatiotemporal patterns of ongoing activity revealed by real-time optical imaging coupled with single-unit recording in the cat visual cortex. Journal of neurophysiology, 73(5), pp.2072–2093. Arsenault, J.T., Nelissen, K., Jarraya, B. & Vanduffel, W., 2013. Dopaminergic Reward Signals Selectively Decrease fMRI Activity in Primate Visual Cortex. Neuron, 77(6), pp.1174–1186. Aston-Jones, G. & Cohen, J.D., 2005. An Integrative Theory of Locus Coeruleus-Norepinephrine Function: Adaptive Gain and Optimal Performance. Annual Review of Neuroscience, 28(1), pp.403–450. Aston-Jones, G., Rajkowski, J., Kubiak, P. & Alexinsky, T., 1994. Locus coeruleus neurons in monkey are selectively activated by attended cues in a vigilance task. The Journal of neuroscience : the official journal of the Society for Neuroscience, 14(July), pp.4467–4480. Attwell, D., Buchan, A.M., Charpak, S., Lauritzen, M., Macvicar, B. a & Newman, E. a, 2010. Glial and neuronal control of brain blood flow. Nature, 468(7321), pp.232–43. Attwell, D. & Laughlin, S.B., 2001. An energy budget for signaling in the grey matter of the brain. Journal of Cerebral Blood Flow and Metabolism, 21(10), pp.1133–1145. Bashinski, H.S. & Bacharach, V.R., 1980. Enhancement of perceptual sensitivity as the result of selectively attending to spatial locations. Perception & psychophysics, 28(3), pp.241–248. Birn, R.M., Smith, M.A., Jones, T.B. & Bandettini, P.A., 2008. The respiration response function: The temporal dynamics of fMRI signal fluctuations related to changes in respiration. NeuroImage, 40(2), pp.644–654. Biswal, B.B., Van Kylen, J. & Hyde, J.S., 1997. Simultaneous assessment of flow and BOLD signals in resting-state functional connectivity maps. NMR in biomedicine, 10(4-5), pp.165–170. Boly, M., Balteau, E., Schnakers, C., Degueldre, C., Moonen, G., Luxen, A., Phillips, C., Peigneux, P., Maquet, P. & Laureys, S., 2007. Baseline brain activity fluctuations predict somatosensory perception in humans.