Jan Schnupp

Jan Schnupp

 

 

Jan W H Schnupp,
Professor of Neuroscience, City University of Hong Kong

email: jan_hk@schnupp.net

A. Personal Statement

I have been pursuing research into the question of how the brain makes sense of sound for almost 30 years.

Hearing is the telepathic sense we all take for granted, allowing us to sense the presence of others in our environment with our eyes closed, or, via the medium of spoken language, to teleport our thoughts into another person’s mind via invisible vibrations. We learn at school that the key to these near miraculous abilities are “sound waves”, minuscule ripples of air pressure that radiate out from physically excited surfaces. But there is nothing “wave like” about our subjective perception of sound. Loudness maps in a very non-linear way onto wave amplitude, and while many people are taught that wave frequency maps onto the percept of pitch (that is whether a note is "high" or "low"), experts know that this is such a gross oversimplification as to be almost completely wrong. The timbre of a sound, perhaps its most distinguishing or characteristic feature, has so many perceptual “dimensions” to it that experts still do not agree on how many dimensional timbre is and what the most important dimensions are. Most of us are also able to perceive sound as highly spatial, being able not only to tell the direction where a sound comes from, but also, to an extent, the distance, and whether the sound travels through a small or large room to our ears. We are also pretty good at guessing the physical dimensions of sound sources, as well as what they are made of. And we are so adept at discovering regular rhythms in sounds that we effortlessly extrapolate them into the future or fill in gaps when part of a sound sequence is obscured by masking noises. All of this illustrates the astonishing sophistication of the central auditory nervous system, which uses still only partly understood neural computations to transform neural impulses triggered by physical sound waves into psychoacoustic, perceived qualities of sound such as loudness, pitch, timbre, rhythm, size or spatial direction, and uses those to create key aspects of the sensory world we inhabit. Improving our still rudimentary understanding of the neural mechanisms underpinning these remarkable transformations is not just a fascinating intellectual challenge, it is a necessity if we want to improve our ability to respond to the needs of hearing impaired cochlear implant users, whose ability to perceive the pitch or the spatial direction of sounds is dramatically diminished, or of those of unfortunate tinnitus sufferers who are disturbed by irritating phantom sounds that have no physical origin and appear to be created entirely in their heads.

My own journey into this area began in 1991 with doctoral and then postdoctoral research under the guidance of Prof Andrew King at the University of Oxford, during which I made pioneering contributions to the use of virtual acoustic space techniques and of linear filter models to better understand the processing of spatial cues in the auditory midbrain and cortex (e.g. Schnupp et al. (2001)) After a brief postdoctoral fellowship under the mentorship of Prof John Brugge at the University of Madison, Wisconsin I returned to Oxford, where I was able to start my own group and turn my attention to the study of neural representations of the pitch and timbre of complex sounds (e.g.Bizley et al. (2013)), and of how these interact with the representations of acoustic space, and how these representations are shaped by experience (e.g. (Schnupp et al. 2006)), or are modulated by context (e.g. Rabinowitz et al. (2013)).

In my new research group in Hong Kong, I am continuing to work on these questions, but have also turned my attention to the study of binaural hearing and pitch perception through prosthetic cochlear implants, using studies in animal models (rats) as well as volunteer patients. The research in my laboratory combines a variety of techniques including psychoacoustic, behavior testing in humans and animals, in-vivo electrophysiology in animals under acoustic and electric stimulation, EEG studies on human volunteers, signal processing, and the modelling and statistical analysis of behavioral and neural responses. By integrating this wide range of approaches we are able to tackle overarching research questions from how the auditory brain creates and uses predictive coding to process speech and music to how one might improve neuroprosthetic devices to restore a good sense of spatial hearing to severely deaf patients. 

References

Bizley, J. K., Walker, K. M. M., Nodal, F. R., King, A. J. and Schnupp, J. W. H. (2013) Auditory cortex represents both pitch judgments and the corresponding acoustic cues. Curr Biol 23:620-625.

Rabinowitz, N., Willmore, B., King, A. and Schnupp, J. (2013) Constructing Noise-Invariant Representations of Sound in the Auditory Pathway. PLoS Biol. 11:e1001710.

Schnupp, J. W., Hall, T. M., Kokelaar, R. F. and Ahmed, B. (2006) Plasticity of temporal pattern codes for vocalization stimuli in primary auditory cortex. J Neurosci 26:4785-95.

Schnupp, J. W., Mrsic-Flogel, T. D. and King, A. J. (2001) Linear processing of spatial cues in primary auditory cortex. Nature 414:200-4.

B. Positions and Honors

Positions and Employment

2016-: Professor in Neuroscience, at the Department of Biomedical Science, City University of Hong Kong.

2010-16: Professor in Neuroscience at the University of Oxford.

2008-09:Visiting researcher at the Italian Institute of Technology, Genova, Italy, in the laboratory of Dr Stefano Panzeri.

2005-15: Instructor at the Otto Loewi Minerva International Neuroscience Course in intracellular recording techniques, Eilat, Israel.

2002-10: University Lecturer (associate professor) at the Department of Physiology Anatomy and Genetics, University of Oxford, Fellow and Tutor in Medicine at St Peter's College and Stipendiary Lecturer at Oriel College, Oxford

2001- 02: Retained Lecturer in Neuroscience for Trinity College, Oxford

2001- 02: Postdoctoral Research Scientist (RS2) in the laboratory of Dr. A.J. King, Physiology, Oxford

1998:Author of “BrainWare”, a Software application for sensory electrophysiology, distributed by Tucker Davis Technology, Alachue, Florida.

1998-99:Visiting Honorary Research Fellow at the University of Wisconsin at Madison, collaborating with Prof. J. Brugge, studying the representation of moving and of natural sounds in the auditory midbrain and cortex.

1997-00: Defeating Deafness - Dunhill Medical Trust Research Fellow

1996-01: Research Fellow at Christ-Church, Oxford University. Studying neural mechanisms of sound localisation in mammals.

Honors

2010: Teaching excellence award, Medical Sciences Division, University of Oxford

2007: Runner up in the UK national final of the FameLab Science Communication competition

2002: Servier Young Scientist Award, 22nd European Winter Conference for Brain Research

1993: Wellcome Prize Studentship

1987: Sessional prize in Genetics, Department of Genetics & Biometry, University College London

C. Contributions to Science

My career in auditory neuroscience research spans almost 30 years and over 80 peer reviewed publications. For a full list see
https://scholar.google.com/citations?user=WCTI8v4AAAAJ&hl=en  or 
https://pubmed.ncbi.nlm.nih.gov/?term=schnupp_j

Below I shall give only a very brief and selective overview of some of the main research themes that I have made contributions to in the course of this career.

Studies of spatial hearing

My early work demonstrated the role of NMDA receptors and auditory-visual interactions in shaping the sensitivity of auditory midbrain neurons to sound source direction (Schnupp et al. 1995; King et al. 1998; Schnupp et al. 1998) , and used receptive field modelling, information theoretic analysis and virtual acoustic space techniques to dissect the contributions of different types of acoustic cues to the spatial tuning of midbrain and cortical neurons (Jenison et al. 2001; Schnupp et al. 2001; Mrsic-Flogel et al. 2003; Nelken et al. 2005)

More recently I returned to the subject of spatial hearing in the context of cochlear implantation. After demonstrating that laboratory rats are, contrary to popular belief, highly sensitive to interaural time differences (Li et al. 2019), we have gone on to find strong indications that the inability of human CI patients to use interaural time differences may be due to maladaptive plasticity rather than lacking input during a critical period. If confirmed, these findings may lead to important improvements in neuroprosthetic practices. Several manuscripts detailing this work are currently under review.

Adaptation and gain control

The receptive field modeling approaches we first used to probe tuning to spatial cues in cortical neurons we were later able to develop further to study cortical gain control mechanisms (Cooke et al. 2018) (Rabinowitz et al. 2012) (Rabinowitz et al. 2011) , and to show the importance of these mechanisms in allowing the central auditory cortical neurons to maintain representations of complex sounds such as speech in environments of increasing background noise (Rabinowitz et al. 2013). Arguing by analogy from the visual system we originally suspected that cortical parvalbumin positive interneurons might be responsible for this form of contrast adaptation, but our most recent work has not borne this out (Cooke et al. 2020). My colleagues in Oxford will continue this line of investigation independently.

Studies of periodicity pitch and timbre processing

Another major area of research which I have pursued, and which is most relevant to the current proposal, investigated how the auditory cortex represents perceptual features such as pitch and timbre, and how these representations interact with spatial tuning. Our work was able to show that the activity of primary auditory cortex neurons encodes perceived pitch and not just pitch cues such as periodicity (Bizley et al. 2013) We were also able to show that, contrary to previous reports, periodotopic maps in the cortex (Nelken et al. 2004) and the inferior colliculus (Schnupp et al. 2015) are not a consistent feature of the mammalian brain, and therefore not a possible substrate for the encoding of periodicity pitch. Rather, pitch representations are patchy and multiplexed, and sensitivities to pitch, timbre and space are interwoven until at least second order cortical fields (Bizley et al. 2009; Walker et al. 2011). However some of our recent human EEG experiments suggest specializations of function for pitch, timbre and spatial processing in high-level multisensory cortex that may even be lateralized, but revealing these specializations requires attentional control (Retsa et al. 2018).

Studies of rhythm processing and predictive coding

More recently, my lab and I have also made contributions to our understanding of the role of rhythmic structure of sounds in several aspects of hearing, from generating expectations and mismatch responses (Szymanski et al. 2011) , to the enhanced detection of targets (Lawrance et al. 2014) or the binding of sounds into auditory streams (Rajendran et al. 2013), to retuning of neurons in human cortex through temporal expectations (Auksztulewicz et al. 2019) , and even to pre-processing of sounds by midbrain and cortex neurons for the emergence of musical beat perception (Rajendran et al. 2017; Rajendran and Schnupp 2019; Rajendran et al. 2020).

Science education and outreach

My long and broad experience in researching the central auditory pathway has put me in an ideal position to write, with two esteemed colleagues, the leading textbook on auditory neuroscience (Schnupp et al. 2011) and I have set up an interactive web site, https://auditoryneuroscience.com, which features many interactive examples that make auditory phenomena more accessible to students of this important field.

 

References

Auksztulewicz, R., Myers, N. E., Schnupp, J. W. and Nobre, A. C. (2019) Rhythmic Temporal Expectation Boosts Neural Activity by Increasing Neural Gain. J Neurosci 39:9806-9817.

Bizley, J. K., Walker, K. M., Silverman, B. W., King, A. J. and Schnupp, J. W. (2009) Interdependent encoding of pitch, timbre, and spatial location in auditory cortex. J. Neurosci. 29:2064-75.

Bizley, J. K., Walker, K. M. M., Nodal, F. R., King, A. J. and Schnupp, J. W. H. (2013) Auditory cortex represents both pitch judgments and the corresponding acoustic cues. Curr. Biol. 23:620-625.

Cooke, J. E., Kahn, M. C., Mann, E. O., King, A. J., Schnupp, J. W. H. and Willmore, B. D. B. (2020) Contrast gain control occurs independently of both parvalbumin-positive interneuron activity and shunting inhibition in auditory cortex. J. Neurophysiol. 123:1536-1551.

Cooke, J. E., King, A. J., Willmore, B. D. B. and Schnupp, J. W. H. (2018) Contrast gain control in mouse auditory cortex. J. Neurophysiol. 120:1872-1884.

Jenison, R. L., Schnupp, J. W., Reale, R. A. and Brugge, J. F. (2001) Auditory space-time receptive field dynamics revealed by spherical white-noise analysis. J. Neurosci. 21:4408-15.

King, A. J., Schnupp, J. W. H. and Thompson, I. D. (1998) Signals from the superficial layers of the superior colliculus enable the development of the auditory space map in the deeper layers. J. Neurosci. 18:9394-408.

Lawrance, E. L. A., Harper, N. S., Cooke, J. E. and Schnupp, J. W. H. (2014) Temporal predictability enhances auditory detection. J. Acoust. Soc. Am. 135:EL357-EL363.

Li, K., Chan, C. H. K., Rajendran, V. G., Meng, Q., Rosskothen-Kuhl, N. and Schnupp, J. W. H. (2019) Microsecond sensitivity to envelope interaural time differences in rats. The Journal of the Acoustical Society of America 145:EL341.

Mrsic-Flogel, T., Schnupp, J. and King, A. (2003) Acoustic factors govern developmental sharpening of spatial tuning in the auditory cortex. Nat. Neurosci. 6:981-988.

Nelken, I., Bizley, J. K., Nodal, F. R., Ahmed, B., Schnupp, J. W. and King, A. J. (2004) Large-scale organization of ferret auditory cortex revealed using continuous acquisition of intrinsic optical signals. J. Neurophysiol. 92:2574-88.

Nelken, I., Chechik, G., Mrsic-Flogel, T. D., King, A. J. and Schnupp, J. W. (2005) Encoding stimulus information by spike numbers and mean response time in primary auditory cortex. J. Comput. Neurosci. 19:199-221.

Rabinowitz, N., Willmore, B., King, A. and Schnupp, J. (2013) Constructing Noise-Invariant Representations of Sound in the Auditory Pathway. PLoS Biol. 11:e1001710.

Rabinowitz, N. C., Willmore, B. D., Schnupp, J. W. and King, A. J. (2011) Contrast gain control in auditory cortex. Neuron 70:1178-91.

Rabinowitz, N. C., Willmore, B. D., Schnupp, J. W. and King, A. J. (2012) Spectrotemporal contrast kernels for neurons in primary auditory cortex. J. Neurosci. 32:11271-84.

Rajendran, V. G., Harper, N. S., Garcia-Lazaro, J. A., Lesica, N. A. and Schnupp, J. W. H. (2017) Midbrain adaptation may set the stage for the perception of musical beat. Proc Royal Soc B. Biological sciences 284.

Rajendran, V. G., Harper, N. S. and Schnupp, J. W. H. (2020) Auditory cortical representation of music favours the perceived beat. R. Soc. Open Sci. 7:191194.

Rajendran, V. G., Harper, N. S., Willmore, B. D., Hartmann, W. M. and Schnupp, J. W. H. (2013) Temporal predictability as a grouping cue in the perception of auditory streams. J. Acoust. Soc. Am. 134:EL98-E104.

Rajendran, V. G. and Schnupp, J. W. H. (2019) Frequency tagging cannot measure neural tracking of beat or meter. Proc. Natl. Acad. Sci. U.S.A. 116:2779-2780.

Retsa, C., Matusz, P. J., Schnupp, J. W. H. and Murray, M. M. (2018) What's what in auditory cortices?. Neuroimage 176:29-40.

Schnupp, J. W., King, A. J. and Carlile, S. (1998) Altered spectral localization cues disrupt the development of the auditory space map in the superior colliculus of the ferret. J. Neurophysiol. 79:1053-69.

Schnupp, J. W., King, A. J., Smith, A. L. and Thompson, I. D. (1995) NMDA-receptor antagonists disrupt the formation of the auditory space map in the mammalian superior colliculus. J. Neurosci. 15:1516-31.

Schnupp, J. W., Mrsic-Flogel, T. D. and King, A. J. (2001) Linear processing of spatial cues in primary auditory cortex. Nature 414:200-4.

Schnupp, J. W. H., Garcia-Lazaro, J. A. and Lesica, N. A. (2015) Periodotopy in the gerbil inferior colliculus: local clustering rather than a gradient map. Front. Neural Circuits 9:37.

Szymanski, F. D., Rabinowitz, N. C., Magri, C., Panzeri, S. and Schnupp, J. W. (2011) The laminar and temporal structure of stimulus information in the phase of field potentials of auditory cortex. J. Neurosci. 31:15787-801.

Walker, K. M., Bizley, J. K., King, A. J. and Schnupp, J. W. (2011) Multiplexed and robust representations of sound features in auditory cortex. J. Neurosci. 31:14565-76.

 

D. Research Support (current)

The following are current, 3 year grants awarded to the Schnupp lab by Hong Kong government research funds or by the local government of Shenzhen.

2019: Hong Kong Government General Research Fund (GRF), “Is impaired binaural hearing in cochlear implantees caused by lack of experience or inappropriate stimulation?”, Project Nr 11100219

2018: Shenzhen Science and Innovation Fund, “听觉经验相关的人工耳蜗双耳时间差的大脑可塑性研究” (“Experience related brain plasticity in processing of interaural time differences”), [Grant No. JCYJ20180307124024360]

In these two projects we use behavior testing and electrophysiology in deafened rats to test the hypothesis that the inability of bilaterally implanted human CI patients to use interaural time differences may be due to maladaptive plasticity rather than lack of stimulation during a sensitive period.

2018: Hong Kong Health and Medical Research Fund (HMRF),“Temporal weighting functions for spatial cues and their potential relevance for better binaural hearing with cochlear implants” Project Nr: 06172296

In this project we use behavior testing and electrophysiology in both normally hearing and CI supplied, deafened rats to investigate whether rats show the same onset weighting of binaural cues that is so well documented in human listeners, and whether this onset weighting develops normally in deaf animals after cochlear implantation.

2018: Hong Kong Government General Research Fund (GRF), “Neural mechanisms of prediction signalling along the cortical auditory pathway”, Project Nr 11100518

In this project we use behavior testing and electrophysiology in normally hearing rats and human volunteers to investigate mechanisms of cortical mismatch detection across different perceptual features.

2018: European Community / Hong Kong Research Grants Council Joint Research Scheme “Cortical mechanisms of temporal predictions in processing rhythmic sounds and speech”

In this project we use behavior testing and electrophysiology in normally hearing rats and human volunteers to investigate cortical mechanisms involved in learning regular patterns and rules such as those governing phonotactics in languages.

2017: Hong Kong Government General Research Fund (GRF), “Natural Sound Statistics in Auditory Scene Analysis”, Project Nr 11100617

In this project we use electrophysiology normally hearing rats and statistical analyses to investigate how the central nervous system processes sound textures.