Skip to main content
EURASIP Journal on Audio, Speech, and Music Processing
Download PDF

Electrophysiological Study of Algorithmically Processed Metric/Rhythmic Variations in Language and Music

EURASIP Journal on Audio, Speech, and Music Processing volume 2007, Article number: 030194 (2007) Cite this article

Abstract

This work is the result of an interdisciplinary collaboration between scientists from the fields of audio signal processing, phonetics and cognitive neuroscience aiming at studying the perception of modifications in meter, rhythm, semantics and harmony in language and music. A special time-stretching algorithm was developed to work with natural speech. In the language part, French sentences ending with tri-syllabic congruous or incongruous words, metrically modified or not, were made. In the music part, short melodies made of triplets, rhythmically and/or harmonically modified, were built. These stimuli were presented to a group of listeners that were asked to focus their attention either on meter/rhythm or semantics/harmony and to judge whether or not the sentences/melodies were acceptable. Language ERP analyses indicate that semantically incongruous words are processed independently of the subject's attention thus arguing for automatic semantic processing. In addition, metric incongruities seem to influence semantic processing. Music ERP analyses show that rhythmic incongruities are processed independently of attention, revealing automatic processing of rhythm in music.

[123456789101112131415161718192021222324252627282930313233343536373839]

References

  1. Friberg A, Sundberg J: Time discrimination in a monotonic, isochronous sequence. Journal of the Acoustical Society of America 1995,98(5):2524-2531. 10.1121/1.413218

    Article  Google Scholar 

  2. Drake C, Botte MC: Tempo sensitivity in auditory sequences: Evidence for a multiple-look model. Perception and Psychophysics 1993, 54: 277-286. 10.3758/BF03205262

    Article  Google Scholar 

  3. Hirsh IJ, Monahan CB, Grant KW, Singh PG: Studies in auditory timing : I, simple patterns. Perception and Psychophysics 1990,74(3):215-226.

    Article  Google Scholar 

  4. ten Hoopen G, Boelaarts L, Gruisen A, Apon I, Donders K, Mul N, Aker-boom S: The detection of anisochrony in monaural and interaural sound sequences. Perception and Psychophysics 1994,56(1):210-220.

    Article  Google Scholar 

  5. Barthet M, Kronland-Martinet R, Ystad S, Depalle Ph: The effect of timbre in clarinet interpretation. Proceedings of the International Computer Music Conference (ICMC '07), August 2007, Copenhagen, Denmark

    Google Scholar 

  6. Besson M, Faïta F, Czternasty C, Kutas M: What's in a pause: event-related potential analysis of temporal disruptions in written and spoken sentences. Biological Psychology 1997,46(1):3-23. 10.1016/S0301-0511(96)05215-5

    Article  Google Scholar 

  7. Patel AD, Daniele JR: An empirical comparison of rhythm in language and music. Cognition 2003,87(1):B35-B45. 10.1016/S0010-0277(02)00187-7

    Article  Google Scholar 

  8. Magne C, Schön D, Besson M: Prosodic and melodic processing in adults and children: behavioral and electrophysiologic approaches. Annals of the New York Academy of Sciences 2003, 999: 461-476. 10.1196/annals.1284.056

    Article  Google Scholar 

  9. Schön D, Magne C, Besson M: The music of speech: music training facilitates pitch processing in both music and language. Psychophysiology 2004,41(3):341-349. 10.1111/1469-8986.00172.x

    Article  Google Scholar 

  10. Besson M, Macar F: An event-related potential analysis of incongruity in music and other non-linguistic contexts. Psychophysiology 1987,24(1):14-25. 10.1111/j.1469-8986.1987.tb01853.x

    Article  Google Scholar 

  11. Koelsch S, Kasper E, Sammler D, Schulze K, Gunter T, Friederici AD: Music, language and meaning: brain signatures of semantic processing. Nature Neuroscience 2004,7(3):302-307. 10.1038/nn1197

    Article  Google Scholar 

  12. Patel AD, Gibson E, Ratner J, Besson M, Holcomb PJ: Processing syntactic relations in language and music: an event-related potential study. Journal of Cognitive Neuroscience 1998,10(6):717-733. 10.1162/089892998563121

    Article  Google Scholar 

  13. Koelsch S, Gunter T, Friederici AD, Schröger E: Brain indices of music processing: "nonmusicians" are musical. Journal of Cognitive Neuroscience 2000,12(3):520-541. 10.1162/089892900562183

    Article  Google Scholar 

  14. Regnault P, Bigand E, Besson M: Different brain mechanisms mediate sensitivity to sensory consonance and harmonic context: evidence from auditory event-related brain potentials. Journal of Cognitive Neuroscience 2001,13(2):241-255. 10.1162/089892901564298

    Article  Google Scholar 

  15. Magne C, Astésano C, Aramaki M, Ystad S, Kronland-Martinet R, Besson M: Influence of syllabic lengthening on semantic processing in spoken French: behavioral and electrophysiological evidence. Cerebral Cortex 2007. Oxford University Press, January 2007

    Google Scholar 

  16. Astésano C: Rythme et accentuation en français: Invariance et variabilité stylistique, Collection Langue & Parole. L'Harmattan, Paris, France; 2001.

    Google Scholar 

  17. Di Cristo A: Le cadre accentuel du français contemporain: essai de modélisation: premiére partie. Langues 1999,2(3):184-205.

    Google Scholar 

  18. Pallone G: Dilatation et transposition sous contraintes perceptives des signaux audio: application au transfert cinéma-vidéo, Ph.D. thesis. University of Aix-Marseille II, Marseilles, France; 2003.

    Google Scholar 

  19. Dolson M: The phase vocoder: a tutorial. Computer Music Journal 1986,10(4):14-27. 10.2307/3680093

    Article  Google Scholar 

  20. Pallone G, Boussard P, Daudet L, Guillemain P, Kronland-Martinet R: A wavelet based method for audio-video synchronization in broadcasting applications. Proceedings of the 2nd COST-G6 Workshop on Digital Audio Effects (DAFx '99), December 1999, Trondheim, Norway 59-62.

    Google Scholar 

  21. Puckette M: Phase-locked vocoder. Proceedings of IEEE ASSP Workshop on Applications of Signal Processing to Audio and Acoustics, October 1995, New Paltz, NY, USA 222-225.

    Google Scholar 

  22. Laroche J, Dolson M: Improved phase vocoder time-scale modification of audio. IEEE Transactions on Speech and Audio Processing 1999,7(3):323-332. 10.1109/89.759041

    Article  Google Scholar 

  23. French NR, Zinn MK: Method of an apparatus for reducing width of trans-mission bands. 1928.

    Google Scholar 

  24. Roucos S, Wilgus A: High quality time-scale modification for speech. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '85), April 1985, Tampa, Fla, USA 10: 493-496.

    Article  Google Scholar 

  25. Verhelst W, Roelands M: An overlap-add technique based on waveform similarity (WSOLA) for high quality time-scale modification of speech. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '93), April 1993, Minneapolis, Minn, USA 2: 554-557.

    Google Scholar 

  26. Hejna DJ, Musicus BT, Crowe AS: Method for time-scale modification of signals. 1992.

    Google Scholar 

  27. Laroche J: Time and pitch scale modification of audio signals. In Applications of Digital Signal Processing to Audio and Acoustics. Edited by: Kahrs M, Brandenburg K. Kluwer Academic Publishers, Norwell, Mass, USA; 1998:279-309.

    Google Scholar 

  28. Repp B: Probing the cognitive representation of musical time: structural constraints on the perception of timing perturbations. Haskins Laboratories Status Report on Speech Research 1992, 111-112: 293-320.

    Google Scholar 

  29. Moog B: MIDI: musical instrument digital interface. Journal of Audio Engineering Society 1986,34(5):394-404.

    Google Scholar 

  30. Puckette MS, Appel T, Zicarelli D: Real-time audio analysis tools for Pd and MSP. In Proceedings of the International Computer Music Conference, October 1998, Ann Arbor, Mich, USA. International Computer Music Association; 109-112.

    Google Scholar 

  31. Jasper HH: The ten-twenty electrode system of the International Federation. Electroencephalography and Clinical Neurophysiology 1958, 10: 371-375.

    Google Scholar 

  32. Kutas M, Hillyard SA: Reading senseless sentences: brain potentials reflect semantic incongruity. Science 1980,207(4427):203-205. 10.1126/science.7350657

    Article  Google Scholar 

  33. Besson M, Magne C, Regnault P: Le traitement du langage. In L'imagerie fonctionnelle électrique (EEG) et magnétique (MEG): Ses applications en sciences cognitives. Edited by: Renault B. Hermés, Paris, France; 2004:185-216.

    Google Scholar 

  34. Astésano C, Besson M, Alter K: Brain potentials during semantic and prosodic processing in French. Cognitive Brain Research 2004,18(2):172-184. 10.1016/j.cogbrainres.2003.10.002

    Article  Google Scholar 

  35. Magne C, Astésano C, Lacheret-Dujour A, Morel M, Alter K, Besson M: On-line processing of "pop-out" words in spoken French dialogues. Journal of Cognitive Neuroscience 2005,17(5):740-756. 10.1162/0898929053747667

    Article  Google Scholar 

  36. Besson M, Faïta F: An event-related potential (ERP) study of musical expectancy: comparison of musicians with non-musicians. Journal of Experimental Psychology: Human Perception and Performance 1995,21(6):1278-1296.

    Google Scholar 

  37. Koelsch S, Gunter T, Schröger E, Friederici AD: Processing tonal modulations: an ERP study. Journal of Cognitive Neuroscience 2003,15(8):1149-1159. 10.1162/089892903322598111

    Article  Google Scholar 

  38. Luks TL, Nusbaum HC, Levy J: Hemispheric involvement in the perception of syntactic prosody is dynamically dependent on task demands. Brain and Language 1998,65(2):313-332. 10.1006/brln.1998.1993

    Article  Google Scholar 

  39. Zatorre RJ: Neural specializations for tonal processing. In The Biological Foundations of Music, Annals of the New York Academy of Sciences. Volume 930. Edited by: Zatorre RJ, Peretz I. New York Academy of Sciences, New York, NY, USA; 2001:193-210.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratoire de Mécanique et d'Acoustique, CNRS, Marseille, France

    Sølvi Ystad, Snorre Farner, Gregory Pallone & Richard Kronland-Martinet

  2. Institut de Neurosciences Cognitives de la Méditerranée, CNRS, Marseille Cadex, 13402, France

    Cyrille Magne, Mitsuko Aramaki & Mireille Besson

  3. Psychology Department, Middle Tennessee State University, Murfreesboro, TN, 37127, USA

    Cyrille Magne

  4. IRCAM, 1 Place Igor Stravinsky, Paris, 75004, France

    Snorre Farner

  5. France Télécom, Lannion Cedex, 22307, France

    Gregory Pallone

Authors
  1. Sølvi Ystad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cyrille Magne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Snorre Farner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gregory Pallone
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mitsuko Aramaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mireille Besson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Richard Kronland-Martinet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sølvi Ystad.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Ystad, S., Magne, C., Farner, S. et al. Electrophysiological Study of Algorithmically Processed Metric/Rhythmic Variations in Language and Music. J AUDIO SPEECH MUSIC PROC. 2007, 030194 (2007). https://doi.org/10.1155/2007/30194

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/2007/30194

Keywords

  • Signal Processing
  • Acoustics
  • Automatic Processing
  • Electrophysiological Study
  • Algorithmically Process