News and Announcements
ARTICLE
Sonic immersion with interactive mobile web-application
Show Less
1 Academy of Music, Hong Kong Baptist University, Kowloon Tong, Hong Kong,
China
2 Experimental Studio, Südwestrundfunk, Freiburg, Baden-Württemberg,
Germany
AC, 025210043 https://doi.org/10.36922/AC025210043
Received: 21 May 2025 | Revised: 22 December 2025 | Accepted: 15 January 2026 | Published online: 24 February 2026
(This article belongs to the Special Issue Ubimus Cross-Cultural Approaches to Interaction and Creativity )
© 2026 by the Author(s). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution -Noncommercial 4.0 International License (CC-by the license) ( https://creativecommons.org/licenses/by-nc/4.0/ )
Abstract
Over the past two decades, interactive technologies have gradually transformed audience engagement in live performance contexts. This paper explores audience interaction through a mobile web-app designed for immersive sonic experiences. The app transforms personal smartphones into a distributed speaker array and interactive controller. We examine two case studies: Freedom Collective, a contemporary opera where smartphones diffused pre-recorded sounds, creating a spatialized auditory field; and Exo Signals, an interactive sound experience at the Ubimus conference in Macau, where the audience actively shaped the soundscape by triggering audio samples and applying real-time DSP (band-pass filter) through touch interactions. We discuss the artistic and technological challenges in both cases, comparing the levels of immersion and audience engagement in passive versus active participation models.
Keywords
Mobile web-app
Sonic immersion
Audience participation
Interactive performance
Passive versus active interaction
Funding
The development and production of Freedom Collective were supported by the Fonds experimentelles Musiktheater (fexm) as part of the NOperas! Initiative, jointly funded by NRW Kultursekretariat and Kunststiftung NRW in cooperation with Musiktheater im Revier, Theater Bremen, and Staatstheater Darmstadt. Technical development of the web-app was supported by SWR Experimentalstudio. Travel support for the performance of Exo Signals at the Ubimus Conference in Macau was provided by Hong Kong Baptist University. No additional funding was received for the research analysis or publication of this article.
Conflict of interest
The authors declare they have no competing interests.
References
- Liew K, Lindborg P, Rodrigues R, Styles SJ. Cross-modal perception of noise-in-music: Audiences generate spiky shapes in response to auditory roughness in a novel electroacoustic concert setting. In: Proceedings of the International Conference on Auditory Display; 2018.
- Nyckees J. Distributed Multi-Touch Interfaces for Musical Expression. Master’s Thesis. KTH Royal Institute of Technology; 2017.
- Wang G, Essl G, Penttinen H. Do Mobile Phones Dream of Electric Orchestras? In: Proceedings of the International Computer Music Conference (ICMC); 2008.
- Riionheimo J, Lokki T. Spatial sound, part 1: Exploring the impact of listening to room on perceptual immersion. J Audio Eng Soc. 2024;72(12):913-929. doi: 10.17743/jaes.2022.0179
- Nosenko O. Immersive technologies in the formation of musical audio space. Notes Art Crit. 2024;1(24):23-32.
- Levin G. Dialtones (A Telesymphony); 2001. Available from: https://youtu.be/039_0Olb1WQ?si=oIEwrJp-GGOd60QV [Last accessed on 2025 Mar 01].
- Ciciliani M. SkylAR. Available from: https://www.ciciliani. com/skylar.html [Last accessed on 2025 Mar 01].
- Taylor B. A history of the audience as a speaker array. In: Proceedings of the International Conference on New Interfaces for Musical Expression. Aalborg University Copenhagen; 2017. p. 481-486. doi: 10.5281/zenodo.1176324
- Forgette A, Manaris B, Gillikin M, Ramsden S. Speakers, More Speakers!! Developing Interactive, Distributed, Smartphone-Based, Immersive Experiences for Music and Art. In: Proceedings of the International Symposium on Electronic Art. Available from: https://par.nsf.gov/ biblio/10374099 [Last accessed on 2025 Mar 01].
- Marshall K, Loydell R. Thinking inside the box: Brian Eno, music, movement and light. J Vis Art Pract. 2017;16(2):104-118. doi: 10.1080/14702029.2016.1195073
- Robaszkiewicz S, Schnell N. Soundworks: A Playground for Artists and Developers to Create Collaborative Mobile Web Performances. In: Proceedings of the Web Audio Conference (WAC); 2015.
- Essl G. The mobile phone ensemble as classroom. In: Proceedings of the International Computer Music Conference (ICMC); 2010.
- Hödl O, Kayali F, Fitzpatrick G. Poème Numérique: Large-Scale Audience Participation in Live Music Using Smartphones. In: Proceedings of the International Conference on New Interfaces for Musical Expression; 2020.
- Xambó A, Goudarzi V. The mobile audience as a digital musical persona in telematic performance. In: Proceedings of the International Conference on New Interfaces for Musical Expression; 2022.
- Koszolko MK, Studley T. From site-specific sampling to gamification: An exploration of performative engagement with the environment. Organ Sound. 2023;28:338-351.
- Apple Inc. Human Interface Guidelines: Playing Audio. Available from: https://developer.apple.com/design/human-interface-guidelines/playing-audio [Last accessed on 2025 Mar 01].
- Matuszewski B. A web-based framework for distributed music system research and creation. J Audio Eng Soc. 2020;68:717-726.
- Shaw T, Piquemal S, Bowers J. Fields: An Exploration into the Use of Mobile Devices as a Medium for Sound Diffusion. In: Proceedings of the International Conference on New Interfaces for Musical Expression; 2015. p. 281-284.
- Chang T. An investigation of usability in immersive environments. In: Proceedings of the International Conference on Human-Computer Interaction; 2024.
- Vincze DB, Oeser M. Freedom Collective: Interactive Mobile Web-App for Immersive Audience Participation. In: Proceedings of the Ubiquitous Music Symposium (UBIMUS); 2024.
- Greenhalgh C, Hazzard A, Benford S, Cliffe L, Kelly E. Crafting trajectories of smartphone use at the opera. ACM Trans Comput Hum Interact. 2022;29(6):59. doi: 10.1145/3531007
- Heim C. Audience as Performer: The Changing Role of Theatre Audiences in the Twenty-First Century. London: Routledge; 2015.
- Xambó A, Roma G. Performing Audiences: Composition Strategies for Network Music Using Mobile Phones. In: Proceedings of the International Conference on New Interfaces for Musical Expression; 2020.