Sofia Nunes

This project covers the scope of audio engineering and music, more specifically, 3D sound environments, room acoustics and auralization/binaural listening. We implemented a didactic mobile app to understand and explore audio concepts related with musical instruments and acoustic environment, such as, sound spatialization, reverberation, wave reflection and material absorption coefcients.

The mobile app is launched through a menu that exposes the user to various ways to choose a virtual room to experience the sound of 3D instruments. These options include choosing from a list a virtual room similar to the actual room or by measuring the reverberation time of the real room you are in.

In this virtual world, the user is invited to explore the music from various 3D instruments which are placed in the scene forming an orchestra, observing them in their own room using AR mechanisms via mobile phone. If the user walks around these same instruments, he will hear the diferences in sound in diferent directions, as well as the room, which influences the hearing perception of the music, due to sound reflections in the room and musical instrument directivity characteristics. Therefore, the user can have the perspective of the musician in live performances. This can also be a didactic tool in a way that the user can mute, choose and/or move some instruments in the space, in a rehearsal class.

The project is implemented on Unity3D platform, with ArFoundation and ArCore to integrate AR and C#. The work is supported by Audio_and_Acoustics_Laboratory_of_ISEL.

I am 22 years old. I´ve been to a private feminine school up until college, where they greatly encouraged the world of sciences and engineering, and we were constantly receiving relevant visits and attending related presentations. I have great memories from this time, made friends for life, and even practiced ballet for 4 years! I have always been a greatly active person, physically. In fact, I enjoy fitness and dancing a lot, which is something present in my daily life. When enroling in high school, I chose to follow the Science and Technology course, and obtained great marks in the various national exams (including grade 20 in Descriptive Geometry and 19 in Mathematics A). Three years later, I entered the Computer and Multimedia Engineering course at the Instituto Superior de Engenharia de Lisboa (ISEL), in Lisbon. In this degree, I had contact with several programming languages and methodologies to carry out work related to computer science. Finally, I completed the degree in 3 years, and right after that I entered the master’s degree, where I am now. I have a great passion for arts, free design and graphic design, and want to reconcile that passion with the technological world, and continue to give wings to my unlimited creativity and imagination

Joel Paulo

The installations of interactive sound art in the public spaces poses important challenges for artists and engineers. This aspect is important given the participation of common people, having poor or none musical or technical skills, is essential for the piece of art to be successful. The focus idea is to generate and modify, using information captured by sensors such as microphones and video cameras to analyze types of sound and gestures, the aesthetics of a base music generated by a sound synthesizer, changing some aspects of the sound in realtime, such as, the pitch, reverb, speed, randomness. The resulting audio streaming is sent to a virtual game platform to create immersive sound through an array of loudspeakers or virtual reality headsets, i.e. oculus VR.

The information captured by the sensors is handled by Pyhton language using the OpenCV and PyAudio libraries. A state machine is used to manage all the possible combination of sound and gestures using a dictionary of events. The communication with SuperCollider sound synthetizer is made by OSC messages. The JackRouter, using the Jack engine, is used to connect the SuperCollider to the Unity3D game platform, where the acoustic spaciousness and reverberation of the room selected beforehand is modified by the Resonance Audio SDK. The user can choose diferent environment/scenarios and/or base sound tracks, thus, this information is sent to Python, by OSC messages, to indicate the state machine to change accordingly. The work is supported by Audio_and_Acoustics_Laboratory_of_ISEL.

Joel was born in Almada, Portugal. He received the Ph.D. degree in electrical engineering from the University of Lisbon, Instituto Superior Técnico, IST, Portugal, in 2012, in the fields of experimental room acoustics and audio signal processing. In 1996, he joined the Department of Radio Communications of the Nautical School Infante D. Henrique as an Assistant Professor, and in 2004 he joins the Department of Engineering in Electronics, Telecommunications and Computers of the Higher Institute of Lisbon, ISEL, Institute Polytechnic of Lisbon, IPL, where he is currently a Full Professor. He is the head of the Laboratory of Audio and Acoustics of ISEL and a researcher at the Group of Acoustics and Noise Control of the CAPS-IST. His main areas of R&D interest are: room acoustics measurements, multichannel sound, beamforming transducers, audio signal processing, psychoacoustics, 3D microphones, machine learning methods applied to music and noise, immersive sound environments for VR and performing arts. 3D sound for cinema, television and arts is one of his interest nowadays. Smart Cities, IoT, entrepreneurship and commercialization the results of academic projects are in his scope. He is a member of the Audio Engineering Society, AES, and the Acoustical Society of Portugal, SPA.

Ruth Barranco Raimundo

Between September 16 and 27, 2020, a new edition of the Asalto International Festival of urban art was held in the city of Zaragoza and against all odds.

This multidisciplinary call that was marked by the global pandemic of COVID-19, focused on the popular Barrio de San José the interventions of its invited artists and was used by the digital artist Karto Gimeno (1964, Zaragoza) to experiment in his artistic creation, with the city limits. Starting from his photographic universes, fed from surrealist painting, cinematographic narrative, Media Arts and folk tales, this creator here merged his stages of experimentation in the interior space, with the opening of immersive thresholds in exterior spaces and urban ensembles. Through interventions on the architectural skin, created with high resolution digital impressions applied to facades, he opened eventual interdimensional windows that for the passerby meant a personal narrative to be completed. In these events, psychological outdoor spaces were shown, which emerged like a wormhole, connecting the city with free nature and ruin as a trace of memory, proposing to the observer an immersion in their own and collective unconscious.

His creation, moreover, full of liminal spaces, is full of suggestions and secret corners that flirt with common immersive images in our individual psyche – umheimlich spaces, hijacking the Freudian definition – and the intense relationship between place and person, studied by the philosopher. German Martin Heidegger in the cabin.