OBA
 
The most popular method of producing a soundtrack for games is known as Object-based audio. In this technique, the entire audio consists of individual sound assets with metadata describing their relationships and associations. Rendering these sound assets on the user's device means assembling these objects (sound + metadata) to create an overall user experience. The rendering of objects is flexible and responsive to the user, environmental, and platform-specific factors [ref.].​

​In practice, if an audio designer wants to create an ambient for an adventure in a jungle, he or she needs to use several individual sound objects, for example, the wind rustling through the trees, sounds of wild animals, the sound of a waterfall, the buzzing of mosquitoes, etc. The complexity associated with Object-based renderings increases with the number of sound objects. This means that the more individual objects there are (the more complex the audio scene is) the higher is the usage of the CPU (and hence power consumption) which can be problematic in the case of mobile devices or limitations of the bandwidth during data transmission.  

VSBA
 
A complementary approach for games is Volumetric Scene-based audio, especially if the goal is to achieve natural behavior of the sound (reflections, diffraction). VSBA is a set of 3D sound technologies based on Higher-Order Ambisonics (HOA), a format for the modeling of 3D audio fields defined on the surface of a sphere. It allows for accurate capturing, efficient delivery, and compelling reproduction of 3D sound fields on any device (headphones, loudspeakers, etc.). VSBA and HOA are deeply interrelated; therefore, these two terms are often used interchangeably. Higher-Order Ambisonics is an ideal format for productions that involve large numbers of audio sources, typically held in many stems. While transmitting all these sources plus meta-information may be prohibitive as OBA, the Volumetric Scene-based approach limits the number of PCM (Pulse-Code Modulation) channels transmitted to the end-user as compact HOA signals [ref.].

To summarizing, the advantages of the Volumetric Scene-based audio approach affecting the CPU and power consumption are:
 

• Rendering complexity independent of the number of objects in the scene, decreasing the CPU usage from 100 to 10 000 times in comparison to OBA

The source signals are converted to a fixed number of HOA signals, uniquely dependent on the HOA order, and not on the number of objects present in the scene (even if the audio scene is created from thousands of audio signals).

• Possibility of rendering HOA signals to any reproduction device

A single format of HOA signals (with the addition of a few audio objects, if needed) can be reproduced on any device including headphones, soundbars, mobile etc.

• Lower number of PCM channels sent to the end user

HOA signals can be “spatially compressed” while maintaining high levels of spatial accuracy.

• Personalized interaction with immersive audio

HOA can be reproduced through headphones via the HOA to Binaural rendering and paired to Head-mounted displays to allow real-time adaptation of the sound field to head rotations in VR productions. It also facilitates 360° video applications where the sound field rotation can typically be controlled by moving a finger on the touch screen of a mobile device.
​

• Lower production costs for loudspeaker layout

The production (mixing and mastering) is performed in a single HOA format which can be rendered to any channel-based audio format for any loudspeaker layout (e.g., 2.0, 5.1, Dolby Atmos, …).  There is no need to produce separate mixes for each target loudspeaker layout which reduces production costs and efforts.

One of the most innovative and efficient tools for producing Volumetric Scene-based audio is ZYLIA 6 Degrees of Freedom Navigable Audio solution. It is based on several Higher Order Ambisonics microphones which capture large sound-scenes in high resolution, and a set of software for recording, synchronizing signals, converting audio to B-Format, and rendering HOA files. The Renderer can be also used independently from the 6DoF hardware – to create navigable 3D assets for audio game design.

ZYLIA 6 DoF HOA Renderer is a MAX/MSP plugin available for MAC OS and Windows. It allows processing and rendering ZYLIA Navigable 3D Audio content. With this plugin users can playback the synchronized Ambisonics files, change the listener’s position, and interpolate multiple Ambisonics spheres. The plugin is also available for Wwise, allowing developers to use ZYLIA Navigable Audio technology in various game engines.

Volumetric Scene-based audio and Higher Order Ambisonics can be used for many different purposes, not only for creating soundtracks for games. This format is very efficient when producing audio for:
 

• Virtual, Augmented, and Mixed Realities
• Facebook 360, YouTube Spatial Audio, Google Resonance
• 360° videos

#zylia #gameaudio #6dof #objectbased #scenebased #audio #volumetric #gamedevelopment #GameDevelopersConference #GDC2021 #GDC

We are happy to announce the new release of the ZYLIA 6DoF Recording Application in version 1.0.0 for Linux and macOS. This application is a part of the ZYLIA 6DoF Navigable Audio solution (ZYLIA 6DoF VR/AR set). It replaces the command line toolkit for the recording and synchronization process. For your comfort, this application has a graphical user interface, so there is no need to use the command line anymore.  

This application offers all features of the ZYLIA 6DoF Recording Toolkit such as:​

• Recording of 6DoF content from an unlimited number of ZYLIA ZM-1S microphones. Or up to 3000 channels on a single computer.
• ZYLIA ZM-1S microphones identification via LED.
• Manual or automatic microphones position configuration.
• Import or export of microphone configuration.
• LED control on all connected ZYLIA ZM-1S microphones. It is possible to turn on/off the LED on all of part of the microphones.
• Sample by sample synchronization of recorded material from any number of recordings. It gives the high sampling precision necessary for full VR / 6DoF content production.

Additionally, there are added few new features.

• During recording, the VU meters of each microphone is shown on the recording screen.
• Additionally, during recording, you can take notes about the recorded take, which will be saved in a certain directory together with audio.
• Automatic management of the location of the raw and synchronized signal.
• Disconnecting the microphone during recording no longer causes a crash of the recording process.

For this workflow you will require:

• MacOS system (with connection to Ethernet + WiFi)
• Insta360 One R camera,
• Insta360 mobile app installed on your smartphone
• ZM-1 microphone
• OBS (https://obsproject.com/)
• ZYLIA Streaming application (https://www.zylia.co/zylia-streaming-application-download.html)
• MacOS RTMP Local host application (https://github.com/sallar/mac-local-rtmp-server)
• Facebook profile

This project consists of 3 main parts:
A. Receiving a live 360° video from the Insta360 to OBS
B. Receiving Ambisonics audio from the ZM-1microphone to OBS.
C. Configuring Facebook for live stream of 360° video with Ambisonics audio

​A. Receiving a live 360° video from the Insta360 to OBS
1. Turn On the Insta360 camera and connect your computer Wi-Fi to the corresponding Insta360 Wi-Fi network.

2. Simultaneously, connect your computer to your local ethernet using an ethernet cable. On your network preferences you should be connected with Ethernet and Wi-Fi.
On step number 4 you will need the IP address for the Insta360 Wi-Fi referred on your network preferences.

3. On your MacOS, open the application Local RTMP server (https://github.com/sallar/mac-local-rtmp-server).
This simple application will be used to receive a live stream from the Insta360 camera. It also provides you the local host address for OBS.

5. Open OBS and setup your project:

• On the video resolution you require a 2:1 format such as 1280x640
• Also, select your audio to 4.0 to receive 1st order Ambisonics audio later.

6. To receive the video stream from the Insta360 to OBS:

• On the Sources, click the plus sign and add “Media Source”
• Disable “Local File”
• On the input add the same address as the Insta360 camera, you may also use:
  • rtmp://localhost/live/abc

After adding the source, you should be receiving a 360° video from the Insta360 into OBS.

B Receiving Ambisonics audio from the ZM-1microphone to OBS.

7. Connect your ZM-1 microphone to your computer.
Open ZYLIA Streaming application.

8. In OBS, add source select ZYLIA Streaming application.

You are now receiving 1st Order Ambisonics audio (4 channels) and a 360° video into OBS.

​C Setup Facebook for livestreaming

​On the Settings – Stream, enable “Broadcast this as a spherical video” and also “This broadcast has Audio 360 (ambix4)”.

10. In OBS press “Start Streaming”. You are now live!