ZYLIA - 3D AUDIO RECORDING & POST-PROCESSING
  • ZYLIA Home
  • Blog
  • Shop

How to prepare a 360 video with 3rd Order Ambisonics audio

11/26/2020

0 Comments

 
by Pedro Firmino
This tutorial is based on the solution developed by professor Angelo Farina for preparing a 360 video with 3rd Order audio (source http://www.angelofarina.it/Ambix+HL.htm).

In this adaptation, we will show you how to create a 360 video with 3rd Order Ambisonics audio using:
  • MacOS
  • ZYLIA ZM-1 microphone
  • ZYLIA Ambisonics Converter plugin
  • IEM Binaural Decoder
  • Reaper
  • Modifed version of Google Spatial Media Metadata injector created by professor Angelo Farina.
​
This tutorial consists in 2 parts:
A: Preparing the 360 content with 16 channels
B: Injecting metadata using Spatial Media Injector version, modified by Angelo Farina.


At the moment, only HOAST library ( https://hoast.iem.at/ ) is the only platform which allows online video playback of 3rd Order Ambisonics and therefore the content created from this tutorial is meant to be watched locally using VLC player.

For this tutorial, basic Python knowledge is advised.

For preparing a 360 video with 1st order Ambisonics, visit the link:
https://www.zylia.co/blog/how-to-prepare-a-360-video-with-spatial-audio

PART A
1. As usual, start by recording your 360 video with the ZYLIA ZM-1 microphone and remember to have the front of the ZM-1 aligned with the front of the 360 camera.

2. After recording, import the 360 video and the 19 Multichannel audio file into Reaper.
Syncronize the audio and video.
Picture
​3. On the ZM-1 audio track, insert ZYLIA Ambisonics Converter and select 3rd Order Ambisonics. This will decode your 19 multichannel track into 16 channels (3rd Order Ambisonics).
Picture
​4. On the Master track, click on the Route button, On the track channels, select 16. Now you are receiving the signal from the 16 channels from the audio track.
Picture
​5. Once the video is ready for exporting, click File – Render.

As for the settings:
Sample rate: 48000
Channels: 16 (click on the space and manually type 16)
Output format: Video (ffmpeg/libav encoder)
Size: 3840 x 1920 (or Get width/height/framerate from current video item
Format: QT/Mov/MP4
Video Codec: H.264
Audio Codec: 24 bit PCM

Render the video.
Picture

PART B
After having the 360 video with 16 channels, it is necessary to inject metadata for Spatial Audio.

In order to do this, Python is required. Python is preinstalled in macOS but
you can download Python 2.7 version here: https://www.python.org/download/releases/2.7/

Afterward, download Angelo Farina’s modified version of Spatial Media Metadata Injector, located at:
http://www.angelofarina.it/Ambix+HL.htm
Picture
​The next part:

1. With the downloaded file located in your Desktop, run macOS Terminal application.
2. Using “cd” command, go to folder where you have Spatial Media Injector (eg. “cd ~/Desktop/spatial-media-2/”)
Picture
3. Run Python script “sudo python setup.py install”. Type your password.
Picture
​After the build is complete, type command: “cd build/lib/spatialmedia”

6. Enter python gui.py and the application should run.
Picture
​With the Spatial Media Metadata Injector opened, simply open the created 360 video file, and check the boxes for the 360 format and spatial audio. Inject metadata and your video will be ready for playback using 3rd Order Ambisonics audio.

Picture
0 Comments

How to convert 360 to 2D video with linked Ambisonics rotation for binaural audio

8/11/2020

0 Comments

 
In this tutorial we describe the process of converting 360 video and 3rd order Ambisonics to 2D video with binaural audio with linked rotation parameters.
This allows us to prepare a standard 2D video while keeping the focus on the action from the video and audio perspective.
It also allows us to control the video and audio rotation in real time using a single controller.

Reaper DAW was used to create automated rotation of 360 audio and video.
Audio recorded with ZYLIA ZM-1 microphone array.

Below you will find our video and text tutorial which demonstrate the setup process.
Thank you Red Bull Media House for providing us with the Ambisonics audio and 360 video for this project.

Ambisonics audio and 360 video is Copyrighted by Red Bull Media House Chief Innovation Office and Projekt Spielberg, contact: cino (@) redbull.com
Created by Zylia Inc. / sp. z o.o. https://www.zylia.co
Requirements for this tutorial:
  •  Video and audio recorded with 360 video camera and ZM-1 microphone
  •  ZYLIA Ambisonics Converter plugin
  •  IEM Binaural Decoder
  •  Reaper

We will use Reaper as a DAW and video editor, as it supports video and multichannel audio from the ZM-1 microphone.

Before recording the 360 video with the ZM-1 microphone make sure to have the front of the camera pointing the same direction as the front of the ZM-1 (red dot on the equator represents the front of the ZM-1 microphone) , this is to prevent future problems and to know in which direction to rotate the audio and video.

​Step 1 - Add your 360 video to a Reaper session. 

The video file format may be .mov .mp4 .avi or other.
From our experience we recommend to work on a compressed version of the video and replace this media file later for rendering (step 14).

To open the Video window click on View – VIDEO  or press Control + Shift + V to show the video.
Picture
Step 2 - Add the multichannel track recorded with the ZM-1 and sync the Video with the ZM-1 Audio track.

Import the 19 channel file from your ZM-1 and sync it with the video file.
Picture
​Step 3 – Disable or lower the volume of the Audio track from the video file.

Since we will not use the audio from the video track, we require to remove or put the volume from the audio track at minimum value. 
To do so, right click on the Video track – Item properties – move the volume slider to the minimum.
Picture
Picture
​Step 4 – Merge video and audio on the same track.

Select both the video and audio track and right click – Take – implode items across tracks into takes
This will merge video and audio to the same track but as different takes.
Picture
​Step 5 – Show both takes.
​
To show both takes, click on Options – Show all takes in lanes (when room) or press Ctrl + L
Picture
Step 6 – Change the number of channels to 20.

Click on the Route button and change the number of track channels from 2 to 20, this is required to utilize the 19 multichannel of the ZM-1.
Picture
​Step 7 - Play both takes simultaneously.

If we press play right now, it will only play the selected take, therefore we need to be able to play both takes simultaneously, therefore:

Right click on the track – Item settings – Play all takes.
Picture
Step 8 – Change 360 video to standard video.

Next we will need to convert the 360 video to equirectangular video to visualize and control the rotation of the camera.

To do so, open the FX window on our main track and search for Video processor.

Picture
​On the preset selection, choose Equirectangular/spherical 360 panner, this will flatten your 360 video allowing you to control the camera parameters such as field of view, yaw, pitch and roll.
Picture
Step 9 – As FX, add ZYLIA Ambisonics Converter plugin and IEM binaural Converter.

 On the FX window add as well:
  • ZYLIA Ambisonics Converter plugin and set it to 3rd Ambisonics Order. Make sure to set the microphone orientation how you recorded it in the first place.
  • IEM Binaural Decoder. Here you can choose headphone equalization at your liking.
Picture
Picture
You should now have the binaural audio which you can test by changing the rotation and elevation parameters in ZYLIA Ambisonics Converter plugin.
​Step 10 – Link the rotation of both audio and video.

The next steps will be dedicated to linking the Rotation of the ZYLIA Ambisonics Converter and the YAW parameter from the Video Processor.

On the main track, click on the Track Envelopes/Automation button and enable the UI for the YAW (in Equirectangular/spherical 360 panner) and Rotation (in ZYLIA Ambisonics Converter plugin).
Picture
​Step 11 – Control Video yaw with the ZYLIA Ambisonics Converter plugin.

On the same window, on the YAW parameters click on Mod…  (Parameter Modulation/Link for YAW) and check the box Link from MIDI or FX parameter.
Select ZYLIA Ambisonics plugin: Rotation
Picture
Step 12 – Align the position of the audio and video using the Offset control.

On the Parameter Modulation window you are able to fine-tune the rotation of the audio with the video.
Here we changed the ZYLIA Ambisonics plugin Rotation Offset to -50 % to allow the front of the video match the front of the ZM-1 microphone.
Picture
Step 13 – Change the Envelope mode to Write.

To record the automation of this rotation effect, right-click on the Rotation parameter and select Envelope to make the envelope visible.
Picture
​After, on the Rotation Envelope Arm button (green button), right click and change the mode to write.
Picture
​By pressing play you will record the automation of video and audio rotation in real time.
Step 14 – Prepare for Rendering

After writing the automation, change the envelope to Read mode instead of Write mode.
Picture
​Disable the parameter modulation from the YAW control:
Right click on Yaw and uncheck “Link from MIDI or FX parameter”
Picture
​OPTIONAL: Replace your video file with the uncompressed version.

If you have been working with a compressed video file, this is the time to replace it with the original media file. To do this, right click on the video track and select item properties.
Picture
Scroll to the next page and click Choose new file.
Then select your original uncompressed video file.
Picture
​Step 15 – Render!

You should now have your project ready for Rendering.
Click on File – Render and set Channels to Stereo.
On the Output format choose your preferred Video format.
We exported our clip in .mov file with video codec H.264 and 24bit PCM for the Audio Codec.
Picture

Thank you for reading and don’t hesitate to contact us with any feedback, questions or your results from following this guide.
0 Comments

How can I copy all sessions from ZYLIA Studio to another device?

11/26/2019

0 Comments

 
If you would like to move/copy all sessions and recordings from one device to another, first open the ZYLIA Studio directory. The default location is:

Windows: "C:/Users/<user_name>/AppData/Local/Zylia/ZYLIA Studio"
MacOS: "/Users/<user_name>/Library/Application Support/Zylia/ZYLIA Studio"

Copy the folders “calibration” “sessions” and also the file “ZS.sqlite” to a pen drive/ external disk.
Picture
On the target computer, you can either replace those files and folders on the default ZYLIA Studio location or just create a new folder, paste files and change the default ZYLIA Studio directory to this newly created folder.
The default directory can be changed by clicking the Cog icon followed by Change data location.
0 Comments

New release of ZYLIA ZM-1 driver for macOS v2.4.0

4/11/2019

0 Comments

 
​We are thrilled to present you the new ZYLIA ZM-1 driver with a long-waited gain control possibility. This feature introduced by Zylia Team is the answer for our clients’ request and feedback.
 
The gain control feature allows you to increase a digital gain of ZYLIA ZM-1 input signal up to 70 dB using ZYLIA Control Panel as well as the Audio MIDI Setup application. The gain amount is stored in the system, so if your recording conditions don't change there is no need to set it before the next session.
Picture
ZYLIA Control Panel 
Picture
​Audio MIDI Setup application
​Watch this tutorial for step-by-step instructions.
0 Comments

How to add binaural and Ambisonics sound to a video for YouTube 360?

9/14/2018

1 Comment

 
By Eduardo Patricio

​In this post we present two videos in different formats, but edited from the same source material captured on the 20th of June 2018, at Barigui park (Curitiba, Brazil).

The audio was recorded with the ZYLIA ZM-1 3rd order Ambisonics spherical microphone array while the video was captured by a 360-degree camera (Gear 360).
 
Below, you can watch both videos and find some information on how to achieve the two different results, with focus on preparing the audio recorded with the ZM-1 microphone for each scenario.

Scenario A

Interactive, immersive video with full 3D sound 
​(media components: 360-degree video + Ambisonics audio) 

Scenario B

​Non-interactive video with fixed perspective 3D sound
(media components: Tiny planet” video + binaural audio)
Equipment used
 
  • ZYLIA ZM-1 microphone
  • Samsung Gear 360 camera (2017 model) + micro SD card
  • Laptop (software list below)
  • Camera stand
  • Extension support arm with clamp
  • USB cable (for the ZM-1)
  • Large microphone windshield 
Software used

  • ZYLIA Studio
  • ZYLIA Studio PRO
  • ZYLIA Ambisonics Converter
  • REAPER
  • Insta360studio
  • Adobe Premiere
The microphone and the camera were placed on a single camera stand with a small clamped extension arm (see picture below). Both devices were aligned vertically with a small horizontal offset. We made sure the microphone and the camera always had the same relative facing direction (front of the microphone aligned with the camera side where the recording button is found).

Picture
ZM-1 and Gear 360 placement on a single stand
​with a clamped extension
Picture
Camera and microphone alignment
Picture
Chosen location at Barigui park
​The exact location at Barigui park was carefully chosen for its proximity to the lake that always attracts various birds, and also for being next to a rather busy highway and a helicopter landing pad. 
​For scenario A, we used the regular stitched video from the Gear 360 and a 1st order Ambisonics audio file.

​Scenario A - Basic steps taken:
  • Simultaneously record video footage on Gear 360 and audio on laptop (running ZYLIA Studio);
  • Convert the raw 19-channel file from ZYLIA Studio to Ambisonics, using ZYLIA Ambisonics converter;
  • Edit 360-degree video and Ambisonics audio on Adobe Premiere.
 
Here are the detailed steps taken for the conversion to Ambisonics:

  • Input (ZM-1’s raw) audio file was selected
  • Microphone model selected: ZYLIA ZM-1 model 1D
  • Orientation selected: Upright, 0 degrees
  • Output format selected: AmbiX*
 
* Standard currently (August 2018) used on YouTube.
Picture
For scenario B, we used the video from Gear 360 in ‘tiny planet’ format and a binaural audio track.

Since, the source material is the same as the one from scenario A, we’ll list here only the steps that differ.

Scenario B steps:

  • Process stereoscopic video from Gear 360 on Insta360 Studio to have the ‘tiny planet’ effect;
  • Convert the raw 19-channel file from ZYLIA Studio to binaural, using ZYLIA Studio PRO running in REAPER.
  • Edit 360-degree video and Ambisonics audio on Adobe Premiere.
Picture
Choosing binaural preset on ZYLIA Studio PRO in REAPER
#ambiencerecording #ambisonics #binaural #soundscapes #immersiveaudio #360recording
1 Comment

How to configure ProTools to work with ZYLIA Studio PRO using ReWire

3/26/2018

0 Comments

 
By Zaneta Lejwoda

ZYLIA Studio PRO is an innovative VST/AU plugin which allows you to separate sources from ZYLIA ZM-1 microphone recording in a real-time. It requires 64-bit DAW with at least 19 channels per track. You can create up to 24 "virtual microphones", adjust and automate parameters of them such as azimuth, elevation and polar pattern. Because of this multichannel processing you can't use ZYLIA Studio PRO directly within ProTools but there is a way to mix signals from „virtual microphones” in this DAW – you can use ReWire protocol.
​
ReWire allows you to connect two applications, where first is a master (it's called "ReWire Mixer") and second is a slave ("ReWire Device"). In this tutorial, we will use ProTools as ReWire Mixer and Reaper as ReWire Device. Reaper is a digital audio workstation, it handles up to 64 channels per track and it's perfect to use it with ZYLIA Studio PRO.

TUTORIAL

First of all, we need a host for ZYLIA Studio PRO plugin – Reaper (if you haven't installed it yet – now it's good time to do it).
​Run Reaper, open Preferences – Plugins – ReWire/DX and enable the option "Check for ReWire mixer/device on startup (and automatically enter slave mode if present)", click "OK" and close application (Fig. 1).
Picture
Fig. 1: Reaper, ReWire configuration.
Next, open Pro Tools and make a new session – remember that ZYLIA requires 48 kHz sample rate (Fig. 2).
Create new tracks for all sources („virtual microphones”) you want to use in your mix.
Picture
Fig. 2: Pro Tools, audio settings.
Open Reaper, configure session and import your recording (you can use our template).

​Now, the application should run automatically in the slave mode – you should be able to see "ReWire Slave (or just ReWire) on the menu bar, next to your audio settings.


Open Routing Matrix (View – Routing Matrix)
Delete send of Master Bus to ReWire Output 1(stereo) and send your virtual microphones' tracks to ReWire Mono Output - in this case it's Output 3 to 6 (Fig. 3).
 
Now, you can go back to Pro Tools.
​
Picture
Fig. 3: Reaper, routing matrix.
On previously prepared tracks insert Reaper, click on insert field, choose Multi-mono plug-in – Instruments – Reaper (Fig. 4).
Picture
Fig. 4: Pro Tools, Reaper inserting.
After doing that you will see the window with ReWire logo, in which you can choose ReWire Output (picture below). In this case, Z1 track in Reaper is connected to ReWire mono output number 3, so Pro Tools' Z1 track is assigned to ReWire Output 3 and so forth (Fig. 5).
Picture
Fig. 5: Pro Tools, choosing ReWire Output fo Z1 track.
That's all, now you should be able to hear all your virtual microphones in Pro Tools  – just insert your favorite plugins and have fun mixing!
0 Comments

Ardour session configuration

2/23/2018

1 Comment

 
by Zaneta Lejwoda

In this article, we will show how to configure Ardour session to work with ZYLIA Studio PRO.
Briefly, we connect 19-channel track recorded with ZYLIA ZM-1 microphone with "output" tracks for virtual microphones (and leave out the connection to master) and then route "output" tracks to the master bus.

Mixing

​Create a new session. In Audio/MIDI Setup choose your audio system and driver.
Set sample rate to 48 kHz (1) and press Start.
Picture
Now we can import 19-channel recording. Click on Session menu and pick Import (or just press Ctrl/Cmd+I). Choose your recording. To import it correctly change Mapping to “one track per file” option.
Picture
It’s important to bypass Ardour panner for this track (right-click on panner icon and enable Bypass).
Picture
Add ZYLIA Studio PRO plugin – open Plugin manager (right-click on INPUT TRACK → New Plugin → Plugin Manager), double-click ZYLIA Studio PRO and click Insert Plugin.
Picture
After creating “input” track for ZYLIA Studio PRO it’s time to make some “output” tracks. To add them just right-click on free space in the mixer or editor view and make as many as you need (remember – one track per each ZYLIA Studio PRO virtual microphone).  ​
Picture
Now the most complicated thing – routing. Open Audio Connection Manager (Window → Audio Connections or Alt+P). In this window, you can see a matrix of Sources and Destinations.
​We want to route our “output” tracks to Master Bus, so choose Ardour Tracks as Sources and Ardour Buses as Destinations and connect them (green dots).
Picture
Then, choose Ardour Tracks as Destinations and connect the output of Virtual microphone (“input” track out) with Z1 “output” track in.
Picture
Add some virtual microphones, enable “output” tracks monitoring and have fun mixing.
Picture

Recording

Plug in ZM-1 microphone and create a session. In Audio/MIDI Setup choose ZM-1 as your input device (Windows users should pick ASIO as their driver).
​Remember to set sample frequency to 48 kHz.

Picture
Next, add a track for your 19-channel recording (right-click on free space in the mixer). Choose custom track configuration.  ​
Picture
Now you can add to this track nineteen channels by clicking on “-” button (picture below) and picking Add audio port (unfortunately, you can add just one channel at once).
Picture
To route signal from ZM-1 open Audio Connection Manager (Window → Audio Connections or Alt+P). As Sources pick Hardware tab and as Destinations Ardour Tracks tab and connect 19-channel track with ZM-1 (green dots).
 
Now you can arm your track and record.

Input monitoring is possible for macOS users – create an aggregate device and choose it as your audio device.
Picture
1 Comment

Reaper session configuration

2/13/2018

1 Comment

 
By Zaneta Lejwoda
In this tutorial we will explain how to configure Reaper session to work with ZYLIA Studio PRO software.

Mixing

Audio Device setup

The first step to get ZYLIA Studio PRO plugin to work is audio device configuration.


​Open REAPER Preferences and choose Audio → Device (1). Next, set up your audio system in Audio System section (2).
IMPORTANT: ZYLIA Studio PRO requires 48000 Hz sample rate. You can set it in section 3.
ASIO users can set it in their driver’s control panel
(if not use Request sample rate option).
Picture
Routing
The second step is a proper tracks routing.
​Add at least two tracks. First track is “Zylia Studio PRO” with imported ZM-1 19-channel recording.
Click on Routing button, deselect Master send option (1) and set Track channels to 24 (2).
Then click FX button and add ZYLIA Studio PRO plugin.
Picture
Now it’s time to deal with a second track - “Z1”. This is the “output” track for ZYLIA virtual microphone.
Click Routing button and Add new receive, choose a track with ZYLIA Studio PRO plugin.
Picture
Next, click on menu with types of received signal and set it to Pre-Fader(Post-FX) (2).
Then click on audio input menu (3) and pick Mono source and channel number corresponding to virtual microphone output (in this example it’s Z1). 
Picture
Similar settings are applicable for more virtual microphones – just add more “output” tracks and set them up as above (remember to choose a proper audio input channel number).

​Recording

It is possible to record directly from a ZM-1 microphone into a Reaper DAW.
Audio Device setup
 
Plug in ZM-1, open REAPER Preferences Audio → Devices (1).

If you are using Windows – choose ASIO as your Audio system and as ASIO driver pick ZYLIA ZM-1 (2). It’s important to enable Request sample rate option and set 48000 Hz.
Number of last enabled input must correspond with “Zylia Studio Pro” track’s number of channels.
Picture
MacOS users can pick both ZYLIA ZM-1 or Aggregate Device (more in Appendix) as their Audio System. Main advantage of second solution is possibility to listen to output of ZYLIA virtual microphones and raw 19-channel track (input monitoring).
In Audio MIDI Setup you have to choose 20-channels or more in Format section (4).
​Number of ZYLIA ZM-1 channels must correspond with “Zylia Studio Pro” track’s number of channels.

Picture
Routing
Routing for recording is similar to a mixing setup. You should add track for 19-channel recording with plugged ZYLIA Studio PRO and “output” track for each virtual microphone. Remember to set up appropriate number of “Zylia Studio PRO” track’s channels and assign each “output” track’s audio input to corresponding virtual microphone output.
To start recording choose 20 channel input for your “Zylia Studio PRO” track and arm it (1). 
Picture
You can also record signal from virtual microphones –  choose Record: output (mono) option and set input to None.
Picture

Appendix

Input monitoring using Aggregate Device (available only for macOS users).
Monitoring of ZM-1 input is possible by using Aggregate Device. 
To create it:
 • plug in ZM-1,
 • open Audio MIDI Setup
 • click on plus icon and add aggregate device (1). 

Next enable ZM-1 and your output (in this case it’s Built-in Output) in “Use” section(2).
Picture
Then open REAPER Preferences → Audio → Devices and choose Aggregate Device as your audio device (1).
The last thing to do is enable input monitoring of recorded tracks (2).​
Picture
Picture
Downloads:
 
ZYLIA Studio PRO Reaper Template
  • ZYLIA_Studio_PRO.rpp file contains blank Reaper session prepared for six sources. We suggest to save this file as Reaper project template – you’ll be able to make multiple projects based on this configuration (File→ Project Templates → Save project as template).
1 Comment

Create your own spatial Ambisonics mix with ZYLIA!

1/18/2018

1 Comment

 
By Jakub Zamojski & Lukasz Januszkiewicz

Recording and mixing surround sound becomes more and more popular. Among the popular multichannel representation of surround sound systems like 5.1, 7.1 or cinematic 22.2, especially worthy of note is an Ambisonics format, which is a full-sphere spatial audio technique allowing to get a real immersive experience of 3D sound. You can find more details about Ambisonics here (What is the Ambisonics format?).

Our previous blog post “2nd order Ambisonics Demo VR” described the process of combining audio and the corresponding 360 video into fine 360 movie on Facebook. Presented approach assumes using of 8-channel TBE signal from ZYLIA Ambisonics Converter and converts audio into the Ambisonics domain. As a result we get a nice 3D sound image which is rotating and adapting together with the virtual movement of our position. However, it is still not possible to adjust parameters (gain, EQ correction, etc.) or change the relative position of the individual sound sources present in the recorded sound scene.
In this tutorial we are going to introduce  another approach of using ZYLIA ZM-1 to create a 3D sound recording, which gives much more flexibility in sound source manipulation. It allows us not only to adjust the position of instruments in recorded 3D space around ZYLIA microphone, but also to control the gain or to apply any additional effects (EQ, Comp, etc.). In this way we are able to create a fancy spatial mix using only one microphone instead of several spot mics!
Picture

Spatial Encoding of Sound Sources – Tutorial

In the end of July 2017, using ZYLIA ZM-1 microphone we have recorded a band called “Trelotechnika”. All band members were located around ZM-1 microphone, 4 musicians and one additional sound source – drums (played from a loudspeaker). During the post-production process, we applied ZYLIA Studio PRO VST plug-in (within Reaper DAW) on recorded 19-channel audio track. This allowed us to separate the previously recorded instruments and transfer them into the individual tracks in the DAW. Those tracks were then directed to the FB360 plug-ins, where encoding to the Ambisonics domain was performed.

“Spatial Encoding of Sound Sources” - a step-by-step description

Below, you will find a detailed description of how to run a demo session presenting our approach of recording and spatial encoding of sound sources. Demo works on Mac Os X and Windows.
DOWNLOAD STAGE

  1. Download and install REAPER software. The evaluation version is fully functional and perfect to run with our demo.
  2. Download and install Facebook 360 Spatial Workstation software. It is a free bundle of spatial audio VST plug‑ins. In the session we used version 3.1 beta1.
  3. Download and install ZYLIA Studio Pro (VST3 Mac, Windows). It’s possible to run the demo in trial mode. In the session we used VST3 plug-in.
  4. Download the REAPER session prepared by Zylia Team. It is already configured with our audio tracks and all required effects. Unzip it.
  5. Download 360 movie – two versions are available: high quality [3840 x 1920] and medium quality [1920 x 960].  High quality version sometimes tends to pause in FB360 Video Player on slower CPUs.
  6. Run REAPER and open the session (ZYLIA-Ambisonics-sources-enc.rpp).
TUTORIAL
 
​After opening the session, you will see several tracks:
1. Very important! Please, ensure that Reaper is working with sample rate of 48 kHz.
​

2. ZS_pro track – contains 19-channel WAVE file recorded with ZYLIA ZM-1. Click on FX button located on the ZS_pro track. If everything is correct, you will see ZYLIA Studio PRO VST plug-in. By default, there will be 5 virtual microphones – each one already assigned to one of the instruments: bass, drums, guitar, synth and pad. By clicking on a specific virtual microphone, you can adjust azimuth, elevation, width, and separation mode. Master send in routing panel for track should be unchecked.
Picture
Picture
3. Separated signals from ZYLIA Studio PRO are passing to 5 individual tracks. You are able to adjust the gain, you can also mute or solo instruments, or you can apply some audio effects. A good practice is to use a high-pass filter for non-bass and low-pass for bass instruments to reduce a spill between them. We applied these filters to our session:
Picture
​4. Spatialiser track – receives 5 signals from tracks with separated instruments. Spatialiser allows to distribute sound sources in desired positions in the 3D space.
​      a) Click on FX and choose FB360 Spatialiser.
   b) Click on Load button placed on the video grid. Choose Slave mode and load the provided video clip. You will see a message box “H264 is not a recommended codec” - click X.
​
Picture
   c) Set the video format to Mono and the display mode to Desktop. In Connect to DAW you should be able to choose your computer’s name. If not, try to restart Reaper and repeat the steps. You are ready to click the Open button. Video box will appear.
Picture
   d) Back to Spatialiser view. You will see an equirectangular picture and five circles with numbers. Each circle represents a sound source position in the space. By default, sources are located in the positions corresponding to the real positions of the instruments in the picture, but it is possible to adjust it by clicking on the circle and dragging it around the picture.
Picture
5. Control track – receives a multichannel signal from Spatialiser. Control provides connection with the Video Player, rotates the audio scene and applies binauralization.

   a) Click on FX and choose FB360 Control.

   b)  Ensure that Listener Roll, Listener Pitch and Listener Yaw are properly received from video – controls should be darkened. Open a video box and try to rotate the image - Pitch and Yaw sliders should follow the image to rotate.

   c) JS: Master Limiter boosts the volume and protects from clipping/distortions.
​
   d) 
Master send in routing panel for track should be checked.
Picture
6. Now video is synchronized with audio. Adjusting the location of play-head in REAPER’s time line will affect the video’s time. Tap space bar to play audio and video. Rotation of the video in the player  is tracked by the decoded and binauralized Ambisonics sound.

7.  A good practice is to play video from the beginning of file to keep the synchronization. In some cases, it is necessary to close the VideoClient + VideoPlayer and load 360 video again to recover the synchronization.
​

8.  Now you are able to rotate video across the pitch and yaw axis. Your demo is ready to run.
1 Comment

Using ZYLIA Studio PRO with ReWire

1/16/2018

2 Comments

 
by Maciej Lenartowicz

Integrating new tools into existing work-flows may be a difficult task, especially when those tools introduce innovative features, such as “virtual microphones” in ZYLIA Studio PRO. 
One of ZYLIA Studio PRO requirements is support for multichannel input for the plugin, which not many DAWs offer. Fortunately, there are ways to combine best of both worlds, and use ZYLIA’s technology with a typical multitrack workflow designed for dedicated microphone recording.
​
Such connection can be achieved thanks to ReWire, which is a software protocol allowing applications to exchange control messages and audio data. ReWire works both on macOS and Windows and requires no prior installation or configuration. Find out more at Wikipedia or Propellerhead website.
"Virtual Microphone" is a spatial audio filtering technique based on advanced beam-forming and sound source separation algorithms. It is defined and created by the ZYLIA software which takes into account both phase and amplitude differences between signals captured by the 19 capsules of the ZM-1. Virtual Microphones can have different spatial characteristics, width or shape depending on recording use cases and user needs. Each can be used as a single spot microphone with super-narrow directivity response, or can be combined with other into a variety of multi‑channel surround recording sets. This technology gives musicians and sound engineers a very flexible recording toolbox which can be used in post‑production phase. Additionally, advanced DSP algorithms allow to reduce room echo or to increase the performance of sound source separation.
More in a ZYLIA white paper.

​This article will guide you through the process of setting up a connection between ZYLIA Studio PRO (running within REAPER) and your DAW of choice (assuming it supports ReWire protocol).
  1. Make sure to run your main DAW in the first place. In case of this tutorial, we will use Ableton Live. It will act as a ReWire master (mixer).
  2. When your DAW is up and running go to its audio settings. Select a proper device, sample rate, and buffer size. Keep in mind that ZS PRO will work only with a sample rate of 48 kHz.
  3. Start REAPER, which will be a host to ZS PRO plugin and act as a ReWire slave (device). If it is configured correctly, you should see “ReWire Slave Audio” in the upper right corner of REAPER window.
    If you don’t see this it means that REAPER is not running as a ReWire slave. To fix this, go to “Edit” menu and select “Preferences”. In the list on the left find “Plug-ins” section and select “ReWire/DX”.  Enable “Check for ReWire mixer/device on startup” option and restart REAPER.
Picture
Example session in REAPER with ZYLIA Studio PRO. Notice that REAPER runs in ReWire mode.
4. Now you can create a new project in REAPER or load existing one. Use ZYLIA Studio PRO as you would otherwise, i.e. create a new track for every virtual microphone, with just one exception – these new tracks won’t be sent to master track. To do so, enter “Routing” menu for a track in REAPER and disable “Master send” option.

​To route audio form virtual microphone out of REAPER, you need to select “hardware” output for a track. These are the very same channels that will be available in Ableton. Click on “Add new hardware output...” menu and select one of the outputs from the list. It is advised to start from “ReWire Output 3” (for Z1 virtual microphone) since first and second outputs are usually associated with a master track
Picture
5. Go back to Ableton and create a new track. You will need “In/Out Section” of the mixer visible – if it’s not, use round “IO” button just right to the master track to turn it on. In “Audio From” section select “REAPER” as an input device, then select “REAPER Output 3” as an input channel. When you press “play” in one of the applications the signal from REAPER (processed by ZYLIA Studio PRO) should appear in Ableton. Set “Monitor” option to “In” to hear the sound produced by REAPER. The signal from virtual microphone now may be recorded and processed in the same way as it would be from a normal microphone connected to your audio interface.
Picture
Picture
Ableton session set to receive audio from REAPER.
2 Comments
<<Previous
    Picture

    Categories

    All
    360 Recording
    6DOF
    Ambisonics
    Good Reading
    How To Posts
    Impulse Response
    Interviews
    Live Stream
    Product Review
    Recording
    Software Releases
    Tutorials

    Archives

    August 2023
    July 2023
    June 2023
    May 2023
    February 2023
    November 2022
    October 2022
    July 2022
    May 2022
    February 2022
    January 2022
    August 2021
    July 2021
    May 2021
    April 2021
    March 2021
    January 2021
    December 2020
    November 2020
    October 2020
    September 2020
    August 2020
    July 2020
    June 2020
    April 2020
    March 2020
    February 2020
    January 2020
    December 2019
    November 2019
    October 2019
    September 2019
    August 2019
    July 2019
    June 2019
    May 2019
    April 2019
    March 2019
    January 2019
    December 2018
    October 2018
    September 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    December 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    May 2017
    March 2017
    February 2017
    January 2017
    December 2016
    November 2016
    October 2016

    RSS Feed

Picture

Connect with us

Terms & Conditions   |   Refund Policy   |   Privacy Policy 
© Zylia Sp. z o.o., copyright 2018. ALL RIGHTS RESERVED. 
  • ZYLIA Home
  • Blog
  • Shop