Audio Synchronisation

Latest revision as of 19:21, 28 September 2016

Language:

English

This tutorial explains how to create a simple Synfig animation synchronised to an audio source. All source material is available for download for education purposes.

Out of the box

Sound Layer

With the Sound Layer you can import sound files and played with the animation previews.

Jack audio server

You can synchronized your animation threw the JAck audio server.

Have a look to the Jack Audio Synchronization Tutorial or this video tutorial.

Multiple approaches

Many approaches are possible. We herein propose a very fast way to design a new video for an existing audio track. The audio source is prepared and labeled in Audacity, a well-known opensource audio editor available for Windows, Mac and Linux. A Synfig plug-in will then be used to import these labels into Synfig and design the animation on top. To use Synfig plug-ins you need to install Python first.

A common alternative is to synchronise your audio and video in a third party video editing software (such as avidemux, Openshot, Blender VSE or many others). It is overall more flexible, but very accurate synchronisation is harder to reach.

The example

For this tutorial we'll be creating a bouncing ball animation based on portions of beskhu's Ping Pong Ball sound.

The resulting video can be viewed online or downloaded.

Prepare and label the audio track

adding a label in Audacity

In Audacity, edit your audio to suit your needs (and storyboard), then select and label each section that seems relevant as an animation reference point (or keyframe in Synfig) in your audio track.

In our case, these significant moments would be the various impacts of the ball, and optionally the intermediate top of each bounce. Download the tutorial audio to try it out!

Adding a label in Audacity is done via Tracks > Add Label at Selection (or (Ctrl+b)). See this tutorial for more details on labelling audio tracks in Audacity. Try to make each label name unique, short and meaningful.

Once satisfied with your labels location and name, use the Tracks > Edit Labels dialog to export the labels into a file named labels.txt in the same directory as your Synfig Project.

Import keyframes in your Synfig project

run the plugin on your .sif project

Install Python and the keyframe import plug-in if you don't have them yet.

Open Synfig and create a new project, choose a document resolution fit to your use (360x203 Web HD, 24fps, for this tutorial).

Synfig plug-ins only work on .sif files, not the default .sifz format. Simply rename your-project.sifz into your-project.sif in the > File > Save As dialog (music_sync_tutorial.sif for this tutorial).

Importing keyframes is then as simple as running the plug-in from > Plug-Ins > Import Audacity Labels as Keyframes. It will import the labels from the file labels.txt in the same directory as your Synfig project. Some options can be customized, but the default settings will do just fine for this tutorial.

Check out the new keyframes in the Keyframes window!

Set your scene up

set up the scene

Listen to the audio... in this sample animation we need a ball to bounce, and some wood-like surface to roll on.

Set them up in any way you like, or copy that in the screenshot, it consists of:

• a ball: a circle and curve gradient (pale yellow for the ball, to blue for the background)
• a floor: a brown rectangle with light rotation

Animate along the keyframes

animate the ball position

Switch to Animate Editing Mode(1) and navigate your different keyframes using the seek buttons (2) to modify for each the origin of the ball, either manually in the parameters (3) or by moving its handle (4).

You can easily refer to the active keyframe name (5) to know where to locate the ball if the name of your audio label was meaningful enough.

It may be necessary to add a few keyframes or waypoints. We added one "exit" keyframe at the end of the animation to make the ball roll out of the picture.

Fine tune the interpolation type with a right-click on the waypoint to make the animation smoother. We opted for [linear] on bounce, [clamped] on tops and [Ease_In/Out easy in] to slow the rolling ball on the last (exit) waypoint.

Render

Rendering in Synfig does not currently support the inclusion of audio tracks, we therefore render the animation as .png images, and merge them with the audio using the opensource avconv tool. This whole process is likely to be integrated in Synfig in a future release.

To render the animation frames select > File > Render, preferably locate the target in a distinct directory (render/music_sync_tutorial.png in our case), check the time settings (duration of our audio is 2s 10f) and hit "Render".

Collate the rendered frames and the audio using avconv:

   avconv -r 24 -i render/music_sync_tutorial.%04d.png -i music_sync_tutorial.ogg -c:a copy music_sync_tutorial.mp4

The above options are: -r rate of the following input stream: the fps used in this Synfig project, -i input: the sequence of images and the audio file, -c:a codec for the audio stream: copy the current audio (ogg) file without re-encoding it.

The resulting video can be downloaded.

Going further

To go further you may want to make the bounces more realistic: see the Ball Bounce tutorial, or make a texturized and rotating ball, or a realistic wood floor... Share your creations in the Synfig forum!

Language:

English