Interactive Video Part 1: Preparing the Videos with Adobe After Effects

Written by Gilles on March 7, 2011

Hey guys!

Today, we’re going to make this really cool effect of compositing a Facebook profile picture into the screen of an iPhone. This will allow us to personalise the user experience and make him or her get right into the story.

First, the finished result of the tutorial is available here.

The following pages go all about tracking, exporting, coding, encoding, etc… so we hope that you’re quite at ease with Adobe Flash and Adobe After Effects. No expert skills required here, though. You can download demo versions from Adobe if you need to.

The whole process is a bit long, so we’ve split it up into parts which goes like this :

  1. Preparing the Videos with Adobe After Effects
  2. Developing the Player with the new AS3 API
  3. Getting the User Content from Facebook
To get you started, we got some footage for you to download

Chapter 1 : motion tracking

Let’s start with a New Composition with the following specs :

  • Resolution : 1280 x 720
  • Framerate : 25fps
  • Duration : 00:00:09:05

Import the two provided assets and drag them into the timeline. The JPG picture should be on the top layer.

In the Timeline panel, select the footage and on the Tracker panel, create a perspective corner pin tracker

Drag the tracking points to the four white markers we stuck on the device during the shoot and analyse the target by hitting the Play button on the control panel.

Tracking can be a tricky task, so don’t hesitate to play with the settings to get the best results.

On the Tracker panel, click on Edit Target… and select the test_asset.jpg. Click on apply.

A 4 corner pin effect has been added to the image with all our tracking points.

The asset now fits the device but not the screen. To fix that add another corner pin effect (right click on the asset -> effect -> distort -> Corner Pin) and drag the effect to the top of the Effects’ list).

On the timeline, open the second corner pin effect and update the 4 pin position to make it overlap the phone screen by dragging the x, y propriety directly on the timeline.

Chapter 2 : video layers

In order to have a realistic compositing, the player will display several video layers : one video in the back, one overlay in the front and a video mask will be applied to the image.

Mask layer

Using a mask in that case is useless because nothing crosses over the front of the tracked area, but in typical case this happen. In order to create the mask, create a black solid (right click -> new -> Solid) at the bottom of the timeline (or change the composition background to black) and create a white solid on the top of the timeline. On the Tracking panel, click on Edit target…, select the white solid and apply.

As you did with test_asset, create a second corner pin effect, put it in the front and adjust the 4 positions values to make it match the screen. You should have the mask matching with test_asset.

Front layer

In our example the front layer will display the reflection of the phone screen. Duplicate the white solid layer and the footage layer, move them to the front (the white solid first and the footage underneath) and change tracking-footage.mov‘s TrackMatte to Alpha. Finally, change its blending mode to Screen.

 

 

The result looks like this. (Don’t forget to hide the masks in the timeline, they are only references)

 

Chapter 3 : exporting

The three layers (back, mask, front) are exported to three different FLV files.

Note : We are not using the newer F4V format (h264) for two reasons : there is no Alpha Channel support, so no transparency and it cannot be embedded in the Flash files. (We’ll come to that later).

Front file : mytracking_front.flv

Uses the default FLV settings without sound and RGB + Alpha Channel

 

 

 

Mask file : mytracking_mask.flv

Uses the default FLV settings without sound.

 

 

 

Back file : mytracking_back.flv

Uses the default FLV settings with sound (our file is mute)

Tracking data export

The tracking sequence player developed in Flash uses XML data to display the 4 corner pin polygon in real-time. We build a conversion script in PHP to convert Adobe After Effects’ output file to the format Flash requires. Feel free to use it online.

The first field is the name of the tracking sequence you want to create. in some case we have several tracking sequences and each one has a name to be recognised by the player. Let’s call the sequence my_sequence.

The second field is the 4 corner pin data.
In Adobe After Effects, open the first corner pin details of test_asset,  select the four properties, copy it (CTRL + C or CMD + C) and paste it into the text area.

The third field is the position data.
In Adobe After Effects, open the transform details of test_asset, select the position property, copy it (CTRL + C or CMD + C) and paste it into the text area.

Keep default values for the other fields.

Click on the submit button for the XML file to be generated.

Create a new xml file called tutorial-tracking.xml with your favourite text editor and create the root node named tracks.
In that node, copy the trackedSequence node you get from the converter and you should have a xml file like this.

All assets are now ready to be integrated into Adobe Flash.

You can download all the source files here.

Want to know how we can get these three videos together?
This is for the next tutorial : Interactive Video Part 2: Developing the Player with the new AS3 API