Friday, April 29, 2016

Apple Motion: 3D for 2D

Late in 2015 I had the opportunity to produce 2 spots for Children’s Wish Foundation in Canada. The process behind the visuals was a bit unique, so I thought I’d detail it here for those interested.  

Motion is always a lot more verbose an application than people give it credit for.  I'd wager it can do most of what a lot of editors  are using After Effects for.  Honestly I think the biggest thing holding it back is some key 3rd party support by the same plug-in makers that bring a lot of the muscle to the table in Ae.  I’m looking at you, Trapcode and VideoCopilot!

The concepts behind the spots had been scripted and storyboarded during the Summer of 2015, with an eye to a handcrafted aesthetic.  The mandate was for simple colours and textures, and we planned to animate the characters using pretty standard Forward Kinematic puppet techniques.  Early on, we’d considered building backgrounds out of actual construction paper, but we realized we didn't have time for real-world construction. We also considered backgrounds built out of texture elements in Photoshop, but I was looking to add some extra pizazz to the design beyond a simple texture collage.   Whatever route we took, at the time we were given the go ahead- we only had 3 weeks to produce both 30 seconds spots from character design to final delivery; so we had to be very schedule conscious.

Character Designs by Ben Mazzotta

With Approved character designs as our touchstone, I started investigated adding 3D geometry and shading over flat textures.  As a Apple Motion user, the answer was obvious: mObject, the 3D text & object tool from one of the best 3rd party developer’s for Final Cut Pro X and Motion- MotionVFX.

mObject interface

Since it’s release in 2014, mObject has been updated several times with new features, UI enhancements, and overall performance improvements.  And while the 3D Text feature added to Final Cut Pro X 10.2 and Motion 5.2 last April has potentially "sherlocked" one aspect of mObject’s feature set- it’s ability to import and work with full, textured 3D models in either FCP X or Motion continues to make it a compelling purchase.

mObject can import .obj files with high polygon detail and high resolution texture maps.  But for my purposes I wouldn't be needing either of those things!  My plan was to build sets out of 3D objects, and then overlay the shadow information onto simple textures; bringing something extra to the backgrounds that at the same time wouldn't conflict with the approved character design.

One of the most important additions to mObject since it’s release has been the addition of Ambient Occlusion [AO].  This feature simulates the bounce lighting that occurs in the real world, and I remember distinctly when this became a “thing” in 3D animation working with Lightwave3D in the early 2000’s.  It was an immense render pig that created some very beautiful results even with simple models.  And now here we are with simulated Ambient Occlusion in Motion via mObject.  What a world!

I thought Ambient Occlusion would bring a nice feeling of real-world "miniature sets" to the backgrounds.

Ambient Occlusion off (L) and on (R)

For the purposes of this tutorial I’ll break down the first shot in the second of the two spots.  There’s a lot going on so it makes for a good case-study.

Sample storyboard.
Based on my storyboards, I constructed the basics of the room out of Primitives that come built into mObject.  This gave me a quick and simple way to roughly block out the room for the camera, without getting bogged down in a lot of the details.  Because the timeline was so short, it was important for me to be able to supply the animator with blocking so he could get drawing asap. In the end, I actually ended up using much of this basic architecture in the final scene, as these primitives are light-weight... and really, a wall is a wall!

Basic Geometry of hospital room created with Primitives in mObject

Next came finding scene specific furniture and decorations.  TurboSquid was my go-to source for all the models in the spot.  It’s not a free service, but if you’re getting paid for something like this, then a couple of bucks isn’t much to pay for good models considering the time you're saving.  I think the most I paid for any individual model was $30, with a total expenditure across the 2 spots of about $150.

Again, my goal here was finding high quality but simple models.  It almost would have been easier if I was wanting to do something MORE detailed as far as objects were concerned, since that’s what’s assumed most people are looking for.  “Simple” models can often mean low-polygon, and that can be seen in the final renders.  But in the end, TurboSquid’s selection was broad enough that I was able to find something for everything I needed.

One thing to note for anyone who’s planning on using downloadable models in mObject.  Even if the models are in the correct .obj format- it can be a real voodoo whether textures import properly; depending on what program the model was created in, and how it was converted.  Knowing the basics about a good 3D modelling program like the open-source Blender can be a real help here.  I didn’t need the textures, but it can be essential for cleaning up models and deleting elements you don’t need, since there’s no ability to alter model geometry within mObject itself.  After import into mObject, all the models were resurfaced with a plain white texture, with no reflectivity or specularity, since all I was really  looking for is form, and how it casts and receives shadows.

Here’s the scene with all of the final models in place in mObject.

Room with final models.

And here they are in the final shot.

Basic scene setup from Motion camera view.

At this point I provided final reference images to the animator, who could then move ahead while I worked on finalizing the scene.  The characters were created in Photoshop, and animated in After Effects, since that's the program he was most comfortable with.  Some animation for both spots was done in Motion- and technically there's no reason why it all couldn't have been.  Just timing.

Lighting mObject scenes can be done with standard lighting setups created within mObject, or using Motion’s own lights.  Important to note here that for these lights to work, they must be within the same Group as the mObject generator.  In this particular scene, I was simulating a time-lapse shot, so I needed to animate some key lights and overall ambience as the shot cycles from day to night and back.

Lighting setup with Ambient Occlusion turned on.

This leads us to the most complicated part of this project- the 2D texturing.  I’d built up a library of paper, wood, fabric, and subtle metal textures, and mocked up the final texture layout using my temp stills in Photoshop.  This way we could be sure that the background textures and colour pallet wouldn't clash with the character designs.

Originally, I had hoped to apply the textures  in mObject using a front projection map which, as the name implies, projects the textures from behind the camera onto the objects, so that the textures DON’T wrap around the models the way you'd normall want.  Unfortunately, projection maps aren’t amongst mObjects various surface wrap modes.  Drat!

This meant I had to create individual hold-out mattes for each of the object and textures in the scene.  Basically, this meant turning on and off the different objects, or specific surfaces on a given object, and using that as the Image Mask for the raw texture layers.  In a perfect world I could have actually used multiple mObject generators for the Image Mask (and this does work), but having duplicate mObject generators in your Project can cause real slowdowns… and crashing.  Oh, so much crashing….

So, once I was happy with the setup of the scene and any camera moves, I rendered still pngs or ProRes444s movies for each texture element.  This took a while for complex scenes with lots of textures… but hey!  Art!

If there was Camera movement in the shot, it was important to match the Z position of the  texture planes to match the objects properly in 3D space, so that textures wouldn’t “slide” relative to the holdouts. Below is a sample of the textures with the Image Masks applied.  You'll note I ended adding perspective to the floor texture; it's perspective made it look too weird when the texture was completely flat. 

Flat textures with Image Mask holdouts from mObject.

After the Image Mask renders, the final pass from mObject was the lighting pass with camera movement and all models turned on, so that you can get all the interactive shadows and Ambient Occlusion interactions in the scene.

This image or movie would then be reimported and overlaid over the texture layers using the Multiply transfer mode, thereby applying the 3 Dimensional shading to all the flat 3D textures.  This was the toughest shot in the second spot, so in the interest of time the push in was achieved with a simple scale shift in Final Cut.  If we'd had time, an actual 3D camera move like you see in other shots would have been my preference.

Shadow pass overlaid onto flat texture layers.

Characters were then added in, and if necessary some of the texture hold outs were used as Image Masks to put characters “between” layers of the background, for example in this scene behind the bed or out in the hallway.

Final Composite with Characters.

In this first shot, simulating a time-lapse, there's actually no "animation" per se, but lots and lots of still drawings cycled at a regular interval.  A high speed cloud or stars layer was added outside the window, and elements like the blinds, bed cover, and TV were animated to add some more movement to the room. The fast moving shadows on the floor and wall help to sell the effect too.  Finally,  bit of grading with Color Finale to show the shift to night and there you have it.  

We had fooled around with different effects to try and enhance the time-lapse effect, but the characters just ended up looking smeary and out of line with the overall aesthetic.  In the end, turning on Frame Blend in Final Cut Pro X's retime window with a very minor speed change gave us just a hint of a fade as the characters appeared, disappeared and moved around in the shot.

Each spot ending up being about 10 days of total production.  Here's the second one, Izaak Gets a Gaming System, in it's entirety.

I use mObject a lot for corporate work, putting clients websites on 3D models of Phones, tablets and computers.  For that alone it's made it's money back for me.  This was a fun exercise in blending 3D and 2D in an interesting way, and I honestly had no idea what the results would look like when I started.  Surprises are great... as long as the client likes them too!


Szymon Masiak said...

Amazing. Thanks for sharing! Good job!

Rodney Smith said...

I love seeing success stories like this. Nicely done.