2018 UPDATE – This tutorial was written in 2000, and is very outdated now. I’ve left it online for historical interest!. I’m currently working on a vastly more detailed and comprehensive VFX Supervision Online learning course. I’d love to share it with you when I launch in a few months. Join my mailing list and I’ll drop you a line when it’s ready. Don’t worry there’ll be no Spam.
This is a very simple tutorial demonstrating how to take 2D video footage and composite a 3D animated object in the scene. I’m not going to go into every specific MAX feature and option used, Instead I’ll be concentrating more on the actual process and flow of work and files.
There is a separate document that should be viewed before this one which deals with shooting video for compositing purposes which can be found here
As we have discussed earlier when working with effect we want to keep file quality as high as possible, therefore JPEG’s are not the best medium for image storage. In this tutorial we’ll be using TIFF files or TARGA files when we require an alpha channel. Take this TIFF sequence and store it for later.
Unless you’re using a very high end machine you’re not going to want to be dealing with large amounts of files whilst compositing, so take one frame of your sequence and keep it handy. Store the rest of the files somewhere else for later. Keep one frame as a proxy. Make sure you keep your files and directories neat, compositing can be a logistical nightmare with the amount of files you’ll be dealing with. A bit of planning and neat directory structuring can save you a lot of headaches later on down the line.
I’m not going to talk much about your actual animation. I’ll assume you have something simple like a rolling ball or the little car i’m using in my demo. Remember that your camera cannot move in this example (camera matching is supported in MAX, but I’ll not touch on that for this demo, we’ll be working with a simple locked off camera).
So now you have a short animation and a video sequence. The next step is to make the camera in your animation line up with the camera you used for your video shoot.
Now is the time all those measurements and notes you took onset will come in handy .. assuming you did actually make all those notes on set. If not then it’s a very long slow process of setting the viewing angle and the zoom of the camera and shifting it’s orientation until it matches.
To speed this process up it would help if you’d shot a few frames of a real object in your scene, and then keep this for reference along with the dimensions of the object. Then when you come to MAX you can load a frame of this into your background and create the object using it’s exact measurements. Now you can be sure your scale of your 3D objects in relation to the background footage and positioning of the virtual camera will be correct.
I used a video case, took the shot into MAX, created a video box in MAX and then lined them both up perfectly, this way i could be sure where the ground was in my MAX scene, and that the animation would align correctly with the footage.
As you can see above it can take a lot of work to line the objects up correctly, and you may require a few reference objects to ensure the camera is perfectly matched.
Once you’re sure you have your camera matched, freeze it in MAX to avoid accidentally shifting it whilst you continue to work on the scene.
Now you can refer to all those lighting notes you took on set. I tend to use a little cheat and place some white ping pong balls in the scene and shoot a few frames, then when i come back to the computer, i can create some balls in MAX and get the lighting and positioning to match perfectly.
Again this will take time, lighting is an art form. It’s important not to rush this stage as the quality of the lighting will decide whether or not an effects shot works.
Play close attention to shadows and the colour of the light in the scene, also be aware that unlike in computer animation, light in the real world bounces around, so if an object is placed on a white surface light will bounce from the white surface onto the underside of the object. You will have to try to simulate all this with your computer lighting, which can be a very hard task.
The best advice to matching lighting is good observation and note taking during the shoot.
There are many visual clues our eyes need to understand the relationship between objects in an image, but one of the most important is shadows. Shadows help us understand where the light is coming from, how many light sources there are and the intensity of each.
In this example the intensity and blur of shadows will be achieved in post production so I’ve used ray traced shadows to give me perfect black shadows with a hard outline.
Assuming your lights are set up correctly your shadows should work without much alteration, we simply have to place proxy objects in our scene to receive the shadows. Proxy objects will be placed wherever there is an object in our shot that will receive a shadow from our CG object. If you look at the example below you can see that I’ve had to recreate the CPU, the desk, the wall and the cup to receive shadows from my car.
To allow me to control the attributes of each shadow individually in post production I’m going to render each shadow out separately. In my example there are three shadows, a downward shadow beneath objects, and then a long shadow cast behind. I’ve split the render of the long shadow into two passes, one for the vertical objects such as the cup and the CPU, and one for the horizontal objects such as the desk. This is because the horizontal shadows are much lighter than those cast on vertical objects and I’d like the ability to control this in post production.
So now we want to render out out shadows for the whole sequence. MAX comes with a material called ‘shadow/matte’ that will only render shadows cast on the object this is ideal for our purposes. We can apply this material to the objects in the scene such as my proxy CPU, desk, wall and cup.
Now if we set the environment background to white and render out a shot you’ll see you only get the object and the shadows it casts.
The problem is we still have the 3D object in the shot, which we really don’t want when we come to post production. When using ray traced shadows MAX won’t allow you to see shadows without the object being visible, so we’ll use a plug in texture called ‘cast shadows only’ which is available from the nice people at : http://www.blur.com/BlurHome/tech/
This material can now be applied to the object (in my example the car) so that when we render we only see shadows. Now you’ll have to play with the shadow cast/receive options for your proxy objects as obviously we don’t want them to cast shadows on each other.
Now we can render our shadows out one by one. In my case this involved rendering out 300 frames for each shadow, each time turning lights on or off and altering the cast/receive shadows options for various objects.
As you can see above you now have three sets of shadows. Take these frames and store them for later, once again keeping a proxy of each for compositing.
You’ll notice in my example that the monitor should cast a shadow on the car as it emerges from between it and the CPU. This is simply achieved by creating a proxy monitor, and assigning our cast shadows only material to it. Not we are ready to perform the final render of the car element. These frames should be saved as TARGA images as the Alpha Channel will come in useful when we perform the composite.
Once again I’m not going to go into any real depth on the software side on compositing all this together. For this example I used Aftereffects.
You may have been wondering why I reminded you to keep a copy of one frame from each render pass, locally on your machine. This saves us a lot of time and space during the compositing process, you can take say frame 75 from your various background, Cg and shadow layers. you can then use this frame to match the video noise and shadow intensity, then use AfterEffects’ ‘replace footage’ feature to replace these proxy frames with your entire footage.
The process of colour matching, levels, contrast, blur and video noise can be extremely complex but nothing beats a keen eye. As a compositor you should observe closely the footage and make sure you use the tools available to you to match your computer generated footage.
In my opinion you should always add an small amount of noise and blur to CG images, to avoid that hard edged look we always end up with. In compositing blur and noise are your best friends.
All video footage will end up looking pretty poor quality when you zoom in, however your CG elements will always look crystal clear. You should take time to blur and add noise to your CG element so that it matches the background.
You can see from the two images above that the CG element is far cleaner and sharper than a close up of the cup in the background. If left like this the image will look very fake with the CG element will be very obvious to the viewer
Even though you attempted to match the lighting in MAX, you’ll still have to work hard to match the contrast, brightness and colour of the CG element and the background. This is a highly skilled task and takes a good eye for colour, the best advice I can offer you is to keep getting other’s opinions and to keep taking a break then coming back to the image to see if you can see any new visual glitches.
There are a lot of expensive high end packages available to help match two images together, but nothing beats a good eye and sense of composition.
If you’re still having trouble, try turning your head upside down (don’t laugh) and looking at the image, this will help you see the composition with a fresh perspective and to find any errors you missed.
Now comes the really boring part, you’ll have to take a copy of your background plate, and place it at the top layer. Now draw a mask around any foreground elements that may clip your CG element (in my example this would be the monitor). If your foreground elements are moving or very detailed, then oh dear !, you have no choice but to rotoscope the whole sequence, masking out the foreground objects one by one.
Now you can place this foreground element back over your scene as in the image below.
Congratulations !, you’re nearly there. Now you just need to take your 3 shadow elements and take them through the process outlined above to blur, add noise and alter their opacity correctly.
The image below shows my finished frame.
It’s not perfect, the shadows aren’t perfectly aligned, the cup handle doesn’t receive any shadows and I’ve yet to add motion blur. In a perfect world I would have liked to make a better job of the reflections on the car’s body, but this would have pushed render time too high with my rather inefficient mesh.
>