Sunday 23rd Mar 2014

This is Part 6 of my tree building tutorial. you can access the other parts from the bottom of the page.

The plan

I know there are a number of methods to attach leaves to trees. I experimented a little with the particle system approach but found I spent so much time fiddling with all the values to get something workable that it just wasn’t intuitive enough. I also experimented with Object Paint found in the graphite tools. This seeemed a much more controllable and artistic process that would allow me quickly place lots of tertairy branches onto the tree exactly where I wanted them.

I decided to build up a couple of simple units or groups of models that allowed me complete control and the ability to continually edit the tree or perhaps randomise the effect so one tree could become many.

The steps are as follows:

  1. Create a number of leaf textures in Photoshop. I created 4 diffuse maps all with alpha channels for the shapes of the leaves.
  2. Create a rough model of a cluster of leaves and use material IDs to randomise which leaf appeared where.
  3. Then I’d make a copy of this model and colapse it down to a single mesh.
  4. This cluster of leaves could then be baked down to a single plane so I now have 3 models, a super high res with each leaf as a separte model, a high res with everything attached and then a low res with this model projected on as a texture. As long as each model shared the same pivot point and all the transforms are reset I knew that I could write a simple script to quickly replace or randomise leaf clusters once they’re placed on a branch.
  5. I could then do the exact same thing with tertiary branches, scatter leaves over the branch model, copy this group and attach everything together and then create a low poly version and project this branch onto the plane. These tertiary branches can then be painted onto the tree mesh and again I have the flexibility to swap and randomise them later on.

Perhaps this illustration will make this a little clearer.

[insert illustration here]

Creating leaf clusters

I looked at more photographs and illustrations of how beech tree leaves are arranged and decided to work on clusters of leaves connected to branches.

PSM V27 D366 Branch and leaf arrangements of beech and elm trees

To get started create yourself a series of leaf textures with an alpha channel, I based mine on some photographic textures from cgtextures.com. In max create a leaf model by creating a plane with 2 subdivisions, place the pivot point at one end of the leaf and now use either a spline or a cylinder to create a short stem that your leaves will connect to.


Place the leaf so the pivot sits at the end of this stem and make a few copies by rotating them around till you have about 5 or 6 leaves coming from this single stem. Add a bend modifier to each leaf and arrange them so the stem lifts upwards a little but and the leaves hang down. Use the illustration below as a guide.


Once you’re happy with the layout of the leaves copy all the objects and then attach all the meshes to the stem so you have a single model. Ensure the tip of the stem is sat at zero and then use Tools->Reset X-Form to ensure all the transforms are zero’d out. Collapse the stack and then name this object something like ‘LeafCluster_A_high_000’. Hide the separate leaf objects and put them in a layer in case you need to recreate this model.

We’re not going to create a low resolution leaf cluster, it will be a render of the model you’ve just made mapped to a plane. So create a new plane that fits exactly over this leaf cluster, give it two segments and put the pivot exactly at zero. Now pull the verts around until the plane roughly matches the shape of your high resolution leaf cluster.

If we had modeled each leaf instead of using simple planes with opacity maps we could use the Render to Texture feature in 3DS Max, however Render to Texture doesn’t work with opacity maps and so we’ll need to manually project the high resolution leaf cluster onto the low res.

To do this create a camera and place it directly over the high resolution leaf model, set the camera to be orthographic so we don’t get any perspective distortion and then use zoom to frame just the leaf cluster. To give you an idea of what’s going to be rendered set your render resolution to something square such as 1024 by 1024 and turn on Show safe frames in your viewport. Now place a skylight into your scene and ensure this is the only light currently active. This skylight will give you nice clean flat lighting for the baking if you want to a sense of occlusion between the leaves turn Cast shadows on for the skylight. Ensure only your high resolution object is visible and render the leaf cluster. You should end up with something like this.


Save your render out as a TIFF or another format that allows you to save the alpha channel and map it back onto your low resolution plane. Place a UVW map on the plane and roughly scale and move it around until the texture sits in almost the same place as the actual leaf cluster. The most important element is the stem, this must be in exactly the same place on both your high and low resolution models. If you wanted to you could also render out an image for the bottom of the leafs but I created mine in Photoshop with a small amount of editing and changing the levels so the backside of the leaves are less saturated.

Make sure you reset the x-form on this new low resolution mesh before moving it away from zero, it needs exactly the same transforms as the high resolution if the swapping procedure is going to work. Name this new object ‘LeafCluster_A_low_000’ and put it into a new layer along with your high resolution.


Creating the branches

I’ve written a very simple Maxscript to help with the randomising and replacing of all the leaf and branch objects. You can download the script from here and load it into the Maxscript editor ready to run in a moment.

To create the branches we’ll use another spline or cylinder, it can be super low resolution so go turn the sides down to 3 and block out a rough branch shape. Create another fork and then convert both lines to be editable polys and attach them together. Ensure once again that the start of the branch is at zero, reset the xform and collapse the stack.

We’ll now use Object Paint to place the leaf clusters onto the branch. Select the branch and open up Object Paint found in the graphite tools. To get started pick your source object (in this case your low resolution leaf cluster). Then in the Paint on options choose Selected Objects.


Select the branch and start carefully either painting or tapping on the branch to place clusters of leaves, you might need to tweak the spacing settings to avoid having thousands of leaf clusters. You can also tweak the random scale and orientation controls, I found that keeping things fairly even and then going back and and rotating the clusters around by hand worked best. Again remember to fill up the 3D space, you want the branch to look natural but you also want it to avoid looking completely flat.

Once you’re done copy everything and move them to one side. We can now use the Maxscript tool to replace all the low resolution leaf clusters with high resolution versions. Save your Max in case something goes wrong and run the Maxscript from the editor and a small window will pop up. Press the Add button to add a source mesh, pick your high resolution leaf cluster. Now select all the low resolution clusters on your branch (ensuring that you only have the clusters selected and not the branch). Press Process and all your leaf clusters should magically become the high resolution versions.


Finally copy both groups of objects, both stems and all the leaf clusters and shift them up in space or press Alt-Q to isolate them. With one of the stems selected attach all the clusters so you’re left with a single object and do the same with the other branch. I tend to create a layer called Branch_A_setup and through all the parts into those layers and hide them. Then create another layer called Branch_A and put your two new branches in there and name one Branch_A_low_00 and the other Branch_B_high. Put them both back to zero and reset the xforms on both just to be certain.

Repeat this process a few times to create a few different branches, I created 4 different branches to give some variation to the tree.

The video below shows the process of creating a branch and swapping out the leaf clusters.

Placing the branches

Placing the branches on the tree is the same process as placing the leaf clusters onto the branches. Select the tree, go to Object Paint and and rather than having the leaf clusters as the source object pick one of the low resolution branches. The video below shows this process.

Randomising and experimenting with different resolution branches

The Meshswapper tool allows you to supply a number of source meshes, it will randomly swap your target meshes out, so you can instantly add more randomness to your tree by specifying a few different branch types. You can also use the tool to swap your resolutions out. I also experimented rendering one of my entire branches out to an image on a plane. I placed the pivot point at the base of the branch and ran the Meshswapper tool again to replace all my branches with planes. With the level 1 tree this amounts to around 7000 polygons, my mid resolution (with the simple leaf clusters) comes in at about 40,000 polygons whilst the high resolution with the level 3 sculpted trunk and detailed leaf clusters amounted to over 450,000 polygons.

You can see the three renders below.


Wrapping up

These poly counts are very high, especially for the low resolution mesh. I’ll try some experiments trimming them back and retopologising the tree and see how low I can get the model but still get a decent result. I’ll post the results back here in the future.

Whilst the level 1 mesh that I exported from Mudbox  looked fairly good. For my demo render I cheated a little and tried the level 3 mesh to project onto as it had a more pleasing form than level 0 or 1. This was just my laziness as I didn’t have time to build a new mesh to bake bake onto.

If I was to spend more time on this I’d want to create new re-topologised model to bake onto. This is because much of the detail I need from the tree is in it’s silhouette, the various small bumps and details on the trunk are way more important to the form than the polygons up in the branches. So taking some time placing the polygons exactly where I need them would be time well spent. Tools such as Topogun, Max or even using the new re-topology tools found in Mudbox 2014 would help this process.

Another option to create a better result would be to extract a displacement map from Mudbox and then use this to displace the level 1 mesh into a more pleasing form that matches your original sculpt.

Hopefully this has spurred you on to try your own techniques and ideas. I’d love to hear your approaches to tree modelling – feel free to leave a comment.

Jump to other sections:


Part 1 – Polygon modelling

Part 2 – Mudbox sculpting

Part 3 – UV unwrapping

Part 4 – Texturing in Mudbox

Part 5 – Baking to a low resolution mesh

Part 6 – Adding leaves and finishing up