Tool Design with Python

Image 1: A view of the inside of a Daursynka long house, in-game. There are some light artifacts that can be seen in the upper-left of the image. Hopefully, these can be overcome by tweaking the cascading shadow settings. Overall, however, we are very happy with the end result.

The Daursynka long house is done, and we are pretty happy with it (see the featured image for this post, and Image 1). There are many variations for each piece in the kit, to reduce the ’tiling’ effect that can be seen in some games. For example, there are twelve different variations of the main roof’s tile sections, to reduce the repetitive use of each piece. The human eye spots patterns fairly well, and repeating patterns are easiest to pick up on when there are many examples of the same pattern right next to one another. So, this approach was used for all of the pieces in the kit. However, this meant we ended up with over 460 pieces for this kit alone!

When looking at the prospects of exporting each piece, along with it’s collision geometry, it became clear very quickly that scripting the export process was going to be mandatory. When we say scripting in Blender, that means Python. I must admit, I have never been (nor will I ever be) a huge fan of Python. I don’t like the language as a whole, and so I didn’t have much experience with it. That meant first learning Python beyond a quick overview of the languages features. Even if the only scripts that you were ever going to write in Blender were small, relatively simple ‘helper’ scripts, the effort would absolutely be worth it. Especially while creating something as large as an entire modular kit.

After going beyond the introductory lessons on Python at W3Schools, I turned my attentions to writing a script to help with the export. It became obvious very quickly that this script was going to be much more than just a simple little script. I wanted to keep my collection hierarchy in Blender and use it as the folder structure on the hard drive when exporting the assets. This meant a recursive function. I want to be completely honest here. In all of the time that I have ever done programming of any kind, I have never had to write a recursive function. So this was a first for me, and it was an…experience.

Recursive functions are usually deceptively short, and therefore they are deceptively simple. But, at first, I found it very difficult to get my head wrapped around exactly how the function was going to work. I had to look at it in a completely new way (for me), which was so different to anything I had done before. The hardest part for me was thinking about how the function was going to ‘walk’ through the various collections, and the order that it was going to run in. I tried to visualize how the function would be called from the very top level collection, and how it would call itself when it discovered a collection within that top level one. I did figure it out, but I can’t say that I will ever be comfortable writing recursive functions.

Once I had a script that would export my objects, I realized that I had only solved half of the problem. I would still have to import those assets into the Unreal Engine, so I was still staring down an enormous amount of work. Thankfully, the Unreal Engine editor can also be scripted with Python, so I knew that I could write a script to handle the imports. I needed a way to have the editor import not only the 3D objects themselves, but also to create the materials for those objects first. It would do no good to import 460+ objects and then have to apply materials to each and every one. This led down a rabbit-hole that resulted in the project import and export managers pictured below.

Image 2: The import manager in the Unreal Engine editor, showing all of the data for the long house modular kit. This is a read-only UI and is only used to verify that all of the data is correct. There are mouse-over popups that show that asset’s JSON entry, allowing for a deeper dive into the data, if the user wishes.
Image 3: The export manager in Blender allows the user to set up the export data needed to successfully export the assets and generate the JSON file needed by the import manager. Even in a project with 460+ assets, setting up the export data only took around an hour and a half at most.

This tool (or tools if you want to count them separately) is why it took so long for a new post to be added to the website. These were a real trial to get to work, but I feel that the time spent will more than pay for itself when we are making other modular kits for ‘At the Crossroads’ as well as any other games that we create. All of the data entered into the export manager is saved in the .blend file that contains the modular kit assets. So, while it does take some time to enter that information, it only needs to be entered once.

When the data entry is complete, pressing the ‘OK’ button starts the export process. The tool uses the data entered for the materials and textures to generate part of a JSON file describing where these assets are on disk, as well as where they should be saved in the UE project when they are imported. It then ‘walks’ it’s way through the collections contained within the ‘Kit Root Collection’ discovering 3D assets as it goes. As it exports each 3D object, it finds that asset’s collision geometry and exports it along with the asset. So, when the import manager within the UE editor imports each 3D asset, it will have the correct collision. All of the 3D-asset-specific information is added to the JSON file as each asset is exported.

The import manager reads the JSON file and imports the textures, materials, master materials, and instanced materials. I say that it imports the materials, but it is more accurate to say that these are created using the data generated by the export manager and saved in the JSON file. The master materials that are defined in the export manager are created, and then all of the instanced materials are inherited from these. Once this is all complete, the 3D assets can be imported and have all of the correct materials and/or instanced materials applied. The whole import process took just under five hours to complete. Imagine how long this would take to do all of these imports individually. This will be a huge time saver in the long run, as there is very little work that needs to be done from that point.

After the import is complete, the only thing that is left to do is to define the actual node networks for the base materials and the master materials. This has to be done by hand due to the fact that, while it is possible to map some of the material nodes from Blender to the UE editor, it would be extremely difficult to do. Some nodes in Blender, like the math node, do have equivalents in UE, but there are others that do not map across the two applications at all. Also, when you get into the more complex materials in either program, trying to make these base node mappings work gets even more complicated. It was decided that it would be best to just define the node networks in the UE editor by hand. It doesn’t take that much time to do, and we get to use our preferred workflow in Unreal without having to worry about how it all has to be created in Blender to make the translation process successful.

After all of this, were these tools worth it? In the short-term, no. I could have exported all of the assets from Blender by hand, and then import them into the Unreal Engine in less time than was required to write these tools. In the long-term though, these tools will pay for themselves many times over. While the export manager does force a specific work-flow on the artist, it is not that much of a constraint. And, the time saved overall will make these tools valuable assets for us going forward.

Thank you for taking the time to read this, and I hope it has sparked some ideas that you may have for tools to improve your work-flow. Have a great day.

Leave a Reply

Your email address will not be published. Required fields are marked *