A big part of the Skylines: City Generator project was being able to render hundreds of different cyberpunk and futuristic building models in realtime in TouchDesigner. Since I had made the early decision to purchase asset packs for the buildings and use Blender to clean and prepare them before I imported them into TouchDesigner, I needed to come up with a way to optimise the asset loading process on project startup to be able to monitor the process and avoid any unnecessary crashes and also to allow for easy loading of modified or new assets.
Asset nomenclature
Before I could get started in TouchDesigner, I had to come up with a strict naming convention for all the assets that I was exporting in the USD file format from Blender. This would allow me to automate the entire asset loading process for the 80+ models which involved making a few changes to the model, assigning the textures to the material and most importantly assigning the instancing data to the model.
In Blender, I decided to prefix each asset with the name ‘building’ and then used two sets of numbers to identify which group the building belonged to. For example, if a building was named ‘building_02_03’, it was the third building in the second group. I also decided to start the numbering from largest to smallest since that was the order in which the buildings were allocated, therefore models with the prefix ‘building_01’ were the largest and those with the prefix ‘building_11’ were the smallest. I created one folder for each building group and a sub folder for each building model and its associated textures.
Once the model is exported as a USD from Blender, we get a .usd file containing the mesh and material and a folder containing the textures. To ensure that these USD assets would load correctly in TouchDesigner, I had to update the name of not only the exported file, but also the base mesh and the material. I also had to update the names for the multiple textures used in the PBR material to hold the asset name as a prefix for the file.
Time is of the essence
The loading process in TouchDesigner is run using the .create() python command. At the start of the loading process, I loop through the geometryCOMPs that will hold the usdCOMPs to ensure that they are empty and destroy any and all old components before we can start creating new operators for the assets. Once we have empty folders, we can create a usdCOMP for every .usd file in our exports folder.
I am using a timerCHOP with a small length, but with the cycle parameter set to true to iterate through the contents of the subgroups in the exports folder at short intervals. The maximum cycle limit parameter is set to 21 when the assets are being loaded which gives us 22 total cycles, 11 of which are first used to destroy older components and then 11 are used to create new components. To decide whether we are clearing the operators or creating new operators, I divide the cycle value from the timerCHOP by 11 and then subtract that from 1 to calculate the current status of the process which will be 1 for the first 11 cycles and 0 for the second 11 cycles. I also used a modulo operator by 11 to get the current value in a 0 to 10 range to be able to access the appropriate folder from the folderDAT.
I am using a pair of folderDATs to access the contents of the multiple subgroups in the exports folder. The first folderDAT is pointing at the exports folder and we use a selectDAT with the re-ranged cycle value to isolate a single subgroup folder and its path which is used in the second folderDAT to look at the contents of the subgroup folder. As the cycle value changes, the contents of the second folderDAT are updated and I use a datexecuteDAT with the onTableChange callback to create new operators using the contents of the folderDAT. The contents of the datexecuteDAT are shown below.
To be able to automate the setup process once the usdCOMP had been created, I attached a common callback script to all the operators which is called when the .usd file is imported. The setup process starts by creating and connecting a few operators to the importselectSOP that is inside the usdCOMP. I created a polyreduceSOP and an attributecreateSOP to first reduce the number of polygons in the asset to help improve performance and then to calculate the tangents of this newly reduced SOP. We can then assign the instancing data to the geometryCOMP holding the importselectSOP and these newly created operators. I then reset the scale of the imported model to 1 to ensure that it fits in our scene. Finally, I assigned the textures to the pbrMAT in the materials component and made small adjustments to get the look that we desire. The onImport callback script for the usdCOMPs is as follows.
I packaged this entire process up to show a loading bar on the screen when the assets are being loaded and unloaded. The process for unloading is executed before the project is quit from the interface and is very similar to the loading process. The maximum cycle limit is set to 10, which gives me 11 cycles and only the first phase of the process is executed and then the project is saved and quit.
Locked and loaded
Through the process of destroying and creating operators, I was able to automate the asset loading and setup process and save a lot of time that would have taken me to manually import and make changes to the 80+ cyberpunk building models that I was using in the Skylines: City Generator project. Even though I had discovered this powerful method in TouchDesigner, I had to be careful in its implementation since the operator creation and destruction are extremely heavy operations and might lead to project crashes or slower performance.