Back into Unity

Experimenting with VRChats World SDK

About the World

I've been spending some of my time interacting with some of the VR development community online in places like VRChat. I've been wanted to get back into using Unity so I can compare workflows with Unreal, and this gave me a perfect excuse. I wanted to create a world that had some volume to it, but also make sure that the projects scope was still feasible.

With this in mind, I decide to make a modern apartment that had space to it, while also leaving spots for the finer details.

Optimization for VR

Because this was also going to be a VR based project, I needed to make sure I had clean modelling and optimized textures throughout. This project was going to be built for both PC and standalone VR.

This optimization was done through 3 main processes :

Creating clean topology for meshes in the scene.
Manually reducing texture scales.
Creating custom shaders for the Built in Render Pipeline.

1. Clean Topology

I modeled everything in this scene, with plans to update more in the future. I usually start with working from reference, and build low poly models to start with. Then working up into higher poly models that I can either carefully reduce or retopologize manually depending on the objects complexity.

The only object I didn't start with myself would be the Star Wars A-Wing that I found on Sketchfab from artist Daniel. I used his high poly model to retopo off of and then used it to generate textures in substance painter. Overall reducing his model from over 32k faces down to only 1k.

2. Texture Scaling

Texture scaling and compression are key for making sure the VR experience runs optimally. For VRChat, the world can be built for both PC and for Quest separately, which means that I could optimize assets when needed for mobile.

I made sure most textures would be within the 512 to 1024 resolution, while larger assets, like the floor, were held at 2-4K. In total only 3 larger texture assets were used, the first and second flooring, and the walls.

I also combined textures that would use similar shaders, like the foliage for example. In the world I have several plants like snake plants or monstera that share the same texture space.

3. Custom Shaders

The hardest part of this project for me was getting some of the custom shaders to work. Because I was using the VRChat SDK, I was limited to the render pipeline they have accessible. The render pipeline they use is the Built-In Render Pipeline, or BIRP.

This meant that I had to manually script the shaders in C# in order for them to work properly.

The Mirror Shader

This shader was actually the easiest, as VRChat already has a base mirror shader. The problem I was running into was that I wanted to have an opacity pass so that I could use the mirror like a realistic reflection without relying on a basic cube map reflection. The benefits to this would be a more accurate reflection of the scene and dynamic reflections of players. After tinkering with adding the properties and transparency variable into the shader, I actually used Chat GPT to help with the final tweaks.

Default Mirror

Transparent Mirror

The Building Shader

This shader was a process to make. I wanted to make a shader that uses cubemaps that fakes the interiors of buildings, much like how the shaders in recent games like Insomniac's Spider-man does. I went through several iterations of getting the shader to work. I first actually worked in Unreal because the shader graph is more intuitive for me, and the math made enough sense for me to dive into it. Then using the HDRP pipeline I was able to replicate it in Unity. But again, VRChat uses BIRP, so I had to manually script the math that before was in the visual node form. I got it to work with either world space or object space myself, but could not get component space.

After doing some more digging online, I found a base shader from SimCity that finally made the component space cubemap calculations work. This was a very simplistic fragment shader, which included the room math with some room rotation variation. I then worked on adding several feature to it so that I could use it more dynamically. This included features like the masking for facades, which uses a 2D base-color textures alpha to cutout the window for the room, or emissive controls like color, multiplier, and room randomness with a secondary emissive color. I want to do more work on the shader to implement PBR features like metal, rough, and normal in the future.

The best part of this shader though, is that it works both for PC and standalone quest, which was a big thing I wanted to make sure moving foward with the project.

Unity Moments

Throughout this process there were errors, breaks, and a lot of relearning different procedures. The last time I legitimately touch Unity was when I was back in highschool. In the 7 years since I've been using Unreal, which became my bread and butter. Jumping into Unity again was struggling at times, especially with the combination of the VRChat SDK. With that said, I liked having a challenge again with interacting and thinking in new ways to come up with creative solutions.

Sometimes there were breaks along the way with relatively simple fixes. For example, the image here shows what happens if you have the VRC SDK targeted to Android, while trying to bake specifically the lightmaps with directional ambient occulsion. Couldn't tell you why it decided to nuke the scene, but I can say a retarget to windows was the proper fix. All in all though, the bugs weren't too much to overcome.