Echoes of Somewhere

As the story is about water, I wanted to use some AI accelerated realtime fluid simulations.

Zibra AI, whose smoke & fire I used before, also provides a very fast AI accelerated fluid sim solution. As the first episode’s storyline also revolves around water, I wanted to take advantage of this cool tech!

https://zibra.ai/zibra-liquids-pro/

Creating & setting up the scene

As per usual, I started with prompting. I knew that I wanted a scene that had a container of some sort for water, and that the scene was from a top down angle. And the space was large. With AI, you really do not want to set any more requirements to a scene as you never know what you are going to get. I used Midjourney, testing the new v5.1 to see if it would give me good results.

I am really not liking the sharp 3D render vibes V5 & 5.1 are giving me. The more organic look from V4 is much better in my opinion. I am sure I could find a look that is a closer match to the V4 stuff that I have used before, but why bother when the older version of Midjourney is already perfect for me!

I chose the first image of the first batch of the V4 generated images. It had open play area, some cool equipment and a pool that I could try to make overflow with the realtime simulated fluids. I had no idea if that would even work, but it was worth a shot.

Then I did the usual steps. I opened the file in photoshop and painted the light away from the ground and loaded up the image in fSpy and reverse engineered the 3D camera parameters.

Read about the shadow shader.

Now with the prep out of the way, it was time to model the scene in Blender.

As the fluid simulation collisions happen against the scene geometry SDFs (signed distance field) the scene mesh needed to be watertight. Also, as I wanted the fluid to collisions to look accurate, I needed to model in way more detail than I otherwise have put in these scenes. I modelled way more than before. Also the fluids would go in all of the nooks and crannies and be obstructed by a lot of the details, I new I had to me more thorough than before.

I also had not tried the water sim collisions agains a mesh like this before, so I wanted to make sure it would work as well as possible.

In Unity, the scene is complex enough to allow for some camera 3D movement if I wanted to. But this time there would only be some simple shake. The complexity would only be used to occlude the water and allow it to collide with as many details as possible.

I duplicated my special shader version of the scene mesh (that has the custom shadow renderer and placed a simple default lit shader on it and changed the render to shadows only. This is to make the environment cast shadows on the player character when it is moving in the scene. It now also casts shadows on itself as well, but it seemed to work ok as long as I lined up the light with the shadows that were in the AI image.

The fluid rendering also requires a reflection probe, so I placed on in the scene and created an otherwise invisible mesh contraption that fills the probe with something for it to reflect.

Setting up the fluid

To control the fluid, I first created the SDF from the scene geometry. This would be the base the water collides against. Zibra AI tools contain scripts that allow you to generate an SDF from either rigid, or skinned geometry. This allows their fluids or smokes to collide with complex, even animated geometry. You do not need to worry about colliders or anything really. Just make sure the mesh is watertight (it does not have holes).

I was lucky with my scene and the SDFs just worked out of the box! The water has pretty nice collisions with the scene mesh. Everything in the scene is stored in the same mesh, there are no colliders, just polygonal geometry. Unreal Engine uses SDFs extensively and they allow so many advanced features. I wish that Unity supported SDFs out of the box like that as well and they were easily accessible for the shader- and the VFX graphs.

Now that the scene is all set up, it was time to take a look at the water.

The water is rendered using a custom fluid render scriptable pipeline pass by Zibra. there are plenty of options to tweak the look. When I was tweaking the options, I found out that most natural look comes when the index of refraction was set unrealistically low and the scattering to 0 and the absorption to some low value. With a dark color, this made the water look pretty natural to me. You can also render the water as a normal mesh with any custom material you like, but I found out that this made the performance dramatically worse.

I did not time myself too carefully, but the full creation of this scene from scratch with AI imagery, modeling the scene, scripting, C#, fluid setup and unity scene building took about one full work day. Naturally in small chunks in the afternoons.

Now that I have gotten familiar with the fluid systems, I want to design more puzzles that use dynamic water!