As a fun little prototype, I wanted to see how much work would it be to use AI character design and graphics as much as possible for a 2.5D point and click style game.

Final in-game capture

I figured the backgrounds would be a given, as the game would pretty much be 2D, but the characters were more complicated. I would need to have a lightning fast pipeline, a pipeline that allows me to use mocap data so creating animations would not be a bottle neck. I needed a way to translate AI generated art into form that can accept motion capture.

The best way seemed to be to generate character concepts with AI and model them into 3D.

I have been working in the games industry for 20+ years, modeling characters and backgrounds for a large part of it. So it would be interesting to see how AI would enable me to work faster. And by how much?

I planned to use the characters in wide shots only, so the possible low quality or messed up details would not matter as much, but the AI tech has come a long way since and the coherency of the art is way better than it used to be mere months ago.

Demo download

Project presentation

My presentation of this project after working on it for almost a year.

Prompt building

I started by ordering the AI (Midjourney in this proto, but I use stable diffusion more) to make me an AI character design model sheet with turnaround images of a character. Similar to one you would have at an animation studio. That would be a good starting point for a thing like this.

Midjourney generated AI character design reference turnaround sheets
cyberpunk point and click adventure game character model sheet turnaround –v 4 –ar 3:2

The first result was very promising. Naturally it was not good for me as it was in black and white, but model sheets usually are. I try to not use any reference to any living (or dead) artists. But with AI it is impossible to not get influences from a bunch of artists. This way I just wish the waters are more muddled and there is no direct reference to anyone specific.

Midjourney generated AI character design reference turnaround sheets
cyberpunk point and click adventure game character model sheet turnaround full color –v 4 –ar 3:2

The second try was not great either. But it was a step in the right direction.

After some tweaking I had a prompt that produced art that was usable for me to use as a base for the 3D step:

The images had surprisingly good coherence for the different angles. These are very usable for modeling and texture projection. There is one major issue though: the images have perspective shift and are often from odd angles. But I had some ideas on how to overcome these issues.

As these images are created by an AI, no one has a claim to their copyright. That is the “contract” you sign when working with AI at the moment. They can be freely used by anyone, as replication of these images is pretty trivial. At least this is how I feel about AI art.


I chose the AI character design that had a good front and side view, but a challenging back view. Just to get to solve that problem and see how it affects things. is it even possible to extract textures for the final mode from this?

I started by using the side and the front view to get a first draft of the character going in Modo. Once the character was done, it was pretty clear to see that the different texture projections did not match properly on the mesh.

To fix this, I created morph maps for each projection, making sure the front, side and back thirds view matched on the mesh as well as possible.

Next I needed to do an old school UV unwrap on the mesh. It is time consuming and tedious, but had to be done properly so blending the textures together in photoshop would be easier to do. After the UV was done it was time to project all the different images on the UV using their respective morph maps.

The finished model

The model is now finished and ready for rigging, for this I used Mixamo. In the past I have not used any auto-rigging tools, but in the spirit of efficiency I wanted to see if there would be a way to make this process as quick as possible. Mixamo did a pretty good job at rigging the character, but I needed to do some touch up on it. Especially the beard.

Background image

The background image was a lot simpler, as it would be used as a 2D image, not a 3D model.

Midjourney generated location for a 2.5D adventure game prototype
cyberpunk point and click adventure game screenshot marketplace exterior –v 4 –ar 3:2

Even though the image is 2D, it requires some work to merge the 3D character in it beautifully. First, we need to get the camera data from the image. I used a free tool called fSpy to reverse-engineer the camera.

Midjourney generated location for a 2.5D adventure game prototype - spy
The image in fSpy

fSpy has an import add-on for Blender, so it was very easy to get the image and camera to blender for modeling a rough representation of the location.

The 3D version of the location is required for the perspective shift on the character to match, and to allow the character to pass behind objects. It is also useful for setting up the lights in 3D space and to get shadows from the location on the character.

Blender also has a great tool for generating UVs from camera projection, so it was the perfect tool for this step, although I needed to learn everything from scratch as the software is very alien to me.

Putting it all together

Now that I have the all the pieces of the puzzle, it was just a matter of putting them together. I used Unity for this. I created a simple animator for the character locomotion, scaled the 3D location to match the character and set up some lights to match the scene lighting so the character would not stand out so much.


I spent 18 hours working on the 3D assets and unity scripting for this prototype. 12hours on the AI character design/model/uv/morph/textures/rigging – 3 hours on the location camera & mesh – and the rest on game scripting – not including any of the AI prompting.

On minimum the AI saved me 2 days of work on the character and 3 on the location. As per my estimation.

The end results are not perfect by any means. Closeups of the character are not ideal. The location is hard to art direct and can be very random. But if you accept the AI’s shortcomings and work around them, creating a full game using AI as a co-worker is perfectly possible. It enables the creation of games that would otherwise be left unmade because of budget or time constraints for sure!

13 responses to “AI assisted graphics: Character modeling”

  1. Avatar

    Hi – thanks for posting your work, which is absolutely mind boggling!

    I’ve also been an artist for 20+ years and I find this whole AI stuff really fascinating – not so much as an end result tool, but more of a starting point for further design refinement.

    I am really interested in how you made those character sheets – what’s the process behind it, what are the AI inputs, which AI have you used and how…the results are absolutely mind boggling – how the AI maintains the style throughout the different characters, from different views. Bonkers!

    Looking forward to seeing more from you!

  2. Avatar

    This is so inspiring and super work! ๐Ÿ˜€ Nice Job! ๐Ÿ˜€ Keep it in follow! ๐Ÿ˜€

  3. Avatar

    this is VERY inspiring. keep up the good work, looks amazing!

  4. Avatar
    Mika Sarfati

    Thank you so much for sharing your pipeline of AI assisted production!
    It would definitly be a huge asset for prototiping lookdev and aestethic in game or animation.

    Very Enlightning!!

  5. Avatar

    Hi! I’m not sure if its an issue on my end but the pictures aren’t showing up for some reason. When I click on them it just says “Placeholder” and the caption underneath it. Thank you for breaking down your workflow!

    1. Avatar

      So I wanted to add to my previous comment. I just checked the website on my phone using chrome… and it looks like it works. So its definitely something on my end.(I should’ve checked that beforehand.. Sorry!)

      1. Jussi Kemppainen

        Oh no, I have the same problem on my laptop. I will look into my image compression plugin.

        1. Avatar

          Oh nice thank you! Everythings working great now.

          1. Jussi Kemppainen

            Thanks for reporting the issue!

  6. Avatar

    Could you tell me the details of the procedure for “Creating a projection morph map”?
    I also read articles about modeling robot characters, but I didn’t understand.

  7. Avatar

    Hey really nice work! Can you share the prompting for the turnaround design of characters? Thank you so much!

    1. Jussi Kemppainen

      Way ahead of you! You can find all the prompts in the image subtitles

  8. Avatar

    it’s fascinating to see how technology is shaping the way we approach the creation of video game character sheets. Do you think traditional methods of character sheet design will eventually become obsolete with the rise of AI in this field?

Leave a Reply

Your email address will not be published. Required fields are marked *

Recent Posts


Social Links