Character / setting continuity in generative AI images

January 7, 2024 8:38 AM

I've started playing with DiffusionBee on my Mac and enjoying it (thanks gregr!). But one limitation is that each time I run a prompt, it generates a new person, new setting, etc. Are there AI image generation tools that let me develop characters & places, and then use them in different prompts?

For example, with DiffusionBee I might say "boy sitting on the ground looking up at the stars". Every time I do that it's a different boy, in a different location. I can put words in prompts to generate specific styles and moods, but it always generates the person and the scene from scratch.

What I'd like to do is develop a person, and show them in different settings, or in the same setting doing different things.

The ultimate goal would be something that would, for example, do the drawing of a graphic novel if I put in a prompt for each square. I don't think generative AI is quite up to that yet, but I'm wondering if there's anything accessible on the path to that point.

I would prefer to run something on my Macintosh (M2, 24GB RAM), but could also do something in the cloud. I'm willing to pay a subscription fee for the right tool (say $10/month).

Does anything exist that will let me do this?

posted by Winnie the Proust to Media & Arts (4 answers total)

If you want to create images of the same character in different poses (and avoid the ethical pitfalls of generative image production) I endorse Hero Forge.
posted by Faint of Butt at 8:42 AM on January 7 [2 favorites]

The terms you should search for are "textual inversion" for creating a repeatable model of a person, and then ControlNet for having more control over how they are rendered. ControlNet works with Stable Diffusion.
posted by siskin at 9:06 AM on January 7 [4 favorites]

To keep it simple and not get into textual inversion or ControlNet, simply give the boy a name instead of telling the app to generate "a boy." Harold Shabadoo, or whatever. It will pretty reliably create images of the same "person" if you tell it to actually create an image of a person instead of just a type of person. Adjectives are helpful, too. Height, size, ethnicity, etc.
posted by emelenjr at 8:01 PM on January 7 [1 favorite]

Response by poster: Thank you for these responses. I don't think Hero Forge is appropriate for the kind of story telling that I want to do (think graphic novel). But the other two point in some productive directions, if I want to put the work in. Leaving this open, in case other people have suggestions.
posted by Winnie the Proust at 2:37 PM on January 8