Beyond AI Art: Image generation for broader creative applications

1980s-style repeating graphic of colourful computers

Over the past year, there’s been an explosion of image generation tools, like VQGAN+CLIP, Midjourney, DALL·E, and Stable Diffusion, enabling artists to generate images from text prompts. These tools offered new and often strange forms of artistic expression — either as interesting stand-alone visuals or poetic prompt+image combinations. In the early days, the results were kind of quirky, niche images with an obvious AI-art aesthetic.

Some early, not very convincing AI images, from left to right: “keanu reeves centaur photorealistic”, “the relationship between mind and matter”, and “the movie the matrix featuring cats” [Images by hannahj with the original VQGAN+CLIP colab notebook]

But with recent advances, the technology got good — maybe too good. The resulting art actually started to look like art and some artists grew concerned. In my opinion, openness is the way forward, but it’s worth considering the place of image generation in our culture, and how that role might evolve in the future. While we debate the rights and responsibilities of artists, researchers, and developers, a broader creative revolution is brewing under the surface that I doubt will be any less controversial.

When interacting with image generation tools, many of us start with fun prompts, like: ‘Keanu Reeves Centaurs’. If you’re an artist (or always wanted to be one), this tends to evolve into more elaborate visual art creation. The ability to quickly and easily externalize the people, places, things, and concepts we imagine is exceptionally powerful. But not everyone can draw or paint. Not everyone has the skills or dexterity necessary to move from idea to visual representation, in even the roughest capacity.

Digital tools provide some support in this regard, with specialized software helping less experienced designers bridge the gap and design something without many of the formerly required abilities. For example, SketchUp lets users more easily 3D model furniture and construction projects. It works quite well for its intended purpose, but tools like this are also limited. They assume that you arrive with a relatively concrete idea in mind.

OK, so I’m in the tool, now what? – Screenshot of SketchUp: from https://app.sketchup.com/app?hl=en (Accessed Aug 18, 2022)

If you don’t yet know what the thing you want to make ought to look like, you’re likely to need a different set of tools. You might browse images online for inspiration. You’ll probably put some keywords (prompts, if you will) into a search box somewhere. In the not-too-distant future, you might instead prompt an image generator. Suppose you could then combine, edit, reshape, and steer those images to something you want to build. A much wider range of people could use technology to help not just execute fully-baked ideas, but to generate and explore them in the first place.

To a certain extent, this is already possible across an incredibly wide range of creative pursuits.

You can explore design ideas for a kid’s birthday cake…

Cakes designs: Penguin [Image by qedtherese with Stable Diffusion] and Captain America + Hot Wheels [Image by Arun Nagarajan with DALL·E], and Robot [Image by hannahj with Stable Diffusion]

Or for a more elegant occasion…

Cakes designs: Fractals [Image by Kynthic with Stable Diffusion] & Baroque [Image by bachstan with Stable Diffusion], elegant raspberry [Image by origindrift.ai with Stable Diffusion]

You can re-imagine interior designs for a room in your home.

Interior design of a waterfront living roomand grey baby’s room [Images by BlitzStryker with Stable Diffusion] and Cabin [Image by Ulysse with Stable Diffusion]

If you’re into carpentry, image generation might help you get ideas for some custom furniture.

Wooden rocking chair [Image by hannahj with Stable Diffusion] and minimalist table [Image by is0be1 with Stable Diffusion]

A budding gardener might imagine new things to try in their garden.

Garden design sketch [Image by stu eleventhirty8 with Stable Diffusion], and outdoor garden kitchen [Image by rashbit with Stable Diffusion], and garden rendering with plants native to Ottawa, Canada [Image by hannahj with Stable Diffusion]

Maybe you’d use it for sewing inspiration.

White dress [Image by crypticsymmetry with Stable Diffusion], Sparkly menswear Image by Neoth with Stable Diffusion], and futuristic dress [Image by contraflecha_ with Stable Diffusion]

If that’s not your thing, perhaps you’d like to invent some more comfortable shoes.

Fluffy and futuristic sneakers [Images by StanyeWest with Stable Diffusion] and colourful sneakers [Image made with Stable Diffusion by a user who wishes to remain anonymous]

You might need some jewelry…

Earrings [Images by Jinny Lee with Stable Diffusion] and wooden ring [Image by hannahj with Stable Diffusion]

… or other accessories.

Futuristic mask [Image generated by faezehrazavi with Stable Diffusion], shoes designs [Image by papuga_lab with Stable Diffusion], and iridescent bag [Image by Victor_vhn with Stable Diffusion]

If you’re good with wool, you can design new knitting patterns.

Sweaters featuring Keanu Reeves and avocados [Images by hannahj with Stable Diffusion], and a knit toy cat [Image by WaterMaster8 with Stable Diffusion]

With Textile Diffusion, a custom-trained model trained on an original dataset of textiles by KaliYuga, you can generate ideas for tapestries, beadwork, patches, and other detailed embroidery projects.

Generated images might prevent you from making a more permanent mistake, say by first testing out your tattoo designs.

Tattoos: Koi fish [Image by imaigenart with Stable Diffusion], geometric design [Image by Ragnakim with Stable Diffusion] and bicycle [Image by hannahj with Stable Diffusion]

If you’re into board games, maybe you want to brainstorm game piece ideas.

Game pieces: Dragon [Image by lolguppy with Stable Diffusion], medieval warrior and pigeon [Images made by Midjourney users who wish to remain anonymous]

Fortunate enough to know someone with a printer? You can DIY your own colouring book pages.

Colouring book pages: wolf [by Atlas Smith with Stable Diffusion], Keanu Reeves [Image by hannahj with Stable Diffusion], Robot [Image by KnobleSavage with Stable Diffusion]

Artist Helena Sarin has been generating some incredible images of pottery. Generate you own and test them out on the kiln!

No kiln? No problem. Design some origami instead.

Origami: [Image by Dodi1010/Khalid Sonbul with Stable Diffusion], house [Image by hannahj with Stable Diffusion], astronaut [Image by COMEHU with Stable Diffusion]

For those who want to create their own films or games, you might explore new environments or imaginary locations…

Scenes: winter village [Image by Cameron Hyde with Stable Diffusion] & underwater [Image by Craig with Stable Diffusion]

…or generate character concept images.

Tree person [Image by BeardedWhale with Stable Diffusion], Kid robot [Image by Crhys with Stable Diffusion], and character rendering [Image by surea.i with Stable Diffusion]

Many instructions, explanations, and other forms of communication could benefit from accompanying visual representations, especially for kids.

These tools can help express dreams or nightmares, either to share with others or process individually.

Recurring dreams: A tree [Image by squidelephant with Midjourney] and greenhouse [Image by stu eleventhirty8 with Stable Diffusion]

We can even use AI to help us imagine safer, more people-friendly cities.

Most of the images above, while incredible, are not polished or final designs. They often have significant flaws or physical inaccuracies that would render them impractical or impossible to build out in real life (assuming we even wanted to). One day, for better or worse, AI might enable this—and more. In the meantime, the generated images can still help us explore, externalize, and communicate internal concepts and spark entirely new ideas and creative works.

Published
Categorized as UX