How to Create Consistent Characters in Midjourney

In this newsletter, read about:

• 🕵️‍♀️ Consistent Characters in Midjourney

• 🗞 News and Top Reads

• 📌 AI Art Tutorial: Sketch-to-Photo with Midjourney

• 🎨 Featured Artist: Claire Silver

• 🖼 AI-Assisted Artwork of the Week

• 🤓 How to Get Started with Generative AI?

🕵️‍♀️ Consistent Characters in Midjourney

I had been trying to avoid delving into this topic for some time, but the results of the poll in the previous newsletter issue clearly showcased an overwhelming interest. So, without further delay, let's dive into the exciting realm of generating consistent characters in Midjourney – or maybe not that exciting 😂 

The reality is that Midjourney is not currently designed to generate consistent characters, at least not yet. The development team is working on this feature.

So, what do we have instead? We have various tricks, methods, and strategies that can bring you closer to achieving "consistent characters," although it may require a considerable amount of patience and some luck. However, it's important to realize that the results may not be perfect.

Now that we have set more realistic expectations, I can share some valuable tips and tricks that can help you achieve a higher level of consistency. These strategies are primarily derived from the insights provided in the #prompt-faqs responses in Midjourney Discord.

1. Choose simple characters.

If you seek consistency in your Midjourney results, consider prompting for the characters that Midjourney already offers out of the box, such as a cute red-headed girl or a red-furred kitten. When dealing with complex subjects, Midjourney may interpret them in various ways, leading to significantly different results each time. By focusing on established characters, you can enhance the consistency of your generated images.

When only starting these experiments, I was very optimistic and created a cute and original character with the help of ChatGPT and Midjourney. Unfortunately, it turned out to be too complicated for getting consistent results.

2. Name your character.

Give your character a distinctive name to encode it into the prompt. By incorporating a name, you can maintain the visual identity of your character while adding a touch of uniqueness.

3. Try keeping the same number of words for your different scenes.

For optimal results, aim to have an equal number of words in each text prompt describing your characters in different scenes. This helps maintain consistency in token count and ensures the AI algorithm performs more reliably. Therefore, the variable part of your prompt, which describes various scenes with your character, should exhibit grammatical similarity – for example, “in an empty house”, “in a mystical forest”, etc.

4. Use a reference image.

I suggest creating a reference image of your character but keep in mind that the actual character in the generated image may not be an exact replica. Midjourney's goal is not to recreate the reference image precisely. Instead, it seeks distinctive objects, features, and styles from the image and incorporates them into the generated output along with your text prompt. As a result, the subject in the generated images will only bear some resemblance to the reference image while still being quite distinctive.

Still, using the reference image is perfect for stabilizing the results. It is often beneficial to start with your prompt without an image and then incorporate it while experimenting with variations in the Remix mode. I will cover this later in this article.

5. Use character style sheets.

Instead of relying on just a single reference image, you can enhance consistency by creating multiple reference images using the character style sheet.

It is recommended to use a gray background instead of a white one, as it tends to work better. Ideally, the background in the reference images should align with the scene where you intend to place your character.

In my personal experiments, I utilized Photoshop's Generative Fill to add trees to the background. Subsequently, I divided the style sheet into four separate images.

The next step involved extending the images using Photoshop's Generative Fill, ensuring that the character in the reference images occupies a similar space as intended in the final images. However, I encountered a surprising obstacle when Photoshop's algorithms deemed the extension of the background trees as a violation of their terms and conditions. Fortunately, we now have an alternative solution in the form of Uncrop by Clipdrop (Stability AI), which proved to be effective.

Here is one of the reference images after all the tricks.

6. Leverage the Remix mode to add more reference images.

Now, with multiple reference images at hand, you can progressively incorporate them using the Remix mode in Midjourney. Activate this mode through the /settings command, and each time you request variations or a re-roll of the image set, you'll have the opportunity to modify your prompt. Take advantage of this feature to include additional reference images and fill in missing details at the beginning of your prompt.

In my case, I began by using the reference images from the style sheet and the following prompts to depict my character in two distinct settings: sleeping under a tree and climbing a tree trunk.

After experimenting with various re-rolls, prompt variations, and different reference images, I arrived at the following images.

In addition to using the initial reference images, I further explored the technique of using an image from one scene as a reference for another scene. I applied this trick multiple times to both scenes in order to enhance consistency.

As you can observe, the two depictions of Bobby in these images exhibit a considerable resemblance, although some differences can still be noticed. It might be possible to achieve even closer alignment between them if I had more time and patience 😃 

You can also observe that the final character deviates quite significantly from the reference image, but this was expected. As mentioned earlier, the purpose of the reference image is to stabilize the results rather than being replicated.

7. Experiment with the prompt formula.

To stage the same characters over and over again in different scenes, you may also experiment with the following prompt formula, suggested by Clarinet.

It is recommended to use this formula together with the reference image. So, I started with creating the reference:

Next, I used this reference image together with the following prompts:

With just a few re-rolls, I got the following results:

Indeed, the resemblance between these four depictions of the woman is quite noticeable. While the result may not be perfect, I was pleasantly surprised by the level of consistency achieved with just a few minutes of experimentation. Probably, I just had very low expectations when it came to achieving consistency with photorealistic characters.

8. Consider using the InsightFace Bot

Another approach to achieving consistent characters in different settings is by using a face-swapping tool like InsightFace Bot. You can learn about how to use this tool, in this video tutorial.

To leverage this tool, you'll need a reference image of your character, which can be a photo of a real person or an AI-generated image. Then, you'll generate multiple images with different scenes that feature subjects resembling your character. Finally, you can swap the faces in these generated images with the face of your character.

While I haven't personally used InsightFace Bot, I believe this technique can be particularly effective with frontal close-up shots. However, it may not be as useful for complex scenes, or subjects positioned further away from the camera, particularly with side views.

Wrap-up

I hope you find these tips and strategies helpful in your quest for consistent characters in Midjourney. While they can bring you closer to your desired results, let's keep our fingers crossed for the future when Midjourney brings us the long-awaited feature of generating consistent characters effortlessly.

🗞 News and Top Reads

• Clipdrop by Stability AI launched Uncrop, an AI-powered outpainting tool.

  • You can use it to change the aspect ratio or composition of any image by expanding the background.

  • The tool is powered by Stable Diffusion XL, Stability AI’s text-to-image model.

• Adobe announced a set of generative AI tools for Adobe Illustrator.

  • Designers and marketers can now effortlessly transform the colors, themes, and fonts of their graphics with a simple text prompt.

  • Dubbed Generative Recolor, the tool is available in beta in Adobe Illustrator.

• Meta plans to grant researchers access to certain components of their groundbreaking "human-like" AI model.

  • The company claims that its model, I-JEPA, uses background knowledge about the world to fill in missing pieces of images, in contrast to other generative AI models that only look at nearby pixels.

  • Earlier, the team also revealed that they are working on a new Instagram feature that will allow users to modify their photos via text prompts.

📌 AI Art Tutorial: Sketch-to-Photo with Midjourney

In this video tutorial, Christian Heidorn goes step-by-step over the process of transforming sketches into photorealistic renders with Midjourney’s Remix mode.

🎨 Featured Artist: Claire Silver

Claire Silver, an artist, collaborator with AI, Cryptopunk enthusiast, and firm believer in NFTs, has embarked on a journey to create art that transcends boundaries. Her works aim to evoke a profound truth that defies words. Claire has been exploring the realm of AI art since 2018, an era that feels like an eternity in the world of AI art. Discover more of her awe-inspiring artworks by visiting @clairesilveraiart.

🖼 AI-Assisted Artwork of the Week

🤓 How to Get Started with AI Art?

1. DALL-E: Creating Images from Text – introduction to text-to-image generation.

2. The DALL-E 2 Prompt Book – a guidebook by OpenAI that explains how to effectively right prompts to generate images across different domains (e.g., photography, illustration, art history, 3D artwork).

3. All-In-One Guide For Midjourney: The Art Of Prompts – a comprehensive guide on creating effective Midjourney prompts by Karol Andruszków.

4. Best Midjourney Prompts – a guide that covers the basics of Midjourney prompts (e.g., which keywords to use to create abstract art, surreal art, minimalism, etc) as well as some more advanced options (e.g., keywords related to camera lenses and filters, imitating certain artists and photographers without using their names). Finally, they provide a list of 600+ creative text prompts for image generation.

5. Stable Diffusion Prompt Book – a prompt book prepared by OpenArt. The book discusses ideal prompt format, using modifiers to change the style, format, or perspective of the image, applying ”magic words” to improve image quality, adding negative prompts, and adjusting Stable Diffusion parameters.

Share Kiki and Mozart

If you like this newsletter and know somebody who might also like it, feel free to share this newsletter. Let’s have more people learn about AI art!

If you have been forwarded this email and you like it, please subscribe below. And welcome to the world of AI art!