Tips and Tricks on Using the New Character Reference Feature in Midjourney

In this newsletter, read about:

• 🕵️‍♀️ Character Reference in Midjourney

• 🗞 News and Top Reads

• 📌 AI Art Tutorial: Consistent Characters

• 🎨 Featured Artist: Caspar Jade

• 🖼 AI-Assisted Artwork of the Week

• 🤓 A Comprehensive Midjourney Guide

🕵️‍♀️ Character Reference in Midjourney

Midjourney has finally released a tool for creating images with consistent characters. It’s still in the test mode and doesn’t always perform reliably, but it’s definitely fun to play with.

To set the expectations right, it is worth noting that the precision of the tool is limited and it will not copy all the dimples and freckles throughout the images, but you can get a very high resemblance when putting the same character in different settings.

In this post, I will briefly introduce the feature for those who are not yet familiar with it. And then, I’ll provide some tips and tricks on how to use it more effectively. Let’s get started!

Intro to Character Reference

The feature works similarly to Style Reference. You just type --cref at the end of the prompt and then, add a link to the reference image:

• You can use the --cw parameter to modify reference strength from 0 to 100 with the default set at 100.

• Note that --cw is not an image weight, but rather defines whether you want to transfer all attributes, including clothing and hairstyle (--cw 100) or just a face (--cw 0).

• You may reference several images in --cref.

• When generating images through the website, you can simply drag the reference image to the imagine bar and then choose the corresponding icon (a little figure) to use this image as a character reference, rather than an image or style reference.

• The feature works best with characters generated using Midjourney and is not intended for use with real people.

Now let’s move to some useful tips and tricks.

Best Practices

Below are a few recommendations that should help you get the most out of the character reference tool.

1. Choose the right reference image.

It’s easier to keep a resemblance across different images when a character has some distinct features. So, choose a character accordingly. Make sure that in your reference image, all the distinct features are visible, you can clearly see the person’s face, and there is only one person in the image. The surroundings should be uncomplicated, but you don’t necessarily need a plain background.

2. Choose your approach to the prompt based on your reference image and goals.

When writing a prompt to place your character in a different setting, you may reinforce the distinct features by listing them once again or you may just describe the new setting. Even if you don’t mention any person in your prompt, Midjourney will place the person from your character reference into the image. But if you need a different pose or change some clothing/hairstyle details, you’ll need to mention this in your prompt. Otherwise, all the details will be just copied from the reference.

3. Experiment with the --cw parameter depending on whether you want to transfer all the character details or just the face.

When generating the image above, I didn’t specify the --cw parameter, and so the default value of 100 was applied. As a result, my fictional character Jake is wearing the same outfit. To change the outfit, you’ll need to reduce the --cw parameter and specify the new details.

We’ve got the new outfit, but the resemblance is not so good. Let’s see how we can improve this.

4. Vary the image with different --cw values to find the right balance between a new setup and character resemblance.

With the Remix mode and Low Variation mode switched ON, re-roll the image you like with different values of the --cw parameter. For example, if the resemblance is too low, try to increase this parameter gradually.

If this doesn’t help, you may always use the Vary Region feature. Select the face area and re-run the prompt with the --cw parameter set to 100.

For instance, here’s the result of this “Vary Region” trick with --cw 100 applied to the previous image. The resemblance is much better.

5. Use several character reference images to get a specific pose or certain attributes.

To keep character consistency across different poses and angles, it might help to include several character reference images.

Alternatively, you may try using a reference sheet, where your character is shown multiple times in many poses. This approach doesn’t perform better than 1-3 good reference images but might be helpful in certain cases.

6. Experiment with combining character reference with image reference and style reference.

Often, it helps to supplement the character reference image with an image reference to get a certain image composition or style reference to achieve a specific style.

For example, I struggled to get my character depicted sitting in the bar and drinking beer because the new composition needed to be very different from the reference image. Though I managed to get a few images with good composition but relatively low resemblance.

I used one of these images as an image reference while adding the initial reference image as a character reference.

Then, again the “Vary Region” trick with --cw 100 helped to further improve the resemblance.

7. Increase the stylization parameter when the output images look incoherent or broken.

You may have noticed that I use quite high stylization parameters for the images in this post. As recommended in the corresponding #prompt-faqs post in Midjourney’s Discord, when an image looks incoherent or broke, increasing stylization to --s 800 or even --s 1000 often helps.

8. Use --cref to change fictional characters to photos and vice versa.

The character reference feature works quite well across different image styles, and you can actually use it to transform fictional cartoon-like characters into realistic photos or vice versa.

That’s it for today! Have fun with a new Midjourney feature! And may your characters be consistent! 😆 

🗞 News and Top Reads

• In case you missed it, last week Midjourney decided to ban all Stability AI employees from Midjourney discord.

  • Apparently, one of the rival’s employees attempted to scrape prompt and image pairs in bulk using bots.

  • This caused a 24-hour outage for Midjourney.

• Pika Labs introduced sound effects for video generation.

  • Users have the option to specify the desired sound effects or allow Pika to automatically generate them based on the video content.

• ChatGPT rivals are getting stronger:

  • Anthropic unveiled its latest Claude 3 model family, introducing the high-end ‘Opus’ variant that surpasses leading models such as GPT-4 and Gemini Ultra in key benchmarks.

  • Inflection has launched Inflection 2.5, a new model that performs within 94% of GPT-4 benchmark scores, but only took 40% of the compute to train relative to GPT-4.

📌 AI Art Tutorial: Consistent Characters

If you prefer a video format, check out this video where Nolan reviews the new Character Reference feature in Midjourney. See how to apply character weight and how to combine character reference with style reference.

🎨 Featured Artist: Caspar Jade

Caspar Jade is an AI alter ego of Hannes Caspar, a portrait photographer from Berlin. AI technology has made it possible for Caspar to express himself creatively in a different way. He is particularly interested in cinematic images with high narrative power. Check out his AI artwork on Instagram @caspar.jade.

🖼 AI-Assisted Artwork of the Week

🤓 A Comprehensive Midjourney Guide

Share Kiki and Mozart

If you enjoy this newsletter and know someone who might also appreciate it, please feel free to share it with them. Let's spread the word about AI art and introduce more people to this fascinating field!