Bring Your Images to Life with the Image-to-Video Generator

In this newsletter, read about:

• 🕵️‍♀️ Runway: Image-to-Video Generation

• 🗞 News and Top Reads

• 📌 AI Art Tutorial: AI Video Effects

• 🎨 Featured Artist: Stephan Vasement

• 🖼 AI-Assisted Artwork of the Week

• 🤓 A Comprehensive Midjourney Guide

🕵️‍♀️ Runway: Image-to-Video Generation

I’ll start with a disclosure – this post in NOT sponsored by Runway. I have been following this company for some time and consider them to be a leading player in the video generation niche.

Initially, they released their Gen-1 video-to-video tool, which allowed users to change the video style based on a reference image or text prompt. Subsequently, they introduced the Gen-2 tool, which enabled the generation of a video from scratch using only a text prompt. Recently, they also unveiled an image-to-video generator, which has proven to be an excellent tool for bringing Midjourney images to life.

So, it sounds like a perfect opportunity to take a brief look at how this tool works and how you can experiment by combining Midjourney and Runway.

How to Use

It’s very easy to start using the Runway video generator because it is available right inside your browser and you can start experimenting for free.

Go to Runway App, sign up for free, and you’ll get credits to generate up to 30 four-second videos from your images. The paid plans start from $15 per month with monthly billing and allow the generation of about 125 seconds of image-to-text videos with higher resolution and no watermarks.

Once you are in the Runway Dashboard, choose “Image to Video” or “Text/Image to Video” tool.

On the next page, you can choose whether you want to generate a video from your image OR your text prompt. Unfortunately, they are not working together yet.

• For an image-to-video generation, upload an image by clicking on the image icon and then click Generate.

• For a text-to-video generation, write your prompt, and then choose Free Preview to have a look at several options for the first frame. You can then generate a video from the preview of your choice.

The videos that have been generated will be visible in your Assets folder, allowing you to download them.

In this introductory guide, I only experiment with image-to-video generation, using my Midjourney-created images. Let’s see how Runway performs with different types of images.

Midjourney + Runway Experiments

First of all, let’s bring to alive this terrifying tiger.

It appears intriguing, but there isn't much action, rather the camera zooming in. Furthermore, there are noticeable irregularities in the movement of the forefoot.

Now let’s see how this image of a man playing saxophone will turn up.

It actually looks pretty good besides minor distortions around the eyes, and obviously, the right hand 😁 

Now what about this image of an underwater world?

Lots of movement, which is great! Additionally, due to the abundance of colorful fish and other creatures, it becomes more challenging to identify distortions. However, you can still observe a coral that transforms into a fish, and some weird fish metamorphoses.

Summing up

Runway's image-to-video generator is still in its early stages, like Midjourney was over a year ago. The results are often unreliable and contain distortions and abnormalities. However, you can still produce beautiful videos with this tool.

Here are a few recommendations that might help you get better results:

• Start with images that contain dynamic scenes. Using still life or architectural images will likely result in some zooming, without actual movement. However, this might be what you want for some videos.

• Experiment with surrealistic and unrealistic images and illustrations. Photographic images tend to show distortions more prominently.

• Consider doing several re-rolls with the same image. However, keep in mind that the number of credits in Runway is very limited, even in paid plans. Save your credits for other ideas.

• Think about creating a small video clip by combining several 4-second videos on the same topic. For example, you may use the last frame of the Runway’s video as a reference image for the next video, thereby creating longer videos with the same subject.

People are actually creating entire movie trailers with Midjourney & Runway.

We live in exciting times, when anyone can experiment with the latest technological advances without any technical background. It's an amazing opportunity to learn and create.

Happy prompting and have fun!

🗞 News and Top Reads

• MIT built a tool to protect images from AI.

  • MIT researchers have developed a technique called PhotoGuard that can protect images from unauthorized manipulation by generative AI systems.

  • PhotoGuard works by adding invisible protective masks to images that disrupt the output of AI models. This makes it difficult for AI models to edit or manipulate images without leaving telltale signs.

• McKinsey released a report on generative AI and the future of work in America.

  • The report examines the jobs that will be in high demand, those that are declining, and those that may be the most challenging to fill.

• OpenAI rolled out ChatGPT App for Android in all countries where ChatGPT is supported.

• Instagram is reportedly working on an AI tool that would allow users to remove unwanted objects from photos in the Story editor.

📌 AI Art Tutorial: AI Video Effects

In this video, Matt Wolfe experiments with a handful of AI video-generation tools, including Runway. Check out the tutorial for some interesting tips, tricks, and video effects.

🎨 Featured Artist: Stephan Vasement

Stephan Vasement is an artist exploring the possibilities of AI in various art forms. Drawing on his background in commercial animation, Stephan consistently challenges misconceptions about AI art by emphasizing the importance of inner content in creating powerful and standout works. His “Europe After the Rain” collection is dedicated to the aesthetics of post-war Europe, in particular France.

🖼 AI-Assisted Artwork of the Week

🤓 A Comprehensive Midjourney Guide

Share Kiki and Mozart

If you enjoy this newsletter and know someone who might also appreciate it, please feel free to share it with them. Let's spread the word about AI art and introduce more people to this fascinating field!