TL;DR Working with both ChatGPT and Leonardo.Ai at the same time can boost your creativity and creative output a lot.
1. Introduction
1. Introduction
This is a follow up to a previous tutorial: https://laibyrinth.blogspot.com/2024/12/how-to-combine-chatgpt-leonardoai-and.html
While the first tutorial explored the idea of creating a montage video made up of still images, this tutorial is about creating a fully animated video.
I used this technique to create visuals for a little "music video" that runs for 2 minutes and a half in only 4 minutes.
Of course you can create even much longer videos with this, too!
And you could use more time, and get an even more polished video.
Reading the first tutorial helps understanding the following tutorial.
So let's go on.
2. The tutorial
My workflow is this:
a) First, I explain the project to ChatGPT:
I tell it that I want to create an animated video to accompany an electronic music track by me. That I want to generate images using Leonardo.ai, which I will then animate using Leonardo's Image2Motion tool. And then create a complete video out of these, using the "montage" technique.
b) I ask ChatGPT to give me ideas for the type of visuals that might be suitable for the project and this type of music.
[Note: I also add that Leonardo's image2motion is still a bit glitchy when it comes to human animation, so I ask ChatGPT to consider this.]
c) ChatGPT gives me a large number of concepts / topics for the visual animation.
I quickly choose 6 that I like best out of these, and copy them to a separate text file.
d) I tell ChatGPT my choices, and for each one I ask it to give me a suitable prompt that I can use with Leonardo's image generation.
I copy the 6 prompts into the text file and assign them the numbers 1 to 6.
e) I launch Leonardo and go to "Flow State" - an image tool that generates a large amount of different images quickly.
f) I copy the first prompt Flow State - hit generate - and out of the "endless images" that are generated, I quickly choose 10 that I like best, save and download them.
g) I do the same thing with prompts number 2-6.
After all images are generated and chosen,
h) I go to the library of my images on Leonardo. The newly generated images are all there. I start to animate them one by one, using image2motion.
(note: if you run out of credits now - continue the project the next day when Leonardo resets the score).
i) I check and download them. I got 60 video clips now, running 4 seconds each. 4-5 of them are 'too glitchy', I discard these. The others are as smooth as ice-cream.
j) i go to my video editor, load all of the video clips, load the audio clip, edit them according to the montage technique, and -
k) voilà - everything is finished!
3. Further Uses
BTW: the track of mine was actually produced 21 years ago and was quite the banger when I played it at the Tresor club in Berlin, but that's completely unimportant ;-) you can use this technique in any way you want.
You could, for example:
a) Create a video for your own music.
b) Visually aid the narration of a short story
c) Use it for a different video project (e.g. as a middle part to spice up a YouTube video of your own)
d) and and and....
the possibilities are endless!
And let me tell you one thing; I'm doing music since decades; I've also tried to create video sequences to my music for a long time.
Usually, this takes me several weeks - or even several months. And these are not even "high quality" productions.
Now, with this thing here, and these AI tools... from conception and idea, to the video being finished and uploaded to YouTube... i.e. having everything wrapped, clean and done... it took only around 45 minutes of "working time"!
This is truly an exciting new era for creative folk.
4. Addendum and Examples
you can watch the finished video here:
https://www.youtube.com/watch?v=XpEgdvh5VKM
example for one of the visual ideas ChatGPT generated for me:
"2: Abandoned megacity: Ruins of a cyberpunk metropolis, overgrown with glowing moss or bio-tech flora."
example for one of the image prompts ChatGPT generated out of a visual idea:
"A surreal alien desert with vast, glowing sand dunes, their surfaces embedded with crystalline formations that emit a faint luminescent glow in shades of blue, purple, and green. The sky above is a deep, ominous blood-red, streaked with dark, wispy clouds. The horizon glows faintly with the eerie light of alien moons, casting long shadows across the rippling dunes. The scene is hyper-detailed, mysterious, and otherworldly, evoking a sense of awe and desolation"
5. Further explanation and disclaimer:
Note: I used a movie technique called "montage" for the video. This was, and is still considered a high form of art. While being ubiquitous in movies and media, in more mainstream type of movies, "montage" sequences are usually delegated to a lesser role, such as openings, dreams, moments of romance, travel sequences... (remember Harrison Ford in "Blade Runner"?)
montages often lack traditional narratives, or structures, and can be dream-like, "stream of consciousness" cuts.
Despite of this: I'm not a top notch video producer, it's meant to be rough and gritty, and the visuals were done in under 4 minutes.
So I'm certain that someone with skill, more time and patience could create something for better and more stunning :-)
Alas, it's a tutorial to built on - for you!
And this gets us to the next point:
This is not some "one prompt fix" automated AI video generation where you can sit back and relax.
It's meant for creative people and to show how AI can *help* with creative projects - not to replace it!
6. The End
I hope you enjoyed this little tutorial, and that it might be useful for you in some way.
if you have further questions - feel free to reach out to me!
No comments:
Post a Comment