The Introduction:
Making of UFC 287 AI
I'm a huge MMA fan and the colorful UFC 287 poster inspired me to create this animation with help from some of my favorite AI applications.
This was a great opportunity to play with some AI apps I was familiar (and not so familiar) with. During the course of this experiment, I got to experience how AI could be used to help out in the creative process.
Anyone working in creative will know, we designers/animators are usually given assets like keyart, photographs, style guides, music, VO, scripts, and various stylistic elements to create with. With the help of AI, I was able to generate these assets myself.
While generating assets using AI sounds great, I soon realized how much work and time it takes to create each of these assets. At least right now (April 2023), commercially available AI (that's directed at taking on creative tasks) isn't at a state where they get things right the first, second or even third time.
I will go through which AI tools I used and talk about some of the challenges I faced when using AI to create UFC 287 AI.
The Art: Fighter & Background Illustrations
AI PROGRAM USED: Midjourney
The first thing I wanted to do was define the look and feel of the animation. I really liked Cyberpunk: Edgerunners so I was going for a hightech, cyberpunk, colorful illustrated look. I started exploring this on Midjourney.
Midjourney is a text to image AI Discord application where you type in a description of what you want and it spits out an image. Sounds easy enough but there were a couple challenges I encountered when I started this process and I'll walk through each of them below.
Challenge #1
Defining a style.
Refining the prompt to get what I was looking for took a lot of trial and error. It took about 2 days to narrow down the style I liked. When I began, the first batch of generated images looked too much like a kid's anime or a photograph. Here are some of these images.
* As you can see in a couple of the images, there's always a chance of deformities appearing on the generated characters. It would be helpful if they had a negative prompt like Stable Diffusion.
3K+
Images Generated
660+
GPU Minutes Used
~55
Images Actually Used
$30
per Month (Midjourney)
Challenge #2
Making the character resemble the fighters.
Midjourney doesn't really know exactly what the fighters look like so this became a challenge. I included some generated photos of the fighters below. I think Jorge looks the closest (there's a mix of Jon Snow in there). Midjourney had no idea who Alex was. They look decent for the most part and if I were shown these images, I could probably guess the fighter but compare this to known people like The Rock and there's a huge gap.
Close but no cigar.
Known people like The Rock are very accurately generated. The bottom one on the left does look a little like Vin Diesel though but it's still pretty close.
There's an option in Midjourney called IW (Image Weight) where you can include an image to influence the generated image. This is the closest thing you can do to "train" Midjourney. Even with the IW option though, I would only get about 1 in 40 generated images that looked similar to the fighters.
Here are some samples of generated images that met the style requirement and looked amazing but didn't really look like the fighter.
Challenge #3
Needle in a haystack.
In the beginning, it was really exciting looking at the results generated by Midjourney, but after the first hundred or so images, the process started to feel very tedious.
The biggest obstacle I feel was Midjourney not knowing what the fighter looked like. This resulted in countless generations of images that didn't resemble the fighter.
When pitching marketing ideas for the movie industry, me and my business partner were at times given access to photos taken during the production of the film. There were thousands of photos we combed through and the few dozen we pulled would be used in the pitch.
This process felt exactly like those photo pulls. There were thousands of images to choose from, but only a handful were actually used.
*Stable Diffusion would have been another option since it's possible to train it. When I first tested Stable Diffusion, I wasn't able to find a model with the same style that the Midjourney characters had. I'll have to play around with V1.5 some more as well as Openjourney and Inpunk Diffusion. I really like the ControlNet option in Stable Diffusion so I'll definitely explore that option next time.
I created this stylized version of The Rock with just 4 generations (4 images each) because Midjourney knew exactly what he looked like.
Background Exploration
Absolutely Beautiful.
Generating the background images was such a satisfying experience. Midjorney really does a great job of creating abstract shapes, lighting, and colors.
I started off testing abstract neon paint textured backgrounds, similar to the UFC 287 poster but ended up going with the high tech, futuristic look.
Here are some of the backgrounds that were generated.
The Design: Style frames & Social post comps
PROGRAM USED: Adobe Photoshop
Now that the assets (fighter and background illustrations) were generated, I started designing the style frames. Style frames are similar to storyboards where you visually show the sequence of what occurs from the beginning of the animation to the end.
The idea for the animation was to showcase the fighters in the main and co-main event of UFC 287. I didn't know what the exact messaging (text) would be but I wanted to showcase 3 different designs per fighter. I comped the fighter's name temporarily (FPO) for the text.
For the overall look of the design, I wanted to use big type with textures overlayed on top. I really liked the main title for the show Preacher so I used that as inspiration. I used the UFC font which worked beautifully for this. I also wanted a version where the fighters stood out so I removed the textures and replaced them with solid colors.
I started off designing Jorge Masvidal's frames first since I really liked the illustration of the fighter Midjourney generated. I'll only show the process with Jorge but I used the same process with the other 3 fighters.
Style Frame #1
Jorge Masvidal, the Heavyweight with the Diamond Fist
50+
Style Frame & Comps
300+
Layers
$54.99
per Month (Adobe CC)
"Who the fook is that guy?" Yes, he's too big to be Jorge but it looks amazing : )
These were the background assets and fighter illustrations used for the 1st frame.
This image was used as an overlay for the text.
This image was used as the background.
I didn't like the fist of Jorge on the left image so I replaced it with the diamond fist from Jorge on the right image. I also got rid of the extra finger on his other hand.
*Move around the line in the middle image to view the before and after.
This video shows where the assets are being used in the design.
I originally had a skinnier Jorge image in the comp but the bigger Jorge looked more dynamic so I replaced him even though he looks more like a heavyweight than a welterweight.
*Move around the line in the middle to view the before and after.
Here is the version with the textures removed and replaced with solid colors.
*Move around the line in the middle to view the before and after.
Style Frame #2
Jorge Masvidal, the Samurai
I can see this Jorge in an anime version of The Last Samurai.
These were the background assets and fighter illustrations used for the 2nd frame.
These images were used over the shard shapes in the background.
This samurai looking image of Jorge was used like this for comp #2.
Style Frame #3
Jorge Masvidal, the Man Bun
This generated Jorge was one of my favorites. It's also the one that looked most like him.
This was the background asset and fighter illustration used for the 3rd frame.
Making Edits
Gil and Izzy Head Transplants
As I stated in the previous section, it was tough generating fighters with their likeness so I used the best generated face and replaced it on some of the frames. Midjourney did a great job with everything else so there weren't too many edits I had to make. These were probably the biggest ones.
Some color correction and a head replacement on Burns and Adesanya.
Style Frames
What we have so far
Here's what the styleframes are looking like. Things are starting to take shape.
Social Graphics
Resize, Resize, Resize
Anyone who has designed for entertainment knows this task very well. Resizing and repositioning huge keyart files into countless formats for social headers, social posts, app banners, web banners, websites, Hulu, Roku, etc, etc : )
Making a whole social campaign out of these would have been fun. I could see a batch of posts exploring Jorge's roots in Miami, similar to what they did on Embedded and Countdown.
❌ Under normal entertainment standards, there's no way we're allowed to flip the characters and make all of the changes like I did with these. : ) They'll also usually have a landscape and portrait version of the keyart.
The Script: Based on the UFC 287 Cold Open
AI PROGRAM USED: OpenAI ChatGPT, Google Cloud-Speech-to-Text
This should have been the first thing I worked on but I was too excited about the art. : ) I wanted the script short, simple, and something that described the fighters. I was going to ask Chat GPT to write me a short intro but then I thought about the UFC Cold Open videos.
The Cold Open is a video the UFC plays before their PPV events to hype up the fights. It's high energy and gives a great summary of the fighters and their history in about 3 minutes. It also serves as a marketing tool as the video is released a day before the fight to drive people to buy the PPV. The narration is done by Ron Perlman. The late Anthony Boudaine used to do the voice overs for the UFC intros as well. John Malkovich did an amazing job for the voice over on the Khabib vs McGreggor fight. Anyhow, I decided to use the narration from the Cold Open video to create a short summary for my script.
400+
Words Transcribed
3 min
of Narration Transcribed
$0
Free
Transcribing audio using AI
Good but not perfect
The first thing I did was clip the audio from the UFC 287 Cold Open. I then used Google Cloud-Speech-to-Text website and uploaded the audio to start the transcription process. You can transcribe 1 minute for free so I cut the audio into 4 clips and uploaded them one at a time.
As you can see from the screenshot below, the transcription wasn't perfect. It also ignored certain parts of the audio commentary from Joe Rogan, Daniel Cormier, and Jon Anik. I think it may have just focused on Ron Perlman's voice since it was the clearest and most prominent. The punctuation was also off and often would end sentences incorrectly.
Even though it wasn't perfect and there were some fixes I needed to make, it definitely was better than transcribing the whole thing manually. I feel it did a decent job considering there were a couple different voices as well as the music. It was also free so no complaints there : )
You can view the cold open below. For some reason, it wasn't on YouTube anymore so here's an embedded Twitter post.
ONE DAY AWAY!
— UFC (@ufc) April 7, 2023
Get fired up with a preview of Saturday’s #UFC287 Cold Open 🔥🔥 pic.twitter.com/OLpB4W2V1Y
Summarize using ChatGPT
Good but still needs work
Now that the transcription was done, I put the Cold Open text into ChatGPT and asked it to summarize it for me in about 40 words.
Here are the results it gave me. (I also asked it to break down some keywords. I can see this being useful to generate metadata for a website.). ChatGPT also thought Masvidal and Burns were fighting for a title which was not true.
These summaries lacked the excitement that was in the original Cold Open intro so I asked ChatGPT for something more exciting, something that you would hear in a commercial. The results were a little better but some parts were a bit cheesy.
The Final Copy
Keeping it simple
I decided to mix and match what ChatGPT gave me for the final script:
In a highly anticipated rematch, UFC 287,
The Last Stylebender
Israel Adesanya
faces his kickboxing nemesis Alex Pereira for the UFC middleweight championship of the world.
King of Miami Jorge Masvidal fights Brazilian Powerhouse Gilbert Burns.
It's a heated rivalry between the division's best.
UFC 287 starts now.
Updated Styleframes
It's all coming together
I made the updates to the styleframes with the new script copy. I couldn't find a place for one of Gilbert's frames so he only has two spots : (
The VO:
AI Ron Pearlman
AI PROGRAM USED: Elevenlabs, Lalal.ai
Now that we had our script, I thought it would be awesome to have Ron Pearlman's voice narrate it! I started doing some research and found Elevenlabs.
Elevenlabs
The deepfake of voices
The site can clone voices based on the audio file you train it on. After training the voice, you can type in what you want the voice to say. Uploading a clean VO file generates the best results so I started uploading audio files of interviews with Ron Pearlman.
The first thing I noticed is that there's a huge difference in the tone of his voice when he's narrating a UFC intro vs him talking about his latest movie. The VO ended up being unusable and I needed another option. I needed Ron Pearlman's voice from the Cold Open.
14K+
Characters narrated
$5
per Month (Elevenlabs)
$15
per Month (lalal.ai)
This is the first line of the script with the voice trained from an interview.
I had to make a version with the original Cold Open text, it's so great.
Lalal.ai
Background music be gone
The issue with using the Cold Open voice over was that the background music would interfere with his narration. I needed a clean version of Ron Pearlman's voice narrating and that's when I found Lalal.ai. This site could remove background music from an audio clip. I fed the Cold Open audio into the site and this is what it did.
This came out pretty good but it wasn't as clean as I wanted it to be. It sounded like he was hissing through the narration so I decided to process another Cold Open from UFC 242.
This audio came out a lot cleaner. It wasn't perfect but it sounded like it would work.
Elevenlabs Attempt #2
It's alive!
After training the voice using the Cold Open audio, I entered the first line from the script. It's all downhill from here I thought. I was wrong.
The audio came out pretty dull.
This definitely sounds more like the Ron Pearlman from the Cold Open but there's a couple things wrong with the clip. One of the first things I caught was that he says two-hundred eighty seven. It should be two eighty seven. He also incorrectly pronounces Alex Pereira's name. The pace and emphasis on certain words was also off. Some tweaking was needed.
I decided to cut the narration down into parts. The risk of something sounding off would grow if the sentence were too long so I started with the line "In a highly anticipated rematch".
It's gonna be a long night.
Since I was creating this in parts, to keep a consistent feel, the energy level at the beginning of each section needed to start high. I broke down the section like the below and generated each part until I got one that sounded good:
In a highly anticipated rematch
UFC 2 87...
The last stylebender Israel Adesanya...
faces his kickboxing Nemesis...
Alex Pehera...
for the U F C middleweight championship of the world.
King of Miami…
whorhay Masvidal…
Fights Brazilian Powerhouse…
Gilbert Burns.
It's a heated rivalry..
between the division's best!
U F C 2 87..
Starts..
now...
I phonetically spelled out Pereira and Jorge (sometimes it got the name right with the correct spelling but why risk it) and I added ellipses so that there would be a short pause after saying the line.
This took some trial and error but it was fun to make and the finished product sounds great.
The final narration.
Elevenlabs works!
But nothing compares ♪♫♪♫
The final narration sounds great but nothing compares to the real Ron Pearlman. I can feel the emotions come through his narration in the Cold Open where with the AI voice, there's something's missing. Ron Perlman could probably knock them out faster than it took to create the AI generated clips as well. Nonetheless, Elevenlabs is very impressive and I'm very happy with the final product. Things are just going to get better. I'm interested to see what happens in the next couple of years (famous last words).
The Music:
Tried AI but failed
SITE USED: Adobe Stock
I attempted to create some music on a couple of AI platforms but it was either too complicated for me or I couldn't find any tracks I liked. I've heard some great music made with AI but unfortunately for me, it still takes musical talent to make great tracks 🤔.
Ultimately, I found a stock track on Adobe Stock that I was happy with it. The only thing I had to do was cut it down to 30 seconds. It was a great timesaver.
1
AI Track Generated
1
Non-AI Track Used
$29.99
per Month (Adobe Stock)
My AI masterpiece.
The Animation:
A little bit of UFC flash.
PROGRAM USED: Adobe After Effects
Now that all the assets were created:
- The Style Frames
- The Script
- The VO
- The Music
It was time to start animating.
The UFC Text Effect
Text animation
When watching a UFC event, the animation style they use to reveal the fighters name is really nicely done. I like this style of animation and wanted to apply this to the names of the fighters. Check out the UFC version of the text animation below (*done by an awesome animation studio, Troika. I got a lot of inspiration from them.):
5hrs+
Rendering Time
100GB+
Cache Used
$54.99
per Month (Adobe CC)
The original UFC text animation in real time.
I slowed it down so the effect can be seen a little better. There are 2 fonts and the space in-between the letters (kerning) is also animated.
I created a similar animation style and applied it to the text.
Here's the text effect in real time.
The animation slowed down.
I used this technique a lot to animate the text into frame:
Here's the text effect in real time.
The animation slowed down.
The UFC Fighter Effect
It's Morphin' Time!
The UFC fighters have a cool transition when coming into frame. They start cut up in sections with different colors and glide into place. The colors come off revealing the fighter. I added the original UFC animation below.
This is the original UFC fighter effect in real time.
Slowed down version.
I created a similar animation style and applied it to the fighters.
Here's the fighter effect in real time.
The animation slowed down.
Adding the Music and VO
The end is near!
Animating to the narrated audio was a little difficult as it would go pretty fast. That made the animation a little bit faster than I liked but overall, I was very pleased with the results.
I originally didn't have the last line "UFC 287 starts now" because it didn't make much sense but after watching it end on the previous line "It's a heated rivalry between the division's best", it was too abrupt of an ending so that final line smoothed it out.
I also wanted the textures to be on screen a little bit longer but it was probably for the best as the textured text became too busy to read when animated against a textured background. They also look very cool as still comps so that was nice.
The Conclusion:
Will AI take over?
Various AI apps have just started becoming accessible to the public and seeing what it does, it's easy to think that it could literally take over everything. To the jaw dropping art it creates, to mimicking Kanye West, it might be a question of not will it take over, but when. However, as of today (4/2023) our AI overlords aren't ready to take over, not today.
5
AI programs used
30
Seconds of Animation
~180
Hours Time Spent
$130
Total Spent
Working with AI
It can't do everything, at least not yet.
Without AI, I wouldn't have been able to create the illustrations and background art. Without the AI art, there wouldn't have been any assets to designs with. I also couldn't have created a voice over with Ron Pearlman's likeness.
AI right now is at a point where it's helping (*I understand this really depends on who you ask). It helped me create beautiful assets that helped me create cool designs and animations. It did however take time to create each part.
Normally, there's a team or person dedicated to working on these individual tasks (the art, VO, script, music, etc). Although I was able to create them all with the help of AI, it took time to refine each one. It actually took more time and effort than a traditional project where the assets are provided.
Trust me, I'm a Professional
Yeezy!!
How AI is used also makes a pretty big difference since so much of what it creates just isn't good. For example, I thought AI Kanye West sounded amazing. This was because the voice that AI Kanye was replacing was a good rapper himself.
As an experiment, I replaced AI Ron Pearlman's voice with AI Kanye to see how it would sound.
This is the AI Ron Pearlman intro.
This is AI Kanye replacing AI Ron Pearlman.
AI Kanye sounds off and a little British for some reason.
I remember Tyron Woodley releasing a rap song a couple years ago so I thought I would test AI Kanye out on his song. This is where AI Kanye should shine.
This is the first verse to Tyron's song
Money and the Life.
AI Kanye replacing Tyron's voice.
It's not perfect but it works a lot better.
I really believe that the amount of experience determines the quality of the work even when using AI. AI Kanye would sound pretty bad if I were to provide the rap vs a professional rapper. The script may have been better if I had a professional writer come up with something on ChatGPT. An audio expert may have created a better audio track.
There are certain things that are better when done by a professional. Although this line is quickly blurring, I think it's going to take some time for AI to completely take over everything... probably : )
Final Thoughts
Take Care of Yourselves and Each Other.
Having the ability to create art is huge. I remember going into a meeting with a movie studio and seeing a PDF file with hundreds of variations of a movie poster. I never realized how many versions of a poster were created. It takes a decent size team to make all that art. In the near future, will departments only need one or two people to get the job done using AI?
One of the drawbacks of AI right now is that it takes time to look over the variations and tweak them along the way. I think the next step may be tools (specifically for creatives) that involve the same process we go through. Midjourney can create a variety of beautiful UX/UI/Storyboard/Syyleframe images but it would be amazing if we had some input in the process so that we could re-create a specific style instead of hitting the regenerate or make variation button. For example, the design process could involve: picking a style, selecting a color palette, selecting messaging, selecting the font, generating the art and generating various designs from the selected art with the option of generating a layered PSD or exporting to Figma. The same process could be done with animation. This does currently exist in the simplest form from sites like Microsoft Designer and Designs.ai but they have a long way to go.
It's also very possible I may be underestimating the timeframe. If AI could teach itself everyday, I can't imaging it taking too long to generate the perfect graphic, animation, poster, etc itself. I do think we're a ways off from this but who knows, technology moves very fast.
Whatever happens though, I'm really enjoying using AI apps right now so I'm going to continue using them as a tool to help me and hope to enjoy the ride while it lasts : )
*I didn't touch on the copyright issues because that's a huge topic that would make this long read even longer. Something needs to be figured out but what that looks like, your guess is as good as mine.
** I was going to have ChatGPT re-write everything but after a few tests, my wife told me that it sounded too much like a research paper so I left it as is. : )
CONTACT ME
Los Angeles, CA