Filmmaker Sabrina Doyle has been experimenting with AI for about a year after seeing the work a friend created using the technology and recognizing the opportunities it may offer in terms of previs, world building and enhancing her own sizzle reels. She had an opportunity to further explore its possibilities via a collaboration with singer/songwriter/guitarist David Haerle, with whom she’s partnered in the past. The pair would connect when both were in Los Angeles, but now she’s splitting her time in London, and was there when the musician offered her the chance to create accompanying visuals for one of his latest songs.
The result is a music video for “Tucumcari Tonite!” — a song that Haerle was inspired to write based on formative experiences that have stayed with him over the years, including long summer drives through the American Southwest. Doyle spent a little over a month producing the piece, using AI tools to create the desert landscape and the dinosaur-themed roadside attractions that Haerle may have come across. The project didn’t call on a production designer, cinematographer or team of visual effects artists. In fact, even the on-screen talent — a young boy who resemble Haerle, and his beloved grandmother — were created using AI.
The video, with its desert monuments, meteor strike, dinosaur characters and roadside attractions could have run up quite a tab, but Doyle pulled it all off for a few hundred dollars — not including her fee as creative director and editor. Here, she shares her experience working on Tucumcari Tonite!, the power of Google’s Veo 3.1 and her thoughts about the future of traditional production.
Photo: Sabrina Doyle
How did you come to work on this project?
“David Haerle is an LA-based musician. I’m between LA and London, (and) had collaborated with David in the past on other music videos, but was in was in London in the fall of last year and was obviously sort of remote at that point. We were thinking about what we could do together…and I had started experimenting with AI as a tool for my features to kind of previs and create sizzle reels to persuade investors (and) give them a sense of tone. Sort of a 'look book,' but enhanced. You’re giving them a sort of teaser trailer, even before they’ve started.
“What we’re seeing is that people want more and more fleshed out detail. A script is not enough anymore. I had sort of experimented with AI to create a sizzle reel for a screenplay of mine called Aster’s Rites, which is a science-fiction screenplay set in a future world. A filmmaker friend of mine used AI to do a sizzle for his screenplay and my jaw was on the ground. This was about a year ago. He was sort of right at the forefront. I couldn’t believe what I was seeing! I was like, ‘What did you use? How did you do this?’ I had so many questions for him and I decided to sort of experiment with it myself, and was able to cut together a teaser trailer for my screenplay,
Aster’s Rites. That really captured something. (It) actually felt emotional. I think that’s the key, right? There’s obviously a lot that’s scary, and a lot that is disempowering for us as filmmakers in this process that we have to kind of navigate very carefully. There is nothing like working with a real actor. And this is where editing comes in — just knowing how you want to cut two images together and knowing that two images together create this meaning is very powerful. I don’t think the AI can do that. I think only the human can do that.
“I was able, with the sizzle for my feature, to create a world and show that I could create a world and sustain a tone. Something that I would never have been able to achieve without millions of dollars, (in) a week-and-a-half of my own time and a subscription to Veo 3.1. That was astonishing to me. So I pitched the idea to David. He sent me a selection of his songs and said, ‘Which one speaks to you for this kind of video?’ And I was like, ‘Well, this one does because it feels like it’s not something that we would shoot for real. It feels like something that we could only do with visual effects, or AI, or something like that.’
“He had other ideas, and I was like, ‘No, just go out and shoot that with a real camera, real actors, real people.’ I don’t think it should be a substitute for doing something for real if you can do things for real. But if you can create something that would otherwise never have been made in the real world, and AI gives you the means to do that, either in terms of scale or budget, then that becomes a viable case to make for using AI as a tool. It was about this kind of ancient land, where dinosaurs roamed, and also his childhood. It felt like (it) had a mythic kind of quality about it, almost like a hallucination, a dream.”
The consistency from shot to shot is very impressive. Can you talk about the prompts that you wrote. Did you have any help?
“I did everything, so it was a complete one-man band — or one-woman band. (I) did the prompting. I cut it together. I did everything. The art is all in the prompting. And generally, the way it works is, you find with trial and error. For example, I was finding that I (was) prompting about a desert landscape, and it was giving me very traditional cactuses, which you don’t find in that bit of the world. So, I had to very specifically talk about the vegetation that I wanted — juniper — and I found a language through trial and error. I found kind of a description of the language that I put into every prompt.
“And then I described the child in exactly the same way, every time, as well. What I found is that the order you write stuff in matters. So you want to kind of front load it with what’s really important. You might get different results if you bury something in the middle of the prompt versus at the top. So sometimes the trial and error is like the order of stuff. And then sometimes it’s even sort of sentence length, because I think it kind of reads the prompt in chunks, so it might lose a little. It might read a chunk, execute, then read another, so sometimes just varying your sentence structure can kind of really help.
“Veo 3.1, which is what I used, it will give you an :08 clip. So you write your prompt, which is usually a few paragraphs — not an insignificant amount of description — and it’ll give you a clip. And based on that :08 clip, you can figure out what it’s understanding and what it’s misunderstanding about what you’re asking...I was figuring out, ‘Oh, if I express it like this, the AI isn’t really understanding it. It’s interpreting what I’m saying this way.’ It was about finding language that was not confusing, but was clarifying. Generally, I was finding that it took me about 20 attempts to get a good shot.”
I was going to ask you how many iterations it takes to get what you are looking for?
“I would say 20. Less on a very few occasions. I would get lucky on the first, second, third time, and it would do something unexpected, as happens on-set, where you have to leave a door open. You know what you want, but then on-set, something happens that you didn’t anticipate. And that’s cool! ‘Oh, I haven’t thought of it like that. But that’s really cool. That’s a lucky accident. I like that better than what I was thinking.’ It’s very much an iterative process and it’s kind of tweaking and tweaking until it’s closer and closer. And then it becomes a case of diminishing returns, because if you put too much detail into the prompt, it starts to confuse it, honestly.”
As it goes on, does it get easier? Are you then copying and pasting what has worked for your next prompt?
“Yes. In terms of the environment, in terms of like the look, in order to get the consistency of the environment and the consistency of the character, you have to put the exact wording in. If you mix up the wording, like if I described the boy differently in one prompt versus another, I would get a different-looking boy. Things are evolving so rapidly that I worry that anything I say might be out of date. But at that point, I wasn’t using a reference image for the boy. The consistency I got with the boy isn’t perfect. To me, I can see the differences, shot by shot, but I’ve shown it to people, and said, ‘Does this look like the same kid to you, shot to shot?’ And they’re like, ‘Yeah.’ But it really isn’t.”
Did you use a photo of David to generate a younger version for the boy?
“I did not…I spoke to David and I was like, ‘Are you okay with not actually being in this video? I have an idea, which I thought we could put you in as a sort of cameo on a billboard and on a flyer.’ Stuff like that. And he was like, ‘That’s cool.’ So I used a current picture of David as an adult, and put that into the billboard, the flyer on a lamppost and then a flyer on the roadside. It appears three times because I always like three as a kind of thing.”
There are quite a few human characters in there. Were they based on real people?
“The only other character that is based on a real person is (the grandmother). I showed it to David. I said, ‘Do you have any notes? He said, ‘Well, I’ve one note.’ The grandmother character that I had originally. The woman that I had sort of crafted as the grandmother was kind of this larger-than-life, flamboyant character with big hair and big shoulders — big kind of dress. And he’s like, ‘You know, that doesn’t really remind me of my grandmother. Would you be open to trying with a real picture of my grandma?’ And I said, ‘I said if you’re open to it, I can try, but it might be a bit uncanny valley for you. I’ll do my best.’
“So he sent me a picture of his grandmother and I reworked that one shot so that it was a likeness. I wouldn’t say an exact likeness, but a likeness of his grandma in that shot. And I think that really moved him. I think those childhood road trips to Tucumcari, New Mexico, had really moved him as a child, and really stuck in his memory and really felt like seminal kind of life experiences for him. And to be able to kind of pay tribute to his grandmother in that one small way felt really meaningful to him.”
Are you keeping the aspect ratio in mind at this point, or is that determined later in the edit?
“I did that in post. What I find is, the aspect ratio, you can’t really control that. You can ask for a certain aspect ratio, and it won’t necessarily always give it to you. It’s not necessarily obedient. So sometimes you’ll get a perfect shot, but it’ll give you the wrong aspect ratio. So what I found was to just focus on getting the shot that you want and then deal with the aspect ratio yourself afterwards in post. Because it’s desert, because it is landscape and it’s epic, it felt right to sort of do it the way I did it…with the bars that I put in.”
What else did you do in terms of affecting the image?
“The other stuff I did was, I added a grain layer, and then I softened the image quite a bit, because I think the sharpness is an AI giveaway, so I just added a blur effect…to give it a softness. I did a sort of uniform color pass on everything to kind of make it all look the same, and then a softening, and then a grain layer and then the aspect ratio.”
What were you using for editing?
“I edited it in Resolve.”
What was the cost of all the AI prompting and exporting?
“I did not do a per-cost export. Now, what I will say, with the :08 sequences, is that it is very rare that the entirety of that :08 is usable. Usually, it will give itself away. You get every frame you can, and then suddenly something will happen that gives it away. Someone grows an extra finger or something weird, where you’re like, ‘Okay, that’s not usable.’ And then you’ll have to cut out at that point. Sometimes your editing decisions are sort of informed by that — usable footage.
“With Veo 3, you have a certain number of credits. I had a subscription. It’s a few hundred bucks to subscribe. It was the Google AI Ultra subscription plan. If I remember correctly, you get a certain number of credits for that subscription, which was enough for me to do this. Obviously, my time is separate, but in terms of the subscription, it was a few (hundred dollars) in actual hard costs — not my labor.”
It’s pretty incredible to think that such a creative video could be produced for that amount.
“That’s the thing. David is an independent artist, and to make this video would be millions of dollars. I mean to do everything we did in this video would be millions of dollars, and it would never have happened. So that’s the thing, and that’s how I see it as a filmmaker. It is a tool to tell a visual story. Create images that work in tandem with the music (for a) particular emotional effect. It would never have happened were it not for this technology. That’s the way I see it…I think for the independent filmmaker and the independent sort of music artists, and people like that, it’s exciting because it kind of opens up things that traditionally were only open to people with very, very large budgets, very large VFX budget and very large production budgets. So suddenly you’re able to tell those kinds of stories with those kinds of images. It’s a very complicated issue and I think we must tread carefully. But I do think the technology is here to stay. And I think it’s important for filmmakers to kind of educate themselves about it. We all will arrive at a different perspective on it.
“Obviously, I think (there’s an) artisanship of a VFX artist. I’ve worked with amazing VFX artists who have blown my mind. There’s a wonder to that process as well. And then you’re engaging with a human, and you’re bouncing ideas off each other. That’s a wonderful process. But that was on a project where there was budget to do that.
“On this project, where you’re working with an independent music artist and the budget just isn’t there, but you want to do something that sort of feels epic and feels like the enormity of the imagination is matched in the visuals, which it kind of is…There’s a sort of expansiveness to the human imagination in this story and in this song that you want the visuals to match.”
What was the timeframe to complete this project?
“It took me a good month, maybe a bit longer. Six weeks maybe. Not necessarily doing it all day every day, but chipping away at it every day I would say. I was doing it full time, I would say sort of a month for something like this.”
What are your thoughts of the results?
“As my friend said, ‘This is the worst the technology will ever be.’ Isn’t that crazy?”
How often do you get to work on music videos compared to your own work?
“Most of my work is longer-form work. I’m a feature filmmaker mostly, but what I find really wonderful about music videos and kind of short-form projects is that you get to try something new, so this is kind of a really good opportunity to try something I would never probably have tried without this excuse to try it.”
What’s next for you?
“I’m currently writing a horror film called In Bloom, which is about an algae bloom in New England. It’s a red bloom — red tide — so the ocean turns red and there are neurotoxic effects that happen because of the bloom, but also sort of veers then into horror basically. It’s an eco/horror, which I think is a genre that kind of, in literature, has been very well explored, and in film, less so. So that’s what I’m writing.
“I always do a look book for my scripts, so I’ve done a look book for this, but I’m wondering whether it would help people imagine the world? Would it helped to do a sizzle — a little, short one-minute something with AI to kind of show people what that would look like? So I’m thinking about whether I should do that as well?
“It’s a great tool for previs. It’s a great tool to pitching — for writers and directors who want to kind of pitch an idea and help people imagine what the end product is going to look like. People are increasingly risk averse and they want to know more and more about what something is going to be like at the end of the day. So it’s a good that for that. I think it’s indisputable as kind of previsualization/pitching tool. It’s a great app for filmmakers to have access to. And then I think the broader question is in terms of art made using AI. I’ve seen film festivals — programs (with) AI films — and responses have been very mixed. People have been like, ‘Why are you programming these films and taking a spot away from (humans)?’ I think that’s where the gray area is. As a film community, we have to figure that out, right?”