Phil Rice 01:00
welcome to And now for something completely machinima, a podcast about machinima, virtual production and related technologies. My name is Phil rice, and I'm here with my co hosts, Damien Valentine and Tracy Harwood, hello, hello. Ricky says, hello from Los Angeles, somewhere in Los Angeles, probably, we suspect decorating his home for Halloween, it's a pretty massive project that he does there. So this week, we're going to be talking about Tracy's pick. And I've done a little bit of homework here to help us Tracy, because the name of this film is something that the meaning and pronunciation has evaded us. It's it's called a rideo, tarajo, or tarajo, we're not really sure. So it's spelled a, r, i, d o, t, a, u, r, a, J O, so we were just talking prior to recording that. We're not sure. You know what this how this pronounced. So I leveraged Google's AI, the one that comes through Google search, and did a search for arito tarajo and I came up with some answers for us. So it turns out that arito Tara, Joe, or tarajo, is an operatic Music Video created through human AI collaboration that blends the digital world of the video game world. Oh, so it's, it's the video that we're reviewing. Yes, we already, we already knew that. And when I scroll down the rest of the page, all of the references to that phrase are to the film we're going to talk about. So it may be that this is a singular entity in the universe. When I put it into Google Translate and say, detect language, it doesn't know how to detect the language. So with that help that I've just provided, Tracy, tell us about this film that's

Tracy Harwood 02:55
really generous. Yeah, well, I'm assuming tarajo tauraggio, or however one pronounces that is actually from the World of Warcraft. World. Is it okay? Well, we'll come to that then. This is a machinima that has been directed and produced by Chantal Harvey, and it was actually released on her channel in 2023 but it was made quite a bit earlier than that, and I'll tell you about that in a minute. But by way of a little bit of background context to this, I am currently working on a case study about the contribution that Chantelle Harvey, otherwise known as mama schema, has made to creative and cultural computing history for for a European funded project. And this is a what they call a cost action called the grassroots of digital Europe, or grade for short. Now, among other things, the project is exploring the roles of women and minorities in computing history. So over the last few weeks, having got the agreement of the grade team to present this as a case for their open educational resources, I've spent some time working with Chantal to understand a bit more about her story, her background, as well as her back catalog of creative work, which many of you know, those folks in the in the world of machinima will have assumed was almost entirely focused on Second Life. Now, clearly it is not, and this is a particular example that I think is worth talking about. So as well as being a major player in the development of creative communities, particularly in Second Life, Chantal has, over the years, created something like 130 films and documentaries and. This week's Pick is one of those films where she is basically documenting and representing the work of others, but with her own unique twist on the process. And in fact, when I've written up the case study, because I'm also producing a series of short films to go with that, with the resources that I'm writing, I'll be really happy to share those with folks here, because she's got this really fascinating story to tell about how she goes about her work, and it's basically because she is a camera woman and a documentarian first and foremost. So it's a really interesting approach that she takes. So in this music video, this is actually a project that Chantelle worked on some years ago with an academic at the CADRE laboratory for new media, which is part of San Jose State University. The project was led by a guy called James Morgan, who's actually an artist that teaches new media and game studies. The short is a song an aria, which tells the story of a female warrior called Dahlia from World of Warcraft, struggling to balance her family life with the call to adventure. Now, James worked with colleagues who created a generative AI system they called robocini. Now that was a songwriting system. They trained on Puccini operas. They then collaborated with Robert Chini to create this work, which was actually translated into Italian by a guy called mattio Leva, who's a writer and director based in LA who specializes in stories that explore the kind of intersection of life and technology. Now the aria was performed by Maya Ackerman, who is, as well as being a very talented and semi professional soprano, also a leading expert on AI and computational creativity based at Santa Clara University. Now, her earlier research when she was studying for a PhD is basically reflecting her personal interest, interests which were which were kind of musically derived. While she was writing her PhD, an AI based PhD. Actually, she also learned to sing opera as a hobby. And what she was doing with this sort of earlier research was was was basically exploring how to solve a particular problem about how to turn text into melody, so you know the way the words could be sung and what have you now. She's also the CEO and founder of an organization called wave AI, which was an advanced musical AI startup in 2024 but which has since seemingly disappeared. The app she and her colleagues created was called Alicia, which basically allowed anyone to create original lyrics, melodies and vocals in minutes. And Alicia, I'll give you the link to that on the show notes, Alyssa, it stands for automated lyrical songwriting application, and it is still available to use today. The music for this was produced by somebody called Nicholas mounters. He's a computer scientist and a music tech specialist, and then obviously chantals role working with these guys was basically as a cinematographer. She did the filming and editing of the piece in World of Warcraft, and she puppeted the characters with James and another person, and then the melody itself for the song called Arida Taurajo was created by Robert Cheney. Now it was first performed in the Passio Festival at the hammer Theater in San Jose on March the eighth, 2017 2017 so it predates a lot of what we've seen over the last three or four years on generative AIS and how they've become popularized in their in you know, well, virtually, we see an update on them on a on a weekly basis at the moment. So this predates all of that, and I'll share some of the background links and information on the show notes, so you can explore a bit more about how it's come to be kind of at your leisure, really. So let me just talk a minute about the piece. The story is in typical kind of Puccini dramatic style, a tale of a woman who has to overcome some challenges now Dalia. She is a she's a guild Raider and a. Mum and she needs to get home. So the story shows her greeting Dora's and organizing a taxi home from a raid on this Windrider. She's singing as she kind of crosses the walls of the city and flies out over the fields. She's navigating the barons of Azeroth, the you know, the landscape there on her way to have dinner and to say goodnight to her child and her husband, before she then goes out raiding with her guild for the for the evening, or that's her intention. Now, as I said at the beginning, you know she's, you know she's, she's a woman on a mission, really. And she, in fact, says right at the beginning of the of the song here, I'll basically be back. It's a, it's a film that's been inspired by the fishmonger's Tale, which is a story, really, about local people and life. And obviously the local in this context is the World of Warcraft. I think there are some amazing lyrics in this, and I love the fact that it mentions Chuck Norris, which I've got to say, was totally unexpected the first time i i watched this. Now, the words are actually really quite central to this. The music and the rhythm of the wings of the rider are beautifully synced and edited. I think it's a really poignant piece, because I, in my view, it really turns World of Warcraft into something quite magical and far more than something fantastical, which is, of course, what it's known for, the scene at the end, and I think in fact, all of the shot choices portray really well what's going on with Dalia, speaking of the love for her home life and her conflicting passion for raiding. Thankfully, it doesn't end in tragedy such as her sort of dying on Route, as one might really expect, if it were to have been based on some of the other composers, other than Puccini I suspect, and certainly it conveys very well the sense of high stakes emotional drama and these kind of grand passions that are really typical of Opera's high art form. So much so that I really love to see this as a full performance. I think that would be a very interesting development of this work. But when I was talking to Chantal about this, it's it's one of the works that she's super proud of, not least because of the extent of the collaboration that was put in place to create it in the first place. And of course, as part of she's basically a second life expert. She quite openly tell you she doesn't have a lot of experience of World of Warcraft.

Tracy Harwood 12:54
And she would actually tell you that the hardest part about about making this was getting to the point in the game, which I think she said was something like, it's level 70, I think, I don't know, but that's the level that they actually wanted to do the filming at in order to get the scenes that they wanted. So she says, after about three weeks of trying to advance her character and getting continually killed, she sort of realized that she was just never going to reach the level she needed in order to sort of make this film. So in the end, she had to say to them, Look, you're going to have to provide me with a character and a login that's going to let me do the film, because I can't do it. So she was, you know, quite open about her World of Warcraft skills, which I think is very interesting, but the way she's created it, and the way, you know, there's the shots that she's taken, I think, are really amazing. Now, I think why this is interesting is because over the last few months, we've talked an awful lot about AI and creative practice, and this one, I think, is quite timely in in the sort of flow of the discussions that we've been having, not least because it is a machinima. And a lot of the ones that we've been looking at are what I would call my cinema, using the the general generative tools themselves, as a playful platform. Well, this one isn't doing that. It's using the platform as a truly creative process, but illustrating it with game footage that's puppeted to match the story that's being told. It's evidence of some very talented folks coming together, not a single person like we see with neural physics work. Of course, I think one of the hardest parts about producing something like this is keeping it going. You know, this is a, this is basically a one shot piece. It's a it's one ARIA produced, not. Whole opera. And that's not quite the same as what neural visits doing is that that workflow has allowed him to continue to create and build on the law and tell more story. This process isn't that at all, and I think essentially what it is is an academic exercise. But I'm bringing it here because I think it's so much more than an academic exercise. It's also a perfect illustration of Creative Technologies, practice and collaboration, and frankly, my congratulations to all of the team involved in this. It's it's really stunning. I really enjoyed it. I have no idea what you guys will think to opera, but over to you guys.

Damien Valentine 15:44
So I'm going to talk about the title first. I was just trying to do a little bit of digging while you were talking there, just to try and confirm my theory. So the main character is a cow, like being and in the game, those are called Torrens. Ah, okay, so I was trying to work out if that's the Italian the title tereso, or tereso is the Italian version of that. But I couldn't find out, because all the websites coming up about, well, the Warcraft are all in English. So that wasn't exactly but in the credits, there is a part about Italian translation, and someone did that. So I wasn't sure if they translated the name, or if they just put some kind of spin on the name. Touring for the, you know, the main Well, I've done, yeah, yeah. So we'll have to do a bit more digging there, or maybe listening to this, you can give us the answer. Alright, so this is one of the best looking World of Warcraft films that I've seen, because a lot of the ones we've talked about before, they're fairly old, so they're using older versions of the game with on computers that couldn't run the game at this kind of high level graphic settings. And obviously the game has had graphical upgrades throughout its very long life. So yeah, definitely looked one of the most visually stunning. Well, the Warcraft games we've had. I was looking at the description of how the song was created. And so this, this film is five years ago. So this is before the AI craze really took off.

Tracy Harwood 17:25
I think it's closer to seven years ago, eight years ago, okay,

Damien Valentine 17:28
so long before anyone really thought about AI for any kind of generative use the way we see now. So it must be one of the first in the world to have used AI this way that, because now you go online, there's lots of AI videos, but yes, it'll be the first, but it still performs the song written by written by AI, but it's still performed by a real person. And obviously the video itself was made by real people. It's not AI generated because that technology didn't exist seven or eight years ago or even five years ago when it was posted. So it's an impressive feat of combining those two different types of creating content. You've got the one for the song, and then the real people making all happen. I think so before, I don't really enjoy AI content so much because there's just so much of it now. But I did enjoy this because I could appreciate the its uniqueness for when it was made and the way it was put together. You can read the credits, and you see there's a lengthy credit sequence of all the different people who worked on it and explaining what they did. So I really enjoyed this film. I'm glad you got you to choose it for us. Tracy, I'm kind of intrigued now about the title. I got to keep digging into that, because I want to know what the answer is so but yeah, I really enjoyed this. So thank you.

Phil Rice 19:00
Well, I did a little more digging while you guys were talking and found out maybe some more information here. So Okay, first of all, in one ear, I I found a spot in the lyrics where she sang that word tarahu, tarahu, and that's how she pronounces it when she's singing. It is tarajo. So it's a J like is as in soft Spanish. Yeah, Spanish language, right? And here's what the here's why Google Translate had a little trouble with it. So ARIDO, the first word that is an Italian word, you know the English equivalents arid, right? So it has to do with dry torajo is, according to what I found, it's a Torin outpost, just like Damien mentioned in the southern barons of World of Warcraft in the game's lore, the camp was destroyed by the Alliance during the cataclysm. Expo. Mansion as someone who's not played the game, I have no idea what that means, but if you've played it, you probably go, oh yeah, you know. But anyway, so that's where are the that's why Google Translate was, was stumbling, is because it's a place. It's basically two, yeah, it's, it's an Italian word and then a proper name that apparently is pretty unique to to World of Warcraft. So it's not directly derivative from from somewhere else, which is kind of interesting.

Tracy Harwood 20:32
Torin, right? It's a cow, isn't it? Taurus?

Phil Rice 20:35
Well, yeah, Torah, yeah, absolutely. That's that is derivative. That the the prefix part of the word Yeah, the place name is tarajo is apparently a unique spin on that, so at least according to what search is turning up. So yeah, Damien, the first thing that I noticed, too was the visual fidelity. There's a crispness to it, because so much of the World of Warcraft. Content that we've been exposed to over the years is from 10 plus years ago, right, sometimes closer to 20. So yeah, it's, it's, it's definitely got a very fresh and crisp look. It's kind of premium quality visuals, which is nice. I happen to think that you mentioned, is it Chantal that actually did the World of Warcraft? Filming, yes, filming, yeah, yes. And, and so you mentioned that she, she said that she doesn't have much experience with it prior to this, I think that's a plus, probably, because that means that she wasn't steeped in the conventions and the ways of, you know, there's, there's a way that World of Warcraft films are made and look, right, yeah, and, and if you were part of that community during the heyday, it's almost inescapable, right? You're going to create within that vibe and that feel. And she, I think it's a plus that she had no real knowledge of that. So she approached this very freshly, and as a result, you get this sequence here, which I think it's the most serene thing I've ever seen come out of this game, like it is just totally, a totally different feeling than any other World of Warcraft thing that I've seen. Now, I've not seen them all. Obviously. I don't know if anyone has seen all of them. There's so many, but yeah, it doesn't feel like a World of Warcraft film, even though it it has the look, it has the esthetic, the characters still have the same motions and all that, but it just, even without the the effect of the music, which I'm sure plays into that as well, that's that's a pretty unique sound juxtaposed with those visuals. But even so, it's just there's a real serenity to it, that that I just haven't seen from this game before. And that's, that's, that's what she brought to it, or was able to bring out of it, which is great. So, yeah, let's talk about the music, because that's really the that's, that's the centerpiece here. Maya's voice is just wonderful. I can't believe she's not full Pro. You said she's like, semi, pro, Semi. Great voice, just gorgeous. The lyrics and the music part is, as a musician, it's confusing to me, because it's like, there's kind of a chicken and the egg scenario there, of which came first. Then, you know, like they've got this robocini that that can generate a melody in the style of Puccini trained on Puccini's approach, right? So I assume that the melody came first, but then you've got to have syllable by syllable, line up with those notes. And typically, when you're writing a song, it's the lyric comes first, and the melody gets paired to the Lyric, or they're written simultaneously. You know that the way a turn of phrase happens the melody will be in this case, the robocini had no, from what I understand, had no concept of what lyric was going to be there. It was just focused on melody. So how in the world did they pair up Latin, some syllables with that and have it make sense? So that's there's more to that process than they probably can convey in a short little description, like, like they did Yes, yes, but it's a fascinating there definitely had to be some Interplay there, like it hasn't been written in. It wasn't, yeah, it wasn't just that this melody was spit out and then we just threw some words there and they fit. No, that's not how songwriting works. It's challenging to get the melody. In the lyric to be married, you know, it's, it's, it's, it's a real skill. So I think that there was a really interesting interplay there. And again, it's between human and AI, right? So that is, that is intriguing, and it's funny how we have to look into the past to see an example of what we both envision is the best future use of this tech? Yes, right? Yes. We have to look backward to find it right now. The craze is type in some words and just let AI spit something out. But this, this is interesting and has real promise and and it even does it ethically. Yes, you know the ethical issues we've talked about with AI, the fact that this was specifically trained on just a very specific set of things, this is what, this is the kind of use of AI that real illusion is encouraging for use with their AI rendering that you actually make your own data set to keep character consistency visually. It's the same kind of concept here. It's not, you know, just making a big land grab from the world of content. It's, you know, it's a carefully curated set of legally and ethically acquired content. So that's encouraging to see that that was going on this many years ago.

Tracy Harwood 26:21
Alicia is still available. So okay, so if you get a chance to have a play, I might fiddle around with that. If you can come up with something. I don't know whether it's that won't necessarily be trained on the opera style, but I bet one of the styles that you could apply to it would be opera. So if it can create lyrics and melodies, I would imagine that it's able to, yeah, I mean, it's just well

Phil Rice 26:46
for me, I would, I would probably be as self centered as my first inclination would be to train it on my own music, yeah, and see what happens, you know. So I'm sure that that's, that's an interesting thing. But yeah, I think, I think the way it goes is Alicia is the the base platform that does the computation. Robuccini is their implementation of it based on the training on the Puccini stuff,

Tracy Harwood 27:10
I think so that's how I read it as well. Yeah, yeah.

Phil Rice 27:13
So, yeah. I mean, I've got a lot of positive stuff to say about this. Short, the closest to negative that I'll say is probably going to come as a surprise, given what I've said so far, but the part that I felt like was the weakest of this was the melody. And it's not that that it doesn't sound vaguely Puccini ish, it does, but an aria, typically, the music, this is so hard to can. It goes somewhere, you know, and and emotionally, it builds and it rises and falls, and it's, it, there's, there's heart swelling moments to it. And this was like, Ah, good point emotionally. It was on Zoloft or something, you know, yeah, like, just no big peaks and no big troughs at all, just all kind of right there in the middle and again, nothing wrong with the notes. They all do fit, but the melody lacks heart, and that's that's kind of disguised a little bit, because Maya sings it with heart, but the melody doesn't give her a place to go that really would have more emotional impact. That's not a criticism of anybody involved with this. It's, I think it just highlights one of the challenges, one of the the limits that AI generated stuff has, is it AI doesn't even know what we're talking about right now. You know, I mean, some people don't even know what we're talking about right now, but AI definitely doesn't. Has no concept. What do you mean? Should I get louder or something? Am I so saying higher? It can only think in those mathematical terms, right? But a Puccini opera, I mean, it'll, it'll, if, if you're really in the moment and taking it in, it'll, it'll make you tear up. I mean, it's, it's, it's really heartfelt, and the lyric seems to have elements of that human emotion, right? But it's, again, it's Zoloft. It's just flat, kind of just meandering around this middle ground emotionally and never really having a sense of direction. And you can it's especially noticeable, I think, at the end of the song, at the end of the melody, part of the song, it just kind of just stops, almost like it's mid sentence, like the melody does not the lyrics, but the melody just, kind of it starts right here and then just ends right here, and there's no Coda at the end, there's no intro, there's no build, and then a rise and fall and things like that, which is. Just, that's just a fact of you know AI, the best that even today's IA can do is imitate that, but it can't really craft it. Maybe, Nobody's really sure why. But anyway, so that was, that was the part that was. It wasn't disappointing to me, but it was noticeable because I create music, I think is probably but my my antenna is a little bit more more apt to tune in on something like that, but it's a wonderful achievement. And again, if it weren't for audio wise, if it weren't for Maya's voice, it would have been, it would have been a really disappointing thing. Audio wise, oh yes, she was just magnificent. And I kind of look at it as like, if you take a you take Meryl Streep, and you give her a not very good script, it's still going to be a pretty damn good performance, but it's not going to be her best, right? That's kind of how I feel. Like the relationship is here between that melody and the singer is that she really put her all into it, and that saves this, the audio portion this video. Visual wise, there was nothing to say. Chantal did a beautiful job. Again, it's, it's serene in a way that I've not seen from this engine before. But yeah, so that's, that's kind of what jumped out to me. Melody wise, great pick. Very thought provoking, and the tech is interesting. I'm gonna, I'm gonna investigate it, just to see what makes it tick.

Tracy Harwood 31:44
Yeah, really astute comments. Phil, really thank you.

Damien Valentine 31:50
Yeah, not having much of a obviously, I like listening to music, but I have, I'm completely clueless at playing instruments or composing or anything like that. So that went way over my head. I understood what you were talking you explained it really well, you know, I didn't think about it that when I was watching the view, watching the film, to think about it, because I had no idea. So, you know, picking up the Yeah, there was something like you said the heart. I didn't really know how to explain it, but I could feel like there was something missing, which is, I think that's a fairly common thing with any AI generated content. Obviously, Maya brought this to life in a very good way, but someone didn't have a voice or talent wouldn't dare to do that, and I hate to remind you to have the this particular video, but that Batman film we talked about where he used AI to write the script, yeah, and then AI to perform the voices, right? You could text text to voice, yeah. And that is like the complete opposite scale of how that could be, yeah.

Phil Rice 33:02
Yeah. Yeah, yeah. It's, it's, I think what gives me a little bit of comfort there is, I think that anybody that was involved with this musically, I don't think there's any way that they were not aware of what I observed, like, of the reality of that, but in similar what like, I ran into some similar stuff when I was working on that, Dr CRI, our neural viz, parallel experiment, right? And there were points in that I've made a documentary on that. If you're interested, we'll link to it, the description about the making of that, and I talk about some of this in more detail, but there were points where it's where I was kind of my inclination was to step outside of the process that I'd committed to because I knew I could get a better result or what I really wanted to happen if I didn't have aI who I described as the employee Who doesn't do what you tell them to do. That's kind of what the experience of working with AI was for that film. Is that I'd hired someone, I'm paying them, and about 75% of time they don't really listen to what I say. They just do what they're going to do. So there's an element with AI to that, right? And I think that the musicians involved with this probably knew that, but I think that in a similar way, they were committed to the experiment. Let's see this through. Let's make the most of this that we can. And I'm sure that anybody who had a compositional bone in their body while working on this were probably tempted to go, you know, if we just changed what rubaccini gave us just a little bit here. So in a way, it's to their credit that they didn't cop out and do that, because the point of this was the experiment, right? And I think it works really well as one. So yeah, great pick Tracy. I'm glad, I'm glad you brought this to us. I I would have never heard of this one, not just because I didn't know what tarajo was, but.

Tracy Harwood 35:01
Yeah, yes. So

Phil Rice 35:02
audience members, what did you think? Let us know in the comments on this video or audio, or wherever you're listening or watching, or drop us a comment at talk, at completely machinima.com We'd love to hear what you think about all of this. Am I just full of hot air? Am I being snooty about the music? What's going on with that? Let us know. So my name is Phil, and on behalf of my co hosts, Tracy and Damien, we bid you a good day and we'll see you next

35:29
time bye, bye. You.