Subscribe: 🎥 YouTube | 🎧 Apple Podcasts & Spotify
Welcome to the Urgent Futures podcast! This podcast tries to clarify the chaos, from culture to the cosmos. Each episode, I sit down with leading thinkers for big idea dialogues about the research, concepts, and questions that animate their approaches to reality.
Today my guest is Asad J. Malk, CEO of Jadu AR.
Asad is an augmented reality trailblazer whose critically-acclaimed narrative storytelling projects Terminal 3 and A Jester’s Tale premiered at Tribeca and Sundance Film Festivals, positioning Asad as a visionary in the space before completing his undergraduate degree. He has directed next-generation AR experiences featuring icons like Serena Williams and Lil Nas X. Asad was named one of Variety’s 10 Innovators to Watch, Rolling Stone’s Future 25, Forbes’ 30 Under 30, and Adweek’s Young Influentials.
Read on for the episode transcript (edited and condensed for clarity).
Ever since Minority Report and Iron Man in the aughts, augmented reality has been touted as being the next big thing just around the corner. But the reality of bringing the medium to the mainstream has proven much more difficult than expected. Looking at you, Google Glass.
True, Pokémon GO made a big splash, but that was 7 years ago. And their developer hasn’t been able to duplicate the success since—even partnered with major names like Harry Potter and the NBA. They flat-out canceled projects with Transformers and Marvel. I don’t mean this as a knock to Niantic—they’re doing important work in the development of the AR ecosystem. All I’m trying to say is that AR is hard.
That’s obviously true on a technical level—think about the amount of computing power and bandwidth required to translate digital objects and information seamlessly, instantaneously, and believably into your physical space. Lots of folks are still plugging away at those problems, making AR faster, leaner, less likely to burn your face, etc. But in my mind, the bigger challenge to AR is a social one. AR is a whole new medium. It’s not just another platform or set of apps. When done well, it has the potential to become a new language, redefining our relationship to both digital and physical space.
This is something that Asad has really understood since he started developing AR experiences in college in the mid-2010s. He has seemingly always had an intuition for what the innate potential of the medium is, alongside a willingness to pivot in totally new directions to learn from what participants, players, and the public at large want to experience in AR. The advantage startups have over major incumbents like Niantic, Apple, Google, and Meta is speed. Asad has made use of this advantage, steering Jadu through different expressions of AR: holograms for social media videos, web3 integrations, and now a fighting game. And that’s not even mentioning the original work he did as a director focusing on AR headsets, creating era-defining installations at film festivals. On paper it sounds like a weird trajectory, but having witnessed its evolution, I can say that it makes sense in the social context of the medium.
We recorded the following interview in Asad’s office right before the game launched, but Jadu is officially out now, and already cracked the top 50 on the entertainment chart on the App Store. This feels to me like an early confirmation that the public is still interested in AR. They just need an experience that works, where the fun is not that it’s a new shiny gadget but rather in that it’s just fun on its own terms and offers something unique.
My thesis, which I’ve developed in part by observing the work of Asad and Jadu, is that the future of AR will be formed by how people use it. I know it might seem weird to talk about a fighting game as a medium-defining moment, but I see it as having the potential to inform the early interaction mechanics of AR on a public scale, and more broadly the expectations for how AR fits into our daily lives. Obviously the Apple Vision Pro is going to play a big part in the development of AR—as is the Meta Quest 3, and a host of other notable devices. But we’re still a long way away from everybody feeling comfortable donning headsets and other doodads. In the meantime, the language for how people use AR is emerging organically on mobile games like Jadu.
So this is why I was super excited to get into it with Asad. We look back on his early experiences, reflections on the early days of AR, the formation of Jadu, his thoughts on the state of the medium, and much more.
Find the episode transcript below (edited for length and clarity):
Jesse Damiani: Asad, thank you for being here.
Asad J. Malik: Thank you, Jesse. Here we are.
JD: Here we are. So as I said in the intro, you have a storied career in augmented reality. Tell me a little bit about what first attracted you to the medium.
AJM: So, I'd been working in technology since I was a teenager in Pakistan. Initially I was just making a lot of websites and trying to get paid here and there for some search engine optimization and whatnot. After doing that for six or seven years, when it was time to go to college, I decided to go to an art school instead of an engineering-oriented school. I went to Bennington, where I realized that not only did I want to work in technology, but I wanted to do work that was expressive and creative in nature.
The three mediums that were up on the table that I thought would be significant in my lifetime were AI, VR, and AR—and AR was the one that really caught my imagination. On the one hand, VR already had a lot of development that was going on into it. It was already the ‘empathy machine’ and Oculus was already a big deal.
But AR as a medium of expression was very underutilized. People were doing enterprise-oriented things, or warehouse-oriented things, but there wasn't really much happening with AR as an expressive medium. And I thought that augmenting your reality and changing the context of your surroundings with digital objects was just really compelling.
So I went ahead and I got myself a Microsoft HoloLens developer kit while in college, and...just dove into it, and initially it was just a bunch of one-off, concept-oriented projects where, you couldn't really use them, but we would make videos and put them online, and those videos really did well, and were spread around, and it became clear to me that people were very interested in what is possible with this medium. We just doubled down from there.
JD: What do you feel like people were attracted to even at that time when you were sharing these videos?
AJM: On a very basic level, just the idea of seeing 3D objects in the real world—the physical lives we're used to connected to our digital lives—is a really compelling idea.
Without getting too conceptual with it, just purely seeing a hologram of something in your space is really strong. I think people want to imagine exciting things in the future and this starts to paint a picture of what the world could look like once some of the world that we're building digitally starts to interact with the world we know physically.
JD: Your breakout project, Terminal 3, made the rounds in the festival circuit [in 2018]. Could you share a little bit about what that project was?
AJM: The origin of Terminal 3 was: one of my professors at Bennington encouraged me to bring my own story and background into my work. We were doing work that's interesting from a technical perspective; it's a new medium, there are a lot of new things that you're trying to discover in it, but in order to give it compelling content, he thought that my own life story, which was me growing up in small town, Pakistan—Osama bin Laden was killed in the town I was born in. Moving to the U.S., I went through a lot of interrogations, the FBI interrogated me once; they actually showed up to my school in Vermont and targeted me, which was definitely pivotal. I thought that was really compelling subject matter for a technology like AR that is so deeply rooted in scanning technologies, which also is something that is used in the military a lot.
I thought that was an interesting parallel, so Terminal 3 was a project in which the viewer would interrogate various holograms of supposedly Muslim passengers in a secondary screening room in an airport. We premiered the project at the Tribeca Film Festival in 2018. Viewers would walk into an interrogation room that we'd built and they would put on the Microsoft HoloLens and meet these holograms of real people that we had interrogated. And you would play the role of a customs officer actually asking questions and going through that narrative with them.
JD: I recognize this may be sensitive, but are you able to share what the FBI interrogation was about?
AJM: I went to a high school in the Netherlands, which was really a life-changing moment for me, getting a scholarship to go to the Netherlands and study at UWC [United World College], which is an international high school where they have kids from 90 different nationalities, all 16-year-olds who have left their country to go to school together in a boarding school. We had people from royal families and we had refugees all in the same school, and so because of that I had friends in Libya who were involved in the revolution when Muammar Gaddafi was overthrown.
What we realized was in order for new mediums to develop you need to build for audiences.
So I found myself in an interesting place after arriving at my college in the U.S., where I helped them get a license for a conference to be held in Libya, and they invited me to speak. I wanted to go see what a country post-war looks like. So I went and stayed in Libya for two months, and when I returned—this was in 2016—unsurprisingly, the FBI wanted to speak to me because obviously a Pakistani kid coming to the U.S., staying here for three months and then suddenly going and living two hours away from an ISIS training camp, you know, triggers some some alarms. That's why the FBI came to Bennington to interrogate me.
JD: You, you brought up this episode that I was actually totally unaware of from your life of going to high school in the Netherlands with all these different people from all over the world. Did that factor in at all to what led you into augmented reality?
AJM: Absolutely. Not many people get an experience like that, where there is no dominant culture. Because every country you live in, there is some dominant culture, right? Like if you're in Pakistan, you're involved with a lot of Pakistani culture. And in the U.S. that's very much the case because, you know, people have strong political opinions here and people assume a lot about each other's identities based on one thing they know about them. So going to a high school like this where there were kids from every different country, you saw various cultures emerge and various aspects of life. So for example, a lot of people would use Arabic slang, even people from other countries would use Arabic slang in my high school because that was the slang that somehow rose to the top. And that was the case with a lot of other aspects of life there; it just creates a really global perspective.
I knew that I had to be involved in technology with that kind of global mindset, and make sure that that comes through in the work that we're doing. So a lot of the early projects being political in nature were tied to the fact that I went to a school that was also very political in nature.
JD: So Terminal 3 was definitely a social impact-oriented project. Can you situate us in what the VR/AR industry or scene felt like at that time and what the conversations were then? Cause I think it's an important preface that we should set for the audience.
AJM: Yeah, it was an interesting scene. This was when there were a lot of 360° videos in VR. A lot of them were very empathy driven. Chris milk had coined the phrase ‘empathy machine.’ And there were a bunch of projects that people were recording in refugee camps in Syria and, you know, the earthquake in Haiti.
And so that was definitely the predominant idea floating around in VR. There was fiction stuff as well, very Pixar-esque animation, 10-minute long projects. But there wasn't really anything of that nature happening in AR. The reason I was interested in tackling similar subject matter, but in AR, was because I felt like VR just felt so removed from your immediate reality. It just felt like a lot of people would have this notion of, “Hey, I'm putting this headset on and I'm in a refugee camp in Syria now. And when I take it off, I'm in a film festival in New York in a very liberal, progressive, privileged setting.” And you would have this kind of delusional sense of moral superiority that you now know how it feels to be in the shoes of someone. I don't think there's anything wrong with that per se, but I thought that the opposite, where someone comes to you was more compelling. So you're in your space, you're grounded in your reality. Nothing else is changing about it, but now a foreign presence is in your space, and you have to address that and deal with that and face the challenges that come with it. I thought that was a more interesting frame for similar subject matter. And it suited the medium really well.
JD: And then the next year, you had A Jester's Tale, which—insofar as all creative acts are political—was not specifically directly political the way Terminal 3 was. I think a lot of people felt like you were staying within AR and you were still at the edge of what AR could do, but in terms of content, it was extremely different. What was your decision making in terms of jumping from the more social impact-oriented, representation-oriented work—that thread you're describing in the VR community, the empathy machine—and then going to this project that's really dark and gritty and experimental and strange.
AJM: I felt like I had said what I wanted to say about group identity-related stuff. Like, I'm from Pakistan, this is my background. Here are things you can confront that I can give you, with this frame of it being a values-driven project. But eventually I just wanted to make work in AR, and I think any work I make in AR is inherently political because I'm from a very different background at the end of the day, transplanted in a very different culture. It doesn't have to be inherently that; it just is that by nature. With A Jester’s Tale, we just wanted to look into the technology itself, and the story that we made was really bizarre and abstract, which is how I think new mediums should emerge, and it was also somewhat of a response to what was generally happening on the Magic Leap and on the HoloLens and also on a lot of VR headsets at the time. Work that was incredibly clean, family friendly, very Pixar-esque. That's what people thought was the content that was necessary to get the medium to the popular consciousness. And I didn't think that was the case.
I thought that early mediums should be messed around with, that we should do random things and take bold steps with it, try to do nonlinear weird stuff that doesn't work in existing mediums. That's how you test the boundaries of a new medium. So that's what we were attempting to do with A Jester's Tale.
It wasn't so easy to sell, honestly. It wasn't so easy to understand as a concept. I think that is what made the project more compelling. And the reason I could do that was because Terminal 3 had been successful enough where we had raised money for a second project without the financiers caring about what the project or the content of it was. I could just take more bold steps with it. So it was a weird piece, I still think about it sometimes. We did bizarre things in it.
JD: Can you share a little bit about it?
AJM: It was framed kind of as a reverse Turing test or a reverse CAPTCHA almost, so you start the experience and it's marketed as a bedtime story, but when you start it, it gives you a CAPTCHA test. So it says, you know, “Say you're not a robot.” We used voice recognition to trigger that pop up and then it says use your thumb, say you're not a robot again with your thumb. And then the CAPTCHA test just never ends. It keeps becoming more and more bizarre, and suddenly these holograms show up, and they're trying to manipulate you emotionally, there's this kid who's sitting on a bed with a bunch of rats. We had a physical bed that the kid's hologram was on, and you're talking to the kid, and suddenly the kid becomes the CAPTCHA test, and so it's switching between these really bizarre scenes, and towards the end, we were doing this thing where the real kid who's hologram we have made would be sitting inside the physical wall. We would open this panel and there was a cage inside the wall in which the kid was sitting. And the last moment was you confront the physical child and you're still looking through the AR headset. So for a split second you're not sure if it's a hologram or a physical child—or at least, that was the concept.
In order to do that, just the logistics were [tough]. We had to look into child labor laws in Utah, and had to get three doppleganger kids and get them the same haircuts that were rotating their shifts sitting in the wall. It was a really bizarre thing and honestly, Sundance, shout out to them for taking the risk because the project wasn't done til the day the festival started. In fact, we missed press hours because it wasn't ready. It wasn't working. We were staying up all night behind the booth, trying to just make sure things worked. It was a bizarre attempt, and it was done really quickly. So, it didn't work a lot of the time, but when it did, I think it really did open people's imagination of what is possible in the medium. And the Magic Leap had just come out, so we got tons of press and critical acclaim out of that project. All that was only displayed once.
JD: I also remember at the time you were talking about—I think this was 2019—why Billie Eilish was so popular among young people. There's this honesty about the darkness in the zeitgeist, which I see in what you're building in Jadu as well. But even thinking back to that time. You would see horror pieces that were outright meant to fit into an existing lineage of horror, but you were actually leaning into the weird with immersive capacities in ways that mirrored what was happening in culture, which wasn't being talked about in XR.
AJM: Yeah, it's really—as I was saying earlier—a response to the fact that everything that was happening in the medium was so clean cut. There was this clear distinction between people that made content and people that make technology, and it felt like a lot of the content that was being built was being built by people that make technology and the people that make content were stuck making 360 videos or other forms of immersive that were less technically challenging to produce.
Between those three buckets, I think AR on mobile developed a stigma where people feel like mobile AR can never be compelling enough and headsets have to be popularized before AR becomes like a thing. I don't believe that's the case. I think mobile AR should have another large moment before headsets.
We thought that we were right at the center of it all because we were leading with the technology. We were developing ourselves and we were downloading the SDKs and messing around with things and trying to see what the possibilities are and just constantly trying to overdo it, being like, “Let's also add in voice, let's also add in gestures. Let's add in as much as we can and then tie it up with this conceptual frame and the things we want we wanted to say.” We weren't so clean cut and organized, honestly, even in terms of storytelling, we didn't have the capability to build a lot of great 3D models and we didn't have the resources to hire out. So we were relying a lot on capture, which also was very glitchy and dirty and dark just by nature of it. We were just leaning in on it and also leaning in to our own influences and that was the result. The nice thing is we were in college, we were like 19, 20, so we didn't have to cosplay as young people trying to make work that relates to young people; we were just doing things that we found interesting. Which was really rare at the time.
JD: Well, that brings us to Jadu because there's this, there's this transition moment with the company. I've heard you talk elsewhere about how Jadu was always this germ in your mind, but at the time 1RIC was the studio that you were operating under. And then there's this handoff where Jadu becomes the main focus.
AJM: Yeah. The word Jadu means magic and when I started working in AR, it was early enough that I felt like I could be as ambitious as possible and I wanted to build whatever the predominant content platform would be for AR. Now, I didn’t know what that meant or what shape that would take, but as a concept it existed and I was scribbling the logo in my notebook for years before it actually started to happen.
With 1RIC, we took the form of a studio. And the reason for that was because, first of all, me and Jack Gerrard and a couple other people who were working with us, we were in college. We didn't feel like we were ready to take on something that was substantial and a product, something that would take years to develop fully. We wanted to do a lot of shorter sprints on three-month production timelines for a project that goes to a festival, push ourselves try to do a lot in a short amount of time, get some gratification out of a release, and just iterate on a couple cycles of that.
When finally we graduated, it made sense that it was time to now take on a fuller project that we could realize. Some of the factors that went into Jadu starting to take the form that it was taking were: 1) we had realized that we couldn't work on headsets anymore, which was a real shame because the whole thing with AR for us was headsets. We thought AR headsets were incredibly compelling and we thought that using a flat rectangle on your palm is not how people will interact with technology in the longterm.
JD: Do you still believe that?
AJM: Yeah, I would say so. In the longterm. However, right now, phones are how we interact. It's a medium that has matured a lot. All iPhones look the same now. People complain about it; I love it. I think it's like it's been perfected. You can add incremental adjustments to it, but it's really in a good place right now. Headsets are going to take some time.
What we realized was in order for new mediums to develop you need to build for audiences. You can't just work in a lab on the best form of the technology forever. You'd rather work with a good enough version of the technology, but at least have an audience that's interacting with it that gets you further.
So that was one of the key theses with Jadu. And the other thing that we were realizing was that The HoloLens and Microsoft honestly did not care about content and creativity and entertainment. They had some pieces they would showcase at events, but realistically they didn't care about it because they treat it as an enterprise product.
So when the Magic Leap came out, I was very excited because their whole thing was, this is a creator platform. So we were creators and we built something on it really quickly. We were mid-production when we pivoted to the Magic Leap. It was really challenging. A lot of things that were necessary to make A Jester’s Tale didn't exist yet. They couldn't play volumetric footage, for example, we had to commission a custom plugin and do a lot of things to get it to work, but we did it because we wanted to make a mark on the headset that was for creatives. And we did, we launched a project, got into Sundance, got great press—got better press than a lot of Magic Leap projects that were at Sundance—and they didn't end up putting it on their platform.
We were in conversation with them for quite some time, it went up to the CEO. And they decided not to. The reasoning I got was because the subject matter was too niche or not family friendly enough, something along those lines. And I realized that we needed more control over how we publish stuff. We can't publish on these emerging platforms that are not ready to take risks with content. So we wanted to build our own platform on mobile, and we wanted to do a project that we could iterate on for long periods of time, because doing something for three months, launching it, and then knowing the shortcomings but not really having any financial or other motivations to fix them was just not the right setup. We knew iteration was necessary. So that was the founding of Jadu.
JD: You're causing me to think about one idiosyncratic piece of the XR puzzle earlier on, which is that there was a lot of energy around it because a lot of big companies wanted to not miss this wave the way they had missed the mobile wave. But that also meant that there was very little indie participation that wasn't in some way funded by those big companies—or smaller big companies, studios and things like that. Nobody was outright censoring content, but I think there were ways in which it had a quieting effect on certain types of content that didn't fit in comfortable buckets.
AJM: Absolutely. And I think you can see it very clearly if you look at the content that came out on VR headsets during 2017, 18, and 19. You can see, “Okay, this is the era when Verizon is funding. This is the era when Oculus is funding stuff.” You know what I mean? You can see content formats and subject matter shift around based on what kind of funding was going into the space. And AR was getting none of it anyway, for the most part. Obviously we had RYOT and Jake (who ended up with us), they took a bet on us, which was awesome. We got to make the early experimental crazy stuff that we got to build. But another thing that was happening with AR: there was indie participation, but it was on face filters, like there was tons of interest from creatives building face filters for Instagram, TikTok, and Snapchat. That has been the case for the last five, six, seven years now.
We've always done the best we can with the resources we have at our disposal, while keeping the core the same, which is we're facing AR and we're creating new interactions in AR, ideally for a large number of people, and ideally in a way that builds on top of itself versus restarting every three months. That's the foundation.
Personally, I think that it's been somewhat detrimental—not entirely detrimental—to the space because indie creatives who could have worked with more fleshed out game engines and produced stuff that was really compelling, which they would own, ended up instead building filters for these platforms using tools that were really simplified in exchange for followers. There was a time period where you could make filters on Instagram and get hundreds of thousands of followers because there weren't that many filters on Instagram and your name would pop up next to the filter and people would go to follow you. So many people went in on that wave, but I think that they were distracted from the possibilities of working on Unreal or Unity and actually building more fleshed-out AR products.
JD: Do you think that had an impact? That era of filters being the main access point for augmented reality? Do you think it impacted what the public thought augmented augmented reality was?
AJM: For sure. I think it has had a similar effect to what Google Cardboard and 360 video did for VR. People had heard about what VR was, but they never experienced it til the New York Times shipped a bunch of Google Cardboards everywhere. And then people were plugging in their phones into a cardboard box, watching 360 videos that didn't have six degrees of freedom (6DoF). They were in a sphere and they were then getting motion sick and this and that. That just put a taste in people's mouth of what VR is supposed to be, which wasn't very accurate, based on what actual 6DoF VR can do.
I think a similar thing has happened with AR. A combination of face filters and Pokémon GO. Face filters are not designed for you to have a spatial experience. They're designed for you to record a 2D video. And Pokémon GO where, you know, it was popularized as an AR game, but the predominant mechanic that you use in the game is geolocation. The actual AR was quite limited. You're throwing a ball and trying to catch your Pokémon. It felt relatively flat when it first came out. And that was how a lot of people experienced AR.
And then the third bucket was basically marketing activations on the web, a lot of AR marketing projects that were also really quickly made, pushed out by agencies, with no iteration—just whatever random novel mechanic that came up to ship it. Between those three buckets, I think AR on mobile developed a stigma where people feel like mobile AR can never be compelling enough and headsets have to be popularized before AR becomes like a thing. I don't believe that's the case. I think mobile AR should have another large moment before headsets. Headsets are still a bit far out.
JD: You've talked about this idea of ‘ghostly presence,’ and I think that one of your approaches as an artist and storyteller that I'm seeing translate into your work in Jadu is really activating the space that you're in now. This differs from Pokémon GO, which is geolocated—you have to partake at this place that you've been told to go, which induces a certain degree of control on the part of the gamemaker. You've taken the opposite approach with Jadu, but also with Terminal 3 and A Jester’s Tale, where you're trying to actually more deeply immerse people in whatever physical space they're already in.
AJM: Absolutely. It used to be volumetrics—which, for people who are not familiar, volumetric is a technique where you use a lot of cameras or at least use some depth cameras to record a person in 3D as a moving video. We were doing tons of that; A Jester's Tale and Terminal 3 and the early iterations of Jadu were all volumetric video. I thought that that was going to be the case forever because that was one of our strengths.
But at some point we were running into limitations with how interactive volumetric video can be and how much effort that takes. So we shifted to what we're doing now, where you have 3D models of characters that you're interacting with. But the thing that that ended up doing was, we were already really familiar with human figures and characters in AR [through volumetric capture], and that became a big differentiator for us. Generally, AR is considered to be a first-person medium. Most AR is: you're the player, you're interacting with your surroundings. You don't need another character in the middle of it to facilitate.
In our case, we did add these avatars in the middle that you're then controlling. So AR becomes a third-person medium in that scenario, which makes it a lot more digestible, in my opinion. You can sit down and move your avatar around, you don't have to be physically moving all the time, which is cool and novel, but exhausting; people don't want to do that all the time. This gives you that optionality. It also adds this concept of the ghostly presence, which is something I have felt for a long time that was strong about AR, that in an AR experience, you can have a person in your room that you're seeing through your phone, but it's still your room. You're perceiving it as your room. So even when you're not looking at it from your phone, it feels like that person was in this space earlier. That's a really unique thing that VR and other mediums can ever accomplish, is give spatial presence to another human being in your space, and for you to feel like that presence is lingering after you're done with the experience.
JD: And that literally changes your sense of that physical reality. It stays with you.
AJM: Absolutely.
JD: When you think about that first moment of Jadu, where you had holograms of performers and it was designed for social video…actually, I have two questions. First, I remember watching you direct at Metastage and taking in the weird considerations that you had to make when you're directing somebody for volumetric capture. I'm wondering if you feel like that laid a foundation for where you've landed with Jadu. And then secondly, how do you feel about those holograms you captured. Do you see any future life for that style of social video, even though it's not where you're prioritizing your focus right now?
AJM: I love those holograms. We have so many of them. We're storing raw data and cold storage from those holograms. We ended up working with Serena Williams and Lil Nas X and we have holograms of them that we've built. Tons of great capture with 106 cameras. Also like artists like Bea and Omar Apollo and artists that have since really blown up and have significant audiences—and we caught them when they were just bubbling up. We have some great photos from that time that, over time would be interesting for a lot of people. So we want to bring it back in some form, even within Jadu, just creating opportunities for it. It's not going to happen in the near future because that's probably very confusing for our audience. We want to establish the scene first.
But I definitely think those clips will have a life later on even for us. Recording for volumetric established me as an AR director because like, initially I didn't feel like a director, I didn't go to school for any kind of direction. Film terms were being thrust upon us because we were going to film festivals. But when we started doing volumetric recordings was when I started feeling like a director because you're on set and there's someone and you're telling them what to do. Very quickly we realized that certain things work, certain things don't.
Part of it was the limitations of the capture itself. For example, for A Jester’s Tale, when when we recorded the kid, he had to be in a huge green screen room alone and give a moving performance while talking to the viewer who wasn't even there, looking like at nothing. It was really hard for him to deliver that. So we ended up hiring what we called an ‘interactor,’ where we hired an actor to play the viewer that we removed in post-production—just so the kid can have someone to bounce energy off. So there are a lot of those kinds of techniques that were just tried for the first time, just because the medium that we were building for was brand new.
I think a lot of that way of thinking translates over to what we're doing now, because we're not just bringing the traditional gaming approach. We have always had an intuition for these considerations of AR such that we can face the medium dead-on versus shying away from its challenges.
A lot of people do that as, “Okay, we're making an AR experience, but there are chairs here and now there's a wall. Let's just put it all on the table and make it small.” You know what I mean? It solves the problems, but it doesn't face them. It doesn't create new interactions and it doesn't embrace the medium in the way that we try to.
JD: Before we get to the current state of Jadu, there's a middle phase. It seems that you've really followed different audiences and communities and learned from them as you've been developing what Jadu has ultimately become. Between the social video moment and the current mobile game moment—or mobile is maybe the wrong word, mobile AR game moment—you built community in web3. I'd love to hear you talk a little bit about: 1) what the web3 strategy was, and 2) how you're thinking about it now with this broader-focused game on the way.
AJM: We've never had too much of a preconceived notion about what the end product is like in a longer span of time. We've always done the best we can with the resources we have at our disposal, while keeping the core the same, which is: we're facing AR and we're creating new interactions in AR, ideally for a large number of people, and ideally in a way that builds on top of itself versus restarting every three months. That's the foundation.
The result of that equation has been different at different times. We were doing holograms earlier with music. The reason for that was because the pandemic had started, we knew we could do volumetric clips, but we couldn't do volumetric clips that were really long because that was really heavy. So we had to do short volumetric clips in the middle of the pandemic and TikTok was blowing up and musicians weren't touring, so we found a unique opportunity there to work with musicians, record short clips that could be used in TikTok videos as holograms. It made a lot of sense. That got us far, it got us out in the world. We had an app. People were using it. We got to around 100,000 downloads on that app. There were moments when it was really trending. There were millions of views on videos that were emerging out of that. But it wasn't very clear how we were going to monetize or where this was going to go long term.
There's a sliding mechanic we've been testing that we really like where you can slide from one button to the other, which is something smartphone screens are good for versus controllers.
It's expensive to record holograms. It's not compelling enough for people to pay for it. It was enough to get us going, and this was around the time that web3 and NFTs were really just starting to pop up, and there was a skepticism from all kinds of groups around the technology; I was parallel to a lot of people that were working in NFTs, people that had early drops.
What I was starting to realize was all the things people were saying about, “Oh, you make your art, you put it out there, you can earn commission on it long term,” it's not just a technological concept. It's an idea. It's like you're saying that, in this space, we value artists and we want you to gain commission out of your thing. In the long span of time, we think that's a better model than what exists currently. That was compelling, you know? That made sense to me.
So, because it made sense, we followed it a bit. I had directed the only music video I've directed so far for Pussy Riot. We had it, we were ready to release it. We'd spent a bunch of money making it. It was not going to make any money. It was just purely because we wanted to show off the technology and we wanted to make something cool. But with Nadya, who's a good friend of mine, we decided at the last minute to release it as an NFT and it sold for—I don't even know the exact amount anymore because it was ridiculous—like a few hundred thousand dollars, and on YouTube it made $50.
Suddenly I was like, “Okay. There is an emerging space, they think about things differently, they're asking the right questions. We are an emerging platform or an emerging medium that's not found its form of monetization yet, this could be compelling for us. We could differentiate from the Niantics and Snapchats of the world because they won't take this risk. So we said we're gonna do AR and web3, and that was a really bold move on our part, it was a big pivot. But we went in on it and we released jetpacks, which were our early collection of assets that you can use in AR, which you own, that also represent your interest in the medium.
You can use it with other avatars that are interoperable and you can be part of a community that also has this shared interest in this company. And we sold around $400,000 worth of jetpacks in seconds. Seconds. And that was the most significant financial event we had had in the history of all the years working in AR displaying stuff and shipping stuff and raising money and building projects.
Suddenly we had made $400,000 in seconds, you know? So based on that, we were like, “Okay, this is, this is the direction for us.” And the surprising part about it was the community because I kind of knew that, yeah, you have a community, people buy your assets and then they rally behind you, but I didn't realize how insane it would be in practice because suddenly there are all these people that own these assets that showed up in our discord and were like, “We're here. We're here every day. We're going to show up every day. We're going to make this thing a thing.”
I went from having a small team to now thousands of people that were absolutely cheering for us, and that's something that we maintained for a long period of time. We released multiple collections and over time we did over $30 million in volume on our assets, and ended up raising a seed round and then eventually a series A around that, and gained a lot of credibility.
The investors gave us money because they wanted to invest in the team and AR as a technology, not necessarily web3 as a technology, but the support we were getting from our audience was critical and us hitting those milestones. So, that's why we went in it and I think we really benefited and learned a lot from it in terms of having a community, building a product that is shaped with the interests of thousands of people that really want to see it succeed.
The reason we didn't continue going in it as strongly as we were initially planning on was because of a bunch of things that have happened over the last year. Obviously, volume and interest went down tremendously. The stigma around Web3 and popular culture…I don't even need to say anything about that.
And generally the space just didn't go in the direction I was hoping for. I really thought that we were getting to a point where people interested in new technology and building new things are going to come into this. There were so many artists and creatives that had such compelling ideas about where this could go, but it did go downhill. It did just end up being a space with tons of scams and people taking advantage of each other all over the place. So many people I knew lost so much, not because of assets coming down, but because of hacks and this and that, the user experience was horrible—getting a new person to understand how to even get a wallet to engage with this stuff was such a large hurdle.
We were so deep in development, and we were building something that was unique for everyone that it felt like web3 was actually becoming a hindrance in onboarding people versus something that was a benefit. So we ended up removing wallet logins. We ended up making a game that was something people can play without having any association with web3 or having any assets. But we also did it in a way where the assets that people did purchase early on with us, they do work in the game and we have intentions and plans for them over time. But what's important for us right now is that we have a few years of runway and during this time, either we're going to build a very successful AR game or not.
If we build a very successful AR game, we have a lot of tools and leverage to bring value to the team and the investors and also people that engaged with us as early supporters, purchasing NFTs. We will have leverage to support that without relying purely on whether web3 as a space succeeds or not. And the other scenario is we only focus on a small audience that we try to appeal and pander to them and then hurt our chances of actually having something that is sustainable and large and successful and has an impact.
JD: There's a quote that I was thinking of in preparing for this interview. It's actually from Steve Jobs, which I know is a cliché, especially so because our friend Gabo Arora is imagining you as the Steve Jobs of this industry, but it hits on this way in which you not only segued out of focusing on Web3, but have journeyed as you've developed Jadu.
“People think focus means saying yes to the thing you've got to focus on, but that's not what it means at all. It means saying no to the hundred other good ideas that there are. You have to pick carefully. I'm actually as proud of the things we haven't done as the things I have done. Innovation is saying no to a thousand things.”
AJM: Absolutely. We've never done more than one thing ever, and the whole thesis of going into building a product and not going by an agency or studio model was that we had to do one thing and give ourselves a setup where we can iterate on it. That's really why we're moving into what we're moving into now, which is building an AR fighting game.
When people think of the metaverse, both in terms of VR and AR, they have this generalized vision of a social space where many things take place and when everyone's trying to build some kind of generalized thing, nothing specific emerges. So we learned this roughly a year ago and we said, “You know what? We have a setup. We have these avatars that you control. It's a third-person game. We're building multiplayer functionality, remote multiplayer. What should these characters do when they're with each other? What's one thing they can do, and what's the best thing they can do that people will understand that will give them an intense experience and just get them into it?”
And we thought fighting was the way to go. So we are now building an AR fighting game, which doesn't sound as grandiose as we're building the AR metaverse. It doesn't sound as large of a concept—it sounds kind of small. It's a contained mobile game. That's not that big of a deal, but I think this contained structure allows us to just do a really good job at it. And I feel like that's something that's been missing in AR is: no one's just grabbed a basic concept and gone deep enough with it, and have polished it enough, given it story, given it consistency develop it over a longer period of time. So I think we're really lucky that we are getting to do that right now.
JD: The last big reference of AR gaming that anybody points to is Pokémon GO, and however much anybody respectively likes or dislikes that, they really nailed the geolocation function. And that of course was explosively popular and continues to be popular to this day. Was there a particular moment that you remember deciding to do a fighting game or was it the result of conversations over the course of days or weeks or months?
AJM: Yeah, it was something that started bubbling up and initially, you're just throwing random ideas with your team and people you meet and just seeing what sticks, what people kind of respond to.
And I started bringing up fighting as a mechanic where it's trying out because it's such a simple, people want it eventually. I know there's going to be an AR fighting game one day, you know? And so when I started bringing it up with some people on the team, there was some skepticism because on, once again, we didn't really have an idea of what we were building at this time.
There were pieces and there were some mechanics, but the overall thing was not quite there. And people, everyone on the team had their own version in their heads of what this could become. And I think some people thought maybe like a Tamagotchi style thing, or people still had maps in their mind. So fighting was kind of a weird one.
And also there was a lot of skepticism because. Fighting games are a very particular thing that very particular type of people usually indulge in, and they're very developed now. So it's actually hard to build something that's an AR fighting game that's novel. And the way I started kind of selling my team on it was like, Imagine all the things we can do, it's in your space, imagine when you get close you can freeze frame it and do some crazy things.
I was just coming up with all these novel mechanics. And it's funny because, we were like, “Okay, let's build a quick prototype.” So we built a prototype in two weeks and shipped it in beta to our Discord audience. And this was the first time there was somewhat of a loop. There was something to do.
And people played a lot. Like we did a thing over last Christmas where we had some cash prizes and whatnot. You just had to play for two weeks and people were playing six hours a day. Like top people were just playing this basic prototype we built in two weeks, a lot.
So we felt like there was something here, something was starting to happen. And then while starting this year, we said, you know what? This is what we're going to do. We're going to rebrand. into an AR fighting game, and just make that, and make it really good. So we really doubled down on it. And it's funny because the mechanics initially were like, gonna do all these crazy novel things.
Ended up actually being really simple. You press a button, and it punches. You press another button, kicks. And you press them a bunch, you get some combos, and you get some finishers and I think that's enough. I think you don't need like, some crazy gestures and weird things happening in order for this to be compelling.
It's already in ar it's already compelling. There are already a lot of novel things happening, so what we needed was actually just the quickest inputs possible and the simplest mechanics possible, and I think we've landed in a really nice place with it. Hmm.
JD: Do you see yourself in the lineage of Mortal Kombat, Street Fight, Tekken, Smash There's a long history of very successful fighting games across different platforms. Jadu of course is entering into a new reality—somewhat literally—but do you see yourself as extending those lineages or are you imagining it more as a hard fork into a different direction?
AJM: Whether we like it or not, we are now a part of that lineage, which is not something I imagined or expected. And that was the case with the film festival circuit. That wasn't a lineage I thought we would be interfacing with. And same goes for music and holograms. We injected ourselves in these different places and had to level up to exist in that lineage because with something like AR there are a lot of intersecting lineages, and that's just going to be the case. As someone building in it, you have to be somewhat aware of a lot of them in order to do it justice and do a good job. So this year for me has been a crash course in fighting games.
I'd played Street Fighter and Tekken as a kid and arcades in Pakistan. That was a big thing. We had arcades that people would just go in and just play a lot of Tekken. In fact the Tekken scene in Pakistan—I don't know if you're familiar with this, but the world champion right now, for the last four years in a row, the Tekken world champion is a Pakistani kid, who we’re working with.
JD: I had no idea.
AJM: Yeah. So, we're making a fighting game. The best fighting game players in the world are Pakistanis right now. And the way that happened is also really weird because they were just playing in home-built cabinet arcade systems in Pakistan without any international exposure. The whole competitive scene was run by Japanese and Korean players, and then a couple years ago, Arslan Ash got a visa, and they sent him to one of the big tournaments, and he just destroyed everyone. His technique was brand new, he uses fingers like this [gestures] to hit the buttons and everything and everyone was just completely taken aback—and then they realize it's not just this one guy. There's a whole scene. It hasn’t touched the global stage yet. So I'm very proud that this is in my background. That's why we started working with Arslan. He's been advising on our gameplay mechanics.
Talking to the Tekken world champion, four years in a row now, I've had to learn a lot about fighting games, and analyze Tekken fights and Street Fighter fights and King of Fighters fights. How many punches do people land in a fight? How many blocks take place? And then compare our setup and tweak it throughout all of this. We've redone our entire combat system so many times now. Our animation team has grown a lot, learned a lot. Me personally. We've been doing a lot of work where we bring in consultants, so we don't actually have an in-house game designer at the moment. I do a lot of it. The animation and gameplay teams obviously contribute a lot. But generally we do work with a lot of consultants.
We want you to face these characters and their reality head on, in an almost realistic manner.
We bring someone on who is an expert, let's say, in fighting games or free-to-play games. And we do sessions with them on a weekly basis where I workshop ideas with them, bring them concepts. They give me best practices. We tweak things for what we think will work best. It's an incredibly hands-on process. I feel very lucky. I have a really nice job where, every time pivoted, I got to learn a lot about a certain lineage, and not only understand how things have been done, but contribute something new. And it's easy to contribute something new because our starting-off point is a new medium that we have been working on for a long time.
JD: What are some of those contributions? Like, obviously it's an AR, that's a new contribution, but looking one level in at the mechanics, the creative development concept, et cetera—what are some of the new contributions you're imagining Jadu logging in this history?
AJM: On the AR level, like I said earlier: third person, that's one thing. On a deeper level you get to remote multiplayer, which is something we're really proud of. We had this idea for a long time and early this year, we weren't sure if we would get it working. But we were operating under the assumption that we somehow will, and we did and it feels great.
When people think of multiplayer in AR, traditionally they think of local multiplayer where both people are in the same place and they're having matches together, which is technically quite challenging to get two sessions to sync up in a local space. So what we did was remote multiplayer because we think that people want to play more with people that aren't physically next to them. Because if you're physically together, you should just fight at that point in real life versus in AR. So the remote multiplayer setup means that both people see each other's avatars in their respective locations.
The third piece is the combat itself, which is where we get into the lineage of the fighting games. What we're doing is not as extensive, quick, and frame-data oriented as most console and PC fighting games like Tekken—but it's also not as simplified as most mobile fighting games or most mobile games in general, where you just have one button you're smashing again and again, or the mechanics are really simplified.
In our case, we have multiple action buttons. There is a punch, a kick, a projectile or a special, and a shield. We tried a lot of formats. We had five-button formats, six-button formats. We've landed on this four button format with three attack buttons and one shield. And for different characters, there will be different things. Those buttons can do, like, a grappler would use a special button for throws, whereas an all-rounder would use it for projectiles, things along those lines. You can tap them multiple times to get to combos. And there's a sliding mechanic we've been testing that we really like where you can slide from one button to the other, which is something smartphone screens are good for versus controllers. Sliding feels great with some haptics. So those are some of the things that we've been working with.
On top of it, a big part of AR is: you gotta frame your fight, which is a big challenge because in most fighting games, you're constantly seeing both players on the screen 100 percent of the time. There's no moment where someone's off screen, whereas in this case players can go off screen, you have to constantly be moving and adjusting your physical position in order to make sure the combat is taking place in a comfortable distance from you, which initially I think a lot of people get a bit turned off by. They're like, “Oh, what, where are the characters going? I have to constantly keep track of them. That sounds exhausting.” So we've had to do a lot to minimize that, lock the characters to each other, make attacks land more easily. Make the buttons more exciting. So just with a bunch of taps, you can get to specials to compensate for that, because that is a big part of AR. You do have a spatial understanding of your fight, which is the whole point of what we're trying to achieve. We're also doing things like using your physical surroundings, which is hard, and multiplayer, but with the single-player modes that we're developing, we're going to do a lot more of that, where you can smash someone into a wall or climb on a table and jump off it with your avatar, things along those lines.
JD: When you're thinking about the progression of a player in their own arc with Jadu, what skills or techniques elevate them to expert status? I'm thinking of Mortal Kombat, for instance, it's being able to do really funny fatalities. What are the things like that here, the red meat for the experts?
AJM: It's shifting right now because we're still tweaking and building and learning. In the more initial iterations we had leaderboards, and we have people that play a lot. Grinding was actually a big part of being on the top of the leaderboard and understanding the meta because the game was always broken in beta, like every time we pushed a build, there was some button you could spam too much and it becomes an advantage. So, when people are playing, they're trying to find what that is.
When we ask for feedback or when we look at what people are saying on Twitter and Discord, often it's things like, “I just had a brand new experience with my phone.”
And every time we did tournaments, someone would figure, “Okay, the projectile button is spamming too much. Or the light kick, you can just keep going on it and the other person can't do anything.” We've been fixing a lot of that stuff. I think now we're at a place where it's a lot more balanced. So finding that meta and grinding was a big part of getting to be on top of the leaderboard.
I think that's gonna shift with this release. We've added rankings and things like that that don't require playing a lot, but they require winning and playing a lot with high ranked players and then being able to defeat them. A big part of it is understanding the controls. I think a big aspect of it is also going to be content creation in general. People recording videos of their fights and whoever records more compelling videos and more compelling context would get some gratification from sharing in the community and social media and whatnot.
Going into next year we're introducing a lot more characters, and with the characters we have single player and campaigns, which is something we're going to be doubling down on because initially when we started building this the idea it was, how do you get to a substantial amount of content in this quickest amount of time? And PvP was the way to do it because building a lot of campaigns and story takes time, so we thought if we just build a quick setup where people can fight each other and give them tools to customize their character, there's suddenly a lot of content, because you could go into PvP, match up with random people, and have a new experience every time.
But now we're at a point where that's established, and we can actually flesh out the single player campaigns where you fight as a character, you go through some story arcs, you have a bunch of fights, you're trying to rescue the next character, you get to the next character, now you can play as the next character. Then you can play as both the old character and the next character together in certain scenarios. And so a lot of possibilities and variations start to emerge there. And that's when people can start to collect certain items and build their character and indulge in the lore. All those aspects start to become more meaningful.
JD: You referenced characters. Obviously, storytelling isn't the first thing that comes to mind when people think of fighter games, but of course there's a lot of story that's imbued—particularly manifest in the characters, and manifest in a way that's in many cases more crucial because you have such limited time to really communicate the story of that character in the gameplay context. You don't have an open world and dialogue and people walking around to give the player a sense of who that character is. Walk me through a little bit of that storytelling and concepting process.
AJM: Story has been a really weird part of our process over the last year or so because, on the one hand, story was always really critical. We started with short-form stories that we told in film festivals. So it was a really important thing to us. It was one of our strengths, and a lot of people on the team wanted to do stuff that was storytelling heavy. When we started the fighting game, a lot of people were slightly turned off because the possibilities for story were slightly more limited.
And I was also a bit allergic to going too deep into story for the last year. Part of the reason was because every time we would have Mac, who does our worldbuilding, build out some elaborate axioms and establish a scene, we would then pivot and suddenly it would be kind of retrofitting and trying to make sense of how the story is going to operate in the new setup.
So we always kept it open-ended enough, where there were universal ideas that we could toy with, but not commit too much. So it was kind of an awkward place for story, and that goes for characters as well, because we were building a fighting game and we started building new characters, but at the same time we were not committed enough to the fighting game, where we still wanted to keep the optionality open that if we pivot again, the characters could work in a lot of contexts. So we ended up building characters that were relatively generic in the sense that we wanted our users to put themselves as the characters, where you can actually customize your character and build it out and get new shoes for it, get new gloves for it, hair, this and that.
And that's what we're shipping right now. I'm really proud of it. I love our character creator, I go in and just mess with it because it feels so good. We have an audio track playing in the background, every time you even swipe on the character the audio shuffles around, which is the kind of attention-to-detail that most mobile games usually don't get to, but we’re adding that kind of polish in every aspect.
So the character creator is really exciting and you get to build your own character and we're doing a lot of item drops and bundles and things that you're getting for your character. That's like an exciting thing that we want to keep developing. But, now that we're more committed to fighting, we are now also creating fighting game-oriented characters, where now story is coming into play more significantly.
We are trying to bring a really fresh take to mobile gaming. We're saying, “You know what, mobile gaming—just completely forget about what it is currently. Imagine if it's mobile. Imagine if you have to move to play it.”
So we have new characters. We're going to be releasing early next year, honestly just in three or four months, where those characters are global, they're from a lot of different parts of the world. They have a lot of story and background imbued in them, and you get to explore them through these campaigns that you go on and the kind of process we're using for these characters is, first of all, we want them to be gritty and grounded in reality. We're not trying to make them so fantastical that it's kind of retreatist. It's not fantasy, you know? Or it's not Smash, you know what I mean? It's also not as indulgent as let's say Mortal Kombat, where the violence and the craziest possible fatalities are what makes the game exciting.
In our case, we want you to face these characters and their reality head on, in an almost realistic manner. That's one of the things that we're following with the characters. And we want them to be global. We want them to reflect what's happening globally in terms of like, Bollywood's massive now, K-pop's huge now, Reggaeton's huge now. We're not living in a Hollywood-only world anymore, and we want the characters to not just be token characters but actually be super compelling and representative of all these things that are taking place in the world. That's actually a big reason why our team is incredibly global. We're covering around 15 nationalities within our 40 person team, which is an expensive thing to run. We have to pay taxes in all these countries and [manage] all kinds of paperwork and logistics that go into running it, but we think it's worth it because the end product then reflects that.
JD: It's funny because in this subtle way—even though somebody might look at your career and say, “Oh, Terminal 3 is Asad’s most political work,” what you're describing has a definite political angle to it.
AJM: Absolutely. The most political thing in a weird way is the thing that has the largest impact, reaches a lot of people, and actually affects a lot of people's lives. What we're trying to establish now is something that young people that are just starting off, getting into things that they like and care about, giving them a new medium to engage with each other on. I'm really proud of the fact that this is an immersive, spatial experience where you meet someone on Discord and you find some friends on the internet and you text them and see their videos and photos—but in this case you feel their presence in a space, in your room, in your world, and I think that that's really powerful. Being able to do that with characters that are coming from all over the world, and the setup that we're positioning ourselves for fighting itself is political. People punch and kick and scream for the things that make them punch and kick and scream in the real world. And we're leaning in on those things. We're leaning in on the angst that emerges from the political state of our planet. So I think that just doing a piece that works for white liberal audiences in a film festival setting is I believe not as political as something that actually reaches a lot of people from a lot of different backgrounds and allows them to mix and merge with each other around something that is compelling, and is directly related to the future of where things are heading.
JD: And, not to speak for all young people all over the world, but generally young people have much less agency than adults, and as a function of that, spend more time in a routine set of spaces, often their room. What you're doing is capturing some of what made going to the arcade special—you referenced the arcade with Tekken—it resurrects that social component in space, but instead of it being in a separate space somewhere else in the world, it's in a space that's very special to you, presumably.
AJM: Yeah, exactly. With A Jester’s Tale, this was a big concept. It was framed in a bedroom. So what we were trying to accomplish was that, if you see a really insane story take place on your bed, you're likely to dream about it, you know?
And we're like, “How do we get into people's dream with content?” AR gives you additional tools for that. And I think that's the case here as well, people playing in their familiar spaces and seeing unfamiliar things and presence of other people from other parts of the world showing up in familiar places. I think that that's really powerful and gets into people's psyche for sure.
JD: Have you experienced those types of dream invasions?
Asad: Have I dreamed in AR? Hell yeah.
Jesse: Can you share any of that? Any moments where that happened, or any anecdotes?
AJM: No, honestly. More recently I don't remember them because there's so much going on work-wise that we're just really tunnel vision in the day-to-day of making things happen. As soon as I wake up, my brain automatically is like, “Okay, what are things that have happened in Tokyo last night? And then all other parts of the world where our team is situated.” And so I get fully immersed in it directly. It's been a long time since I've been remembering my dreams.
JD: Well, I'm sorry to hear that. I hope after the launch, you’ll have time to sleep and have some deeper dreams. Speaking of your team, it seems to me you're describing a context for Jadu where you've got team members all over the world. You've also been on this ride as a company. Talk to me about that experience as somebody leading a startup with, now 40 or 50 people and bringing them on this journey. Because one thing that often gets discounted in these discussions is how brutally difficult running a startup is of any kind—but then much less when you're describing one that is attempting to be global and operate in a relatively new medium.
AJM: That aspect has been one of the most significant learning experiences for me over the last year-and-a-half, just forming a team and relying on a team to get the job done. It's an interesting push-and-pull of, how much agency do you give people, how much do you trust people right off the bat, how much do you step away from things that you feel like you've been an integral part of developing and let other people fill those shoes?
We hired a lot last year. We raised a lot of money really quickly. It was a very euphoric market where it felt like the only limitation was talent and not resources. We got the team up to around 50 towards the end of last year and very quickly realized that that was a mistake because a lot of roles we filled were either roles that weren't actually needed, which were actually now a deterrent to us moving quick, and other roles that we didn't hire well for, and ended up with people that we thought weren't doing the job the way it was supposed to be done. We had around the layoffs last year that, me as a CEO for a young company, it was a pretty insane, like, just hiring rapidly and then having to go with layoffs and dealing with everything that comes with it and then rescaling up this year, but now doing it with more learnings and having a clearer sense of what we're trying to do, having a better organizational structure.
One of the big things that I've had to do multiple times over the last couple of years is restructure the team til it works, and a couple of months ago we did a big restructure in the middle of our sprint to get the app out, which wasn't planned. I was just noticing that certain aspects of how the team was organized was not working as well because there was often wasted work. Someone would do something that someone else was doing in a different way and both things wouldn't work harmoniously together. So with that, we arrived at what we have currently, which is a structure I'm incredibly proud of the org chart. It looks incredibly symmetrical. We have five pillars that report directly to me, which are: the app team, which is essentially responsible for everything that happens on the backend, everything that the app itself, the UI, the campaigns, the leaderboards and profiles and connections—all of that is under the app team.
And then we have a game team. The game team does everything that is AR. The game team doesn't do any 2D UI or any other things that the app team takes care of, it is entirely AR game play focused.
I hope that people treat the game as a fight for this medium to become compelling over our lifetimes and join us for that.
Then we have an art team that takes care of all the 3D asset creation, models, cinematics, web, all that kind of stuff is on the art. And these three teams are structured in a way where usually we have a lead developer, a lead designer and then essentially a product manager or someone who's making sure everything's working. And those three people report to me directly. I have syncs with all of these pillars on a weekly basis where all three stakeholders are present. And the designers that are responsible for app don't touch game, we've created these clear divides.
And then the two additional pillars are ops and growth. Those are obviously critical to have. We have really strong ops team and everything's really cleanly done. I'm very proud of our code base. It's incredibly modular. It's very thoughtful. We're constantly refactoring. Everything's just really nice. There's very little technical debt, everything is in figma, and everything connects to each other, a strict design system. Everyone's in sync about what's happening.
I'm really proud of the organization we've been able to make really quickly, and each of the teams knows that we're building in a new medium and knows that our setups never should just be how setups at other places are. We start from scratch every time. And we ask ourselves the right questions till we arrive at a structure that works.
Jesse: Long way from 1RIC.
AJM: Man, if I tell you about the way some of those projects look like in the back end, it's like, Jack and I joke about it all the time. We were taking these to festivals and people are writing about us and whatnot, and like, the Unity projects were just…horrendous. Just tons of game objects laid up with sim. We had a script called a simple timer and we would just time them on and off to make them work. It was very put together, but the user experience came across polished because we were good at that, but now it's kind of ridiculous. I'm incredibly proud of the code base on the design systems and everything. It's my dream.
JD: I like to think of when people take the leap to building a startup, they're observing something about the way the world works and they're making an argument that it should work differently. How are you thinking about that relative to Jadu?
AJM: I think there are a lot of different aspects in which that happens. On the one hand, it's like, how is the team structured? We try to not have too much of a preconceived notion. Similarly, AR as a medium constantly just means that whatever we're doing, we have to think of it differently. So we're operating in the mobile gaming space now, right? If you think about mobile gaming, it's the most significant market; it's $90 billion a year quickly growing from here. But if you think about most mobile games, a lot of them are glorified slot machines or underwhelming versions of what's possible on PC and console.
One of the consultants we're working with recently told me that 70 percent of people that play mobile games have never in their lives played a PC or console game. They don't have the context of gaming culture in the same way. So many people just play mobile games and don't even think of themselves as a gamer necessarily. We are trying to bring a really fresh take to mobile gaming. We're saying, “You know what, mobile gaming—just completely forget about what it is currently. Imagine if it's mobile. Imagine if you have to move to play it.”
JD: It's literally mobile.
AJM: Yeah, it's literally mobile. And that's just clearly distinct from what's possible on PC and console. It is original to this platform. It's native to this medium. And so that gives us a brand new frame. Everything we think about in terms of mobile gaming is: how do we make a mobile game, you know? That's just one of the many aspects in which I think the way we work is different.
JD: One reason that I think Jadu meaning “magic” is really apt is: you're imbuing these phones with a new kind of magic by prompting people to physically move and think of the device differently. Of course you could walk around with your phone, but you're not thinking of it as a portal into a gaming environment and you're kind of flipping that on its head.
AJM: That's actually been my favorite feedback so far from people that have played our beta. We've had around 10,000 users playing around 100,000 matches while we were in beta. When we ask for feedback or when we look at what people are saying on Twitter and Discord, often it's things like, “I just had a brand new experience with my phone. And I thought I could never have a brand new experience with my phone again because I've had my phone for a decade now, and it does the same things every day. And now suddenly I'm using it to interact with my space and move around and try to follow things.” It's not behavior people are used to when they're just slouching on their phones, scrolling around.
It's introducing mechanics with something we're really familiar with, but mechanics that are brand new—which I believe are using the device for its full potential, because modern smartphones have a lot of computing power, way more than is required to just watch a bunch of videos. They have gyroscopes and LIDAR sensors, those are the things that make this medium special, and things that other gaming mediums have not had access to.
Novelty consoles have existed and portable consoles have come out at various times, but they've never reached the kind of scale and ubiquity that mobile as a platform has. So I think that if we are able to do what we're trying to do here, which is use a mobile phone to its full potential, use all the things that make it unique and offer an experience that people have never had, I think a lot of people would want to engage with that experience.
JD: Obviously this is being released as a mobile game, you'll get it on the app stores. But do you differentiate in your mind between what you're describing with the Candy Crush-ification of mobile games versus what you're offering?
AJM: Our starting point was so different and we knew nothing about games or mobile games. Our background was not in games; initially we had a lot of new things to offer. I think that at this point we are circling back to the Candy Crushes of the world and actually trying to understand how we operate within the context of what mobile gaming has made work so far. That's why single-player campaigns and level design and you complete a level that there is a core mechanic, which is you fight. You try to win, you get a reward at the end. And now we repeat this mechanic in the form of various campaigns and stories, which is the foundation of how most mobile gaming works. They have a core loop that you duplicate a bunch of times to form levels, eventually campaigns. Our core loop is very distinct and very different. It engages your space and does everything in AR, but now we're able to learn from these existing games in terms of establishing campaigns and structures.
Give people things that they can go through. We left that part till the end. We didn't want to have our process be biased based on what people had already built on this platform. So we started from scratch and put our blinders on, but now that we have something that we feel is novel and compelling, we're going back to best practices that already work in the space and trying to work them into our system.
JD: Are you encountering any misconceptions—like when you're engaging with your Discord community, with folks that are on the beta—are you encountering any misconceptions about what augmented reality is or what you believe its potential is?
AJM: Absolutely. I think a lot of people have a reaction of, “Oh my God, that looks so sick, but I'm going to wait for the headset. I bet it's not that good on mobile.” It takes it till people actually play it to be like, “Wait, what? Mobile phones can do this? I had no idea that this was even possible.” A lot of times when people see it, they do approach it with skepticism. That's just a default.
We have to win them over by showing them things that are really compelling. That's why I think younger audiences are way more open to it, because they have less of a preconceived notion about what this is supposed to be and they're willing to try it out a bit more.
And obviously a lot of people are constantly confused about whether it's VR or AR, communicating 3D “reality” oriented experiences in the form of photos and videos on the internet is never easy. People always end up having a different idea what it actually is, so we we also do a lot of promo videos and cinematics that we build that are more full AR, where we do a lot of work in post—but people have negative relations with those kinds of things. Obviously the Magic Leap whale comes up. But sometimes those things are quite necessary to communicate conceptually what is taking place.
JD: Do imagine that there would come a point in time where Jadu would function on headsets?
AJM: Oh yeah. AR as a medium, at the end of the day, needs your hands to be available, among other things. That's a really big one, looking through a hole, like trying to look around, it's not the way it's going to be forever—but I think there is a lot you can do with it right now. We are now a venture-backed company that has a certain runway and we’ve got to make this thing work during that time—or at least get it to the next level during that time. So we can only engage with headsets to a limited extent because that's going to take time. We're not quite there yet, but the Apple Vision Pro is something I'm insanely excited about.
Imagine from my perspective: everyone's talking about how there's going to eventually be an Apple headset. And everyone always just assumed it's going to be VR because no one cared about AR anyway, AR never really had crazy funding rounds or crazy hype around it. It was always a relatively ignored medium. And for me to just stick to it anyway for seven years, and then Apple finally releases the storied headset and it's AR-first, and that's what they're leading with: it's AR, not VR. You can go into VR parts of the time, but for the most part, it's passthrough AR, and they make all these decisions in it that are super in line with the way that we imagine this technology.
One of them is that Meta always shows their headset being a bit more active. People are getting up and playing fitness games and Beat Saber and walking around. Whereas Apple’s concepts were more people sitting on their couch and using their hands and gestures to move around, which is very much in line with what we're doing because we're doing a third-person thing where you're controlling your character rather than moving around so much yourself. So that's just one of what I feel like are many decisions that Apple ended up making that reinforced some of the mechanics that we were developing, so I’m really excited about the Vision Pro. We are in communication with Apple, they're very generous in the sense that we're discussing what the possibilities could be, I think it's more likely that, rather than seeing the full version of Jadu as a game work on the headset, we will probably see more storytelling, like, imagine a backstory of one of our characters and a 10-minute narrative experience on a headset.
Those kinds of projects will likely happen before we get to the full game, because it's still a bit further out in terms of the headset, finding proper mass market, but we're going to do these storytelling experiences. We're eventually going to build a game available on the headsets. We're going to make it cross-platform. There's a lot that we're just really excited about diving into with this. In fact, we have a developer who's full-time focused on the Apple vision pro SDK already, so we're doing some early exploration, but I can tell you that it's going to take some time.
JD: I believe it. These are critical moments, though. People often discount that in these super early days when it's the most confusing, that's when these interaction languages are being developed.
AJM: Absolutely. It's not just a few years. It's often decades. That's why just sticking to it has worked for us, it's been seven years now. It's kind of crazy to think of. We've been doing AR exclusively for that amount of time and have never lost track or focus. We've never touched VR. Every time we get pitched a VR project, we've always just been like, “We just don't do VR. It's not our medium. Our medium is AR. They're not the same thing. Don't confuse them. They're distinct. One is about your existing reality. And the other one is about transporting you somewhere else. Just conceptually different.”
JD: So, just to state it baldly: why should somebody take the jump to play Jadu on day one?
AJM: Because you really are going to get to see the trajectory of how it evolves. Because we at this point have a pretty insane team. We have really crazy product velocity. Not only are you going to get an experience that is incredibly rich, in terms of multimedia, you're moving around, it's great music, great sound, great VFX, great sound effects, haptics on every moment…there's real craft in what we're building.
It's not a quickly spun-up mobile game that's trying to hop on a trend or something. We're really putting a lot of love and attention into it and you experience it in the product. And it's only going to get crazier from here, the characters we're building, the new mechanics that are coming out. It's a frame to experience what AR is going to be for the rest of our lifetimes. A lot of the work we did early on is now taught in new media departments at USC and UC Berkeley and whatnot. This may come across as, “Oh it's just a game,” but every single move and mechanic that we're building showcases what techniques are going to be used in storytelling and games in the arc that we're going to be witnessing for the rest of our lives. So I think that's a compelling reason to play a game on your mobile phone.
JD: So many of the bigger companies you referenced have taken an inductive approach, like you referenced the metaverse. So the inductive approach to it is: the metaverse will be a thing, and we're going to now create the container for the thing to happen. You're coming at it from a deductive approach, which is: let's make a really excellent fighting game and the behaviors and social dynamics and culture will emerge and tell us what the metaverse—or you call it the bloom will—become.
AJM: Absolutely. It's not even a new concept, right? The more cliche example would just be Fortnite, you do have a simple core mechanic, but then when people are spending enough time in a game it becomes a social space to start spending time. Then you expand it, you add more social functionalities and features and creator tools. Suddenly you have something that is more comparable to what people imagine when they think of a metaverse. And I really think that this is an interesting trend that's happening right now where—Epic announced a bunch of layoffs and it seems like they're doubling down on Fortnite and stopping a lot of side projects that might be actually quite successful.
Same's happening at Niantic. They're also laying people off, stopping a lot of side projects and doubling down on Pokémon GO. I'm seeing a lot of organizations double down on their main offering and try to make it more of a social experience where they add more mechanics that appeal to larger amounts of people, because having a lot of people in one place engage and sustain is what starts to form a metaverse-like space.
So it's critical for us to establish a mechanic and have a reason for why people are coming. And then once they're staying, give them the additional creator tools and the types of things that are gonna form community and form social interactions.
JD: It’s like that “Saying no” quote; saying no to get to the yes of the metaverse—if there will be a yes. I have a few recurring questions that I do with every guests given that the show is called “Reality Studies.” They're lightning round-style questions about reality. What's one thing you wish people paid more attention to?
AJM: I think the answer for me would just be reality itself, because at this point, our screens and everything are so compelling, they really are. And there are so many distractions that are available to us that I think our perception of reality has become really skewed, especially in the U.S. It's something that I experienced. There's a distinction when you come from another country to the U.S. where, over here, everything feels like a simulation because of the amount of marketing and fashion and the amount of meta systems that have existed for such long periods of time. And the way that it's taking place in other countries is also really crazy now because everyone has smartphones and often living standards in other countries really suck.
So not focusing on reality is really compelling. That's why disassociation and retreatist ideas around technology are so prevalent, and that's one of the strengths of our medium to begin with, that it allows some of the excitement of the digital world to filter into the real world and actually bring attention back to reality and the things that often go overlooked.
JD: Given the work you're doing with your global team, given your background growing up in Pakistan and now working with folks in Pakistan the aforementioned Tekken champion, is there any insight or learning about contemporary digital culture that you think the West is totally missing?
AJM: One of the things that's happening is a lot of people in parts of the world that otherwise have been just very disconnected from the rest of the world are now sharing their sense of humor and their realities with everyone. My instagram Reels feed is really weird because it's constantly shifting between American stuff and Pakistani stuff.
I don't know how they decide when I'm in what mood, but whenever it goes back to the Pakistani reels, a lot of them are actually like, random workers in a farm joking around with each other, making weird videos—I'm sure you might have seen some, it's kind of a joke of how Indian and Pakistani videos are really bizarre—and people are just putting random filters and weird dynamics over it. There’s one right now where it’s these Indian stories that people are telling with some kind of game engine creator thing, and they're so abstract and bizarre the way that they're told. People laugh at them because they are so removed from the traditional storytelling structures that people are used to. I don't know if there is any deep insight there, but I do think the democratization of what content looks like and how it can be so different all over the world is really compelling to look up.
JD: Final one. What's one moment where your sense of reality was disrupted?
AJM: I came back from school and I turned on the TV, and the news said that Osama bin Laden was killed in Abbottabad, which was my hometown. And that was by far the moment that I felt like the world was just this weird simulation, because I couldn't believe it. I grew up in Abbottabad. It's a vacation town kind of thing, a small hill station town where a lot of people from the south go to in the summer when it gets really hot and it was just incredibly peaceful. Nothing ever happened there, and seeing that Osama bin Laden was killed five minutes from the spot in which I was born. I called up my cousins and they saw the helicopters crashed that evening.
Growing up Osama bin Laden was the most wanted man on the planet. No one could track him down, the myth around it was so insane that the idea that he was found five minutes from my house was reality-shattering.
JD: Wow, I can imagine. There's so much that you have to say about AR and Jadu that I'm sure I we could only scratch the surface in this interview, is there anything that we missed that you feel is important for people to consider about any of the aforementioned?
AJM: Play the game. Obviously it's my job to promote it and make sure that people got to it. I'm really curious to see how audiences engage with it. If you're a listener who's going to go and actually play and engage, put yourself in the shoes of the people making it. It's an interesting exercise. I try to do that, but most media that I consume just happens automatically at this point. But especially for a new medium, if they are thinking about, “Hey, why did they make this choice? Why is this thing here? Why is this indicator pointing to me here? Why does this event take place before this thing?” I think it will get your brain thinking about the medium. In a way that makes your gameplay experience actually way more compelling than if you're not thinking about the medium.
I would love for our audience and our players to be in it not just for the story and the fighting and the experience that is on the surface, but also because of what this means, what this is trying to be. We're trying to be the fighting game of the medium, in which a lot of the other people who are building stuff are not building stuff that's angsty and trying to be intense in this format. So I hope that people treat the game as a fight for this medium to become compelling over our lifetimes and join us for that.
Reality Studies is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
This podcast is edited and produced by Adam Labrie and me, Jesse Damiani. Adam Labrie also directed, shot, and edited the video version of the podcast, which is available on YouTube. Music is by Eaters, sound effects by Eric Medias at soundimage.org. For more information, please visit realitystudies.co. And if you appreciate the work I'm doing, please consider liking, subscribing, and sharing it. Until next time.
Asad J. Malik: Fighting for the Future of Augmented Reality | Urgent Futures Podcast Ep. 2