Nexus Studios’ augmented reality experience for the White House is a big step forward.
We are coming towards the end of a US presidency that’s been almost universally admired by the creative and communications industries. Barack Obama’s administration has been the techiest ever and (drone killings aside) much of that technology has been used for good. The White House quietly recruited the finest minds in tech from places like Google and Facebook and used their big brains to improve government services and solve American citizens’ problems.
The latest example of the Obama team’s technophilia comes in the form of an augmented reality experience called 1600.
Created by Nexus Studios’ innovative Interactive Arts division and featuring animation and design from Nexus Studios director Jack Cunningham, the free mobile app allows the public to explore the White House. The experience tracks a stunningly intricate AR animation off a one-dollar bill and tells the story of one year in the administration.
We spoke to Luke Ritchie, Head of Interactive arts at Nexus Studios, about why this project is such an important moment in the timeline of AR.
The Beak Street Bugle: How did the White House approach you?
Luke Ritchie: We’ve been doing a fair number of AR and VR projects in the last few years and one that caught notable attention at the White House was The New Yorker, which was an AR experience we did with Christoph Niemann, quite a well-known illustrator who does covers for them. We got to work with him on the innovation issue in May. It’s a cute little experience that starts off in 2D, which is amazing, and then breaks into 3D.
The White House had seen that experience through gave us a phone call to see if we would be interested in working on developing an AR experience for them.
BSB: What was the brief?
LR: To begin with it was pretty loose. They were very interested in a message about the White House’s role in democracy.
President Obama and his administration, and Michelle Obama, had been working very hard to try and dispel myths about the building itself. They’ve done lots of interesting things, from opening up rooms that were previously closed, inviting everyone to have a festival on the lawn of the White House for South By South Lawn. You could Google Street View through all the rooms and stuff.
The key word that went around was transparency. The hope for the project was to enlighten people about what goes on there.
I guess there was always an educational angle right from the start, but it was always to be bathed in a creative experience first. It was meant to captivate you visually and then take you on a journey and if you walked away learning something from it then that would be an added benefit.
BSB: How did it develop from there?
LR: It didn’t begin on a dollar note. We did explore doing something on the tour itself. We talked about using the building façade as a marker to scan on, so you could stand outside the building and then you could see inside it.
Then as the project grew there was a requirement for it to reach more people, and that’s when we started trying to think what everybody has that doesn’t cost them anything to buy and is fully accessible by most Americans. And we landed on a dollar bill.
It was actually a 20-dollar bill to begin with because the White House is on the 20-dollar bill. As visual storytellers we had the idea of the White House emerging out of the bill, which would have been really great, but it was fairly pointed out that a lot more people have access to a dollar bill.
BSB: How do you think it compares to other AR experiences out there?
LR: I think in general it’s still early days in AR. We had an opportunity to experience AR a few years ago, but the problems back then were we didn’t have the tracking potential back then, or camera lenses, or software, and we definitely didn’t have the power that we have in a mobiles now. Most people didn’t even know what a game engine was five years ago. So it was a different place.
Remember QR codes and the importance of the contrast and definition? AR has a much more magical ability now. Lots of things can be markers that never could be before. That is a real opening point in terms of how advertisers may decide to use it because there’s a lot more freedom there.
There’s a lot of pressure coming into a job like this with the amount of people that are going to see it. It does need to look as good as it can within the limitations. So I’m pleased with some of the shaders we’ve built and the optimisation to get as many characters animating and moving as possible. I think we’ve made some real advances there.
People from a technical background are surprised by the quality of the render. They didn’t realise we could do things like that now. We worked extremely hard to make sure that was the case.
The other bit is, I think, still early stages but we are getting into narrative storytelling in AR. We’ll see a lot more of it if we all end up wearing headsets [like the Microsoft HoloLens]. You know that scene in Her where the character steps out of the wall? You have things like that because you can use the physical room you’re inside. And I hope that the White House is an interesting step in that direction.
Even though it’s loose enough for you to navigate around and explore, it’s a year in the White House and there are events unfolding, so it has a structure to it and we do encourage you to keep looking, keep exploring, do it again. See if you found all the presidents or not, open the Oval Office or touch the roof of the White House. There are elements in there for exploration.
Hopefully it stands as a benchmark for other AR projects that can come. And I genuinely think it can. I know that once I introduce it to a lot of people they always talk to me about how shit the last experience they saw was.
BSB: What were the biggest challenges?
LR: The technical challenges are always there the minute you want to try and push the boundaries. If I’m building for the newest iPad it’s a lot easier, but if I’m building it for more or less everyone in America to access it without a problem, then I have to work to older devices too. It needs to look as amazing on an iPhone 7 as it does on an iPhone 5.
The pressure to create something good for that kind of a stage was extremely important. You know it needs to be good.
We have a lot of love for Obama and his administration and I think there was a pressure of making sure that it represented him well and this notion of transparency that they were working on. We obviously didn’t anticipate the future, so it ended up having a bit more relevance.
One thing we did was build a pause week into the schedule so everyone stopped and we reassessed and talked about whether we were happy with it, where it was at that point and what changes we might want to make in the remaining four weeks of the job.
That was vital because I think we’d managed to answer a lot of what the White House needed but we hadn’t managed to get the charm and sense of humour into it yet.
Another thing we did that was really important was by the end of the first week we had the app built. It was a crude, square, White-House-looking building with a few bouncy 2D characters in it. What that allowed everyone to do was to allow all the stakeholders on the job to have the app and then week-by-week we all get to review it in the app, not on 36-inch screens in the studio.
I thought that was brilliant because it allowed everyone to have a go in the evening in their kitchen, have a think and in the morning they might come back with more thoughts.
BSB: What have people’s reactions been like?
LR: You never know, but we’d hoped it would have a relatively large release because the White House were going to talk about it. I think the hopes outside of that were to do with mainstream people trying it out and learning a bit about AR.
We’ve seen teachers in classrooms showing their kids. The majority of people don’t know where the Oval Office is. Maybe this project, if it is seen by millions, would almost guarantee that everyone knows that at least. And from the WHHA’s point of view that would be a huge thing.
It’s obviously been picked up a lot by people from Sky News in the UK to Jimmy Fallon in the US. That’s been great. I don’t think we could have hoped for a better result.
BSB: What has this project taught you about the direction interactive technology is moving in?
LR: I recently saw an article which I’m sure there will be a few of, which is all about how 2016 was meant to be the year of VR, but it was actually the year of AR. That rang very true.
AR is much more social and it doesn’t require any expensive hardware. There are big implications there for why it can be so useful for messaging.
I’m a big fan of the next phase, which is much more about computer vision and [the technology] understanding our environment. If you look at the concept art on the Microsoft Hololens website, I think that begins to get you excited about what the potential is. I can absolutely see the storytelling possibilities for us.
That will be interesting. Right now I still think it’s much more interesting how that’s relevant to the mobile. Google Tango is an amazing bit of kit that’s now in the Lenovo phone. And that allows us to remove the marker [the dollar bill in 1600’s case], so I can look around the room and it understands what a person is because it can track the skeleton. It understands what a door, wall or table is.
We talk a lot about how it’s important that things like Pokemon Go happened because they raise everyone’s interest and investment into an area. So you can guarantee that just from that one hit there’ll be ten more coming.
When I asked the White House why AR, the answer was because they’d tried everything else but didn’t know how to access millions of people with low to no investment. I don’t think it’s gimmicky. I think good AR experiences have a place because they don’t require any investment from the user. Right now if everyone’s getting close to Christmas and thinking ‘Oh my God. Am I going to spend 1,500 quid on a VR setup?’
This is an easy way to reach a lot of people and it still has the emerging technology feel. It still is exciting. It’s not like everyone has seen AR. It’s still in its infancy and it comes with that excitement.
I’m pitching to everyone but I’d love to do a graphic novel. You could stare at the page, follow a 2D story, when you touch it it animates or comes to life and you watch it. Or maybe your character breaks out into 3D, opens the book and turns the page for you and you follow them to the next page. There are some really unique opportunities in that space for storytelling. A lot of AR has been pointing it at this thing, seeing a 3D thing appear and that’s it. Now we’re getting into the area where I can delve deeper; I can explore something or touch on something to open another window and dive into that and have another experience.
I think that gets away from it being gimmicky at that point because whatever you want to do you can do. That’s the point. Everything is interactive and you can engage with whatever you want.