This year, as part of the Next Reality 30, in partnership with Snap, we're spotlighting an up-and-coming innovator in augmented reality. Specifically, we were searching for an independent developer or creator who embodies the bleeding edge spirit of AR and its ability to leverage machine learning. This year's honoree is Aidan Wolf.
• Don't Miss: Next Reality's 30 People to Watch in Augmented Reality for 2020
When Magic Leap first made its way into the hands of developers, an informal posse known as the Leap Squad quickly formed around the platform. One of its members was Aidan Wolf, an indie developer who has gone on to leverage machine learning to publish some of the more unique augmented reality mobile apps out there under his imprint Kevaid.
In those early days, Wolf worked on an application called Project Play, a 3D modeling, animation, and game creation tool. While he didn't work with Magic Leap as an employee or even a contractor, he did receive support from the company, including a Magic Leap One headset and development assistance.
Ultimately, Project Play did not make its way to the Magic Leap World app store, but Wolf would eventually leverage the work for the development of other mobile AR apps.
"It was fun during the rise, but after Magic Leap moved from consumer to enterprise the spirit powering the community really flamed out, so I haven't really been in touch with my fellow Leapers since," says Wolf. "I still see the amazing things they've been up to, so I'm happy to see the spirit is still very much alive but in other places in the XR industry and beyond."
A native of rural Connecticut, Wolf leaned on gaming to pass the time in his youth. "I'm from a small, sleepy farm town in western Connecticut, and having the luxury of boredom growing up, wasted a lot of my time making computer games," says Wolf. His interest in gaming eventually evolved into game development
"I don't have a formal background in anything I do, never graduated school or landed a cool job, but I have been able to make games and apps for a living since I was 16," says Wolf.
Wolf got his start in AR with Capsule, a geolocation-based social app he developed through an accelerator program in Kentucky. After reaching beta, Capsule ran out of funding and shut down in 2015.
"Though that company didn't work out, the experience greatly altered my life's trajectory and brought me to where I am today," says Wolf. "For the last six years, I've been thoroughly obsessed with AR and luckily have had a couple of cool projects work out."
One of those projects is Blue Sky Paint for iOS. The app takes the well-tread concept of AR doodling but uses the sky as its canvas. Wolf wrote a machine learning algorithm that identifies the sky and segments it for AR projection, with ARKit and sensor APIs assisting in the effort.
"Basically, [the algorithm accounts for] really general things that are true about the world most of the time: the sky is always up, so if your phone is pointed up, it's likely looking at the sky. The sky is generally blue (even the clouds), while the ground is usually lit directly by the sun, giving it a yellow tint," says Wolf. "GPS and compass orientation are what's used, but we do have a re-projection algorithm which redraws the drawing to match different perspectives and distances. The fun thing about using the sky is we can get away with a lot of GPS distortion since the sky is 2.5 km off the ground."
His follow-up, DoodleLens, built on the algorithm for Blue Sky Paint and his work on Project Play, was mixed with some inspiration from another creator.
"It was a Saturday night and instead of working on my side project I was procrastinating on YouTube watching a video from one of my favorite creators, James Rolfe (a.k.a. YouTuber Angry Video Game Nerd).
The video covered various cameras that Rolfe has used, including a '90s camcorder with a feature called "Memory" that could burn monochromatic images into film for making title cards and other graphics. Rolfe used it to add effects to his one-man movies.
"I instantly saw the translation to AR," said Wolf. "After picking my jaw off the floor, I immediately cracked open Unity, tweaked my Blue Sky Paint sky segmentation algorithm to detect paper, and ended up making the first version of DoodleLens in just a couple hours. I posted it that night before I went to bed, and the rest is history."
Like Blue Sky Paint, DoodleLens (formerly called Doodle Cam) uses a machine learning algorithm, but, instead of identifying sky, it uses a compute shader to segment the camera view based on color contrasts and cuts out the areas with the darkest colors. This enables users to cut drawings, logos, text, and other images from the real world, modify them in the app, and then paste them elsewhere in the physical realm. Wolf has continued to add new features to the app, enabling users to create AR animations, record videos, and share them across the social media landscape.
Wolf has also delved into the relatively new AR platform known as Snapchat's Lens Studio. Wolf and his creative partner, Kevin Habich, were accepted into the Snap AR Creator Residency program, which provided the opportunity to learn from Snapchat's internal creative teams and receive some funding to develop their project.
"The Snap AR Creator Residency was awesome, I had never done a residency before so I'd describe the experience as a sort of 'Lens Accelerator Program,'" said Wolf. "I basically went from knowing nothing about Lens Studio or how Snapchat's AR platform operates to knowing it's the future of the AR space, and have since gained a healthy addiction to creating Lenses."
The result of the residency was Glam Quest, which is available now on Snapchat. The Lens isn't your typical face or world effect, but rather an AR game featuring a dozen animated characters, dialogue cinematics, virtual environments, and a hand-tracking combat system, all managing to fit within Snapchat's Lens limit of 4MB.
"With this Lens, we became obsessed with obliterating people's preconceived notions of what a Lens could be," said Wolf. "Now we're rewinding a bit and are planning to break Glam Quest into smaller, bite-sized gameplay chunks in a new concept we're calling Adventure Cards."
His next project, code-named Troubletown, seeks to subvert preconceived notions of AR yet again. It's a cartoon series that unfolds in the real world via AR rather than through the boundaries of a two-dimensional screen. The pilot series concerns Blorp, an alien and member of Beam Bois, an "intergalactic boy band."
"They ultimately need your help putting on the greatest show ever, on a planet whose government is hellbent on making sure no one knows they exist," said Wolf. "It's got a cool physical component that enables players to anchor gameplay and story moments to specific real-world locations, with our ultimate goal being to blur the lines between games and reality and get people outside playing and adventuring together."
Aidan Wolf
*Editorial Disclaimer: Snap had no role or input in the selection of this year's honoree, the selection was made solely at the discretion of Next Reality's editors and writers.
Cover image via Aidan Wolf
Comments
No Comments Exist
Be the first, drop a comment!