“Immersive experiences” is one of those buzzphrases that describes an age-old part of being human: the feeling of being inside of a story, whether it’s verbal, written, or otherwise. But with the rise of virtual reality and augmented reality, storytellers have new tools for transporting people to other worlds.
The only problem is, they require clunky hardware, like VR headsets, or they force you to view the world through a screen, as is the case with using augmented reality on smartphones. But a new prototype points toward a world in which you don’t need any kind of screen at all to enjoy new types of storytelling. Called Lumen, it’s a projector built in the form of a flashlight that uses machine learning to recognize objects around it and can project information onto different surfaces. For instance, you could use it in a museum, shine the light on a painting, and watch an animation of the artist appear on the wall to tell you about the work.
Lumen is the brainchild of Sweden-based interaction designer Arvind Sanjeev, who created it as his design thesis while studying at the Copenhagen Institute of Interaction Design. Sanjeev, who previously started a series of companies focused on automation, wearables, and DIY, wanted to create an “immersive experience” that didn’t have all the worst parts of today’s current AR and VR technology. “I started thinking about, how can I make something that can immerse people in these kind of digital, alternate realities without confining them to a headset,” he says. “How can a group of people collectively be immersed together?”
While other projects use projection mapping to transform the world around you– like changing your clothes while you’re wearing them or even turning your whole home into a screen–Sanjeev wanted to create something portable that doesn’t require a room filled with sensors. With these design constraints in mind, he wrote a story with a magical flashlight at its center: in his fiction, a time traveler owned the flashlight but lost it in the 1940s; in 2017, a family discovered it and learned how it unlocked places in their world through its enchanted light.
Then, he set about making this story a reality by talking to interaction design experts about which interactions the make-believe family had that were most compelling, and then figuring out a way to build them. Sanjeev created a rough prototype by duct-taping together a Raspberry Pi computer, a projector, and a battery pack. Now, he wants to build out a storytelling dashboard, where designers can use an online portal to create interactive experiences within physical space—programming Lumen to react in certain ways to particular objects in an environment.
“background graphical interface that will serve as the story builder and dashboard. ”
The idea has several applications: in museums, in education, and in projecting the apps that live on our phones into the physical environment. For instance, Sanjeev imagines Lumen could project the Spotify app onto a real radio, and you could control the volume and song by touching the physical surface instead of the digital one.
But for now, he’s focusing on the museum application. This week, Sanjeev won the Playable Museum Award, the first of its kind given by the Museo Marino Marini in Florence, Italy, to develop his prototype into a demo for the museum, which focuses on contemporary art. The challenges ahead lie in making the prototype fully functional within a beautifully designed, flashlight-like object that’s intuitive for people to use. Sanjeev plans to relocate to Florence for a few months to continue developing Lumen.