Innovation, entrepreneurship and building metaverse

Jeremy Hartmann successfully defended his dissertation in December 2021 and launched an entrepreneurial venture that took his academic research to the next level.

As a graduate student in the Cheriton School of Computer Science, Hartmann's PhD was a study of merging the real and the virtual, which his new company MTION is putting into practice as a virtual reality streaming service and digital clubhouse.

While pursuing his PhD, Hartmann was involved in the Waterloo innovation ecosystem and was a winner of the Concept by Velocity startup challenge for graduate students. He completed internships at Microsoft Research in 2018 and Adobe Research in 2019, and developed the technical side of his current venture in the Human Computer Interaction Lab, supervised by computer science professor Daniel Vogel.

"My research program took up different types of extended reality," Hartmann said. "I've been working to build and understand different interaction techniques and different systems to enable things like augmented reality, virtual reality and mixed reality."

Augmented reality is layered on top of our everyday physical reality, such as through a pair of digital glasses that may show information or overlaid images in physical space. Virtual reality is a more immersive experience, such as through VR headsets that create a digital space. Mixed reality and extended reality are terms to describe the combination of various forms of augmented reality, virtual reality and physical reality.

A continuum of realities

"The way I like to think of it is as a continuum," Hartmann continued. "On one side you have unmediated physical reality. That's the space we occupy with our physical bodies. On the other side of the continuum we have a completely virtual reality, with things like VR headsets.

"The more senses that are immersed in the virtual world, like visual, auditory and touch sensation, the further on this side of the continuum. Then you have things in the middle, like augmented reality, where you have a blending between the real and the virtual."

What Hartmann's company MTION is doing is somewhat unique, even in this continuum of realities, in that it builds a virtual reality space that then incorporates the physical reality into that space, both through the design of the space and through sources of video, audio and other feeds.

Hartmann's virtual avatar within the VR space, looking at a screen showing Hartmann in physical reality. Image courtesy MTION.tv.

"You still get the benefits of being immersed inside the virtual world but then you can also take aspects and pieces of reality and impose them into your virtual space," Hartmann said.

The main customers for the service and the clientele the company caters to are currently video-game streamers. MTION's virtual clubhouse, as Hartmann calls it, changes the way that video games are spectated by allowing the audience to essentially be inserted directly into the game, such that they can interact with the streamer and other viewers around them and move through a constructed 3D game-like space.

An evolving metaverse

Because of its unique approach to virtual spaces, Hartmann's company and his research are contributing to the creation of the metaverse, a concept that has come to public attention recently. The slogan for Hartmann's company is "building the metaverse, one virtual brick at a time."

In a general sense, the metaverse points to these various forms of blended and mixed realities, to something that amplifies, makes more of, or transcends a particular way of being. The metaverse, as it is being created by entrepreneurs like Hartmann, is a conjunction of physical and digital worlds. It is not just reality or virtual reality or augmented reality, but an intermingling of all these kinds of reality.

A piece of concept art for metaverse spaces that map or overlay some typical and not-so-typical physical spaces. Image courtesy MTION.tv.

"The metaverse is really a collection of a bunch of different things," Hartmann explained. "Virtual reality and augmented reality is one aspect of it. But there's also the issue of how we use these spaces and how they complement or overlap with everyday life. A big part of it, at least right now, is about play and entertainment, and so that's where MTION is positioning itself."

Asked about the future trajectory of the metaverse and the technologies that make it possible, Hartmann said it's all about creating things that in some way improve people's lives.

"I think what people fundamentally want is to live meaningfully. Any versions of mixed reality and any emerging aspects of the metaverse are going to have to be things that people can get something from. It all has to be in the service of creating meaningful and authentic human experiences."

Overview of the MTION studio streaming software, showing the ways users can create and manipulate a 3D space that incorporates a video feed.