Immersive Entertainment for Hospitalized Kids
  • Client: AT&T Foundry Palo Alto
  • When: 2017
  • Team: Product Manager, Virtual Reality Development Engineering team (Quantum Interface), and Non-profit with connections to musician and hospital (Melodic Caring Project)
  • My Role: Product Manager
Overview

As an Innovation Lead at the AT&T Foundry, I work on applying emerging technologies to make the world a better place. In 2017, I led a technology for good project to create immersive entertainment experiences for children in hospitals as a form of distraction therapy, through collaboration with Quantum Interface (technology startup) and the Melodic Caring Project (nonprofit).

AT&T is transforming into both an Entertainment and Telecommunications company. Projects such as this highlight the need for a strong 5G use case where there is a need for high throughput and bandwidth to support a quality experience for users. By applying emerging technologies to real-life problems, we are able to solve business needs, uncover new markets, and help change lives.

Immersive Entertainment for Hospitalized Kids: Amos Lee Concert in Virtual Reality on Google Daydream VR Goggles UX Mockup

Immersive Entertainment for Hospitalized Kids Poster

Background

In 2016, I initiated and spearheaded our exploration of the Augmented and Virtual Reality market to propose strategic business plays for AT&T and Ericsson. As I engaged with our ecosystem, I discovered two amazing companies, each experts in their own field — one a technology company specializing in virtual reality and custom interfaces (Quantum Interface) and the other a nonprofit that streams concerts to children in hospitals (Melodic Caring Project).

It occurred to me that if I connected Quantum Interface’s next generation, interactive interface for 3D experiences together with a 360 virtual reality video of Melodic Caring Project’s live streaming of personalized concerts to hospitalized kids, we could create something truly memorable. So, we did just that.

Live Streaming

After introducing everyone, the project took off immediately! Quantum Interface brought in SubVRsive to make the Virtual Reality capture a reality, and Melodic Caring Project brought in singer-songwriter Amos Lee and team as the content creators at the core of this experience. Together, the team used the magic of 360 live streaming to transport hospitalized children to an Amos Lee concert experience that they could share with their loved ones as they received encouragement from near and far!

Our partners were pivotal in making our live stream a success. Fresh off this achievement, we quickly moved onto the next step — building an interactive, immersive experience.

“Watching the concert as a VR show completely swept Maya away. She was feeling badly from recent chemo and having the glasses and the 360 experience took her away from the yuck that she had been feeling.”

– Maya’s father (Maya is in the photo below)

VR Virtual Reality 360 Video Amos Lee Concert Experience for Maya

Hands-free, Interactive Experience

After the live stream, our team created a post-produced 360 experience with an interactive layer on top of this recorded concert. Children are not only able to be immersed in the concert, but also can interact with the environment and control playback using only small head movements, all completely hands-free! This is huge for patients who may not just be confined to their care facilities, but may also be bedridden or tethered to medical equipment with limited mobility.

We have been fortunate enough to share our interactive experience with countless viewers both inside and outside of hospitals, and the response has been overwhelmingly positive.

Instructions for Immersive Entertainment for Hospitalized Kids
AT&T Shape Conference attendees enjoying the Immersive Entertainment for Hospitalized Kids

Quantifying the Impact

We started by collecting anecdotal evidence that we are helping patients escape from the discomfort of their hospital rooms. Now, we are quantifying the impact our immersive entertainment experience is providing as distraction therapy through clinical trials.

With the use of quantifiable biometric sensors and qualitative surveys, we are able to correlate the two to assess how effective this experience is as distraction therapy and what factors are necessary to build an effective distraction therapy experience.

Streaming a Catalog of Content

As we widen our audience reach, we are also expanding the catalog of content accessible within our video player. Using AWS, we are building a repository of content pieces that a user can draw upon. Patients are able to navigate hands-free through a library of options to suit their needs, whether it be a desire for an outdoor adventure to a relaxing meditation by the fire.

Amos Lee Concert being recorded backstage in 360 degrees for Immersive Entertainment for Hospitalized Kids
Amos Lee on stage during the concert for Immersive Entertainment for Hospitalized Kids

Press Release

Austin – March 7, 2017 – Philadelphia-based singer-songwriter Amos Lee played a sold-out concert at the world-renowned Austin City Limits Moody Theater on Saturday, February 25, 2017. This time, Amos’ sold-out show didn’t just reach the crowds of the Moody Theater. With the support of partners including Quantum Interface, SubVRsive and the Ericsson team at the AT&T Foundry, the Melodic Caring Project, a nonprofit that bridges the gap between music, technology, and patients battling serious illness by live streaming personalized concerts to kids and teens in the hospital, live streamed its first-ever multi-camera, virtual reality 360 video show directly to kids’ hospital/home care rooms.

Since Melodic Caring Project’s founders, Levi and Stephanie Ware, began this journey seven years ago, the nonprofit has broadcast approximately 400 concerts to nearly 5,000 children all over the world and has partnered with some of the world’s biggest artists, including The Black Eyed Peas, Jason Mraz, Andra Day, Alabama Shakes, Rachel Platten, Daughtry and The Head and the Heart. The nonprofit gives the performing artists a list of kids (aka rockSTARS) who will be watching the night of their show. Then, during the shows, the artists call each rockSTAR out by name, offering support and words of encouragement.

The project all started at the AT&T Foundry, a place where corporations, technologists, start-ups and content creators come together to incubate ideas and test concepts. With a mission to take the viewing experience to the next level, a three-way collaboration was created between The Melodic Caring Project (MCP), Quantum Interface and AT&T Foundry/Ericsson. Quantum Interface and MCP crafted a vision to live stream a concert in fully immersive 360 Video and, by drawing in other parties from their networks, were able to make that vision a reality. With the support of Ericsson and thanks in large part to AT&T Foundry’s network of partnerships and the leadership vision that pioneered this collaboration along with Amos Lee’s desire to get involved, that vision came to life in Austin’s famous Moody Theater.

On the night of the show, SubVRsive, which handled the end-to-end production and distribution of the stream, placed cameras in two spots on stage and streamed the entire concert straight to the VR headsets of the evening’s rockSTARS. As the kids watched the concert, they were able to stand on stage with Amos Lee and the band, look out into an audience full of supportive faces, and leave the sights and sounds of the hospital behind to truly feel present at the show. Saturday’s concert in Austin was the first Melodic Caring Project concert that used state-of-the-art VR technology to give kids an opportunity to virtually be on stage with Amos and his band.

“One of the rockSTARS watching the show in Austin was a girl named Maya,” said Evan Blackstone, VP of Melodic Caring. “Maya had previously taken to Amos after watching one of his concerts and afterward, Amos paid her a hospital visit in Seattle. On Saturday, Maya lay on her couch in her hospital room, yet was front row at the show getting the full live experience of hanging out with her buddy Amos. He spoke directly to her and the other rockSTARS throughout the night and, although he was miles away, they were all able to be right there at the show, no longer in the isolation of their hospital rooms.”

“It’s been said that virtual reality is the ultimate empathy machine because it allows users to not just look through a frame, but step into it,” said Austin Mace, CCO of SubVRsive. “Being able to let these kids step through the frame and be on stage with Amos was a really powerful thing and we’re grateful to have had the opportunity to leverage 360 live streaming for such a great cause.”

The final stage of this project will be to combine the video with a virtual interactive layer, which Quantum Interface will be contributing. In this way, the amazing 360 video will become an interactive experience, so the children will not only be able to be immersed in the concert, but interact with encouraging messages, the environment, and other virtual content using only gaze, completely hands-free. This is a significant and first in world experience that will bring the 360 experience to life even more.

“Music is a celebration of life. Musicians have such a unique, magical ability that deeply moves people. We live in a time and place that needs this energy more than ever. Especially the kids that Melodic Caring Project works closely with. Levi, Stephanie [Ware] and crew are world class. We look forward to supporting their process and helping them tell that story”.

– Amos Lee

Learn More

Check out AT&T’s featured article on the project. AT&T’s LinkedIn post. AT&T’s Tweet. AT&T’s Innovator Series post.

Check us out on the Ericsson Tech For Good Blog.

Read SubVRsive’s perspective on our project.

Read our partner Melodic Caring Project’s perspective on our project.

We are also receiving international coverage.

Check out one of our demos at the following locations: AT&T Foundry in Palo Alto, AT&T Foundry in Houston, Ericsson Experience Center in Santa Clara, Ericsson Experience Center in Plano, Ericsson Experience Center in Sweden, AT&T Forum in D.C., Quantum Interface in Austin, and Melodic Caring Project in Seattle.