2017 2018

August 20, 2019

2017 – 2018

Randall Rode

Spring break offers students an opportunity to decompress from the stress of mid-term exams and other academic deadlines. An especially cold and snowy winter in the northeast had many Yale students dreaming of an escape to warmer climates.  This year’s 2018 research trip to HP Labs in Palo Alto, CA offered that opportunity and a lot more to one administrator, three faculty, and three students involved in the Yale – HP Blended Reality research project. 

HP Labs’ Alex Thayer played host, helping the group connect with Silicon Valley product teams at Google Daydream and Intel RealSense, as well as arranging a full day of conversations with the HP Labs Immersive Experiences team.  The trip agenda included a full day at Stanford University, sharing research insights at the Virtual Human Interaction Lab and with teams at Stanford Medicine anatomy, instructional technology, and simulation lab areas. The goals of the trip are to expose the Yale faculty and student researchers to commercial development approaches; introduce them to the latest advances and successful applications of blended reality technologies; and, most important, help them build relationships and partnerships with other research/product development groups.  By every measure, the trip was a roaring success.  

The Yale team landed in San Jose at noon on March 19 and headed right off to meet with Intel’s RealSense team.  Our hosts, Brian Pruitt and Anders Grunnet-Jepsen, demonstrated the capabilities of the stereo depth cameras that are supported by a rich software development kit (SDK).  Visual input is a technical challenge across the Yale teams. For instance, Yale Department of Music faculty member Konrad Kaczmarek often has students use hand gestures to control electronic music performances.  As at each of the stops, the Yale group shared highlights of their own work, helping Brian and Anders better understand the needs and opportunities within a higher education environment. To further the Yale team’s research, Intel has provided three of the RealSense D435 stereo depth cameras; the cameras will be used by teams working over the summer on a range of experiments.  

After a good night’s rest, day two started at HP Labs. Alex Thayer, director and chief experience architect of the HP Immersive Experiences Lab, was joined by his colleagues Tico Ballagas, Alex Ju, Ji Won Jun, Hiroshi Horii, and Kevin Smathers.  The Yale team included Justin Berry, Johannes DeYoung, Konrad Kaczmarek, and Randall Rode and students Lance Chantilly, Jack Wesson, and Valentina Zamfirescu. One of the research areas the Yale Blended Reality teams have been exploring this year focused on ways to make the user interface of virtual reality environments more intuitive and natural.  Movement and interface within virtual environments typically requires the use of hand-held controllers, with little standardization across programs and apps; each app can require its own learning curve. The Yale teams envision a future with standardized interfaces for virtual reality experiences, with a greater reliance on physical body movements rather than clicks of a controller.  

Through the day’s discussions we learned the HP teams have similar interests. A day of productive and informative conversations about these ideas ensued — conversations that will be continued over the coming year in a planned series of check-ins to share research findings.  

A man in VR in front of a model of the human skeleton.

Stanford University was our host for the final day’s research visits.  At the Human Virtual Interface Lab we viewed a range of their VR projects including walking over a chasm on a virtual plank, feeling an earthquake’s rumble in an emergency planning application, and having an embodied experience to empathize with the challenges of homelessness. Through several labs within Stanford Medicine the group discussed challenges in producing medical simulations, the delivery of virtual anatomy models within a classroom, and supporting online delivery of 3D teaching assets. Once again, the Yale team shared examples of their work leading to engaging discussions with our Stanford peers, and a number of possibilities for new collaborations.  

Two people standing beside a hospital bed with a dummy in it.

A college campus can be an insular experience, with everyone heads down in the day-to-day demands of the academic schedule.  This Blended Reality team research trip offered an opportunity for team members to take a step out of the daily routine, and gain a broader understanding of the range of use cases and techniques being employed across the virtual, augmented and 3D reality technology fields.  New partnership opportunities were formed, and existing relationships were strengthened. What we learned on this trip is already influencing project outcomes, and will continue to invigorate the research and project work over the coming year.