Virtual Reality at UCSF

Updated February 4, 2020

Project Overview

We are developing new, cutting-edge methods for the visualization and analysis of molecules and cells using virtual reality (VR) technology for analyzing important experimental data. These novel methods will have significant impact in our understanding of important aspects of structural biology, including drug design, the functioning of complex molecular machinery, and basic cellular properties such as motility. We aim to bring VR into routine use in labs at UCSF, with suitable analysis problems in cryoEM and light microscopy, and to advance VR capabilities for analyzing ligand-receptor interactions. This work will pioneer the way so that other research labs at UCSF can more easily adopt this technology.

News

Research Applications

Drug binding sites
Matt Jacobson lab
Weekly group meetings using VR to analyze ligand binding using ChimeraX. Contact Wilian Cortopassi Coelho.
p97 inhibitors
Michelle Arkin lab
Inhibitors of p97 involved in protein recycling and possibly neurodegnerative disease. Virtual reality demonstration for UCSF Campaign.
Antibiotic resistance
Danica Fujimori lab
Mechanism of post-translational methylation of a ribosome nucleotide causing antibiotic resistance. Shown in VR for Byer's award reception.
Cell motility 3D light microscopy
Dyche Mullins lab
ChimeraX VR displays 3D microscopy time series. Neutrophils crawling through collagen filaments.
Calcium signaling in neurons
Bo Huang lab
Dan Xie observes calcium signaling in 8000 zebrafish neurons using 3d light-sheet microscopy time series. Demonstrated in VR for Byer's award lecture using ChimeraX VR.
Neurodegeneration in zebrafish
Steve Finkbeiner lab
Neurodegeneration in zebrafish in 3d light microscopy time series from Jeremy Linsley at Glastone Institute shown with ConfocalVR.
DNA origami virtual lab
Shawn Douglas lab
Interview with Shawn on the Foo Show about his VR nano-machine lab simulation.
Brain disorder treatment
Adam Gazzaley lab
Neuroscape.

VR Software for Research

ChimeraX VR
Tom Ferrin lab

The ChimeraX visualization package can display atomic structures and microscopy data with the vr command. A tutorial explores mutations in a scorpion toxin.

Collaborative structure viewing
Tom Ferrin lab

ChimeraX VR meeting command allows multiperson viewing of structures and microscopy data.

ConfocalVR
Tom Skillman and Caroline Stefani at Benaroya Research Insitute

ConfocalVR displays multichannel 3D light microscopy data.

Protein Databank VR viewer
Tom Skillman at Benaroya Research Institute

AltPDB collaborative viewing of Protein Databank molecular structures.

Nanome
from Nanome, Inc.

Nanome allows display and analysis of molecular structures of proteins including multiperson collaborative sessions.

syGlass
from IstoVizio, Inc.

syGlass displays volumetric and microscopy data, designed for VR.

Software for Education

Molecular Zoo
Alan Brilliant in Tom Ferrin lab

Molecular Zoo is a biomolecule education VR program that lets children handle fully flexible molecules of life: water, aspirin, saturated fat, carbon dioxide, caffeine, ATP.

Peroxiredoxin Fish Tank
David Doak at Norwich University of the Arts
Nick Young and Bianca Haux at the Centre for eResearch at the University of Auckland
Juliet Gerrard and Michael Barnett from the University of Auckland School of Biological Sciences

Fishtank is a STEM education application showing the pH dependent formation of protein dimers, rings and tubes of an antioxidant enzyme. Currently (Sept 2018) only Unity source code releases are available at Github, but a binary release for Windows is coming.

Protein Backbone Explorer
David Doak at Norwich University of the Arts

Work in progress: interactive polypeptide backbone viewer for teaching using Unity and Oculus SDK. Currently (Sept 2018) only Unity source code releases are available at Github, but a binary release for Windows is coming.

Nano Simbox iMD
Interactive Scientific

Nano Simbox iMD allows exploring flexible molecules with molecular dynamics computed in real-time on a server. It has four scenarios, handling Buckminster fullerene, threading methane through a carbon nanotube, changing the screw-sense of an organic helicene molecule, and tying a knot in a 17 amino acid peptide. Developed by Interactive Scientific in collaboration with the Glowacki lab at University of Bristol, and described in this article.

Water VR
Jonas Boström and Magnus Norrby

Water VR shows water in gas, liquid and solid phases for teaching chemisty, specifically aimed at 7th grade students. Developed by EduChemVR.

VR Equipment

Shared VR equipment
Tom Ferrin lab

An HTC Vive virtual reality headset for viewing your molecular structures or microscopy data is available at the Visualization Vault in Genentech Hall at UCSF Mission Bay. Contact Tom Goddard.

Headset Notes

We use Windows 10 Home on our VR computers. We use primarily SteamVR to interface to all headsets. Wikipedia headset comparison. VRcompare headset comparison.

  • Vive Pro ($1400, released April 2018) - Higher resolution (1440x1600 per eye) makes reading text easier. More expensive ($1400 with hand-controllers and version 2 base-stations).
  • Samsung Odyssey ($400, released Nov 2017) - Most portable, using inside-out tracking instead of base stations. Higher resolution (1440x1600 per eye). Shorter cord to computer limits range of motion. Hand controllers have no recharge port. Needs bluetooth used by controllers. Works with Windows Mixed Reality runtime, or SteamVR runtime.
  • HTC Vive ($500 + $100 for headstrap, released April 2016) - Best tracking system. Deluxe headstrap is easier to adjust, more comfortable, and provides audio. Resolution (1080 x 1200 per eye).
  • Oculus Rift ($400, released March 2016) - Best hand controllers, not as bulky, extra button, better button placement. Tracking is a pain, best to have a 3rd camera sensor, uses lots of USB ports. Resolution (1080 x 1200 per eye).
  • Oculus DK2 - Historical developer kit 2 headset, released July 2014. Positional tracking using a single camera, loses tracking when facing away from camera. Resolution 960 x 1080 per eye. Frame rate 60-75 Hz.
  • Oculus DK1 - Historical developer kit 1 headset, released March 2013. Only rotational tracking. No external base stations. Oculus SDK required app GPU code to correct lens distortion and chromatic aberration. Resolution 640 x 800 per eye. Frame rate 60 Hz.

Graphics Card Notes

We have extensive experience with the GPUs noted below and have also been experimenting with others, so this list may be a bit out of date:

  • Nvidia GTX 1080 - Main card we have used at UCSF. No problems observed. 8 Gbytes of memory.
  • Nvidia Quadro P6000 - Primarily because of its high price, we recommend this card only for special applications. We use this for both LCD shutter glasses and VR. The Quadro cards provide a 3-pin DIN connector for the left/right eye synchronization with shutter glasses. The IR emitter used with shutter glasses is overpowered by Vive (version 1) base stations so shutter glasses cannot be used with Vive base stations plugged in. This card has 4 display port outputs and no HDMI output. The Samsung Odyssey VR headset would not start using an HDMI to DisplayPort adapter with the Windows Mixed Reality setup program giving an error (Sep 2018). 24 Gbytes of memory. This card was generuously provided by NVIDIA Corporation's GPU grant program.

Projects

Virtual Reality in the San Francisco Public Libraries - pre-proposal to setup VR systems for informal STEM education in the 27 branches of the San Francisco public library. Full proposal due November 17, 2018.

Analysis of Molecules and Cells in Virtual Reality - Setup research VR systems in 5 UCSF labs for cryoEM structure refinement, 3D light microscopy, and multi-person VR collaborations. Funded Aug 1, 2018 for 1 year.

Exploring Drug Binding using Molecular Dynamics and Virtual Reality - proposal by Tom Goddard, Wilian Cortopassi Coelho, Matt Jacobson, Michael Grabe, Tristan Croll, and Tom Ferrin to study mutations effecting drug binding sites. Funded. January 2018.

Virtual Reality Molecular Playground - proposal for STEM education VR application to familiarize students with biomolecules. Not funded. January 2017.

People

Steven Abbott - Developer of many WebVR apps related to solvation.

Marc Baaden (Institut de Biologie Physico-Chimique) - Developer of UnityMol library for representing molecules in the Unity3D game engine.

Alan Brilliant (UCSC) - Developer of Molecular Zoo biomolecule education application.

Wilian Cortopassi Coelho (UCSF) - Studies drug-receptor interactions using VR in Matt Jacobson's lab. Publishing 3 drug studies in July 2018 using VR analysis.

Gareth Denyer (University of Sydney) - Developed Peppy protein structure teaching app with David Doak. Teaches structural biology at the Sydney Immersive learning laboratory.

David Doak (Norwich University of the Arts) - Developed peroxiredoxin fishtank STEM application and is working on a protein backbone explorer app.

Shawn Douglas (UCSF) - Developed VR lab simulation for DNA origami research.

Tom Ferrin (UCSF) - Provides shared HTC Vive VR equipment in the Visualization Vault.

Nicolas Floquet (U Montepellier) - Manages the MUSE-VR education facility in the Institut des Biomolécules Max Mousseron.

Christopher Fortney (NIH) - Manages the Virtual Reality Consulting Service at the NIH library.

Zack Gainsforth (Space Sciences Lab) - VR of electron microscopy of cometary dust.

Tom Goddard (UCSF) - ChimeraX VR developer.

Matt Jacobson (UCSF) - Analyzes drug binding, weekly use of VR in group meetings.

Meghan McCarthy (NIH) - Using ChimeraX VR with radiologists at NIH.

Dyche Mullins (UCSF) - VR visualization 3D light microscopy of crawling neutrophil cells.

Mike Schmid (Stanford-SLAC CryoEM Center) - VR visualization of EM tomography of bacteria, and atomic resolution virus capsids.

Tom Skillman (Immersive Science) - Founder of Immersive Science, developing VR science applications ConfocalVR for light microscopy, AltPDB for collaborative molecular structure viewing, and FCS-VR for flow cytometry analysis.

Caroline Stefani (Benaroya Research Institute) - Developer of ConfocalVR for light microscopy, working with Tom Skillman.

James Tyrwhitt-Drake - Developer of speech recognition command input from ChimeraX. Worked with Meghan McCarthy at NIH on virtual reality applications.

References


Tom Goddard / UCSF / goddard@cgl.ucsf.edu