March 26, 2001

NC State Researcher Fine-Tunes Robotic Control with Virtual Reality

Subjects participating in an experiment to investigate
the relationship between presence in virtual reality
and subjects’ allocation of attention to visual displays
and their surroundings. The goggles worn by the
subjects create a three-dimensional image.

Have you ever identified so strongly with a movie character that you felt the action was happening to you — that youwere the one in a bus going over a cliff or being chased by a gigantic boulder? Did you grip the armrests of the movie theater seats with your heart pounding? Understanding that effect is the goal of Dr. David B. Kaber, assistant professor of industrial engineering at NC State University. Kaber and his graduate students want to characterize this phenomenon as it occurs in the use of virtual reality systems so they can apply that understanding to the design of graphical user interfaces for remote control of robotic systems, which have practical applications from medicine to space exploration.

Telepresence is the technical term for the feeling that you are actually present at a site far removed from your physical location. Many of us have accidentally experienced this, but if researchers can determine exactly what triggers this sensation, then they can design virtual reality systems and interfaces that assist operators of remote robotic manipulators, or teleoperators, who need to be fully immersed in the experience for maximum efficiency.

Kaber’s studies link cognitive psychology and engineering in experiments to determine the degree of telepresence human subjects experience in synthetic tasks simulating real-world teleoperations (remote control robot operations), such as landmine disposal and hazardous materials-handling. Psychological factors suspected to influence telepresence include subject concentration, allocation of attention and "situation awareness," a measure of one’s perception and interpretation of physical surroundings. Engineering technology can enhance the sense of telepresence through virtual reality interfaces integrating head-mounted displays with high fidelity, three-dimensional models of remote robot/task environments and live video feeds on remote manipulator action. According to Kaber, the quality of the interface is critical to telepresence and, therefore, high quality performance of remote system operators. "We’re looking at system factors, but we’re also looking at psychological factors, and I think that’s a nice blend of engineering and psychology in a research program," said Kaber.

This multidisciplinary approach to virtual reality research is the focus of Kaber’s four-year National Science Foundation (NSF) Career grant. Four consecutive projects will each be the focus of one year of the study. Years one and two, in which technological and human factors influencing telepresence were identified, are complete.

The current third year is focused on development of an objective measure of the strength of telepresence. So far, researchers have only been able to use subjective methods, such as surveys, to measure presence. Kaber’s objective measure involves determining the distribution of virtual reality user attention to virtual and real stimuli; specifically a ratio is to be computed based on the number of signals a person perceives in a real control setting versus virtual displays of a remote environment. The ratio is expected to show how the subject allocates attention between the real world and virtual environment interface. According to Kaber, "If we can objectively quantify attention distribution between the virtual reality display and attention to the real control environment, then telepresence may be significantly predicted."

The fourth year of research, scheduled for next year, involves integration of technological and psychological factors into a comprehensive, predictive model of telepresence that can serve as a framework for future telepresence research. The goal of Kaber’s research is to design systems to boost human subject performance in virtual environments and the control of teleoperators/telerobots. "Ultimately, we want to create systems that will allow people to control a robot’s arm as they would control their own arm," said Kaber.

Practical applications for this research cover a broad and diverse range of disciplines in which effective transportation of human sensory, cognitive and motor skills to a remote environment is important. Advanced telerobots designed for telepresence could allow expert surgeons to perform operations on patients from miles away. Such systems could also enhance planetary exploration by allowing scientists to control telerovers on the surface of Mars in collecting geological samples and to examine them from Earth. Telepresence-based design of telerobots could allow for effective remote maneuvering of robots through mine fields and deactivation of land or ocean mines without a human ever being in the area.

-- rudd --

Technical Contact: Dr. David B. Kaber, 919/515-3086 or dbkaber@eos.ncsu.edu
Media Contact: Linda E. Rudd, 919/515-3848 or linda_rudd@ncsu.edu

/ News Index / News Archives Index /