Figure 2. The average heart rate for the baseline and trial sessions at each diffi  culty level was compared. The trial heart rate was always lower than its respective baseline heart rate. The error bars represent standard deviation.

While the participant was playing the pong game, the virtual avatar’s responses were recorded (Table 1 and Fig. 3). As the anxiety level increased, verbal re­sponse also changed according to Table 1. The most numerous response was 0 where the avatar declared that the subject was not anxious.

Table 1. Actual response assigned numerical values for easier representation in Figure 3.

Figure 3. The virtual avatar gave different responses based on the subject’s average heart rate over each minute of each trial at each difficulty level in the pong game.

DISCUSSION.

Although having the heart rate sensor mounted on a clothespin did alleviate some noise, it did not completely clean up the output signal from the sensor. Voltage spikes from electrical noise caused the instantaneous trial heart rate to exceed the baseline heart rate. In order to account for these occurrences, the avatar was coded to respond only if increased heart rate lasted for about 30 sec­onds. Voltage spikes resulted in a false indication of increased heart rate but because they did not last for 30 seconds, they had minimal effects on the virtual avatar’s response. This demonstrates an effective interface between the heart rate sensor and the virtual environment.

Although previous studies have shown that cardiovascular activity and anxiety are linked [10], this relationship was not strong in this study. In comparing the differences between trial and baseline heart rates between the difficulty levels, we saw significant differences between Difficulties 1 & 2 and 1 & 3 but not 2 & 3 (Table 2). This led us to conclude that modifications in the pong game to induce anxiousness in the subject were insufficient.

Table 2. Using a two sample t-test, the average differences between trial and baseline heart rates between difficulty levels were compared.

The task presented to the subject, the pong game, was not strenuous enough to cause noticeable increases in heart rate compared to a baseline. In fact, the task seemed to relax the subject instead of making him anxious seeing as the aver­age trial heart rate was consistently lower than average baseline heart rate (Fig. 2). This is consistent throughout the results seeing as the virtual avatar’s most numerous response was that the subject was not anxious.

CONCLUSION.

This study showed that a psychophysiological signal, heart rate, can be used as a stimulus for an intelligent virtual reality system. A virtual avatar was able to provide varying responses on the anxiety level of a subject as expressed through changes in the subject’s heart rate. Furthermore, as exhibited by the results, the entire interface between the heart rate sensor and the virtual environment was functional and versatile. It took into account voltage spikes from noise and was capable of giving three distinct responses on the subject’s anxiousness.

For intelligent virtual reality systems to be widely used, more tests need to be conducted to gauge the effectiveness of the systems. A commercial heart rate sensor should also be employed because it can provide cleaner signals. One cru­cial step to improve this overall experiment is to design the trials to maximize the differences between baseline and trial heart rates. This can be achieved by changing the stimulus to which the subject is exposed. Instead of a pong game, the subject could watch a video clip to induce shock, thus increasing heart rate. In addition, more trials should be conducted to increase the confidence levels of the results. Other questions that need to be asked before long term deploy­ment of this technology include:

What is the most optimal stimulus to which the virtual reality system could adapt?

If multiple stimuli are applied, how could they be interpreted to return the best output possible?

How feasible is a system that adapts a task instead of just returning comments?

While heart rate is a possible psychophysiological signal to use as a stimulus, other signals such as perspiration levels, cardiovascular and electromyogram activities can be used as indicators of emotional state [10–13]. By finding the advantages and disadvantages of these different indicators, they can be linked together to create one conglomerate where the benefits of all the signals can be used to provide the stimulus for the intelligent virtual reality system. Doing so would improve the overall performance of the system and make it more viable to be used as part of autism intervention paradigm.

In this study, a virtual avatar gave responses while a task was being undertaken but in the future, there should be a push towards having a system where the task itself is adapted. In context of the pong game, a future virtual reality system could be used to adapt the game difficulty in real-time to increase the subject’s engagement. Having such a system can lead to new innovations in areas such as autism intervention where self-adaptive systems can increase the engagement and reduce the frustration of autistic children. This would cause the children to learn better from the intervention programs.

ACKNOWLEDGMENTS.

I would like to thank my PI, Dr. Nilanjan Sarkar, for his consistent support in this project. I would also like to thank Dr. Uttama Lahiri who provided me with the utmost assistance throughout this endeavor. In addition, I want to acknowl­edge the Robotics and Autonomous Systems Laboratory for providing me the necessary materials to complete this study. I would also like to acknowledge the School for Science and Math at Vanderbilt for giving me the opportunity to carry out this research, and Dr. Chris Vanags for his guidance in this project. The project described was supported by Award Number R25RR024261 from the National Center for Research Resources. The content is solely the respon­sibility of the authors and does not necessarily represent the official views of the National Center for Research Resources or the National Institutes of Health.

REFERENCES.

1. “Introduction to the Arduino Board,” Arduino, 2008.
2. D. Zieve, et al., National Institute of Health, (2010).
3. M. E. Dawson, et al., J. T. Cacioppo and L. G. Tassinary, Eds. New York: Cambridge Univ. Press, 295 (1990).
4. A. Pecchinenda, et al., Cognit. Emotion, 10, 481 (1996).
5. J. T. Cacioppo, et al., Handbook of Emotions, M. Lewis and J. M. Haviland-Jones, Eds. New York: Guilford, 173 (2000).
6. J. F. Papillo and D. Shapiro, J. T. Cacioppo and L. G. Tassinary, Eds. New York: Cambridge Univ. Press, 456 (1990).
7. C. Rice, Center for Disease Control and Prevention, (2009).
8. L. Tarkan, New York Times, (2002).
9. S. J. Rogers, J. Clin. Child Psychol., 27, 168 (1998).
10. “The Aurora Project Introduction,” University of Hertfordshire.
11. J. Groden, M. S. et al., Focus Autism Other Dev. Disabil., 20, 244 (2005).
12. G. J. Compton, Pico Technology.
13. K. Dautenhahn, I. et al., Proc. IEEE Int. Symp. Comput. Intell. Robot. Autom., 577 (2003)