Recalibration and Spatial Recall in an Immersive Virtual Environment
ABSTRACT
Spatial memory, the judgment of an object’s position in space and where we are in that space, has been studied with arrays, displayed in real or immersive virtual environments (IVEs). In our study, we used an IVE to observe how spatial memory is affected in multiple dimensions and conditions. In the environment, participants were asked to memorize 9 objects that were distributed around the room at a constant height. Then, the environment was occluded, and the participants completed a locomotion condition–where they physically rotated–and were asked to face target objects at different orientations. Participants also completed an imagination condition–where they imagined one rotation before facing target objects relative to the imaginary rotations. Performance was recorded by turning errors (TE, angle between the target position and where the participant faced) and latency (time taken to complete the trial). We tested adolescents (12–15 year-olds) and young adults (18–21 year-olds). The results showed that all participants had fewer TEs and less latency in the locomotion condition. No difference was found in TE or latency between age groups in both conditions. Our results will add to the research on spatial memory development and how humans interact with the world around us, in both virtual reality and the real world.
INTRODUCTION.
In the realm of virtual reality, researchers have been dedicated to continuously improving the realism of immersive virtual environments (IVEs) to resemble the real world. This process involves mimicking people’s behaviors in real environments and smoothly transitioning these behaviors into virtual environments. Scientists have tested behaviors of recalibration, gap affordances, distance perception, spatial recall, and other cognitive functions. These functions are tested in both real and virtual environments and compared to see how accurate virtual reality can be in mimicking the real world.
Spatial memory has been studied with arrays, displayed in real or virtual environments [1]. Spatial memory refers to the recall of the location of objects in an environment. This is one factor that helps scientists determine how close to reality virtual environments have become: by comparing ideas of spatial memory in both the real and virtual worlds. Egocentric distance is often underestimated in virtual environments [2]. This underestimation of an object’s scale leads to distance perception being misjudged, which affects the participant’s perception of the object’s location. Previous research has suggested that people who do better in spatial memory tasks in the real world and in 2D virtual environments also do better in 3D environments, which our study examines [3].
We were interested in testing spatial memory within an IVE in order to determine how object placement and participant movement affects our participant’s spatial recall [4]. We measured performance through turning error along the y-axis and latency (the time taken for each trial). One difference in performance we expect to see is through sex. In general, women outperform men in spatial recall tasks in the real world [3]. This provides a basis for women to outperform men in virtual environments as well. We expect to find differences in spatial recall as a result of different object positions. Research has also shown that height differences do not seem to influence spatial memory, so we kept the height constant for all the objects in the environment [5]. We also expect to find an advantage to performing the locomotion task first instead of the imagination task and expect that higher performance will be associated with beginning with the locomotion task. However, since spatial memory is thoroughly developed by 12 years old, we expect to find similar performance in adolescents and adults in both the locomotion and imagination conditions [6].
This present study looks at the advantage of locomotion over imagination tasks and the comparison between objects placed at different positions within the environment. Additionally, this study also looks at the advantage of sex in both conditions. We hypothesize that all participants will have less turning error (along the y-axis) and less latency (time taken for each trial) in the locomotion environment over the imagination environment and performance in the imagination environment will rise when completing the locomotion task first. We also hypothesize that males will perform worse than females in both conditions.
MATERIALS AND METHODS.
Participants. Eight adults (ages 18-21, M=19.75, 4 female) and eight adolescents (ages 12-15, M=13.625, 4 female) volunteered for this study. This sample size was chosen based on prior experimentation studied in this lab [5]. The local institutional review board has approved this experiment at Vanderbilt University and all participants and legal guardians for minors have signed a consent form. The study occurred at Vanderbilt University in Featheringill Hall.
| Table 1. Demographics and separation by the condition of age groups. | ||||
| Age Group | Age | Gender | Random Order | Condition Started with |
| Adults | 19.75 (avg.) | 50% male | 50% Random Order 1 | 50% Locomotion |
| Adolescents | 13.625 (avg.) | 50% male | 50% Random Order 1 | 50% Locomotion |
Virtual Environment. The virtual environment was created through Unity. This room is walled on all four sides, with an open roof. The roof was not included in the environment, and research has shown that spatial accuracy has been more consistent in an open environment [2]. As shown in Figure 1, nine objects were placed in various parts of the room while keeping the height of the objects the same. The same environment was used for both the adult and adolescent age groups.
Figure 1. Nine objects are distributed at different positions around the room at a constant height.
Conditions. Participants stood in the middle of the room on a circular platform that shows up in the hidden environment (Figure 2a). Subjects participated in two tasks: a locomotion condition and an imagination condition. In the locomotion condition, participants were asked to make rotations of the body to look at a target object that the experimenter calls out. The experimenter says “Turn to look at [object 1]. Now turn to look at [target object].” The participant was instructed to say “okay” after facing each object so that the experimenter knew when to continue the instruction to face the next object or to stop the timer on the final object.
Figure 2. a) The circular platform participants stand on in the middle of the room throughout the trials. b) The reset target participants are instructed to look towards in between trials.
Between each trial, participants were asked to reset their position by facing a target (Figure 2b). In the imagination condition, participants were asked to imagine one rotation towards one object, and then make an actual rotation towards a target object, relative to the position of the first object they faced. The experimenter says, “imagine you are facing [object 1]. Now turn to face [target object] (as if you were facing object 1).” The participant was instructed to say “okay” only after making the rotation toward the target object, which instructs the experimenter to stop the timer. The participant was asked to reset and face the target, as stated before in the locomotion condition. 18 trials were conducted for each condition.
Procedure. Participants used the HTC Vive Pro with a per-eye resolution of 1440 x 1600 and 100º field of view, to be immersed into the virtual environment. They were instructed to walk into the middle of the room to the starting position of the study. Participants were given instructions to look around the room where nine objects were placed. The experimenter pointed out each of the objects to make sure the subject was aware of each of them and knew the names of each object. Then, the participant was given 3 minutes to look around the room and memorize the positions of each object. The experimenter then explained to the participant which condition they would be starting with. The experimenters also made 2 random orders of object pairings, which were assigned to the participants. There were four groups for the experiment: 1) random order 1 and locomotion first; 2) random order 1 and imagination first; 3) random order 2 and locomotion first; 4) random order 2 and imagination first. Participants from the adults and adolescents age group had one male and one female paired into each group. Experimenters gave participants a few practice trials to make sure they understand the task they are completing before going into the real experiment. After 18 trials of the first condition, participants were then given an additional 2 minutes to re-familiarize themselves with the environment. Then, the experimenter explained the second condition where they then completed 18 trials as well. After the competition of each condition, if needed, experimenters can enter “error mode.” This tool is used if a participant took an unusually large amount of time if there was a mistake in the recording of time, or if participants ask if they can redo a trial. Experimenters can manually repeat the trial of their choosing to reduce any faulty trials. These additional trials replace the same trial that was considered faulty.
Measuring Performance. We measured two dependent variables: turning error and latency. Turning error was measured as the angle of error between where the participant faced at the end of the trial and where the target object resided. Latency was measured as the time the participant took to move from the reset target to the final position of the target object.
RESULTS.
Participants were given 3 minutes to memorize the position of 9 objects in the room and were given a few practice trials to make sure they understood the instructions for each condition. In between conditions, they were given an additional 2 minutes to refamiliarize themselves with the environment before moving on to the next condition. Turning error, the absolute value of the error along the y-axis, and latency, the time taken for each trial, were compared between age groups using a two-tailed independent t-test and a two-way ANOVA.
We predicted that there would be no difference in turning error or latency between age groups for both locomotion and imagination. An independent two-tailedt-test was used to compare the turning error between the adult age group (M=15.26, SD=6.34) and the adolescent age group (M=19.5, SD=8.59) in the locomotion condition, which turned out to be not statistically significant (t=-1.12, p>0.05) (figure 3). However, there was a statistical significance of turning error between the adults (M=127.0, SD=13.4) and the adolescents (M=102.7, SD=18.0) in the imagination condition (t=3.06, p<0.05) (figure 3). Surprisingly, the adolescent age group had fewer turning errors than the adults in the imagination condition. There was no statistical significance of latency between the age groups for locomotion (t=-0.468, p>0.05) or imagination (t=-1.32, p>0.05) (figure S1).
Figure 3. Depicts the average turning error on the y-axis of adolescents and adults. An * represents significance between two factors.
The results of the study found that both age groups performed significantly better in the locomotion condition (t=9.129, M=17.38, SD=7.61) rather than the imagination condition (t=23.132, M=114.86, SD=19.86) in terms of turning error (p<0.05) (figure 3). Both age groups also expressed a nonsignificant increase in latency in the imagination condition (t=8.132, M=7.38, SD=3.63) over the locomotion condition (t=13.136, M=5.51, SD=7.38) (p>0.05) (figure S1).
Our study predicted that women would have less turning error and latency in both locomotion and imagination compared to men. The results showed that the turning error between men (M=19.0, SD=8.01) and women (M=15.7, SD=7.3) does not have a significant difference in the locomotion condition (t=0.8454, p>0.05) (figure S2). The turning error between men (M=119.7, SD=21.2) and women (M=110, SD=18.4) was not significantly different in the imagination condition (t=0.9765, p>0.05) (figure S2).
Additionally, the latency between men (M=4.9, SD=1.3) and women (M=6.0, SD=1.9) was not statistically different for the locomotion condition (t=1.2719, p>0.05) (figure S3). The latency between men (M=6.5, SD=1.8) and women (M=8.2, SD=4.8) was also not significantly different in the imagination condition (t=0.9250, p>0.05) (figure S3). This suggests that there are no gender differences in turning error or latency in locomotion or imagination.
DISCUSSION.
The results show that there are no differences in the performance of adults and adolescents in the locomotion condition. This is supported by our hypothesis, which states that spatial memory is well-developed by the age of 12 [6]. Our adolescent age group is 12-15 years old, so their brains should have developed enough in order to accurately recall objects in space at the same rate as the adult age group. However, contrary to the hypothesis, the adolescent age group had less turning error than the adult age group in the imagination condition. This is surprising because it is believed that younger participants have larger turning errors and take more time in all spatial tasks [5]. This may be a result of the small sample size of the study.
On average, both age groups had less turning error and spent less time in each trial of the locomotion condition compared to the imagination condition. This is consistent with the hypothesis. The locomotion condition is considered easier because participants are asked to face objects with their position in the room unchanged. The objects they turn to face are all exactly where they memorized them, so they are accurate with their rotations of the body. However, the imagination condition forces participants to think of the space in between objects. They are forced to pretend that they are facing an object they are not already facing, so the perspective of the room changes. In another experiment, subjects reported that they were unable to imagine facing the object, and instead tried to calculate the angle between the two objects, which may provide evidence as to why the angle error was much larger in the imagination condition [1].
Contrary to what was expected, we found no significant differences in the turning error or latency between men and women in either the locomotion condition or the imagination condition. Although previous research supported that women outperform men in spatial memory tasks, another study supports that there are no gender differences in spatial memory in 3D real environments [3, 7]. These differing results may be a result of the former study being translated from 2D to 3D, so our 3D virtual environment aligns better with the latter which is also tested in a 3D environment. However, the adolescent female age group seemed to have much larger latency and lower turning error than the other groups, which can be further analyzed in a future study.
In the future, the continuation of this study with a younger age group could help find differences in both conditions between age groups. Our results support that most behaviors in virtual reality mimic those in the real world, but a future comparison between the results of this study and a real-world adaptation can help infer how virtual reality can improve to make its way into becoming the next most realistic world adaptation.
ACKNOWLEDGMENTS.
I thank the School for Science and Math at Vanderbilt for providing me with his opportunity. I also thank the Learning in Virtual Environments (LiVE) lab at Vanderbilt for giving me the materials and purpose, and helping me conduct this research.
SUPPORTING INFORMATION.
Jayathilake_Supporting Information
Supporting Information includes graphs for average latency between age groups and gender, and average turning error between gender.
REFERENCES
1 B. Williams, G. Narasimham, C. Westerman, J. Rieser, and B. Bodenheimer, Functional similarities in spatial representations between real and virtual environments. ACM Trans. Appl. Percept., 4, 12, (2007).
2 S. Creem-Regehr, J. Stefanucci, B. Bodenheimer, Perceiving distance in virtual reality: theoretical insights from contemporary technologies. (Not Published).
3 J. Huang and A. Klippel, The Effects of Visual Realism on Spatial Memory and Exploration Patterns in Virtual Reality. 26th ACM Symposium on Virtual Reality Software and Technology, Virtual Event Canada, (2020).
4 J. Kelly, Distance perception in virtual reality: A meta-analysis of the effect of head-mounted display characteristics. PsyArXiv, preprint, (2022).
5 G. Narasimham, H. Adams, J. Rieser, and B. Bodenheimer, Encoding Height: Egocentric Spatial Memory of Adults and Teens in a Virtual Stairwell. ACM Symposium on Applied Perception 2020, New York, NY, USA, 1–8, (2020).
6 H. Adams, G. Narasimham, J. Rieser, S. Creem-Regehr, J. Stefanucci, and B. Bodenheimer, Locomotive Recalibration and Prism Adaptation of Children and Teens in Immersive Virtual Environments. IEEE Trans. Visual. Comput. Graphics, 24, 1408–1417, (2018).
7 T. Iachini, I. Sergi, G. Ruggiero, and A. Gnisci, Gender differences in object location memory in a real three-dimensional environment. Brain Cogn, 59, 52–59, (2005).
Posted by John Lee on Tuesday, May 30, 2023 in May 2023.
Tags: Locomotion, Spatial Memory, Virtual reality