Visual Vestibular Conflict Mitigation in Virtual Reality Using Galvanic Vestibular Stimulation
BACKGROUND: Virtual reality (VR) is an effective technique to reduce cost and increase fidelity in training programs. In VR, visual and vestibular cues are often in conflict, which may result in simulator-induced motion sickness. The purpose of this study is to investigate the integration of Galvanic Vestibular Stimulation (GVS) with a VR flight training simulator by assessing flight performance, secondary task performance, simulator sickness and presence.
METHODS: There were 20 participants who performed 2 separate VR flight simulation sessions, with and without GVS (control). Flight performance, secondary task performance, and electrogastrogram were measured during VR flight simulation. The standardized simulator sickness and presence questionnaires were administered.
RESULTS: Electrogastrogram measures such as dominant power instability coefficient (DPIC) and percentages of bradygastric waves (%B) were lower in the GVS session than the control session in the flight simulation (DPIC: 0.44 vs. 0.54; %B: 21.2% vs. 30.5%) and postflight (DPIC: 0.38 vs. 0.53; %B: 22.8% vs. 31.4%) periods. Flight performance (#hit-gates) was improved in the GVS session compared to the control (GVS: 17, Control: 15.5). Secondary task performance (%hits) was improved with GVS for the Easy task (GVS: 55.5%, Control: 43.1%).
DISCUSSION: This study demonstrates the potential of synchronizing GVS with visual stimuli in VR flight training to reduce visual-vestibular sensory conflict to improve fidelity and performance. These results provide initial evidence, but continued research is warranted to further understand the benefits and applications of GVS in VR simulator training.
Pradhan GN, Galvan-Garza RC, Perez AM, Stepanek J, Cevette MJ. Visual vestibular conflict mitigation in virtual reality using galvanic vestibular stimulation. Aerosp Med Hum Perform. 2022; 93(5):406–414.
Flight simulators are an essential tool used during pilot training and are associated with decreased cost and risk compared to in-flight training. The capability to use simulation in training can aid in the acquisition of skills, development of competencies, and reduction of errors in real environments. Aviators can learn from flight-simulation training and transfer that learning to perform in real-world aircraft flight situations. 1,11 Research has shown that higher fidelity simulation training can be associated with more effective skill acquisition training and better performance in real-world situations. 7,12,20 Achieving high fidelity in traditional flight simulators (i.e., immersive visuals, motion cues, etc.) is expensive and often requires substantial physical space and other resources. Recently, virtual reality (VR) has been used as an effective tool to reduce the cost and increase the fidelity of simulation training 19 and is increasingly being considered for future training platforms.
One major limiting factor in VR flight simulation, however, is a type of motion sickness due to incongruent conflicting sensory inputs such as the visual sensation of self-motion and the vestibular sensation of no motion. The simulation environment such as flight simulators, driving simulators, and other virtual, immersive environments imposes limitations in matching real-world sensory experiences. When there is a mismatch among sensory signals or when input patterns from different senses do not correspond to expected sensory patterns, spatial disorientation may occur. 3 The two primary conflicts occur between the visual and vestibular senses (i.e., intersensory conflict) and within the vestibular system between the semicircular canals and otoliths (i.e., intrasensory conflict). Secondary conflict, however, may come from proprioceptive inputs that fail to synchronize with other sensory cues, particularly visual and peripheral proprioceptors connected to the vestibular system through vestibulospinal pathways. 9 These limitations can manifest in the form of simulator-induced motion sickness (SS), recognition of which has increased in recent decades. 22 Sensory conflicts causing SS remain one of the most persistent issues facing advanced flight simulation development. 21
SS is mainly the result of technological limitations in simulating dynamic environments that create a conflict in the body’s self-motion perception sensors. 14 SS is described as a polysymptomatic phenomenon because of the wide variety of symptoms that are mainly represented by nausea, oculomotor disorders, and disorientation. 13 SS has also been described as “polygenic” since several factors have been identified, including age, gender, simulator features such as lag and field of view, and factors associated with the task performed such as duration and degree of control. 12 Presence of SS has been largely measured with either subjective reporting of motion sickness or objective measures of physiological change, such as electrogastrography (EGG), with some studies finding that physiological changes precede subjective awareness of motion sickness. 10,18
The addition of Galvanic vestibular stimulation (GVS) to simulator immersion may help mitigate subjective and/or physiologically measured objective SS. GVS is the application of low-level electrical current to the vestibular system to induce the sensation of self-motion (i.e., the translations and rotations that occur during flight). GVS is a safe method to alter vestibular self-motion perception and has been used to study vestibular responses for almost 200 yr. 17 Bilateral bipolar GVS, as used in this study, is achieved by passing current through anodal and cathodal electrodes placed on the mastoid processes behind each ear. Currents in the range of 1 mA to 2.5 mA are typically used to achieve perceptions of motion. The current stimulates the vestibular system, which then interprets the GVS-evoked input like a real head movement along the rotational vectors in space. 8 A yaw sensation of rotation can be perceived to the side of cathodal stimulation in the horizontal plane while the same increased stimulation in anterior and posterior canals can signify ear-down roll and pitch. 8 The addition of forehead and nape of the neck electrodes afford the capability of inducing angular motion perception along yaw, pitch, and roll axes with specific patterns of GVS stimulation. 3 Previously, we integrated GVS with a flight simulation program to synchronize visual and vestibular stimulation in near real time to demonstrate the potential of oculo-vestibular recoupling (OVR) 3 to stabilize gastric activity and cardiac autonomic changes altered during simulator sickness. 2 However, no study to date has synchronized vestibular information with visual stimuli in VR flight training while examining the impact of GVS on performance and comfort. While a reduction in simulator sickness through the application of GVS, if achieved, could improve training effectiveness and adoption, another potential benefit of oculo-vestibular recoupling with GVS in VR-based training could be a measurable improvement in task performance. Further, if a visual-vestibular conflict is minimized, creating a more realistic and natural simulation, mental workload may be decreased, and spare capacity may be increased. For this study, we define spare capacity as cognitive resources remaining for additional tasks outside the primary task, as measured by secondary task performance.
Our objective was to assess the effect of GVS on multiple measures of performance and comfort during flight simulator training tasks in VR. We compared objective flight performance, objective secondary task performance, objective and subjective motion/simulator sickness, and subjective simulator presence, measured during VR flight simulation tasks with synced GVS and without synced GVS during easy and hard difficulty flight trials. Our hypotheses include:
- 1)
Flight performance and secondary task performance during flight simulator tasks in VR will increase with the application of GVS synced with VR visuals.
- 2)
Subjective simulator sickness and objective measurements of gastric dysrhythmia using EGG associated with simulator sickness while performing flight simulator tasks in VR will be decreased with the application of GVS synced with VR visuals.
- 3)
Subjective, self-reported sense of presence in the simulated environment during flight simulator tasks in VR will be increased with the application of GVS synced with VR visuals.
METHODS
Subjects
There were 20 participants, (Male:Female, 16:4) enrolled in this study protocol, which was approved by the Mayo Clinic Institutional Review Board (IRB). Only recruits between 18 and 55 yr of age with no history of vestibular disease, migraine, significant balance disorder, or history of severe motion sensitivity were enrolled. A negative urine pregnancy test was required for female participants. Informed consent was obtained from all participants prior to enrollment in accordance with Mayo Clinic’s IRB regulations. Participant demographics showed mean values (± SD) of: age (31 ± 9 yr), height (1.75 ± 0.1 m), and weight (74.3 ± 19.3 kg).
Equipment
Mayo Clinic’s GVS system 4 was integrated with Lockheed Martin’s Prepare3D® simulation software within the VR environment ( Fig. 1A ), using an Oculus Rift headset. The GVS system consisted of a four-channel galvanic vestibular stimulator (Good Vibrations Engineering, King City, ON, Canada). Participants only controlled rotational angular positions (bank, pitch, and heading) of the simulated aircraft (with preset constant speed) in the VR environment using a 3 df Logitech Freedom 2.4 GHz joystick (roll, pitch, and yaw, respectively). The three-dimensional angular displacement of the joystick that controlled the visual rotational motion in the flight simulation was simultaneously inputted to Mayo’s GVS system 4 to generate a real-time vestibular stimulation synchronized with the angular visual motion during flight simulation. The GVS system was driven by the proprietary algorithm that converted the three-dimensional angular displacement of the joystick signals into first-order angular velocities (i.e., roll rate, pitch rate, and turn or yaw rate). And these rotational velocity components were transformed into proportional amplitudes and directions (roll, pitch, and yaw) of GVS corresponding to expected matching multiaxis motions. Since the aircraft airspeed was constant within the flight simulation, there was no linear acceleration or thrust. The combined Prepar3D VR with synced GVS system was designed and intended to allow users to dynamically navigate a virtual world with a corresponding motion perception experience.
Citation: Aerospace Medicine and Human Performance 93, 5; 10.3357/AMHP.5921.2022
The primary flight task was the Ring Target task, where the goal was to fly through the lower green circle target in the VR flight simulation ( Fig. 1B ) using the joystick (Fig. 1A). After successfully flying through the target or missing a target, the next target to fly through would appear in the field of view. Participants were instructed not to fly back to hit a missed target and to keep flying toward the next target. To measure spare capacity, a secondary task was included during the flights. The secondary task ( Fig. 1C ) objective was to press the pointer finger trigger button on the joystick when the red symbol in the field of view met two criteria: 1) the circle was at least half-full; and 2) there were two or more antennae on top of the circle. The symbol changed in a pseudo-randomly generated order during the entire simulation. The order was kept constant across participants.
During the flight simulation, electrogastrogram (EGG) data were continuously recorded with a portable EGG recorder (Medical Measurement Systems) with low and high cutoff frequencies of 1 and 15 cycles per minute (cpm), respectively. The signals were amplified, digitized at a rate of 1 Hz, and transferred to a personal computer for further analysis by a commercially available software program (MATLAB R2019a).
Procedures
The experiment occurred in a quiet, climate-controlled room in the Aerospace Medicine and Vestibular Research Laboratory (AMVRL) at Mayo Clinic Arizona. Participants were asked to consume a light breakfast at least 2 h before the study was started. All participants attended two separate sessions of the experiment on separate days. In one session, participants performed flight simulation tasks in VR with matching GVS (GVS session) and in the other session, participants performed flight simulation tasks in VR without GVS (Control session). The order of sessions was counterbalanced across participants to control for training effects such that half of the participants did the GVS session first. Each session was conducted at least 4 d apart to minimize any carryover of visual or vestibular effects. During the GVS session, four electrodes were placed on the two mastoids (left and right), forehead, and nape of the neck to deliver the electric currents through the galvanic stimulator. For the Control session, four electrodes were placed in the same positions, but the GVS remained off for the duration of the session.
During both the GVS and Control sessions, all participants had six cutaneous electrodes (Ambu Blue Sensor N; Ambu A/S) positioned on the abdomen to record gastric myoelectric signals from the EGG. The first electrode was positioned below the left rib margin, 2 cm from the xiphoid process; the third electrode was placed equidistant between the xiphoid process and the umbilicus; the second and fourth electrodes were placed, respectively, along the left and the right midclavicular lines, 3 cm below the rib margin and equidistant from the midline; the fifth and sixth electrodes were placed, respectively, along the left and the right midclavicular lines, equidistant from the midline and 3 cm below the line of the second and fourth electrodes. A seventh electrode served as a reference electrode and was placed on the center of the left clavicle. Before each electrode was attached, the skin beneath it was abraded gently to decrease electrical impedance.
Before putting on the VR headset, participants were trained on how to conduct primary and secondary tasks in the flight simulations on a desktop computer screen (without VR). All participants were given enough time to practice flight simulation to surpass the learning curve and achieve familiarity. After training and electrode application, each participant completed flight simulator tasks in VR. The flight simulation session included two types of trials (Easy and Hard) based on difficulty. In the Easy trial, participants flew the aircraft at a constant velocity of 250 kph for 12 min (two back-to-back repetitions of 6 min of flight simulation). In the Hard trial, participants flew the aircraft at a constant velocity of 500 kph for 12 min (two back-to-back repetitions of 6 min of flight simulation). The order of two flight trials (Easy – Hard) in all sessions was fixed for all participants.
A modified standardized simulator sickness questionnaire 13 (SSQ) and the presence questionnaire (PQ) were administered to all subjects immediately after completion of the session. The Motion Sickness Susceptibility Questionnaire (MSSQ-Short) was administered to all subjects prior to the first session.
Physiological Measurements
The physiological response of gastric myoelectric activity was recorded by electrogastrogram (EGG) to objectively measure the degree and magnitude of simulator sickness. These EGG recordings were analyzed to derive the following commonly used parameters 2,5,23 : 1) dominant power instability coefficient (DPIC) to express the stability of the power of the dominant frequency – higher DPIC values indicate higher gastric dysrhythmia (in our case, due to motion sickness); and 2) the percentage of recording time with the dominant frequency in normogastric (2.0 – 4.0 cpm), bradygastric (1.0 – 2.0 cpm), and trachygastric (4.0 – 9.0 cpm) ranges—higher gastric dysrhythmia should decrease percentage of normogastric and increase percentages of bradygastric and tachygastric. The percentage of dominant frequency in normogastric range of more than 60% was defined as normal. 23 These EGG parameters were measured and compared across the baseline, Easy and Hard trial of flight simulation, and postflight periods during both the GVS and Control Sessions. The results are expressed as the mean ± SE.
Data Analysis
We analyzed the changes in the physiological measurements of EGG before, during, and after flight simulation periods (four levels: baseline, Easy trial, Hard trial, and postflight) in two different conditions (GVS and Control) using two-way repeated-measures ANOVA to detect main effects (period and condition) and interactions. It was followed by multiple comparisons between conditions across periods for statistical significance (two-tailed) using one-way repeated-measures ANOVA (equivalent to the paired t-test for two groups) and computed Cohen’s d to evaluate the effect size. Bonferroni corrections were not made during multiple comparisons to avoid the likelihood of Type II errors, which could deem interesting physiological trends (EGG) nonsignificant. We also analyzed flight performance measures including number of hit gates, standard deviation of latitude, standard deviation of altitude, and secondary task performance measure of % hits (with arcsine transformation) using two-way repeated-measures ANOVAs to detect main effects of GVS (2 levels; GVS and Control) and difficulty (2 levels: Easy and Hard trials) and interactions. Subjective motion sickness and presence ratings during the GVS and control conditions were compared using the Wilcoxon signed-rank test and we used Spearman’s rho correlations to examine potential relationships between MSSQ-Short and SSQ scores.
RESULTS
Of the 20 participants in the study 19 completed the experiment. One participant was unable to complete the study due to experiencing severe motion sickness symptoms during the Control session of the flight simulation and withdrew from the study before completing the GVS session.
Electrogastrography – Gastric Motility
Fig. 2 summarizes the statistics of DPIC values during all four periods (X-axis) in sequential order for GVS and Control sessions conducted on different days on the same subjects. For DPIC, Mauchly’s sphericity test for two effects (period and interaction) indicated that the assumption of sphericity was met (P > 0.05). There was significant main effect of period (F(3, 54) = 12.6, P < 0.001) and condition (F(1, 18) = 9.9, P = 0.006) on DPIC. But there was no significant interaction between the periods and the conditions. By multiple comparisons, one-way repeated measures ANOVA determined that DPIC baseline values between GVS and Control sessions did not differ [F(1, 18) = 1.642, P = 0.216, Cohen’s d = 0.47, mean difference = 0.048, 95% CI (−0.126, 0.031)]. DPIC values in the GVS session during the Easy trial [F(1, 18) = 4.5, P = 0.049, Cohen’s d = 0.7] and Postflight [F(1, 18) = 4.7, P = 0.04, Cohen’s d = 0.85] periods were significantly lower than the Control session. The DPIC values during the Hard trial period did not show statistical significance.
Citation: Aerospace Medicine and Human Performance 93, 5; 10.3357/AMHP.5921.2022
The baseline percentages of gastric waves for normogastria, bradygastria, and tachygastria were consistent and numerically close for all 19 subjects during both sessions on two separate days showing the reproducibility of measurements in normal conditions ( Fig. 3 ). For percentage of bradygastric waves, Mauchly’s sphericity test for two effects (period and interaction) indicated that the assumption of sphericity was met (P > 0.05). There was significant main effect of period [F(3, 54) = 5.1, P = 0.004] and condition [F(1, 18) = 4.9, P = 0.04] on percentage of bradygastric waves. But there was no significant interaction between the periods and the conditions. By multiple comparisons, only during the Control session, the percentages of bradygastric waves were significantly increased in the flight simulation and postflight periods (Fig. 3B), indicating dysrhythmia [baseline vs. Easy - F(1, 18) = 6.8, P = 0.018, Cohen’s d = 0.7; baseline vs. Hard - F(1, 18) = 7.3, P = 0.015, Cohen’s d = 0.6; baseline vs. post - F(1, 18) = 4.3, P = 0.05, Cohen’s d = 0.65]. In the GVS session, the percentages of bradygastric waves did not increase significantly in the flight simulation and postflight periods as compared to the Control session. For the percentage of normogastric waves, Mauchly’s sphericity test for the interaction (period and condition) indicated that the assumption of sphericity was met (P > 0.05). However, the period main effect violated this assumption (P = 0.02) and so the F-value and P-value for this effect was corrected using Greenhouse-Geisser correction. The percentage of normogastric waves decreased numerically more between the baseline and flight simulation trials in the Control session than in the GVS session. The decrement in the percentage of normogastric waves during the flight simulation in the Control session remained the same even after postflight. However, in the GVS session, the percentage of normogastric waves increased in the postflight period, suggesting a trend toward the stabilization of gastric motility after GVS (Fig. 3A). The behavior of tachygastric waves during Control and GVS sessions was similar except during the Hard trial of flight simulation, where the percentage of tachygastric waves in the GVS session was noticeably larger than in the Control session but not statistically significant (Fig. 3C).
Citation: Aerospace Medicine and Human Performance 93, 5; 10.3357/AMHP.5921.2022
Flight Performance, Secondary Task Performance, and Subjective Ratings
Fig. 4 shows the comparisons between group average flight performance and secondary task measures during the Control and GVS sessions. These measures are means of the two repetitions done for each condition/difficulty pair (e.g., GVS, Easy). There was a statistically significant effect of GVS on number of hit gates [F(1,18) = 4.783, P = 0.0042] with the mean number of hit gates larger by 2.145 hits with GVS compared to Control. There was also a significant effect of task difficulty [F(1, 18) = 24.254, P < 0.001] with the mean number of hit gates decreased by 7.066 gates in the Easy trials compared to the Hard trials. While seemingly counterintuitive, the number of hit gates was higher in the Hard trials due to the faster speed allowing participants to cover more distance and hit more gates than in the Easy trials. There was no significant interaction between GVS condition and task difficulty for number of hit gates.
Citation: Aerospace Medicine and Human Performance 93, 5; 10.3357/AMHP.5921.2022
There was a statistically significant effect of task difficulty on the standard deviation of altitude [F(1,18) = 26.152, P < 0.001] with a mean decrease of 483.006 ft in the Easy trials compared to the Hard trials. The effect of GVS on the standard deviation of altitude was marginally statistically significant (F(1,18) = 4.051, P = 0.059) with a mean decrease of 139.929 ft in the GVS condition compared to the Control condition. There was no significant interaction between GVS and task difficulty for the standard deviation of altitude.
There was a statistically significant effect of task difficulty on the standard deviation of latitude [F(1, 18) = 161.981, P < 0.001] with a mean decrease of 0.009 degrees in the Easy trials compared to the Hard trials. The effect of GVS on the standard deviation of latitude was marginally statistically significant (F(1, 18) = 3.364, P = 0.083) with a mean decrease of 0.001 degrees in the GVS condition compared to the Control. There was no significant interaction between GVS condition and task difficulty for the standard deviation of latitude.
There were no statistically significant main effects of either GVS or task difficulty for the secondary task measure, % hits, with the arcsine transformation applied. However, there was a significant interaction of GVS session and task difficulty [F(1,18) = 5.670, P = 0.029]. The group mean % hits was higher by 12.35% with GVS in the Easy trial compared to the Control but was smaller by 2.32% with GVS in the Hard trial.
SSQ and PQ responses were not significantly different between the GVS (SSQ: mean = 7.5, SD = 4.7, PQ: mean = 115.5, SD = 14.30) and Control sessions (SSQ: mean = 6.6, SD = 5.9, PQ: mean = 110.8, SD = 15.60), as tested with Wilcoxon signed-rank tests. SSQ and PQ scores ranged from 0–24 out of a possible 140, and 46–136 out of a possible 154 across all participants and sessions, respectively. While not a primary research question of this study, we were interested in potential correlation between motion sickness susceptibility, measured by the MSSQ-short, and simulator sickness reported by the SSQ during VR flight sim tasks. Using Spearman’s rho correlation, raw MSSQ-short score and Total SSQ score were positively correlated for SSQ data in the Control session (rs = 0.514, P = 0.024) but not for the GVS session (rs = 0.212, P = 0.384).
DISCUSSION
Virtual reality (VR) is fast becoming a household technology. Not only is it used for gaming, social interaction, and immersive experiences, VR has successfully been used in training, significantly decreasing the cost and accessibility. While an effective training tool, VR can create sensory conflict in which visual, vestibular, and tactile stimuli are not always matched. This mismatch limits the realness and potential transition of skills, for training of motion-centric scenarios and skills, such as flight training. Additionally, this sensory conflict, particularly the visual-vestibular conflict, can result in simulator sickness, one of the most common complaints about the use of VR that impedes its widespread use and adoption. 15 We hypothesized that Galvanic Vestibular Stimulation (GVS) synchronized with visual stimuli may be able to mitigate issues associated with sensory conflict in VR by matching vestibular and visual cues. This study is the first to demonstrate that in VR, GVS can decrease gastric dysrhythmia, an objective measure of simulator sickness, and improve some measures of flight task performance and secondary task performance.
Focusing first on simulator sickness, in this study, our electrogastrography results showed a significant difference in the percentages of recording time with the dominant frequency in normogastric and bradygastric domains between the GVS and Control simulation sessions (Fig. 3). In the Control session, the percentage of dominant normogastric waves decreased by 11% from baseline to simulation, and the percentage of bradygastric waves increased by 16% from baseline to simulation. Conversely, in the GVS session, there was a significantly reduced decrease (3%) in the percentage of dominant normogastric waves and significantly reduced increase (8%) in the percentage of dominant bradygastric waves. These results suggest that the synchronization of the visual and vestibular system in VR simulation using GVS has led to maintaining normal gastric myoelectric activity with a potential reduction of the conflicting sensory inputs that create motion sickness or spatial disorientation. These findings are consistent with previous studies where GVS was used to mitigate simulator sickness, 3 reduce motion sickness symptoms during physical Coriolis stimulation, 6 and ameliorate symptoms such as dizziness, nausea, vomiting, and nystagmus during caloric stimulation by irrigating the vestibular system. 16 However, in this study, we did not observe subjective simulator sickness improvement with GVS. As expected, participants varied in their propensity to experience simulator sickness. While one subject got too sick to complete the study, most scores were relatively low, and some participants did not experience any simulator sickness symptoms at all. Some studies have shown that physiological changes in gastric movement precede subjective awareness of motion sickness. 10,18 Our seemingly contradictory EGG and SSQ results would be consistent with a delay in subjective awareness of sickness following sickness objectively measured by the EGG. While our motivation for this study was to reduce simulator sickness in a practical way that could impact simulator immersion or the subjective experience of simulator sickness, it is also useful to understand the impacts of GVS on the temporal characteristics of simulator sickness, beginning with objective measurements and how they may lead to conscious awareness of motion sickness. More work needs to be done in this area to understand the potential impact of GVS on devolution into simulator sickness and the impact of time in the simulator on both objective and subjective measures. Future research should also aim to achieve higher levels of subjective sickness to help better understand potential benefits of GVS in mitigating simulator sickness.
A possible explanation of the generally low subjective sickness scores in our study was that we used flight scenarios that may not have been particularly provoking. In the simulation, flights took place in good weather over a calm ocean. More complex environments offer different visual orientation points that could increase the discrepancies between visual and vestibular stimuli and therefore increase subjective simulator sickness. We encourage future studies to examine simulator sickness with and without GVS in more challenging, dynamic, terrain environment (i.e., mountains, valleys, clouds, etc.) compared to simpler scenes. We hypothesize that the increased scenario complexity could result in higher overall simulator sickness ratings without GVS and a potentially larger effect of GVS at lowering simulator sickness. Based on the positive correlation between the motion sickness susceptibility (MSSQ-Short) and subjective sickness (SSQ) during the Control session and not the GVS session, it is also warranted for future research to investigate the effectiveness of GVS in mitigating simulator sickness during provocative simulation scenarios particularly for those with a high propensity to motion sickness in everyday life.
Next, focusing on task performance measures, we found that the primary flight performance measure, number of hit gates, was significantly higher with GVS than in the Control session by 2.15 gates. One possible explanation for this is that the synchronized motion cues provided by the GVS enabled subjects to fly with better intuition and control. However, to better understand the underlying cause, future research could aim to investigate how GVS impacts specific perceptual, cognitive, and/or behavioral functions relevant to piloting by measuring the impact of GVS on primary flight performance and spare capacity for a variety of flight tasks and workload states. For the secondary task, meant to provide a measure of spare capacity, there was no main effect of GVS on secondary task % hits, however a significant interaction between GVS and task difficulty for % hits suggests that GVS improved secondary task performance in the Easy trials but not the Hard trials. As evident from significant effects of task difficulty on primary task performance measures, subjects had more difficulty performing the flight task in the Hard trials. One possible explanation for the significant interaction is that, due to the high cognitive demand of the primary task during the Hard trials, there may not have been as much room for improvement in the secondary task measures with GVS for the Hard trials than for the Easy trials.
In this study, participants always completed the easy trials before the hard trials, so results that showed an improvement with GVS in the easy trials but not the hard trials could have been due to a larger beneficial effect of GVS during early training stages. Similar future studies should consider counterbalancing task difficulty levels, if possible. Additionally, none of our subjects were pilots and, therefore, there was variability in flying performance particularly in the hard trials, in which input errors could cause provocative aircraft motions due to the higher aircraft speed. This might explain the increase in gastric dysrhythmia (i.e., higher DPIC score in Fig. 2), drop in the percentage of normogastric waves (Fig. 3A), and sudden spike in the percentage of trachygastric waves (Fig. 3C) from the Easy trial to the Hard trial even in the GVS session. Future work could examine the potentially differential effects of GVS on task performance and spare capacity for expert vs. novice pilots and as training progresses over time.
Both VR and GVS techniques are relatively portable, inexpensive, and safe as compared to fully immersive pneumatic simulator motion systems, making these training enhancements feasible for future widespread adoption. Importantly, the portability of a GVS and VR system would enable refresher training in remote locations for specific scenarios or tasks as needed within an evolving mission. We acknowledge that GVS cannot replace actual g-forces and proprioceptive and tactile sensations felt in real flight and in high fidelity motion simulators. However, GVS may be able to improve on desktop and current VR training and could be further enhanced in the future with portable proprioceptive feedback methods such as inflatable wearables or seat inserts.
Military training can benefit extensively from realistic, low-cost training simulations. This study represents an initial investigation into the usefulness of GVS in VR simulations for pilot applications. With further advancement, realistic vestibular cues from a paired VR-GVS system could potentially go beyond nominal flight training and give pilots experience in specific piloting scenarios which require the use of precise motion perception for optimal performance (e.g., recovery from Pilot-Induced Oscillations, PIOs). Across all domains where VR training is applicable, we expect that a GVS and VR integrated system will become more effective as VR systems technology continues to improve, with the ultimate goal of decreasing unwanted side effects of the training environment for maximum realism, fidelity, and immersion.
(A) Prepare3D flight simulation program in the VR environment displayed in the Oculus headset and controlled by the Logitech Freedom 2.4 GHz joystick. Joystick movements in pitch, yaw, and roll axes performed the Ring Target Task and the pointer finger trigger button on the joystick performed the secondary task. (B) Primary Ring Target task during the flight simulation in the VR environment. (C) Secondary task in the field of view during the flight simulation.
Instability coefficient factor for the dominant power during the baseline, Easy Trial and Hard Trial of VR flight simulation, and postflight periods. Note: * indicates P < 0.05.
Percentages of gastric waves for normogastria, bradygastria, and tachygastria during the baseline, Easy Trial and Hard Trial of VR flight simulation, and postflight periods. Notes: The error bar represents the ± SE; * indicates P < 0.05.
Flight performance and secondary task performance data including the number of hit gates, standard deviation of latitude, standard deviation of altitude and secondary task hit percentage. Notes: The error bars represent the 6 SE; * indicates P < 0.05.
Contributor Notes
