The Air Force has long faced a silent crisis: It can’t train and retain enough pilots. And now, challenged to confront peer adversaries, rather than the insurgents it’s been engaging for the last 20 years, even the pilots the service can train aren’t getting the quantity or quality of hours in mock combat they need to hone their skills, according to former USAF leaders.

A transformative new technology, which combines live training with virtual simulation, can help address that problem, its proponents say. Augmented reality training is “a game changer,” promised Will Roper, who previously served as the Air Force’s assistant secretary for acquisition, technology, and logistics.

The pilot shortfall has been a persistent problem for the Air Force, despite a $1.7 billion annual training budget. But it’s been highlighted anew as the U.S. military has pivoted from the “endless wars” of counter insurgency to confront peer adversaries, especially China, which has its own fifth-generation fighters to match the U.S. F-22 and F-35.

“The gap between how we train our combat aviators and how they’ll need to fight against modern adversaries has continued to grow wider,” said retired Lt. Gen. David A. Deptula, dean of AFA’s Mitchell Institute for Aerospace Studies. “We’re essentially still training our pilots the same way we did 60 years ago.”

Combat training for fighter pilots, with a red team playing the role of adversaries like China’s fifth generation J-20, is expensive and time consuming. Above all, it’s “limited by the availability of air and ground training assets that accurately replicate those peer threat systems,” explained retired Gen. James M. Holmes, former head of Air Combat Command who now serves as Chairman of the Red 6 board. He added that the red teams at major Air Force exercises like Red Flag “resemble 1980s Warsaw Pact threats more than they resemble the capabilities of 2020’s China and Russia.”

Forty years ago, Holmes said, as a young lieutenant, he flew more than 225 training sorties a year, or about 350 hours a year. Now, his son Capt. Wade Holmes, who flies F-16s in the Air National Guard, is “lucky to fly 120 of those [training sorties] a year and almost half of those are flown as red air, providing training support for someone else.”

The Air Force has used simulation technology to try to close that gap, and is investing heavily in its new approach to training: Live, virtual, and constructive. “LVC is widely recognized as the only way, the only cost-effective way, for us to train against fifth-generation adversaries,” said long-time Senate Armed Services Committee staffer and USAF veteran Robert “Otis” Winkler. “We spent a ton of money and a ton of time developing the virtual and constructive portion of it.”

He spoke alongside Holmes and Deptula at a Mitchell Institute virtual seminar March 16.

Simulators and other virtual training tools allow geographically dispersed forces—including foreign allies—to train together, Winkler noted. Simulators also let pilots push the envelope in ways that would be too risky in real-life training, noted Holmes. “You can practice things in a sim that aren’t safe to practice in an aircraft.”

But, Holmes added, “There’s no substitute for live training.”

That’s because simulators can’t accurately reproduce the physical and intellectual stress of actual flight, noted Dan Robinson, a retired Royal Air Force pilot who became a USAF flight training instructor and was the first foreigner to fly an F-22. “It’s one thing to perform a maneuver in a sim, it’s another thing to perform it when you’re pulling 9Gs,” Robinson told Air Force Magazine. “The physics matters … The cognitive load is completely different when the pilot is actually in flight.”

Robinson is the founder and CEO of Red 6, a tech start-up offering augmented and virtual reality solutions as a way to allow pilots to train live against virtual adversaries.

“It’s the best of both worlds,” Robinson said. Pilots can train in real flight—flying with a special helmet and heads-up display that allows them to see their virtual enemies. The enemies, being online creations, can be modeled to mimic the capabilities and profile of any adversary weapons platform—and can be powered by artificial intelligence. “You can create the adversary, define his capabilities, and train against weapons systems that would be too expensive for live training, such as hypersonic missiles,” Robinson said.

But the technology is challenging, he explained. Virtual reality creates an entirely synthetic environment, which is a relatively straightforward challenge. “In augmented reality we are introducing virtual entities, virtual objects into the real world, and making them interact dynamically with us, with our surroundings, as if they were really there and that’s a much more complex set of technical problems to solve.”

The key breakthrough technologies that enable Red 6’s Airborne Tactical Augmented Reality System (ATARS), Robinson said, were in vision tracking—ascertaining where the pilot is looking and shifting the perspective of the virtual objects accordingly—and in the display. “Most VR doesn’t work outside … The environment is too dynamic and the display isn’t visible enough. It’s like trying to look at your cellphone screen in bright sunlight.” VR technology is generally limited to a 60-degree field of vision, about a third of the 180-degree field of vision humans have in the real world. “We are at about 120 degrees right now, and we are working on expanding that,” Robinson said.

AR is a transformational technology, with applications way beyond pilot training, Roper said. “This will disrupt not just Air Force training, but all-domain training … Augmented reality provides a paradigm-shifting opportunity for the military to train at much lower costs and against threats and in environments that cannot be recreated in the real world,” added Roper, who was appointed last month to the Red 6 Advisory Board. “AR technology has major commercial applications as well.”

Red 6 has completed a SBIR II contract from AFWERX and is expecting a SBIR III soon. The Red 6 team is working to integrate its ATARS technology into a T-38 training aircraft at Holloman Air Force Base in New Mexico.

“That is the next step for us,” Robinson said, “to demonstrate how this will work in an Air Force trainer jet.”