2nd Lt. Brett Bultsma, Pilot Training Next student, and Capt. Jeffery Kelley, PTN instructor pilot, prepare to take off in a T-6 Texan at Austin-Bergstrom Int. Apt., Austin, Texas. Photo: Sean Worrell/USAF
The National Defense Strategy predicts artificial intelligence could transform the battlefield of the future, changing the “character of war” in ways not yet imagined.
Explosive growth in commercial AI technologies, fueled by advances in microprocessors, cloud computing, and their combined ability to rapidly assemble and analyze vast data sets, has set off a revolution in commercial markets. AI now directs traffic, routes packages, answers customer queries, and anticipates consumer demand. Its military applications are also endless.
“Structurally, we know AI has the potential to be an enabling layer across nearly everything,” Dana Deasy, the Defense Department’s chief information officer, told the House Armed Services emerging threats and capabilities panel in December. “It means the opportunity to positively transform every corner of the department, from innovative concepts that change the way we fight, to improvements in the way we maintain our equipment, perceive our environment, train our men and women, defend our networks, operate our back office, provide humanitarian aid, and respond to natural disasters.”
Federal investment in unclassified AI research and development is already growing rapidly, up more than 40 percent since 2015, according to the White House. More increases may be ahead. The Trump administration plans to update the 2016 National Artificial Intelligence Research and Development Strategic Plan this spring.
The Defense Advanced Research Projects Agency (DARPA) has been driving AI technology research since the 1960s. Of DARPA’s 250 programs today, 80 involve AI in one form or another, and 25 are focused on breakthrough AI technologies that are yet to emerge from the lab. Currently, DARPA plans to invest $2 billion over the next five years in AI Next, a new campaign focused on three tenets:
- Third-wave AI: Technologies that incorporate contextual reasoning;
- Solving the most vexing problems: As DARPA Director Steven H. Walker said in September, DARPA is seeking to “identify and apply emerging AI technologies to solve today’s toughest security challenges;” and
- Performance Assurance: Walker seeks to “create the deep analysis and understanding of how and why today’s AI technologies work” and ensure “the robust performance guarantees essential for military- and safety-critical systems.”
While DARPA focuses on future developments, the Defense Department is working to field AI more rapidly. Last June, DOD established the Joint Artificial Intelligence Center. Led by Deasy, JAIC (pronounced Jake) aims to accelerate AI deployment to the operational force. As of December, JAIC had a staff of 30—a mix of DOD civilians and military personnel from across the services—and Deasy told House legislators in December he is actively recruiting AI experts to expand the team.
At AFWERX—the Air Force’s innovation hub—AI is a clear focus.
“We think AI will fundamentally change the way the Air Force operates in kind of unseen ways, and that’s why we take this iterative, experimental approach to it,” said Lt. Col. Dave Harden, AFWERX chief operating officer.
Harden said AFWERX looks at AI in four buckets:
- Autonomous deep learning: Analyzing vast data sets to assist with predictive analysis or early warning.
- Human-machine collaboration: Using AI to help human operators make better decisions more quickly.
- Assisted human operations: Streamlining and automating processes so that machines can help humans operate more efficiently.
- Advanced human-machine teaming: Humans working with unmanned systems.
In one recently completed deep-learning project, AFWERX researchers studied 10 years of supply data and five years of maintenance reports for the E-3 fleet and ultimately identified 29 percent savings in unplanned scheduled maintenance. The study also revealed 48 parts “that were highly likely over maintained,” Harden said. In other words, the analysis concluded those parts were swapped out more frequently than necessary.
Using AI to better predict maintenance needs has long been on the Air Force’s radar. But the technology to enable the concepts hasn’t been readily available until recently. This project came about after an AFWERX innovator witnessed USAF maintainers deployed downrange using spreadsheets to track maintenance. Aware that AFWERX was already working on other strategic AI projects, he called back to the States and, within six months, AFWERX had secured funding and USAF leadership support for the project. AFWERX partnered with the Pentagon’s Defense Innovation Unit (formerly known as DIUx) on the project, Harden said.
“This was the toughest AI project because it was all unstructured data and text,” said Harden. Structured data is information organized in tabular form, such as the data in a spreadsheet. Unstructured data is everything else and can include photographs and handwritten documents, PDFs, and more.
Now AFWERX is working to expand the project to other aircraft. “With the other structures we’re adding on, like the F-16 and C-5, we’re working hard to integrate sensor data in real-time, [which] heightens or improves your algorithm capability,” Harden said.
Lt. Col. Justin Elliott, 59th Test and Evaluation Squadron director of operations, runs through a presentation. Elliott wants to use new, cheaper, lightweight data-collection boxes in every aircraft flown by USAF instead of just those assigned to test units. Photo: A1C Andrew Sarver
MAN AND MACHINE
Human-machine collaboration tries to find innovative ways to help operators make decisions. USAF’s Pilot Training Next program, a collaborative effort with Air Education and Training Command, leverages AI to augment instructors in training student pilots.
In the initial test, an “AI coach” observed what students did, how they learned, and offered real-time advice. For example, if a student pilot learning to fly a loop pulled too hard on the stick, the AI coach could tell them to ease up. Also, if a student is more of a visual learner or prefers reading about a task first, the AI coach could personalize the training syllabus for that student.
“It’s an artificial intelligence that’s actually adapting the environment in which you learn,” explained AETC boss Lt. Gen. Steven L. Kwast at AFA’s Air, Space & Cyber Conference in September 2018. “It’s coaching you along the way, so you’re reminded of what you’re doing right and what you’re doing wrong.”
Twenty students—15 officers and five enlisted airmen—participated in the first Pilot Training Next class, which graduated 13 pilots in half the time it normally takes to complete USAF’s Specialized Undergraduate Pilot Training course. The second class, which kicked off in January, will have 24 students.
AFWERX launched an open challenge for the second phase of the program in October, promising to spend up to $300,000 “for the right set of upgrades and solution trials.” It also noted the possible scale of the program, saying it “has the potential to reach $100 million or more in contracts over time.”
In this phase, officials want the AI tutor to “rapidly adapt to accomplished learning objectives,” according to the October challenge. While the initial AI coach “provided game-changing elements of individualized training,” it “could not keep pace with student progress.”
AFWERX Mission Lead, Brian “Beam” Maue, said the next phase aims to better structure data to tailor simulator scenarios to each student’s needs and to improve the biometric sensors used to monitor students’ performance during training.
The first phase utilized eye-tracking sensors embedded in virtual reality goggles, which traced students’ eye movements, providing an indication of whether students were focused on the right things in order to have truly mastered a maneuver. Now, Maue said, AFWERX wants to track even more, perhaps adding headbands that could track body temperature, perspiration level, and heart rate as indicators of students’ stress levels.
“AFWERX engages in experiments, and AETC has been a forward-leaning command with this experimental mindset as well,” Maue said. “So, going forward into any of these experiments we wouldn’t expect we would have 100 percent of the technology identified for something that’s never been tried before.”
Industry interest is intense. AFWERX received 150 responses to its October challenge and selected 15 to pitch their concepts at its Austin, Texas, storefront. Of those, five were awarded $50,000 each to further refine their concept. “When they come back with their results, they will be re-evaluated to see whether or not we can do some form of other transaction authority [or] prototyping contracts to further advance that type of technology,” Maue said.
Exoskeletons support the weight of limbs to reduce strain on shoulders and arms. The vest provides 5 to 15 pounds of lift assistance and would help with overhead work such as aircraft maintenance. Photo: EksoVest
INTELLIGENT ASSISTANCE
Further off—but still within reach—is what AFWERX calls assisted human operations.
“Think of this as kind of like the Iron Man suit,” Harden said. “You know that Iron Man suit is not about the physical metal material that’s flying around. It’s about all the algorithms and software necessary to interpret in real-time all the external sensor data, integrate it, and then create a response that assists the human.”
Say, for example, there are five airmen on the flight line tasked with moving munitions. If you gave one of those airmen an exoskeleton suit, he might be able to accomplish the work all by himself, freeing up the other four airmen for another task, said Harden.
AFWERX is also looking for opportunities to investigate advanced human-machine teaming. For instance, how might a pilot work seamlessly with an unmanned platform in a multi-domain operation?
The ability to operate concurrently in multiple domains is one of USAF Chief of Staff Gen. David L. Goldfein’s top three priorities, and AFWERX has an open MDO Fusion challenge intended to identify innovative concepts that could advance the concept. Harden said his team expects AI to feature prominently in the resulting proposals.
2nd Lt. Charles Keller and A1C Tyler Haselden, Pilot Training Next students, maneuver on a virtual reality flight simulator at the Armed Forces Reserve Center in Austin, Texas. Photo: Sean Worrell/USAF
In science fiction, AI is typically seen as autonomous robots or computers with a mind of their own. In reality, it’s a natural extension of the Information Age: It is what becomes possible when enough data can be captured and enough computer power can be applied to calculate in a timely way things that, before now, were simply unimaginable. AFWERX sees at least part of its role as working to “demystify” the technology—to make the concepts accessible to the people who stand to benefit from what they can offer.
“I think artificial intelligence is much more nuanced than what we see maybe in the headlines,” Harden said. “It’s not about killer robots. … We can help humans find targets in landscapes. We can cut maintenance. We can do communication validation. We can use artificial intelligence for financial data. It’s really taking all those use cases, doing a short experiment, and then maybe you start to see people who maybe aren’t as familiar with tech … say, ‘Oh, I get it.’?”
The applications are endless. But the most effective use cases are the ones where the subject matter experts themselves see the possibilities.
“As soon as you have that aha! moment,” Harden said, “you’re able to move the needle on artificial intelligence: the way we do business and fight in the future.”
Finding those ideal-use cases and proving their value is the challenge right now, Harden acknowledged. “Our only path forward is to rapidly experiment.”