Building Trust in AI: The Air Force’s Make-or-Break Challenge

NATIONAL HARBOR, Md.—As Air Force leadership prepares to grant artificial intelligence an expanding role in critical decision-making, officials are simultaneously focused on earning the confidence of the airmen who will work alongside it.

“A little bit scary, but we need to lean into it,” said Air Force Secretary Troy Meink at the Air and Space Association’s Air, Space & Cyber conference this week. “Autonomy is one of, if not the key technology enabling cost-effective combat power.”

While Meink did not elaborate explicitly, that embrace of AI will demand a fundamental rethinking of how human airmen build trust with the AI assistants, chatbots, and autonomous drones poised to become ubiquitous in military operations. Other corners of the Air Force have already been grappling with this challenge, particularly as the service moves toward fielding 500 highly autonomous collaborative combat aircraft, or CCAs, by 2032.

“There’s a trust and autonomy question,” said Michael Gregg, director of aerospace systems at the Air Force Research Laboratory. “We have psychologists and behavioral scientists in our 711th Human Performance Wing that are actually studying this problem: How do you actually team machines with humans to make the human really really good at what they do and aid their decision-making? That is the future.”

For certain missions — such as defending air bases from drone attacks — the Air Force is even exploring the possibility of entirely replacing human judgment rather than merely augmenting it. The waves of drones striking Ukrainian cities and devastating U.S. bases in the Middle East have prompted service leaders to investigate the use of AI agents capable of taking independent action to streamline the approval process for launching counterattacks.

Such an approach would require humans to define acceptable thresholds of risk for allowing AI agents to make consequential decisions on their own.

“There is an increased push on how we use agents and agentic workflow into everything from fires to our back-office activities, and like you’ll see a lot more of that coming this year,” said Ben Van Roo, CEO of Legion Intelligence, a company that develops agentic AI systems for national-security applications.

The prospect of AI directing “fires” is contentious, as the term implies empowering a software program to launch a missile, fire a weapon, or otherwise produce lethal effects. While certain exceptions exist, military doctrine generally upholds a strong preference for meaningful human control over potentially lethal outcomes.

However, cyber operations, directed-energy weapons, and other non-lethal technologies are providing the military with new options that could be deployed without direct human oversight, said Mike Hiatt, chief technology officer at Epirus, a company specializing in microwave and other directed-energy counter-drone defenses.

“If you can have an option that has lower collateral effects and no collateral damage, you know, which is what the directed-energy weapons offer, you start to be able to change your risk calculation. You start to be able to make decisions where you say ‘OK, I’m a little more comfortable with putting this part of the system on a fully autonomous mode,” Hiatt said.

Van Roo said that the decision-making processes required to permit autonomous cyber operations or non-lethal effects could eventually be accelerated — or even fully automated — through AI.

“We’re really still in the infancy right now of how we think about where we’re going to use agents, what are we going to allow them to decide on? How do they work their way into our tactics, techniques, and procedures, into our doctrine?” he said.

Earning comfort among leaders and operators with AI agents will require new doctrine, updated training, revised permissions, and a high degree of confidence that AI tools will behave in predictable and reliable ways.

“This is going to happen right now in front of us before AI and killer robots take over the world,” he said.

This year’s conference took place against the backdrop of a vigorous national debate over the risks of AI, how it should be regulated, and who should make those decisions.

This summer’s disclosures by Anthropic and OpenAI about surprising and dangerous model behaviors were followed on Sept. 9 by the high-profile resignation of an Anthropic engineer who estimated a 10-percent probability that AI “could kill all humans.” The next day, representatives from those companies and Google intensified efforts to establish an industry safety body while signaling openness to Congressional intervention. Regulation, however, faces opposition from President Donald Trump, who posted this week, “The only control or ‘guardrail’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!”

The Pentagon’s willingness to embrace AI may partly stem from the fact that existing defense systems failed to protect key U.S. bases. But it may also reflect several years of Defense Department efforts to confront questions of intent, alignment, and safe use — concerns that frontier AI labs are only now beginning to address in earnest. Since 2019, the military has maintained a set of ethical principles governing AI development, experimentation, and deployment. It also rigorously reviews processes, experiments, and testing procedures for new technology, including software and, naturally, the most dangerous weapons on Earth.

Yet the rapid pace of AI advancement is quickly reshaping established concepts of how to develop and test weapons. Ryan Tseng, president and co-founder of Shield AI, told attendees at AFA that his company had collaborated with Ukrainian forces to enhance the strike capability of one of its drone weapons by 70 percent through “advanced autonomous behaviors.”

If autonomous drones have defined the war in Ukraine, many military officials expect the next major conflict to be characterized by rapid collaboration among AI agents, human operators, and drones — accelerating the pace of strikes and counter-strikes. That future will demand continuous progress in building trust between humans and AI systems.

“When we think about a future that’s going to have very large-scale deployments of systems that need to interoperate with each other,” said Tseng. “It’s a much larger decision space. A lot more things can happen.”

Source link