Artificial Intelligence in the IC: Culture is Critical
(Editor’s Note: This article was first published by our friends at Just Security and is the second in a series that is diving into the foundational barriers to the broad integration of AI in the IC – culture, budget, acquisition, risk, and oversight.)
OPINION — Several weeks ago, I wrote an article praising the widespread, bipartisan support for the U.S. Innovation and Competition Act (USICA), which would dramatically expand federal government support for U.S. technological growth and innovation in the face of the global AI race.
In that article, I argued that for the Intelligence Community (IC) to take advantage of AI in this supportive environment, it must overcome several critical implementation challenges, and quickly. In particular, the IC must more rapidly and nimbly navigate U.S. government budget and acquisition processes, create a simple but effective risk assessment framework, and work with congressional overseers to streamline engagement and improve the partnership between Congress and the IC. Each of these areas is in dire need of radical re-imagining, without which any one of them could be the Achilles’ heel for AI in the IC. I will address each of these in my next few articles.
To successfully tackle any of these specific tasks, though, the IC must at the same time prioritize an issue much more intangible and nebulous – its own culture. Culture is the ethos of an organization – the beliefs, behaviors, values, and characteristics of a group that are learned and shared over many years. In the IC, there are several predominant cultures, all of which flow from the mission of the IC – protecting the women and men on the front lines, defending U.S. national security, and delivering insights to policymakers at the speed of decision. This mission is a powerful and unifying force that naturally leads to important IC values and behaviors.
IC Culture Today
Intelligence operations – uncovering foreign secrets and protecting assets, for example – are inherently risky; they very often put people in harm’s way. If there is a leak of information related to an operation – if the people involved, or the location or target of an operation, are exposed – not only might the mission fail to collect the desired information, but someone’s life could also be in jeopardy. The extreme consequences of leaks are well understood, thanks to notorious spies like Robert Hanssen and inside leakers like Edward Snowden. But significant damage can also flow from what seem like merely small mistakes. If someone fails to make a connection between relevant information or forgets to check a database of known terrorists, for example, the results can be just as disastrous. Thus, the IC’s high-stakes operations drive an enormous emphasis on security, preparation, and tradecraft, all of which help mitigate operational risk.
This same spirit manifests in “enabling” activities, like budget, acquisition, or human resources, through a focus on certainty of action and predictability of results. Enabling activities – by some considered a negative term but one in which I take pride as a life-long enabler – are somewhat removed from the “pointy end of the spear” but are no less critical to the ultimate success of the mission. Proper funding and resources, the right capabilities, skilled officers, legal approval, and the many other support activities are integral to successful operations.
In the field, risks are unavoidable – operators cannot choose inaction to avoid those risks. Given that risks are inherent in what they do, they must accept the reality that risks are inevitable, and they must learn to manage those risks to get the huge payoff of successful operations. So, the focus is not on risk avoidance, it is on risk management – what level of risk is acceptable for what level of intelligence gain?
The IC’s high-stakes operations drive an enormous emphasis on security, preparation, and tradecraft, all of which help mitigate operational risk.
Back home, where most enabling activities are handled, risks are not seen as inevitable – certainly not big ones. They are seen as avoidable, and subject to being minimized and mitigated. And some believe the best way to do that is by staying with tried-and-true standard operating procedures rather than experimenting with new approaches. Innovation is inherently risky. It can and will fail. Innovation is not mandatory, it is entirely avoidable. Therefore, if the tendency is to avoid risks, in most cases innovation will be avoided.
In addition to this instinct, there are compounding issues that discourage innovative change in enabling activities. First, there are practical difficulties: change is hard, messy, and requires resources that most offices can’t spare. These concerns alone are big hurdles to clear. Second, innovative change means uncertainty – in execution, accountability, and success. And that uncertainty leads to the risk that projects may fail, resulting in loss of money, reputation, or even position. Thus, control, compliance, and trust are paramount, and there is a strong aversion to things “not invented here.” Innovation is not particularly welcome in this environment and introducing new ideas can be an uphill battle, discouraging creativity in areas where it is needed most.
When it comes to budget and acquisition processes in particular – which are critical to the IC’s ability to quickly harness the power of AI capabilities – the IC’s risk-averse culture is also drawn in part from its reliance on Department of Defense (DOD) processes. And while DOD processes have aligned to IC needs in the past, they are based on decades-old approaches used for major systems acquisitions, like airplanes, aircraft carriers, and satellites. These are big ticket items that require minimal risk and significant time, structure, and certainty to move forward, because any failure could have costly consequences. But using this same approach for emerging technology like AI hampers the IC’s ability to promptly obtain and use it before the underlying technologies become obsolete. In these cases, the IC must have the ability to try new things quickly, adjust on the fly, and accept some level of loss and failure as it continues to grow new technologies and processes. The IC must embrace a more flexible, innovative approach for AI – even as that approach introduces more uncertainty.
Opportunity Born from Crisis
It is difficult, if not impossible, to create major cultural change in the absence of a compelling real-world problem. As leading change management expert John Kotter explains, a sense of urgency helps others see the need for change and the importance of acting immediately. For example, prior to the COVID-19 pandemic, telework in the IC was virtually unheard of. Over the past 18 months, however, the IC not only achieved telework for many roles, telework proved to be so productive that many IC elements have now also changed their standing policies to allow it in certain roles going forward. Using that crisis to create a blueprint for new long-term approaches was key to jump-starting a real transformation.
It is difficult, if not impossible, to create major cultural change in the absence of a compelling real-world problem.
Another striking example occurred after the September 11, 2001 attacks (“9/11”). Prior to 9/11, the IC culture around sharing and protecting information was heavily weighted toward jealously guarding information within agency stovepipes. This was a result of many things – including the genuine need to protect sources and methods – but it evolved in the extreme because it is easier to protect sensitive information if fewer people have access to it and, importantly, knowledge is power. After 9/11, though, IC culture slowly started to evolve to one that acknowledged the need to connect the dots and proactively share information with critical partners, even while protecting it. The need to share information more freely within the IC had been documented for decades and executive branch policy already allowed information to go to those who had a “need to know,” but it took the crisis of 9/11 to really start to break through the cultural barriers to information sharing.
Indeed, the IC now has created an entire information technology enterprise architecture – groundbreaking for the U.S. government when the enterprise was started in 2011 – founded on the concept that the IC agencies must work together and share information more easily than in the past. The IC Information Technology Enterprise (IC ITE) has been underway for a decade and is not yet complete, not solely due to its audacious technical goals but because it is breaking down cultural barriers, connecting the IC as never before, and pushing the IC together beyond its comfort level. Culture shifts slowly, and this one has had its detractors. But eventually over time there has become a general acceptance of and willingness to share information in ways that would previously have been unthinkable. The 9/11 crisis and subsequent terrorist attacks (and attempts) brought the responsibility to share information into immediate, sharp focus.
As I discussed in my previous article, one of the IC’s crises today is the ubiquity of and access to AI around the globe. Sophisticated technology like AI is no longer available to only a handful of wealthy governments; it is now abundant and often easy to acquire, enabling smaller foreign governments and non-governmental actors alike to take advantage of it. If the IC does not modernize its approach to AI quickly, it is only the United States that will pay. The IC must turn this crisis into an opportunity to embrace a more flexible and innovative culture that supports the IC’s ability to leverage AI tools at the speed of mission.
IC Leadership Action Required
Culture is driven from the top through leadership actions, behaviors, and priorities that reverberate across every level of an organization. To grow an IC-wide culture of innovation, IC leaders must set the right vision and tone by affirmatively and publicly embracing outside-the-box thinking, articulating acceptable risk and expected failure rates, continuing to back innovators when projects go awry, and setting expectations for collaboration between innovators and practitioners in every area – from budget to acquisition to operations.
In 2018, the Director of National Intelligence (DNI) made IC-wide innovation a top priority by creating an ODNI innovation organization to lead and support, among other things, cross-IC creativity and modernization – to help pioneers find each other, to encourage the IC to leverage each other’s ideas, to provide advice to those hoping to contribute new ideas, and to train officers on innovation best practices. This organization also led the top IC strategic initiatives, including Augmenting Intelligence using Machines (AIM), which drove senior and subject-matter-expert coordination and collaboration across the IC on AI and machine learning activities to better align innovation, acquisition, and use of AI and emerging technology.
The AIM initiative was one of only six priorities the directors of every IC element very visibly embraced as imperative to the future of the IC. As a result, the AIM initiative had relative success in its first few years, bringing the community together and ensuring better coordination across the many AI initiatives. However, the innovation office’s more foundational task of creating a broad IC culture that embraces innovation foundered. Chronic under-resourcing of those activities signaled that they were not, in fact, a priority, and subsequent changes in ODNI leadership and support resulted in minimal progress over the course of two years. That office was recently disbanded, leaving IC innovators to continue to fend for themselves and signaling a lack of strategic support and leadership for IC-wide innovation.
There are glimmers of hope, however. Innovation remains one of the IC’s stated values, and despite the lack of coordinated approach, there are pockets of brilliant innovation across the IC where individual visionaries recognize that new ideas and technology can transform the world of intelligence. Many enterprising and energetic individuals expend herculean efforts to create new pathways, build out novel ideas, and find new solutions to old problems. They are willing to fail and accept the consequences because they believe so strongly in the importance of their work.
IC leaders must harness this grassroots energy and enthusiasm by actively supporting and connecting innovators, rewarding creativity even when a project fails, and empowering the workforce to effectively manage risk. The DNI should designate a senior leader to drive innovation across the IC, starting with a 90-day action plan that takes advantage of the AI crisis to kick-start activities that will help shape the IC culture toward one that embraces and supports innovation more broadly. Strong and steady leadership, clear prioritization, and a willingness to hold the organization accountable for achieving those goals are critical for creating and adapting a culture.
IC leaders must harness this grassroots energy and enthusiasm by actively supporting and connecting innovators, rewarding creativity even when a project fails, and empowering the workforce to effectively manage risk.
As with all organizations, some IC officers will embrace changes quickly. However, people with a long history in the IC often become attached to existing processes and mechanisms, mistaking them for the ethos of the IC. And because major culture change cannot leave all existing processes untouched, it can provoke skepticism and pushback from those folks. But dragging them along or leaving them behind is not an option – officers with time and depth in the IC are critical; they make up the engine of the community, and they support and nurture employees throughout it. Actively engaging them as a part of the process and focusing on small wins that provide tangible benefit will help cultivate buy-in for larger, overarching goals. Proving out a few impactful ideas in pilot programs will show the IC can handle increased flexibility and speed without losing the security, rigor, and accountability required. This will, in turn, refresh the IC’s culture while preserving and championing its strengths, and pave the way for innovation and AI adoption at scale.
Because no matter how many brilliant minds come together to create excellent recommendations to take advantage of AI and promote innovation, the IC will not successfully implement them without addressing the institutional resistance to new ways of doing business that acts as a self-sustaining barrier between the IC and widespread AI adoption. As Winston Churchill admonished, the IC must not let this crisis go to waste.
Corin Stone is a Scholar-in-Residence and Adjunct Professor at the Washington College of Law. Stone is on leave from the Office of the Director of National Intelligence (ODNI) where, until August 2020, she served as the Deputy Director of National Intelligence for Strategy & Engagement, leading Intelligence Community (IC) initiatives on artificial intelligence, among other key responsibilities. From 2014-2017, Ms. Stone served as the Executive Director of the National Security Agency (NSA).