Across the United States, state and local governments are increasingly exploring the use of artificial intelligence (AI) to improve public services. From real-time translation of public meetings to facilitate broader participation, to AI-powered chatbots that help residents navigate government services, agencies are discovering multiple ways to leverage the technology. Yet, as promising as these innovations are, they also bring significant risks if deployed without careful oversight.
A stark example of AI’s potential pitfalls occurred in Michigan, where nearly 40,000 residents were erroneously accused of unemployment insurance fraud due to a flawed algorithm and insufficient human supervision. Incidents like these underscore the importance of thoughtful, cautious adoption of AI in the public sector.
Suresh Venkatasubramanian, a computer science professor at Brown University and co-author of a recent National Academies rapid expert consultation on AI in state and local governments, emphasized the need for a measured approach. “We have to think about a lot of AI systems as potentially useful and quite often unreliable, and treat them as such,” he said. Venkatasubramanian advised leaders to avoid succumbing to hype—both positive and negative—and instead adopt experimental methods to understand AI’s potential in specific contexts.
Venkatasubramanian presented his insights during a National Academies webinar, joined by Nathan McNeese of Clemson University, Leila Doty, a privacy and AI analyst from the city of San José, California, and moderator Kate Stoll of the American Association for the Advancement of Science. The discussion focused on new guidance and resources designed to support responsible AI implementation in government.
Starting with the Problem, Not the Technology
A key recommendation from the webinar was to begin any AI initiative by clearly defining the problem it is intended to solve. McNeese stressed that agencies should ask, “What’s the problem?” or “Which part of the organization could benefit from enhancement?” rather than rushing to adopt AI simply because it is trending. “You do not want to introduce AI if there is not a specific need,” he said. “You don’t want to implement AI because everyone else is.”
Venkatasubramanian reinforced this point, urging leaders to focus first on human needs. “If you have a problem that needs to be solved, figure out what people need to solve it,” he explained. “Maybe AI can be a part of it, maybe not. Don’t start by asking, ‘How can we bring AI to this?’ That way leads to problems.”
This human-centered perspective also means designing AI solutions around people’s needs, motivations, and preferences. McNeese highlighted the importance of involving employees with direct domain expertise—those who deliver public services and understand operational realities—when deciding where AI might be appropriate. “It is really, really important to empower the people who have the expertise to understand the domain,” Venkatasubramanian said.
The ultimate goal, he and McNeese agreed, is to create complementary human-AI teams. AI should handle repetitive or computational tasks, freeing humans to focus on work that requires judgment, creativity, and interpersonal skills. “We’re not trying to use AI to replace humans,” McNeese noted. “We’re trying to support humans in doing what they do best.”
Strategic Implementation of AI
Implementing AI in government is complex, and McNeese outlined several strategies for agencies to consider. After identifying a problem AI might help address, leaders should evaluate whether AI is technically feasible, how it will integrate with existing workflows, and whether it is better to develop AI internally or rely on external vendors.
He also emphasized the importance of understanding the risks associated with different AI technologies. The National Institute of Standards and Technology (NIST) provides an AI risk management framework to guide organizations in this process. Piloting AI in a limited, controlled environment—rather than rolling it out broadly—is recommended to test its effectiveness and identify potential issues before full deployment.
Another essential step is collecting employee feedback. McNeese explained, “You want people to have a voice. You want people to give you insights, and then … you want to adopt, refine, or abandon the utilization of AI based on that evaluation and that feedback.” This iterative, evidence-based approach can help agencies maximize benefits while minimizing unintended consequences.
Venkatasubramanian added that partnerships with universities, civil society organizations, and local experts can provide valuable support. “People at universities are very interested in working with state and local governments because these demonstrate actual use cases in real settings,” he said. These collaborations can help governments refine AI applications in ways that are grounded in real-world experience.
Building a Community of Practice
In addition to individual agency efforts, broader networks are emerging to support responsible AI adoption. The GovAI Coalition, established by the city of San José in November 2023, is one such initiative. Leila Doty described how the coalition grew out of San José’s early 2023 efforts to implement an AI policy and governance framework. The city encountered challenges in obtaining transparency from AI vendors about their systems, training data, and performance metrics. Discussions with peers at other government agencies revealed similar struggles and a shared need for standardized guidance.
The GovAI Coalition aims to create a “community of practice” for public-sector AI governance. Participating agencies collaborate to set standards, demand accountability from AI vendors, and share resources including policy templates and contract guidelines. According to Doty, more than 850 agencies nationwide—spanning local, county, state, and even a few federal entities—are now members.
“The scope of the work in the coalition has really expanded, and now it’s truly become a much broader community of practice around AI in the public sector,” Doty said. She welcomed the National Academies’ recent publication, noting that its guidance aligns closely with the coalition’s principles and practices. “These strategies will be really helpful to state and local governments that are looking to adopt AI responsibly.”
Balancing Innovation with Accountability
Overall, experts at the webinar stressed that AI adoption in government should strike a balance between innovation and accountability. AI offers significant potential to improve efficiency, accessibility, and service quality, but it must be implemented thoughtfully to avoid mistakes that can harm residents and undermine public trust.
Leaders are encouraged to adopt a measured, experimental mindset: clearly define problems, involve domain experts, pilot solutions, solicit feedback, and continuously refine approaches. Partnerships with universities and membership in networks like the GovAI Coalition can provide additional guidance and support.
As government agencies navigate this rapidly evolving technological landscape, the message is clear: AI should serve people, not the other way around. By focusing on human needs, emphasizing transparency, and embracing iterative experimentation, state and local governments can harness the benefits of AI while minimizing its risks.
AI in the public sector is not just a technological challenge; it is a human-centered endeavor. Success depends on careful planning, collaboration, and a commitment to serving the public effectively and responsibly. With the right approach, AI can become a powerful tool for enhancing government services, making them more inclusive, efficient, and responsive to the communities they serve.
Leave a Reply