Artificial Intelligence and Government Leadership: Insights Revealed

Artificial Intelligence Governing Administration: Insights Revealed

In recent years, generative AI has made its way into various government sectors, aiming to automate tasks previously performed by human workers. However, the use of this technology comes with its own set of challenges and errors that require careful consideration.

One significant issue is the potential for generative AI systems to provide outdated or incorrect legal information. Due to a lack of up-to-date data and the complexity of legal nuances and contexts that require human judgment, these systems can often misinterpret laws and produce unreliable legal texts without human review. This can lead to responsibility gaps, particularly in the case of unpredictable autonomous behavior that challenges established legal accountability frameworks.

A 2024 study found that chatbots specifically designed for legal research made factual errors, or hallucinations, 17% to 33% of the time. Lawyers have to carefully vet the language in contracts generated by generative AI due to its unreliability in legal reasoning. Courts can disagree on the interpretation of the law, making the answers to seemingly simple questions unclear.

The tax code isn't always clear what can be written off as a medical expense, making it subject to individual court cases. This complexity adds another layer to the challenges faced by generative AI in providing accurate and reliable information in this area.

In the legal field, retrieval-augmented generation, a technique commonly used by legal chatbots, often produces errors as well. A 2024 study by a law professor at the University of North Carolina, Chapel Hill, did not involve generative AI but did highlight the need for disclaimers informing users that they're not talking to a human and making clear that the output isn't legally binding.

The U.S. Army has deployed CamoGPT, a generative AI tool, to review documents to eliminate references to diversity, equity, and inclusion. This deployment raises questions about the appropriate use of generative AI in government, particularly when it comes to sensitive topics.

Despite these challenges, government agencies are pushing forward with the use of generative AI. The Department of Education, for example, has proposed using generative AI to answer questions from students and families on financial aid and loan repayment. The Department of Veterans Affairs is using generative AI to write code.

The General Services Administration and the Social Security Administration have deployed ChatGPT-esque technology for their workers. Tech companies like OpenAI, Anthropic, and Google are partnering with governments to find useful applications of generative AI in government.

However, the error-prone nature of generative AI may not speed up procurement, especially in negotiations involving large amounts of money. Government agencies need to adopt a clear chain of command showing who is in charge of creating and maintaining these chatbots, according to a law professor at the University of California, Irvine, who collaborated with Osofsky on the study.

The Internal Revenue Service does not currently offer a generative AI-powered chatbot for public use, but a 2024 IRS report recommended further investment in AI capabilities for such a chatbot. The GSA, on the other hand, wants to use generative AI for tasks related to procurement.

The Trump administration released GSAi, a generative AI tool, to 13,000 people on an accelerated timeline without careful deployment for narrow purposes, according to Young. This rapid deployment without proper preparation highlights the need for caution when implementing generative AI in government services.

In conclusion, while generative AI holds promise for automating government tasks, its error-prone nature and potential to misinterpret complex legal and financial information necessitate careful consideration and oversight. Government agencies must take steps to ensure the reliability and accountability of these systems to avoid potential pitfalls and maintain public trust.