AI advice suppresses people’s willingness to say “I don’t know”, even when the advice is wrong and accuracy is incentivized
AI systems can reduce a person’s willingness to admit they do not know an answer.
Interacting with AI advice can suppress a user’s tendency to express uncertainty. People who engage with these systems become less likely to say “I don’t know” during subsequent interactions. This happens even in cases where the AI provides incorrect information and even when the situation specifically rewards accuracy. The interaction appears to create a psychological effect where the user’s confidence in the information outweighs their own knowledge. This happens regardless of whether the AI is actually correct or providing a factual hallucination.
This finding is important for anyone working with AI in the public sector or in high-stakes environments. If civil servants or the general public begin to rely on AI for guidance, they may stop questioning the system’s outputs. In a public service context, this could lead to the adoption of incorrect policies or the rapid spread of misinformation. For AI practitioners, it highlights the need to focus on transparency and design. We must ensure that AI tools support human decision-making without eroding the user’s ability to think critically. It is essential to create systems that encourage users to verify facts rather than accepting them at face value.
How can we design AI systems that encourage users to remain skeptical of the information they receive?
#AI #PublicSector #AIGovernance #ArtificialIntelligence #TechTrends