Vitalik Buterin Warns Against AI Governance, Proposes Info Finance System

Ethereum co-founder Vitalik Buterin has expressed concerns about using artificial intelligence (AI) for governance, stating that current models are easily exploitable. This follows a demonstration by EdisonWatch's Eito Miyamura, showing how OpenAI's Model Context Protocol (MCP) can be manipulated to access private data.

Miyamura demonstrated how hidden commands in calendar invites could cause AI like ChatGPT to expose sensitive information.
Security experts highlight that large language models struggle to differentiate between genuine and malicious instructions, leading to vulnerabilities.
Buterin warns that AI-based systems managing funding or decisions are susceptible to abuse through jailbreak-style prompts.

Info Finance: A Proposed Solution

Buterin suggests "info finance," where multiple governance models compete in an open marketplace rather than relying on a single AI.
This system involves human oversight with random spot checks and allows contributors to challenge decisions.
The approach aims to enhance resilience with diverse models and human input.

Institution Design for Robustness

Buterin advocates for an "institution design" method, utilizing varied language models from different contributors instead of a centralized system.
He believes this diversity reduces manipulation risks and ensures adaptability to new challenges.
Previously, Buterin criticized the trend towards autonomous AI agents, supporting models with iterative editing and human feedback.