AI Economics / No. 035
The Problem with Modeling AI Governance on FINRA
OpenAI, Anthropic, and Google DeepMind have reportedly discussed a FINRA-style self-regulatory body for frontier AI. But FINRA works because federal law makes expulsion fatal. Without statutory backing, an industry standards body is either toothless during competitive races or a private mechanism that raises compliance hurdles for challengers.
Representatives from OpenAI, Anthropic, and Google DeepMind have held recurring working-group sessions since July to negotiate an industry-led standards body for frontier artificial intelligence, according to reporting by The Information. The initiative envisions common protocols for pre-release evaluations, independent audits, and standardized risk benchmarks. In public remarks and essays over the summer, executives from the participating laboratories repeatedly pointed to a specific institutional template: the Financial Industry Regulatory Authority, the self-regulatory body that polices American broker-dealers. Demis Hassabis floated the comparison in July, while Sam Altman argued that major frontier developers should construct an oversight framework themselves without relying on the federal government.
The motivation behind the effort is easy to understand. Frontier model evaluation requires specialized engineering talent, extensive compute clusters, and proprietary testing harnesses that exist almost exclusively within the largest research organizations. Legislative efforts move at the pace of congressional committee markups, and executive branch actions frequently stall amid administrative infighting. Expecting civil servants to draft dynamic red-teaming benchmarks for rapidly evolving multimodal systems is unrealistic. Technical standards organizations like the W3C or the Internet Engineering Task Force have long shown that engineers closest to the infrastructure can write durable protocols far faster than statutory regulators.
The difficulty lies in the analogy. Borrowing the prestige of FINRA ignores the legal architecture that makes securities self-regulation functional.
FINRA is described as a self-regulatory organization because broker-dealers fund its budget and participate in its governance committees. But FINRA does not derive its authority from voluntary consensus. It operates under express statutory delegation established by the Maloney Act of 1938, which amended the Securities Exchange Act of 1934 to authorize registered national securities associations under the supervision of the Securities and Exchange Commission. Under federal securities law, operating as a registered broker-dealer without membership in such an association is illegal. If FINRA suspends a firm, levies a fine, or revokes a license, the firm cannot simply resign from the club and continue trading equities. The state shuts it down.
Strip away statutory authority and SEC oversight, and an artificial intelligence standards organization faces an unstable payoff matrix.
Without legal enforcement, a voluntary consortium confronts a classic defection incentive. Training and running a frontier model requires hundreds of millions of dollars in amortized capital expenditure. If Laboratory A discovers an ambiguous safety signal during a pre-release audit, but Laboratory B or an open-weights competitor is preparing an equivalent launch, Laboratory A bears immediate commercial penalties by delaying deployment. Under voluntary self-regulation, resigning from the consortium carries zero statutory cost. When commercial survival collides with a voluntary evaluation pledge, the dominant financial strategy is to ship anyway and relabel the risk in a technical appendix. Voluntary restraints hold while margins are wide and release schedules are spaced; they dissolve the moment market share is contested.
The alternative scenario presents a different institutional risk. If the three founding laboratories successfully establish their shared testing framework as an informal prerequisite for enterprise procurement and government contracts, the voluntary standard acquires de facto regulatory force. Fortune 500 compliance officers seeking insulation from liability will mandate that vendors carry the consortium’s seal of approval.
Once a private evaluation standard becomes a procurement gate, it functions as a barrier to entry. Designing and executing comprehensive multi-tiered evals across thousands of benchmark variants demands tens of millions of dollars in compute infrastructure and specialized red-teaming personnel. Hyperscale operators and well-funded laboratories can absorb that overhead as a minor line item within multi-billion-dollar budgets. For a venture-backed challenger, an academic spinout, or an open-weights developer, that compliance overhead represents an insurmountable fixed cost. Competitors like Meta and Cohere have already expressed skepticism toward the working-group talks, with Cohere’s leadership openly characterizing the effort as an incumbent alignment.
In financial markets, self-regulation functions only because the legal authority of the sovereign sits directly behind the disciplinary committee. In technology markets, private standards without statutory mandates alternate between cheap talk during competitive sprints and market foreclosure during consolidation phases. Borrowing the FINRA label offers the appearance of disciplined institutional governance while omitting the only mechanism that makes it work.