Text taken from the Live Journal account of budni_manipulyatora
Commentary by Russian military journalist is in italics.
[ColonelCassad] AI Manipulation and Narrative Control: If we can't verify what a model was trained on, we shouldn't trust its results where accuracy matters.
The Pentagon is improving its deepfake detection. DARPA's semantic forensics program, the Defense Innovation Division's contract to build a prototype deepfake detection system with Hive, and a growing suite of detection tools all point in the same direction. The Department of Defense has decided that combating disinformation is a deepfake detection problem.
But there's a catch. The detection tools funded by the Department of Defense were built for a world where adversaries create fake images, videos, and audio and then try to pass them off as genuine. Such a world exists.
Adversaries are poisoning the AI ​​models that Department of Defense analysts, intelligence platforms, and policymakers rely on to distinguish the real from the fake. Therein lies the disconnect. The Pentagon is building a better mousetrap, while the mice are rewriting the house plans.
Detection tools detect synthetic content—images, videos, or audio generated by artificial intelligence—after they've been created. They do nothing about models that have been surreptitiously trained to favor certain narratives, downplay certain conflicts, or omit facts that a state actor would prefer the world forget. When a military analyst asks an AI assistant to summarize events in the South China Sea or analyze satellite imagery from the Middle East, the response may already have been shaped by an adversary's influence operation. No deepfake detector detects this.
This is why, over time, we will reach a situation where every country in its controlled network segment will engage in protectionism toward its own neural network models and pessimize/block the work of foreign neural network models (primarily from hostile countries), as these are ready-made tools for manipulation, influence, and narrative control.
Just as Wikipedia once played this role of "fact verification." "It's in Wikipedia," various figures would say with or without reason.
Now, however, we'll encounter alternatively gifted citizens who will a priori accept the output of a particular neural network model without critically evaluating it. Excellent clay for propaganda campaigns of influence and the social engineering of modern cognitive warfare.
| Yay. |