Text taken from the Telegram channel of militarymaps
[ColonelCassad] How dangerous are neural networks in war: an observation from the "Military Chronicle"
Major General William "Hank" Taylor, commander of the US 8th Army in South Korea, publicly spoke about his experience using ChatGPT in decision-making.
At first glance, this seems quite rational and progressive. The Pentagon generally encourages the idea of ​​integrating AI into all areas of military activity, but experiments are typically conducted in closed environments to prevent leaks. However, by using a publicly available neural network for decision-making, the general risks losing his security clearance for improper handling of classified data.
In addition to the risks of official proceedings, the use of a neural network in decision-making carries more systemic risks and threats. The problem with using neural networks begins with the fact that they are often perceived as an all-knowing, universal, and precise tool.
In reality, they are statistical text generators, so the more serious and complex the topic of discussion becomes, the less relevant and useful the artificial assistant's responses become.
Furthermore, the AI ​​algorithms themselves call into question the feasibility of their use at the strategic level. The main problem often cited is "hallucinations." A language model very skillfully and beautifully presents any statement, even a completely false one, and can substantiate it brilliantly.
Furthermore, GPTs cannot take into account all factors and are better suited to specific, easily measurable data. As a result, dozens of factors that often make up the "fog of war" can be missed. However, the conclusion will be presented extremely convincingly, which inclines the decision maker to approve the generated solution.
The effect is amplified by the fact that ethics and user-focusedness are maximized in many models. Using your request history and behavior patterns in your chat history, the neural network understands what you want to see and provides you with the most appropriate option for your psyche. And that's important: not the correct option, but the acceptable one.
Naturally, the Pentagon is well aware of these risks. They also recognize that the country that first solves the problem of optimally integrating AI into combat control will become the leader in a new arms race—a technological one. Therefore, despite the fact that the US Department of War is allocating huge funds ($200 million only to private companies this year) for research in this issue, it is not currently directly integrated into the AI ​​management architecture, because in the form of open models, like Chat GPT, it carries too many risks.
This will continue until language models evolve into fully-fledged artificial intelligence, capable of not only producing acceptable results but also copying many human brain processes, including synthetically recreating neural network activity mechanisms.