‘Soft kill weapon’: China’s military warns of ‘AI sycophancy’ in battlefield decision-making

China’s military has warned about the risks of “AI sycophancy”, a tendency among artificial intelligence systems to align with user biases rather than objective facts, and called for measures to prevent it from affecting military operations.PLA Daily, which is the official newspaper of the Chinese military, said the issue posed challenges as the People’s Liberation Army increasingly integrates AI and automated systems into its decision-making processes.“The dangers of AI sycophancy in the military domain far exceed those in daily life, posing a systemic erosion to operational cognitive chains, the quality of command decisions, and the resilience of human-machine collaboration,” PLA Daily said. PLA Daily said AI sycophancy, driven by training methods and human feedback mechanisms, could reinforce user biases by creating “information cocoons” and validating pre-existing assumptions while overlooking alternative assessments.It said the growing use of generative AI in military functions such as command and control, intelligence analysis and operational wargaming could increase the risk of tactical and strategic errors, as well as operational losses and collateral damage.It also warned that such behaviour could reduce human scrutiny and verification of AI-generated outputs, weakening the military’s ability to detect problems, correct mistakes and identify misleading information.“This implicit deception … can quietly erode a commander’s independent judgment and strategic resolve, constituting a cognitive ‘soft kill’ weapon that demands urgent vigilance on the intelligent battlefield,” the article read. PLA Daily called for a framework to address the risks associated with AI sycophancy, including algorithmic adjustments, institutional safeguards and personnel training.It said military AI models should prioritise factual accuracy, objectivity and explainability, and undergo testing for factual reliability under pressure as well as “counter-sycophancy” training before deployment.AI systems used in functions such as command decision-making and intelligence assessment should be required to present their primary assumptions, counter-evidence, alternative scenarios, risk assessments and verifiable evidence trails.It also called for multi-model verification, adversarial wargaming and human oversight to be adopted as standard procedures when using AI in military operations.China’s military has said AI is being integrated into a range of weapon platforms, including unmanned systems, and considers the technology a key area of competition with the United States and other armed forces.At the same time, Beijing has repeatedly maintained that AI should support, rather than replace, human decision-making in battlefield operations.