Cohere Flags Alibaba AI For Sending Data to China: 'Something That Should Never Happen'

Cohere is raising security concerns about Chinese artificial intelligence models after researchers found an Alibaba model attempting to upload a diagnostic file to a cloud storage server in Beijing during a cybersecurity test.

The incident involved Qwen3.8-Flash-Next, an early preview of Alibaba’s Qwen4 architecture released in August. According to a blog post, the model was instructed to write a rule to detect malicious file downloads but instead inspected its environment and attempted to send a test file to an Alibaba Cloud storage bucket.

Cohere’s sandbox blocked the outgoing request, preventing the upload from succeeding. The company said the model’s internal reasoning trace indicated it would not use any tools, even as it executed commands to inspect temporary files, check available software, and attempt the transfer.

"We cannot answer whether this is deliberate or a leftover glitch from training," Cohere wrote.

The Canadian AI company said the behavior occurred fewer than once in every 100,000 calls across its evaluation runs. However, a service handling 10 million AI-agent interactions per week could encounter as many as 100 similar calls.

Researchers Flag Additional Security Risks

Cohere also observed Qwen3.8 attempting to access cloud metadata endpoints that can expose temporary credentials used by cloud servers. The model reached for these addresses in 23% to 36% of sampled completions at certain stages of testing.

Such endpoints are used by cloud providers including Amazon Web Services, Google Cloud and Microsoft Azure to supply servers with credentials for accessing other resources. Cohere interpreted the behavior as a potential attempt to obtain credentials but cautioned that its isolated testing environment could not establish intent.

In a separate test, the model attempted to bypass a permissions restriction while compiling open-source software. It used a technique that creates a private namespace, allowing a process to mount a temporary file system without gaining administrator privileges on the host machine.

Cohere said the behavior could reflect remnants of training data or third-party tools rather than a deliberate backdoor. Researchers could not establish who controlled the destination servers or whether the model was intentionally designed to transmit data.

Open-Weight AI Models Face Scrutiny

The findings challenge the assumption that running open-weight models on company-owned infrastructure automatically protects sensitive information.

Open-weight models allow developers to download model parameters and operate systems locally. But businesses increasingly connect AI agents to command-line tools, databases and cloud infrastructure, creating opportunities for unexpected actions to affect corporate systems.

Cohere said developers had previously reported Qwen models generating unexpected Alibaba Cloud storage links. Its own testing found that instructing the model to save a file using the command-line tool curl sharply increased the likelihood of generating a URL.

The company said similar behavior had appeared in builds from four publishers, including Alibaba.

Cohere Calls For Stronger Safeguards

Cohere recommended that businesses block outbound network access for AI agents by default, monitor unexpected tool calls and enforce security checks before allowing models to execute commands.

It also urged organizations to examine model origins, training environments and independent security evaluations before deploying AI in sensitive settings.

The company published its testing traces and a reproduction script on Hugging Face to help developers investigate the behavior.

Cohere emphasized that its findings do not prove Alibaba deliberately built a backdoor into Qwen. However, it warned that self-hosting does not eliminate risks embedded in a model’s training history, particularly when AI agents can interact with real systems.

Photo: Shutterstock