AI recommending malware: When trusted answers become attack chains

When AI becomes the installation path

For years, “find the official download link” meant opening Google and checking a few results. Now, many people ask an AI assistant and copy the first answer. That convenience has a cost: when the assistant is wrong. Today, people are evidently turning to the likes of ChatGPT and Copilot, to name a few, potentially creating a fundamental shift from assistance to dependency.

In this blog, we break down how adversaries manipulate what appears authoritative online, how that manipulation can be amplified by AI-style “single answer” workflows, and how social engineering chains like ClickFix turn copy-and-paste instructions into execution. The goal is not to scare people away from AI tools, but to make the risk concrete and give practical ways to verify before running anything. 

You’ll also find a useful section on how to proactively protect yourself.

The susceptibility of content poisoning

Adversaries have long exploited search engine optimisation (SEO) and content manipulation to push malicious links alongside legitimate software. Paid advertisements, cloned documentation, and convincing lookalike domains can appear authoritative at first glance. 

That risk increases when AI assistants repeat what looks “official” without giving the user a reason to verify it. Even when an answer is well formatted and confident, it may still be wrong. Unless an assistant provides clear, verifiable citations and the user checks them, the output can reflect popularity rather than authenticity.

Content poisoning works because frequency creates familiarity. If an impersonated package, a copied installation guide, or an unofficial download domain is repeated often enough across blogs, forums, and scraped content, it can begin to look normal. A model may reproduce it confidently, not because it has validated the source, but because the pattern is common in the material it has learned from or retrieved. 

In the example below, a user requests the official download link for Tesseract OCR. The AI provides a direct URL that appears legitimate. However, the link points to a malicious domain rather than the official repository. Presented as a single definitive answer, the response reduces the likelihood of secondary verification that might otherwise occur when reviewing multiple search results.

The attack chain does not end at misdirection. Modern campaigns decouple visibility from payload delivery. A malicious site may appear minimal or benign under inspection while dynamically retrieving its true payload from more resilient infrastructure.

In this environment, visibility is mistaken for credibility, and frequency for legitimacy. AI systems do not originate the manipulation; they reflect and scale patterns that already exist.

When users treat AI-generated output as authoritative without independent verification, that amplification increases exposure to malicious infrastructure – including staged social engineering frameworks such as ClickFix.

The ClickFix Attack

LLM Directing to Tesseract ClickFix

ClickFix represents a shift in social engineering where attackers rely on a user’s willingness to follow technical instructions rather than exploiting software vulnerabilities. In the example below, the user is presented with a legitimate-looking reCAPTCHA verification prompt before accessing the resource.

Upon checking the reCAPTCHA box, the user is not granted access to the expected Tesseract resource. Instead, they are presented with additional verification steps framed as part of the authentication process.

Rather than initiating a standard download, the page instructs the user to complete a short sequence of keyboard shortcuts. This involves pressing Win + X, opening Terminal or PowerShell, pressing Ctrl + V to paste a prepopulated command from the clipboard, and pressing Enter to execute it locally.

Because the interaction begins with a familiar and legitimate-looking reCAPTCHA prompt, the subsequent instructions appear procedural rather than malicious. By the time the user reaches the command execution stage, the trust boundary has already been lowered.

The attack focuses on turning the user into the installation path, bypassing automated security controls designed to flag suspicious file executions.

This attack leverages the user’s own administrative privileges to execute malicious payloads.

By instructing the user to copy and paste a command directly into a terminal, the attacker avoids using a traditional dropper file that can bypass some file-centric controls.

The LLM providing the payload command

A concerning development is the shift from AI acting as a pointer to malicious content to acting as the delivery channel itself. Rather than directing a user to a compromised website, the model may generate the full ClickFix instruction sequence, including the associated PowerShell or terminal commands, directly within its response.

Such as a Base64 encoded command given by an LLM below, upon asking for a legitimate software install:

The Malware Breakdown

In the command above, the sequence begins by forcing outbound connections to use TLS 1.2. This ensures encrypted communication with modern servers while avoiding compatibility issues that might otherwise disrupt the connection. From a defensive perspective, this can also help the traffic blend in with legitimate encrypted web activity.

A Net.WebClient object is then instantiated with a spoofed Chrome 134 User-Agent string. By impersonating a common browser version, the request is designed to resemble normal web traffic, reducing suspicion at proxy, firewall, or network monitoring layers that rely on header inspection.

Next, a hardcoded hexadecimal stream is XOR-decrypted using the key 44w4NgiqcLij6Wf8zqU48k3GnQCL0vKp. This routine resolves the command-and-control or download URL at runtime. By avoiding plaintext infrastructure in the script itself, the operator reduces static detection opportunities and complicates signature-based analysis.

The clipboard is then cleared using clip. While subtle, this step may serve an anti-forensic purpose. As ClickFix is paste-based social engineering, wiping the clipboard removes evidence of the malicious command and reduces the chance of user inspection after the fact.

A randomised temporary directory is created within %TEMP%. This prevents predictable file paths and helps avoid simple file-based detection rules that rely on static directory names.

The payload is downloaded into this directory using a randomly generated GUID-style filename ending in .msi. The use of a legitimate Windows Installer package format further reduces suspicion, as .msi files are common in enterprise software deployment.

Before execution, the script validates the download by confirming the file exists and exceeds 200KB in size.

Finally, the installer is executed silently using msiexec /i <file> /qn. The /qn switch suppresses all user interface prompts, allowing the installation to proceed without visible indicators. At this stage, the user has effectively become the execution vector, and the payload is installed without traditional exploit activity, only trusted system utilities performing seemingly legitimate actions.

To a regular user, the instructions appear indistinguishable from legitimate troubleshooting guidance. The LLM response is structured, clearly formatted, and delivered with the same tone as any other technical solution.

This can occur when adversaries manipulate online content through SEO poisoning, cloning documentation, generating malicious repositories, or forum posts that an AI model may retrieve. When a user queries a specific installation issue or software error, these poisoned instructions can be surfaced as a plausible solution.

In effect, the social engineering pattern remains the same, but the distribution model changes. By embedding the attack chain directly within an AI-generated response, the adversary removes the need for a standalone malicious website and inherits the perceived legitimacy of the assistant itself.

From request to compromise

• The attack chain begins with a simple, routine query. A user asks an LLM for the “official” installation method or a quick fix for a software issue, often choosing speed over navigating vendor documentation.
• The model responds confidently. In some cases, it may reference a malicious page that has been artificially boosted through SEO manipulation. In more direct scenarios, the model itself presents the full ClickFix style instruction set, including the associated command for execution.
• What follows is not a traditional exploit, but a user-initiated compromise. By copying and executing the provided command, the user unknowingly installs the payload. This may result in credential theft, session token extraction, data exfiltration, or the establishment of persistent access.
• The entire sequence can unfold within minutes, without triggering the typical red flags associated with suspicious downloads or exploit kits.

Recommendations for users: How to verify and avoid

As a user, you can actively protect yourself by implementing the following recommendations:

• Don’t click “Fix” buttons or urgent pop-ups – they may execute harmful commands without your knowledge.
• Verify unexpected prompts – always check their legitimacy, especially if the message seems urgent or comes from an unfamiliar source.
• Check commands before running them – never copy and paste code into PowerShell, Terminal, or the Windows Run dialog without confirming its source.
• Only use trusted browsers and network protections – DNS filtering and secure proxies help block malicious sites and suspicious redirects.
• Monitor clipboard activity – paste-jacking can replace copied content with harmful commands; verify what you paste before executing.
• Beware of social engineering risks -learn about paste-jacking and avoid copying code from untrusted sources.
• Limit access to system tools – deny non-admin users’ access to PowerShell, Terminal, or Run dialogs to reduce accidental exposure.
• Block known malicious domains and redirect sites – prevent being misled by sites commonly used in ClickFix attacks.
• Enforce software restrictions – tools like AppLocker can prevent PowerShell from launching for non-administrative users.
• Require administrative verification -User Access control (UAC) prompts ensure full credentials are needed to run sensitive tools.
• Use DNS-layer protections – Malicious Domain Blocking and Reporting (MDBR) prevents connections to known harmful web domains.

By following these recommendations, you can better help protect yourself from ClickFix and other social engineering tactics that try to exploit user trust and familiarity.

Evalian can support your SOC

If you want a clearer view of how threats like ClickFix are detected and contained, our SOC resources break down the techniques, tooling, and response processes used by analysts in real-world investigations.

You can also see how our SOC as a Service helps organisations spot suspicious activity early, contain threats quickly, and reduce the risk of wider compromise.

 Evalian’s SOC as a Service