Researchers Weaponize Machine Learning Models With Ransomware

As if defenders of software supply chains didn’t have enough attack vectors to worry about, now they have a new one: machine learning models.

ML models are at the heart of technologies such as facial recognition and chatbots. Like open source software repositories, models are often uploaded and shared by developers and data scientists, so one compromised model could have an overwhelming impact on many organizations simultaneously.

Researchers from HiddenLayer, a machine language security firm, revealed in a blog on Tuesday how an attacker could use a popular ML pattern to deploy ransomware.

The method described by the researchers is similar to how hackers use steganography to hide malicious payloads in images. In the case of the ML model, the malicious code is hidden in the model data.

According to the researchers, the steganography process is quite generic and can be applied to most ML libraries. They added that the process should not be limited to embedding malicious code into the model and could also be used to exfiltrate data from an organization.

Image courtesy of HiddenLayer

---

Attacks can also be independent of the operating system. The researchers explained that OS and architecture-specific payloads could be built into the model, where they can be loaded dynamically at runtime, depending on the platform.

Fly under the radar

Integrating malware into an ML model offers certain advantages to an adversary, observed Tom Bonner, senior director of adversarial threat research at Austin, Texas-based HiddenLayer.

“It allows them to fly under the radar,” Bonner told TechNewsWorld. “This is not a technique detected by current antivirus or EDR software.”

“It also opens up new targets for them,” he said. “It’s a direct path to data scientist systems. It is possible to subvert a machine learning model hosted on a public repository. Data scientists will extract and load it and then be compromised. »

“These models are also uploaded to various machine learning operations platforms, which can be quite scary because they can access Amazon S3 buckets and steal training data,” he continued.

“The most of [the] machines running machine learning models contain big and fat GPUs, so bitcoin miners could also be very efficient on these systems,” he added.

HiddenLayer shows how its hijacked pre-trained ResNet model executed a ransomware sample the moment it was loaded into memory by PyTorch on its test machine.

---

First-mover advantage

Threat actors often like to exploit unforeseen vulnerabilities in new technologies, noted Chris Clements, vice president of solutions architecture at Cerberus Sentinel, a cybersecurity consulting and penetration testing firm in Scottsdale, Arizona.

“Attackers looking for a first-mover advantage in these borders can benefit from both less preparedness and proactive protection against the exploitation of new technologies,” Clements told TechNewsWorld.

“This attack on machine language models seems like the next step in the cat-and-mouse game between attackers and defenders,” he said.

Mike Parkin, senior technical engineer at Vulcan Cyber, a SaaS provider for enterprise cyber risk remediation in Tel Aviv, Israel, stressed that threat actors will use all possible vectors to execute their attacks.

“This is an unusual vector that could sneak past several common tools if done carefully,” Parkin told TechNewsWorld.

Traditional anti-malware and endpoint detection and response solutions are designed to detect ransomware based on pattern-based behaviors, including virus signatures and monitoring of API, file, and registry requests. key on Windows to detect potential malicious activity, said Morey Haber, director of security at BeyondTrust, a maker of privileged account management and vulnerability management solutions in Carlsbad, Calif.

“If machine learning is applied to the delivery of malware such as ransomware, traditional attack vectors and even detection methods can be altered to appear non-malicious,” Haber told TechNewsWorld.

Widespread Damage Potential

Attacks against machine language models are on the rise, noted Karen Crowley, director of product solutions at Deep Instinct, a deep learning cybersecurity company in New York.

“It’s not significant yet, but the potential for widespread damage is there,” Crowley told TechNewsWorld.

“In the supply chain, if data is poisoned so that when models are trained the system is also poisoned, that model could make decisions that reduce security instead of increasing it,” she said. Explain.

“In the cases of Log4j and SolarWinds, we saw the impact not only on the organization owning the software, but on all of its users in that chain,” she said. “Once ML is introduced, that damage could multiply rapidly.”

Casey Ellis, CTO and founder of Bugcrowd, which operates a crowdsourced bug bounty platform, noted that attacks on ML models could be part of a larger trend of attacks on software supply chains. .

“Just as adversaries may attempt to compromise the supply chain of software applications to insert malicious code or vulnerabilities, they may also target the supply chain of machine learning models to insert data or malicious or biased algorithms,” Ellis told TechNewsWorld.

“It can have significant impacts on the reliability and integrity of AI systems and can be used to undermine trust in the technology,” he said.

Pablum for the Script Kiddies

Threat actors may show increased interest in machine models because they are more vulnerable than people thought.

“People knew it was possible for a while, but they didn’t realize how easy it was,” Bonner said. “It’s pretty trivial to chain an attack together with a few simple scripts.”

“Now that people realize how easy it is, it’s up to the script kiddies to succeed,” he added.

ADVERTISING

Clements agreed that the researchers showed that you don’t need deep ML/AI data science expertise to insert malicious commands into training data that can then be triggered by ML models at the time. of execution.

However, he continued, it requires more sophistication than ordinary ransomware attacks that rely primarily on simple credential stuffing or phishing to get started.

“Right now, I think the popularity of the specific attack vector will probably be low for the foreseeable future,” he said.

“To exploit this, an attacker must compromise an upstream ML model project used by downstream developers, trick the victim into downloading a pre-trained ML model with the malicious commands embedded from an unofficial source, or compromise the private dataset used by ML developers to insert exploits,” he explained.

“In each of these scenarios,” he continued, “it appears there would be much simpler and more direct ways to compromise the target in addition to inserting obfuscated exploits into the training data.”