Press "Enter" to skip to content

SolarWinds hackers have a whole bag of new tricks for mass compromise attacks

Getty Images

Almost exactly a year ago, security researchers uncovered one of the worst data breaches in modern history, if not ever: a Kremlin-backed hacking campaign that compromised the servers of network management provider SolarWinds and, from there, the networks of 100 of its highest-profile customers, including nine US federal agencies.

Nobelium—the name Microsoft gave to the intruders—was eventually expelled, but the group never gave up and arguably has only become more brazen and adept at hacking large numbers of targets in a single stroke. The latest reminder of the group’s proficiency comes from security firm Mandiant, which on Monday published research detailing Nobelium’s numerous feats—and a few mistakes—as it continued to breach the networks of some of its highest-value targets.

Abusing trust

One of the things that made Nobelium so formidable was the creativity of its TTPs, hacker lingo for tactics, techniques, and procedures. Rather than breaking into each target one by one, the group hacked into the network of SolarWinds and used the access, and the trust customers had in the company, to push a malicious update to roughly 18,000 of its customers.

Almost instantly, the hackers could intrude into the networks of all of those entities. It would be similar to a burglar breaking into a locksmith’s premises and obtaining a master-key that opened the doors of every building in the neighborhood, sparing the hassle of having to jimmy open each lock. Not only was Nobelium’s method scalable and efficient, it also made the mass compromises much easier to conceal.

Mandiant’s report shows that Nobelium’s ingenuity hasn’t wavered. Since last year, company researchers say the two hacking groups linked to the SolarWinds hack—one called UNC3004 and the other UNC2652—have continued to devise new ways to compromise large numbers of targets in an efficient manner.

Instead of poisoning the supply chain of SolarWinds, the groups compromised the networks of cloud solution providers and managed service providers, or CSPs, which are outsourced third-party companies that many large companies rely on for a wide range of IT services. The hackers then found clever ways to use those compromised providers to intrude upon their customers.

“This intrusion activity reflects a well-resourced threat actor set operating with a high level of concern for operational security,” Monday’s report said. “The abuse of a third party, in this case a CSP, can facilitate access to a wide scope of potential victims through a single compromise.”

Advanced tradecraft

The advanced tradecraft didn’t stop there. According to Mandiant, other advanced tactics and ingenuities included:

  • Use of credentials stolen by financially motivated hackers using malware such as Cryptbot, an information stealer that harvests system and web browser credentials and cryptocurrency wallets. The assistance from these hackers allowed the UNC3004 and UNC2652 to compromise targets even when they didn’t use a hacked service provider.
  • Once the hacker groups were inside a network, they compromised enterprise spam filters or other software with “application impersonation privileges,” which have the ability to access email or other types of data from any other account in the compromised network. Hacking this single account saved the hassle of having to break into each account individually.
  • The abuse of legitimate residential proxy services or geo-located cloud providers such as Azure to connect to end targets. When admins of the hacked companies reviewed access logs, they saw connections coming from local ISPs with good reputations or cloud providers that were in the same geography as the companies. This helped disguise the intrusions, since nation-sponsored hackers frequently use dedicated IP addresses that arouse suspicions.
  • Clever ways to bypass security restrictions, such as extracting virtual machines to determine internal routing configurations of the networks they wanted to hack.
  • Gaining access to an active directory stored in a target’s Azure account and using this all-powerful administration tool to steal cryptographic keys that would generate tokens that could bypass two-factor authentication protections. This technique gave the intruders what’s known as a Golden SAML, which is akin to a skeleton key that unlocks every service that uses the Security Assertion Markup Language, which is the protocol that makes single sign-on, 2FA, and other security mechanisms work.
  • Use of a custom downloader dubbed Ceeloader.
Just came in:  T-Mobile Offers The Best Mobile Internet Experience In The US: Report

In an interview, Doug Bienstock, one of the report authors and a Mandiant manager who has responded to multiple compromises by the hacking groups, said the SolarWinds-related hackers are among the most difficult he has ever encountered. He said:

What is unique and challenging is their toolbox seems to grow every month. Every time we are engaged by a system with a Nobelium-related breach we almost always find a new tactic, a new tool in their toolbox. And that I think is pretty unique, the speed at which this group can iterate, can add new tools, can find new ways to get around defenses. That’s pretty unique, the speed at which they do it and how adept they are.

The report in large part echoes previous research findings. Last December, security firm Volexity detailed how Nobelium, after gaining administrator privileges on a target’s network, was able to bypass multi-factor authentication by using those unfettered rights to steal a cryptographic key from a server running Outlook Web App, which enterprises use to provide account authentication for various network services.

And in October, Microsoft reported that Nobelium-linked hackers compromised cloud service providers in the US and Europe to “exploit existing technical trust relationships between the provider organizations and the governments, think tanks, and other companies they serve.”

One of Nobelium’s trademarks is its top-notch operational security, which makes it hard for defenders to detect attacks until it’s too late, and even then, leaves few traces for defenders to find afterward. Even so, the group, like all hackers, makes mistakes. One such error occurred when the hackers attempted to use binaries to upload files to the Mega cloud storage provider. The tool, deployed in the %TEMP%d folder as mt.exe and mtt.exe, failed to execute because of a bug that occurred when the binary was renamed.

Fascinating details

Monday’s report adds a few new TTPs and further details about other previously known TTPs. For instance the Mandiant researchers write:

Initial Compromise

Compromise of Cloud Solution Providers
Mandiant has identified multiple instances where the threat actor compromised service providers and used the privileged access and credentials belonging to these providers to compromise downstream customers. In at least one instance, the threat actor identified and compromised a local VPN account and made use of this VPN account to perform reconnaissance and gain further access to internal resources within the victim CSP’s environment, which ultimately led to the compromise of internal domain accounts.

Access obtained from info-stealer malware campaign:
Mandiant identified a campaign where the threat actors gained access to the target organization’s Microsoft 365 environment using a stolen session token. Mandiant analyzed the workstations belonging to the end user and discovered that some systems had been infected with CRYPTBOT, an info-stealer malware, shortly before the stolen session token was generated.

There are also some fascinating details about how the hackers use a compromised active directory inside a hacked CSP to gain a toehold in a customer’s network. The technique used RBAC, short for role based access control, which CSPs use to access the networks they serve.

In at least one case, the threat actor compromised a Microsoft Azure AD account within a Cloud Solution Provider (CSP) tenant. The account held a specific Azure AD role that allowed it to use the Admin on Behalf Of (AOBO) feature. With AOBO, users with a specific role in the CSP tenant have Azure RBAC Owner access to Azure subscriptions in their customer’s tenants that were created through the CSP program. RBAC Owner access gives the role holder complete control over all resources within the Azure subscription. The threat actor leveraged the compromised CSP’s credentials and the AOBO feature to gain privileged access to Azure subscriptions used to host and manage downstream customer systems. The actor executed commands with NT AUTHORITYSYSTEM privileges within Azure VMs using the Azure Run Command feature. The Azure Run Command feature allows a user to run PowerShell scripts within an Azure VM using the Azure Portal, REST API, or PowerShell without knowledge of Windows credentials that are valid on the VM itself.

Once inside the customer’s network, the hackers escalated their privileges by accessing internal memory storing cryptographic secrets used by the LSASS, short for Local Security Authority Subsystem Service, which verifies users logging in to Windows accounts, handles password changes, and creates access tokens used to access accounts once a password is entered. The Mandiant researchers wrote:

Mandiant found evidence that the threat actor used RDP to pivot between systems that had limited internet access. The threat actor accessed numerous devices using RDP and executed several native Windows commands. On one device, the threat actors made use of the Windows Task Manager to dump the process memory belonging to LSASS. The threat actor also obtained the Azure AD Connect configuration, the associated AD service account, and the key material used to encrypt the service account credentials. The Azure AD Connect account is used to replicate the on-premise instance of Active Directory into Azure AD. In addition to this, the threat actor obtained the Active Directory Federation Services (ADFS) signing certificate and key material. This allowed the threat actor to forge a SAML token which could be used to bypass 2FA and conditional access policies to access Microsoft 365. The actor stopped Sysmon and Splunk logging on these devices and cleared Windows Event Logs.

The Kremlin-backed groups also were adept at defeating network filtering in place between CSPs and their downstream customers. To identify systems in the CSP network that were authorized to access customer networks, the hackers exported a particular virtual machine that may have contained network configuration information. They then used a combination of PowerCLI — short for the PowerShell-based command-line interface—which people use to manage VMware vSphere devices and custom scripts to perform the export.

Just came in:  Recreational drones banned in United Arab Emirates after oil facility attack

Mandiant explained:

CSPs have network filtering layers in place between their on-premises environment and downstream customer environments as an added security layer. Mandiant identified that the threat actor used the vSphere PowerCLI and custom PowerShell scripts configured to target the vCenter Web endpoint to export the virtual disk image of a specific networking device and copy it off the service provider’s infrastructure. To authenticate to vCenter the threat actor used a stolen session cookie for a Privileged Access Management (PAM) account. Mandiant believes the threat actor was able to analyze this virtual machine and identify devices within the CSP’s network that were specifically allowed to communicate with targeted downstream customers.

Using this knowledge, the actor compromised the authorized source jump hosts that circumvented the network security restrictions of the service provider and downstream victim network. The actor compromised a customer administration account from one of the administration jump hosts used for customer administration within the CSP’s environment. The CSP would connect via these jump hosts using dedicated customer admin accounts to interact with a downstream customer’s infrastructure. The actor then performed lateral movement through RDP and the stolen target credentials towards the victim customer network.

In another case, the threat actor used Azure’s built-in Run Command feature to execute commands on numerous downstream devices. The threat actor used native Windows tools to perform initial reconnaissance, credential theft and deploy Cobalt Strike BEACON to devices via PowerShell.

The actor then used this BEACON implant to persistently install CEELOADER as a Scheduled Task that ran on login as SYSTEM on specific systems. CEELOADER is a downloader that decrypts a shellcode payload to execute in memory on the victim device.

These excerpts are only a sampling of the insights Mandiant has gleaned after responding to hacks by these groups. The report in its entirety should be required reading for anyone defending networks against advanced hackers.

Just came in:  Yakuza director announces new studio after leaving Sega