Nine Chainlink node operators — which play an essential role in powering decentralized finance (DeFi) protocols by providing token price feeds — suffered an attack on August 30 that drained about 700 ETH (worth about $335,000 at the time) from their “hot” wallets, The Block has learned.
One source with knowledge of the matter told The Block that an attacker began sending valid price feed requests, which resulted in operators having to pay a lot of gas fees, or Ethereum transaction fees.
Chainlink node operator Certus One also provided details on the attack. CEO Hendrik Hofstadt told The Block that the attack affected at least nine Chainlink node operators. Hofstadt said Certus One lost 20 ETH, or about $7,500 at current prices.
Chainlink confirmed the attack when reached, but did not comment on how much ETH was drained and how many node operators were affected.
There was a “brief spam attempt on Sunday” that lasted for “approximately two hours,” a Chainlink spokesperson told The Block. “While this spam attempt did require Chainlink nodes to spend additional ETH, this need was quickly removed when the network properly addressed the spam.”
The spokesperson went on to say that spam requests often require networks to initially spend “slightly more” resources until requests are identified as spam. The spokesperson termed the attack as a “failed attempt” to spam the Chainlink network, which had “no meaningful impact on the network or its feeds.”
“The failure of this attempt at spamming the Chainlink network is a testament to how resilient the Chainlink network has grown to become,” the spokesperson said. Still, the attack perhaps illustrates the risks associated with being a node operator and the lengths to which malicious actors in the digital asset ecosystem will go to generate profits at the expense of honest actors.
Oracles like Chainlink feed data on to blockchain smart contracts via node operators. This is because smart contracts, by design, cannot communicate with external systems. Node operators get rewarded for their work. Chainlink, for instance, pays them in its native token LINK.
The malicious actor could successfully perform the attack by using operators’ spare gas to mint Chi gas tokens, which are created by decentralized exchange aggregator 1inch.Exchange.
Chi is a tokenized form of gas and helps hedge against volatile gas fees. It is the “most liquid” gas token, Hofstadt told The Block, adding that the gas taken has “quite some liquidity on Mooniswap [1inch’s automated market maker protocol], so I was assuming that is why they [the attacker] picked it.”
Put simply, once the attacker sends a price feed request, the node responds — thus making the operator in question spend “a lot of gas and [the attacker] eventually managed to capitalize on some of that gas,” Hofstadt told The Block.
Gas prices on Ethereum are denominated in gwei. The attacker exploited the high fees on the network by driving up the gas costs of these oracles and then minting Chi tokens at these elevated levels. These tokens would normally be used to defray high gas costs, but in this instance, the attacker sold them for ETH.
Finally, the attacker used Tornado Cash, an Ethereum-based transaction mixer, to obscure the path of the ETH transactions, according to a source.
The nine affected Chainlink node operators were: T-Systems (a subsidiary of Deutsche Telekom), 01Node, Anyblock Analytics, B-Harvest, ChainLayer, Chainlink itself, Everstake, Figment Networks and LinkPool, Hofstadt told The Block. The Block reached out to Figment and Chainlayer but did not receive a response by press time.
The other affected operators could not immediately be reached. “We assumed they took an alphabetical list of nodes,” said Hofstadt. Also, some of these operators had larger balances of around 50 ETH in their wallets, Hofstadt added. So, these were just “more lucrative targets to attack.”
Smaller node operators have only around 2-3 ETH in their wallets, said Hofstadt, adding that their balances largely depend on their internal policies.
As the nine node operators had drained ETH balances, they could not fulfill data requests during the attack period, which lasted for about two hours.
When a node is drained, it is not able to pay for transactions anymore, and therefore no longer able to respond to requests or serve data on-chain, Hofstadt told The Block.
Until Hofstadt and his team realized that this was the attack, most of the affected node operators continued refilling their nodes “because they thought it was due to the gas spike that night” and the refilled ETH was also getting drained, said Hofstadt.
“My team actually noticed the strange pattern of the gas token being minted and so we immediately reported that to the Chainlink team and they patched their security team,” said Hofstadt. “And within like half an hour to an hour, we had to come to the conclusion that we need to look at this whitelist and that this is the best solution.”
In the whitelist solution, node operators rank the most valuable data requests, coming from the most active DeFi protocols — such as Aave and Synthetix, for example — and fulfill only their requests while blocking all other, non-whitelisted, requesters.
A temporary solution
It is, however, worth noting that whitelisting is the temporary solution. For a permanent solution, Chainlink would need to find “common ground with actual data consumers” before node operators can change the system, Hofstadt told The Block.
Overall, Sunday’s attack did not have any “any real impact” on the Chainlink network, according to Hofstadt, because the unaffected node operators could continue feeding data. For example, the ETH/USD official feed has about 25 or 28 data providers. So when the nine nodes were down, the rest were “still sufficient to update that feed,” said Hofstadt.
Yet during the attack, some data feeds might have been more biased towards the API (e.g. CoinGecko) that these providers were using, according to Hofstadt. Also, if the attack had impacted 50% of Chainlink node operators, that feed wouldn’t have updated until enough came back up, said Hofstadt.
The Chainlink spokesperson also said the attack did not impact its network as it “behaved exactly as it was supposed to in this instance, with node operators paying whatever fees necessary to continue powering Chainlink’s feeds that secure $3 billion in value.”
“Our economic incentive model for operators functioned just as it was supposed to,” the spokesperson added.
Hofstadt told The Block that Chainlink was helping some official node operators that didn’t have enough ETH in their hot wallets. For example, nodes that lost funds in the 10-20 ETH range received assistance in order to refill their nodes, given that they were drained after auto-refill scripts kicked in. Others were offered grants as a “thank you for the quick reaction that night and to help reimburse the damages,” according to Hofstadt, who wasn’t sure whether every node operator has had their damages reimbursed.
The Chainlink spokesperson did not comment on reimbursements.
The higher gas fees appear to be trouble for Chainlink node operators. Hofstadt said Certus One has had days where it experienced “multiple thousands of dollars in losses.”
But he reassured that consumers are “absolutely not impacted.”
“This is an interesting design choice. This whole system, what we as oracles are actually doing is we have smart contracts that are called aggregators and these are unchanged and we update them and the operator of this aggregator pays us. In this case, this is mostly Chainlink itself or consumers of that data, and they essentially pay us for the data and their dAPP can then access that data on-chain for free or actually everyone can access the data on these feeds for free,” explained Hofstadt.
Overall, Hofstadt thinks that the exploit wouldn’t have been “lucrative” for the attacker under normal circumstances, since gas tokens are “not 100% efficient.”
“The attacker would have actually been loss-making until enough requests would have led to a flood of responses from the oracles, which in turn would have pushed the gas prices up to a point where it could be lucrative — similarly to a DDoS [denial-of-service] attack. However, this would have been quite expensive to perform and wouldn’t have yielded any significant profits”, said Hofstadt.
He acknowledged that the elevated gas prices were the “main trigger” for the attack.
© 2020 The Block Crypto, Inc. All Rights Reserved. This article is provided for informational purposes only. It is not offered or intended to be used as legal, tax, investment, financial, or other advice.