HomeCoinsCartesi (CTSI)Upgrading smart contract code and storage layout with EIP-2535 “Diamonds” | by...

Upgrading smart contract code and storage layout with EIP-2535 “Diamonds” | by Guilherme Dantas | Cartesi | Aug, 2022

- Advertisement -

Upgrading smart contract code and storage layout with EIP-2535 “Diamonds” | by Guilherme Dantas | Cartesi | Aug, 2022
Photo by Bas van den Eijkhof on Unsplash

Ethereum smart contract code is immutable: it cannot be changed after deployment. This goes hand-in-hand with the trustless nature of blockchain, but hinders smart contract development and maintenance. In addition to that, smart contract code size is limited to 24 KB.

Apart from code, each smart contract has its own internal storage, an array of 256-bit slots. Solidity, the most popular programming language for Ethereum smart contracts, assigns state variables to storage slots and takes care of the low-level details. However, there are many cases where the developer might want to change the storage layout after deployment, e.g. to add a variable or to optimize for storage read/write operations.

To mitigate these issues, EIP-2535 proposes a sophisticated proxy pattern, which allows the addition, replacement, and removal of functions and virtually removes any constraint on the code size; and an innovative storage layout, which is collision-resistant and very flexible.

In the following sections, we will go through the basics of EIP-2535, and how it enables code upgradability. Then, we will see how it also enables storage layout upgradability. At the end, I will present some caveats related to the task of upgrading dynamic fields and also propose mitigations.

Although smart contracts have immutable code, they can call functions from other smart contracts. In particular, they can also do so while preserving storage context via delegate calls. This feature is fundamental for enabling advanced proxy contracts.

The proxy contract proposed in EIP-2535, namely the Diamond, has very little code and is completely generic. Meanwhile, all the application-specific functions are implemented by contracts called Facets. Upon construction, a diamond exposes only two functions: a diamondCut function, which allows the diamond owner to add, replace and/or remove function implementations, and a fallbackfunction, which dynamically dispatches function calls to the appropriate facets seamlessly.

The EIP also specifies an interface called Diamond Loupe, which allows introspection on the functions that can be called via the fallbackfunction. This can be useful to developers when testing upgrades and to users for making the functionality of diamonds transparent. Moreover, upgradability is an optional feature, as the diamondCutfunction can be removed at any time.

EIP-2535 also proposes a storage layout called Diamond Storage, which groups state variables into structures. To avoid collisions, these structures are stored in locations given by the hashes of very descriptive strings (like mytoken.diamond.storage). This layout was only made possible after Solidity 0.6.4 allowed references to structures in arbitrary storage locations.

The diamondCutfunction also allows the diamond owner to specify an extra function call to be delegated after adding, replacing, and/or removing function implementations. According to the specification, “this execution is done to initialize data or setup or remove anything needed or no longer needed”.

Upgrading smart contract code and storage layout with EIP-2535 “Diamonds” | by Guilherme Dantas | Cartesi | Aug, 2022
Photo by wu yi on Unsplash

EIP-2535 allows functions to be upgraded in a pretty straightforward way, but little is said about upgrading storage layouts. To give an example of what we mean by such, suppose we deploy a contract that adopts the diamond pattern and utilizes the following structure.

struct DiamondStorage1 {
uint32 ts;
uint32 a;
uint64 b;
uint128 c;
mapping (address => uint32) m;

Note that ts, a, b and c are tightly packed in the same 256-bit storage slot. Let’s suppose ts stores the timestamp of the last interaction with the contract. We know that an unsigned 32-bit integer can only hold UNIX timestamps until around the year 2106. So, if we wanted to postpone this event for billions of years, we could represent ts with 64 bits, for example. Taking this into account, a backwards-compatible structure would look like the following.

struct DiamondStorage2 {
uint32 _ts; // deprecated
uint32 a;
uint64 b;
uint128 c;
mapping (address => uint32) m;
uint64 ts; // new field!

It’s important to note that we chose to move ts to the end of the structure instead of expanding it in place, because this could alter the storage location of some fields that follow _ts. In particular, if the mapping m were to be shifted to some other storage location, so would every entry m[k]. To initialize the new field ts, we need to first deploy a contract with the following function. Assume diamondStorage returns a pointer to the structure in the correct storage location.

function upgrade() external {
DiamondStorage2 storage ds2 = diamondStorage();
ds2.ts = uint64(ds2._ts); // expand `ts` in a new field

We also need to replace the functions that reference the deprecated field (uint32 _ts) with newer versions that reference the new one (uint64 ts). However, if there are any functions that expose a parameter or return value of type uint32 because of _ts, we can choose to keep backwards compatibility with contracts that call them.

Because we can always append new fields to the end of structures and replace function implementations that reference them, it is easy to see that adding and deprecating fields, regardless of type, are operations as straightforward as adding, replacing, and/or removing function implementations.

However, changing the type of field is not always so easy. We were able to change the type of ts because integers occupy a constant number of storage slots and thus can be copied and expanded with a constant amount of gas. However, the same cannot be said for dynamic types like arrays and mappings, as they can occupy an arbitrary number of storage slots, and iterating over them might consume more than the block gas limit, which might make such upgrades impractical to be performed in one transaction.

The following list helps organize some types of upgrades that are possible with EIP-2535, being mindful of the scalability limitations of Ethereum.

  • Adding functions
    Specify the function selectors to add and the address of the facet.
  • Removing functions
    Specify the function selectors to remove.
  • Replacing facet address of functions
    Specify the function selectors and the address of the new facet.
  • Adding fields
    Append the fields to the end of some existing structure, or create a new structure with them.
  • Removing fields
    Deprecate the fields. If there are no dynamic fields in the structure or if the dynamic fields are sufficiently small, you can also create a new structure without such fields and copy over the remaining fields.
  • Changing the type of static fields
    (e.g. from 32 to 64-bit integers)

    Deprecate the old field, append the new one to the end of the structure, and cast the contents of the old field in the new one.
  • Changing the type of array elements
    (e.g. from 32 to 64-bit integer arrays)

    If the number of array elements is sufficiently small, deprecate the old field, append the new one to the end of the structure, and copy the array to the new location element-by-element.
  • Changing the type of mapping keys and/or values
    (e.g. from integers to structures)

    If the number of mapping entries is sufficiently small, deprecate the old field, append the new one at the end of the structure, and copy the mapping to the new location entry-by-entry.
Upgrading smart contract code and storage layout with EIP-2535 “Diamonds” | by Guilherme Dantas | Cartesi | Aug, 2022
Photo by Dawn McDonald on Unsplash

Operations on dynamic fields are not always possible to be performed in a single transaction because of the block gas limit. To put this into perspective, an over-optimistic upper bound for the number of storage writes before exceeding the block gas limit is 1500. A possible mitigation would be to split the task into several independent transactions, each not exceeding the block gas limit.

If the dynamic field is not changed by any function, the upgrade is straightforward. For example, let’s say we want to change the mapping field m from the previous example. We can deprecate it and append a new field to the end of the structure, like in the following code snippet:

struct DiamondStorage3 {
uint32 _ts;
uint32 a;
uint64 b;
uint128 c;
mapping (address => uint32) _m; // deprecated
uint64 ts;
mapping (address => uint256) m; // new field!

And then, we can convert the mapping entries little-by-little, without affecting the behavior of the contract, which only takes the old fields into account. To be able to split the upgrade into several transactions, we partition the set of keys for which the mapping has a valid entry into several disjoint subsets. To give some motivation to this upgrade, let’s say we want to multiply each value by some constant c.

function upgrade(address[] calldata keys, uint256 c) external {
DiamondStorage3 storage ds3 = diamondStorage();
for (uint i; i < keys.length; ++i) {
address key = keys[i];
ds3.m[key] = c * uint256(ds3._m[key]);

After initializing the new mapping field m with all the valid keys, we can replace the functions that reference the old field with newer versions that reference the new one.

If, on the other hand, the dynamic field to be updated can be changed by users, we want to avoid read-after-write concurrency. A cheap and dirty solution would be to remove the functions that reference the mapping, make all the necessary changes, and then add the newer versions of the functions. However, this would break contracts that depend on the diamond during the upgrade, and could even be used by a malicious diamond owner as a DoS attack against such contracts.

EIP-2535 enables the addition, removal, and replacement of function implementations, as well as arbitrary code to be executed on the storage context of the diamond after every update. This allows code and storage layout to be upgraded indefinitely and in a flexible way.

There are, however, some caveats that need to be considered when upgrading dynamic fields, since their size can be arbitrarily large, and traversing them might surpass the block gas limit. One possible mitigation is to split the task of upgrading a dynamic field into several transactions, each consuming less than the limit. This technique works perfectly for fields that cannot be changed by users. However, in the opposite case, the diamond owner is forced to prohibit users from changing such fields during upgrades, which might temporarily break contracts and applications that depend on the diamond.

Source link

- Advertisement -
Mr Bitcointe
Mr Bitcointehttps://www.bitcointe.com/
“Fact You Need To Know About Cryptocurrency - The first Bitcoin purchase was for pizza.” ― Mohsin Jameel

Most Popular

Bitcoin (BTC) $ 23,919.41
Ethereum (ETH) $ 1,774.31
Tether (USDT) $ 1.00
Bitcoin Cash (BCH) $ 144.51
Litecoin (LTC) $ 62.82
EOS (EOS) $ 1.25
OKB (OKB) $ 18.49
Tezos (XTZ) $ 1.89
LEO Token (LEO) $ 4.85
Cardano (ADA) $ 0.533488
Monero (XMR) $ 166.36
Stellar (XLM) $ 0.133986
Chainlink (LINK) $ 8.55
Huobi (HT) $ 4.42
TRON (TRX) $ 0.070399
USD Coin (USDC) $ 1.00
Dash (DASH) $ 54.40
NEO (NEO) $ 11.78
IOTA (MIOTA) $ 0.351566
NEM (XEM) $ 0.052917
Zcash (ZEC) $ 75.37
Maker (MKR) $ 1,148.18
Pax Dollar (USDP) $ 1.00
Ethereum Classic (ETC) $ 38.05
VeChain (VET) $ 0.031754
TrueUSD (TUSD) $ 1.00
FTX (FTT) $ 31.64
KuCoin (KCS) $ 10.77
Waves (WAVES) $ 6.23