168 lines
8.7 KiB
Markdown
Executable File
168 lines
8.7 KiB
Markdown
Executable File
# NeoFS block storage
|
||
|
||
Using NeoFS to store chain's blocks and snapshots was proposed in
|
||
[#3463](https://github.com/neo-project/neo/issues/3463). Tutus contains several
|
||
extensions utilizing NeoFS block storage aimed to improve node synchronization
|
||
efficiency and reduce node storage size.
|
||
|
||
## Components and functionality
|
||
|
||
### Block storage schema
|
||
|
||
A single NeoFS container is used to store blocks and index files. Each container
|
||
has network magic attribute (`Magic:56753`). Each block is stored in a binary
|
||
form as a separate object with a unique OID and a set of attributes:
|
||
- block object identifier with block index value (`Block:1`)
|
||
- primary node index (`Primary:0`)
|
||
- block hash in the LE form (`Hash:5412a781caf278c0736556c0e544c7cfdbb6e3c62ae221ef53646be89364566b`)
|
||
- previous block hash in the LE form (`PrevHash:3654a054d82a8178c7dfacecc2c57282e23468a42ee407f14506368afe22d929`)
|
||
- millisecond-precision block creation timestamp (`BlockTime:1627894840919`)
|
||
- second-precision block uploading timestamp (`Timestamp:1627894840`)
|
||
|
||
Each index file is an object containing a constant-sized batch of raw block object
|
||
IDs in binary form ordered by block index. Each index file is marked with the
|
||
following attributes:
|
||
- index file identifier with consecutive file index value (`Index:0`)
|
||
- the number of OIDs included into index file (`IndexSize:128000`)
|
||
- second-precision index file uploading timestamp (`Timestamp:1627894840`)
|
||
|
||
### Contract state storage schema
|
||
|
||
A single NeoFS container is used to store contract state objects. Each container
|
||
has network magic attribute (`Magic:56753`). Each contract state object is stored
|
||
in a binary form as a separate object with a unique OID and a set of attributes:
|
||
- state object identifier where the state index number corresponds to the block
|
||
height, starting from 0 (`State:0`)
|
||
- contract state object interval of uploading (`StateSyncInterval:5000`)
|
||
- state root hash in the LE form (`StateRoot:58a5157b7e99eeabf631291f1747ec8eb12ab89461cda888492b17a301de81e8`)
|
||
- millisecond-precision block creation timestamp (`BlockTime:1468595301000`)
|
||
- second-precision contract state object uploading timestamp (`Timestamp:1742957073`)
|
||
|
||
The binary form of the contract state object includes serialized sequence of
|
||
contract storage item key-value pairs for every deployed contract including
|
||
native contracts excluding native LedgerContract contract state. The binary
|
||
form of the state object has the following format:
|
||
|
||
```
|
||
1 byte: Version of the contract state object (uint32)
|
||
4 bytes: Network magic (uint32)
|
||
4 bytes: Block height (uint32)
|
||
32 bytes: State root (Uint256)
|
||
Variable-length sequence of contract storage key-value pairs, each encoded as:
|
||
Variable-length key (varbytes)
|
||
Variable-length value (varbytes)
|
||
```
|
||
|
||
### NeoFS BlockFetcher
|
||
|
||
NeoFS BlockFetcher service is designed as an alternative to P2P synchronisation
|
||
protocol. It allows to download blocks from a trusted container in the NeoFS network
|
||
and persist them to database using standard verification flow. NeoFS BlockFetcher
|
||
service primarily used during the node's bootstrap, providing a fast alternative to
|
||
P2P blocks synchronisation.
|
||
|
||
NeoFS BlockFetcher service search and fetch blocks directly from NeoFS container via
|
||
built-in NeoFS object search mechanism.
|
||
|
||
#### Operation flow
|
||
|
||
1. **OID Fetching**:
|
||
Searches blocks one by one directly by block attribute.
|
||
Once the OIDs are retrieved, they are immediately redirected to the
|
||
block downloading routines for further processing. The channel that
|
||
is used to redirect block OIDs to downloading routines is buffered
|
||
to provide smooth OIDs delivery without delays. The size of this channel
|
||
can be configured via `OIDBatchSize` parameter and equals to `2*OIDBatchSize`.
|
||
2. **Parallel Block Downloading**:
|
||
The number of downloading routines can be configured via
|
||
`DownloaderWorkersCount` parameter. It's up to the user to find the
|
||
balance between the downloading speed and blocks persist speed for every
|
||
node that uses NeoFS BlockFetcher. Downloaded blocks are placed to the
|
||
block queue directly.
|
||
3. **Block Insertion**:
|
||
Downloaded blocks are inserted into the blockchain using the same logic
|
||
as in the P2P synchronisation protocol. The block queue is used to order
|
||
downloaded blocks before they are inserted into the blockchain. The
|
||
size of the queue can be configured via the `BQueueSize` parameter
|
||
and should be larger than the `OIDBatchSize` parameter to avoid blocking
|
||
the downloading routines.
|
||
|
||
Once all blocks available in the NeoFS container are processed, the service
|
||
shuts down automatically.
|
||
|
||
### NeoFS block uploading command
|
||
The `util upload-bin` command is designed to fetch blocks from the RPC node and upload
|
||
them to the NeoFS container. The batch size should be consistent from run to run,
|
||
otherwise gaps in the uploaded blocks may occur. Below is an example usage of the command:
|
||
|
||
```shell
|
||
./bin/tutus util upload-bin --cid 9iVfUg8aDHKjPC4LhQXEkVUM4HDkR7UCXYLs8NQwYfSG --wallet-config ./wallet-config.yml --block-attribute Block --rpc-endpoint https://rpc.t5.n3.nspcc.ru:20331 -fsr st1.t5.fs.neo.org:8080 -fsr st2.t5.fs.neo.org:8080 -fsr st3.t5.fs.neo.org:8080
|
||
```
|
||
|
||
Run `./bin/tutus util upload-bin --help` to see the full list of supported options.
|
||
|
||
This command works as follows:
|
||
1. Fetches the current block height from the RPC node.
|
||
2. Searches in batches for the blocks stored in NeoFS.
|
||
3. In batches fetches missing blocks from the RPC node and uploads them to the NeoFS container.
|
||
|
||
If the command is interrupted, it can be resumed. It starts the uploading process
|
||
from the last fully uploaded batch.
|
||
|
||
For a given block sequence, it is strictly prohibited to change the batch size.
|
||
The batch size can be changed only if uploading has successfully completed up to
|
||
the current chain block height, otherwise it can lead to gaps in the sequence.
|
||
|
||
### NeoFS StateFetcher
|
||
|
||
NeoFS StateFetcher service enables contract state synchronization for non-archival nodes
|
||
by fetching contract storage key-value pairs from a trusted NeoFS container.
|
||
It serves as an alternative to P2P-based MPT state synchronization ([P2PStateExchngeExtensions](https://github.com/tutus-one/tutus-chain/blob/f7080f28d7088517de1d624dfdaf247f914486d2/docs/node-configuration.md?plain=1#L421)).
|
||
It allows to download contract state objects from a trusted container in the NeoFS
|
||
network and persist them to database.
|
||
NeoFS StateFetcher service primarily used during the node's bootstrap, providing a
|
||
fast alternative to P2P synchronization.
|
||
|
||
#### Operation flow
|
||
|
||
1. **State Object Search**:
|
||
Searches the NeoFS container for objects with the configured `StateAttribute`
|
||
(e.g., `State`), filtering by block height aligned with `StateSyncInterval`
|
||
(e.g., heights 0, 40000, 80000).
|
||
2. **Storage Data Fetching**:
|
||
Fetches contract storage key-value pairs from the identified state object.
|
||
3. **State Synchronization**:
|
||
Passes key-value pairs to the `statesync.Module`, along with the synchronization
|
||
height and expected state root. The `statesync.Module` builds a local MPT trie,
|
||
flushing batches of up to `KeyValueBatchSize` pairs to storage. If the trie’s
|
||
state root matches the expected root at height P, synchronization process
|
||
considered to be completed.
|
||
4. **Atomic State Jump**:
|
||
If the state root matches the expected root, the `statesync.Module`
|
||
performs an atomic state jump. The local MPT trie built during
|
||
synchronization becomes the working trie for normal chain processing.
|
||
|
||
Once state object fetched from the NeoFS container is processed, the service
|
||
shuts down automatically.
|
||
|
||
### NeoFS state uploading command
|
||
The `util upload-state` command is used to start a node, traverse the MPT over the
|
||
smart contract storage, and upload MPT nodes to a NeoFS container at every
|
||
`StateSyncInterval` number of blocks. Below is an example usage of the command:
|
||
|
||
```shell
|
||
./bin/tutus util upload-state --cid 9iVfUg8aDHKjPC4LhQXEkVUM4HDkR7UCXYLs8NQwYfSG --wallet-config ./wallet-config.yml --state-attribute State -m -fsr st1.t5.fs.neo.org:8080 -fsr st2.t5.fs.neo.org:8080 -fsr st3.t5.fs.neo.org:8080
|
||
```
|
||
|
||
Run `./bin/tutus util upload-state --help` to see the full list of supported options.
|
||
|
||
This command works as follows:
|
||
1. Searches for the state objects stored in NeoFS to find the latest uploaded object.
|
||
2. Checks if new state objects could be uploaded given the current local state height.
|
||
3. Traverses the MPT nodes (pre-order) starting from the stateroot at the height of the
|
||
latest uploaded state object down to its children.
|
||
4. Uploads the MPT nodes to the NeoFS container.
|
||
5. Repeats steps 3-4 with a step equal to the `StateSyncInterval` number of blocks.
|
||
|
||
If the command is interrupted, it can be resumed. It starts the uploading process
|
||
from the last uploaded state object. |