Imagine if every user of a decentralized exchange, from individual users to major institutions, could independently verify transactions with the same level of certainty that comes from running a full node. A peer-to-peer decentralized network would form among users, and they wouldn’t need to trust any individual or entity to verify the network.
Unfortunately, with today’s prominent blockchain architectures, it would require everyone to actually run a full node to achieve this. But this doesn’t scale, due to:
- High hardware requirements: Including significant RAM and storage space.
- Technical Complexity: Setup, maintenance and management of a full node requires significant technical expertise to do effectively.
- Resource Consumption: Paying to run a full node incurs additional costs that regular network users cannot justify paying.
- Time Consuming: Most users care about what the app can do for them and have no interest in setting up and maintaining full node infrastructure.
This introduces centralizing forces to networks, and the participants that end up running full nodes typically fall into one of two groups.
- Those with a financial incentive e.g. miners, validators, and network participants that gain a performance benefit from running full nodes, like exchanges.
- Those who are simply experimenting or running them for altruistic reasons e.g. developers.
Irrespective of the reasons, the result is that less than 1% of network participants are able to independently verify the network. Everyone else places different levels of trust into entities within the network, and these entities acquire more and more power over time. This is the reality facing many decentralized networks today, and runs counter to the goal of achieving peer-to-peer, decentralized networks at global scale.
But what if instead of asking ourselves, how can we get everyone to run a full node, we instead ask, what are the assurances that one receives from running a full node, and how can we make it easy for everyone to generate these assurances themselves?
As it turns out, there are three core things that each participant in a decentralized network must check to verify something onchain.
- Execution Correctness: We need to know if all the state transitions were done correctly. A full node does this by re-executing every transaction to ensure it was executed as per the agreed State Transition Function (STF).
- Correct Transaction Order: State transitions might be executed correctly, but full nodes must also verify they were executed in the correct order. A full node can verify the consensus of the protocol to ensure the order of the blockchain is the correct order.
- Data Availability: A blockchain is only decentralized if others can participate. For others to participate the data required to arrive at the latest state, needs to be publicly available. If a full-node can re-execute the STF to arrive at the current state, then it can be certain that the transaction data is available as a result.
As we have seen, running a full node doesn’t scale. Instead, we seek to verify the 3 elements above with the guarantees of a full node, in a way that’s secure, scalable and easy for anyone to run.
The Avail Light Client: A Building Block For Efficient Decentralized Networks
When deploying any blockchain on the Avail Network, the default light client comes included and can be used immediately. The Avail Network is optimized to provide data availability for blockchains. By default, the light client can verify data availability, and the order of finalized blocks on the Avail Network. This provides the following assurances to Avail powered chains.
Data Availability: The light client independently verifies transaction data is available and in its original form, without relying on a full node. This is done through Data Availability Sampling (DAS). Instead of downloading the complete transaction data like a full node, data is secured by the Avail network using KZG commitments, making it mathematically secure. This enables light clients to check random samples of data and verify they match the commitments in block headers. Doing this a number of times for random cells enables the light client to calculate the probability that all data is available. Using this technique, Avail’s light client achieves the guarantees of a full node from taking just 8-30 samples.
Block Ordering: The light client verifies the correct order of blocks on the Avail Network by checking validator consensus.
This alone however is not yet sufficient to provide full node guarantees. Only when the Light Client independently verifies all three elements (data availability, ordering and execution correctness) can it provide assurances comparable to running a traditional full node. While checking execution proofs is not currently part of the standard light client deployment, it will be possible for Avail chains soon. If you want to start using this feature sooner however, you can customize the light client now, and adapt it to verify the State Transition Function (STF) of your chosen execution environment.
What Can You Do With The Avail Light Client?
Remember our earlier example of a decentralized exchange which anyone could independently verify? Some of the major blockers holding this back were the high hardware requirements and technical expertise required to set up and maintain a full node. Let’s explore how this part changes with the Avail light client.
The minimum requirements for running the Avail light client are:
- RAM: 512MB
- CPU: 2 cores (amd64/x86 architecture)
The recommended specification is:
- RAM: 1GB
- CPU: 4 cores (amd64/x86 architecture)
With these hardware requirements it becomes possible for anyone to run the light client with hardware they already own like phones, laptops and even smartwatches. The additional computation required to verify execution proofs is also relatively low. This makes it conceivable that every single participant using a decentralized exchange could verify the network with full node guarantees.
The light client is a flexible building block that can be customized to fit the needs of any particular network or application. In the same way the ERC-20 token can be adapted, extended and branded, developers can freely adapt and deploy Light Clients in ways that make sense for their applications and users.
Light Clients In Action
Sophon, a consumer focused blockchain on Avail, customized the light client to play a key role in enabling non-technical participants from their node sale to take part in securing Sophon. They raised $60 million in project funding by selling 123,554 node sale licenses. Using the light client, Sophon was able to provide their community with an efficient piece of network infrastructure that could be run on almost any device, by anyone. License holders, whether they are technical or not, can run the node and get rewarded in SOPH tokens for the uptime and performance. This not only has helped decentralize the Sophon network, but it also has encouraged participation from a much broader community base and helped contribute valuable work for Sophon’s ecosystem.
The light client has already been adapted to run on phones, in the browser, on a smartwatch and on a RaspberryPi.
Avail Light Client Comparison
We compared the recommended requirements for running an Avail Light Client vs an Ethereum Full Node and Ethereum Light Client, so readers can see how the Avail light client stacks up compared with more familiar infrastructure.
What Makes The Avail Light Client Unique?
Avail's light client enables cryptographically secure, trustless data verification directly on user devices. This eliminates reliance on centralized infrastructure while maintaining mathematically secured data integrity and unprecedented performance. Chains built on Avail can provide independently verified data availability guarantees to end users faster than any other network.
While some data availability solutions use KZG commitments, they do not offer sampling through light clients, which means network participants have no way of independently verifying the data is available. Other solutions provide light clients with Data Availability Sampling, but use Name Spaced Merkle Trees which rely on Fraud Proofs, introducing wait times of 10 minutes or more. The Avail Network is unique in the way it combines Validity Proofs, based on KZG commitments, with Data Availability Sampling, as this combination allows Light Clients to independently verify data availability in around 40 secs. This is ~15X faster than the next closest alternative.
The benefits of using Validity Proofs for light clients include:
- Proactive Verification: Light clients cryptographically verify that a commitment is correct before accepting it.
- Efficient: As the number of transactions increases, the work on the light client stays the same.
- Trustless: The light clients can verify cryptographic proofs and don’t need to assume an honest challenger will detect and report errors.
- Finality Time: Light clients can sample data from the network as soon as the data is finalized (around 40 seconds), without needing to wait for a challenge period.
- Security Guarantees: Validity proofs are mathematically correct, reducing the attack surface.
The Avail light client lets you interact with Avail DA without requiring a full node and without trust assumptions towards remote peers.
The light client also has a hidden trick up its sleeve, and that’s the ability for it to communicate with other light clients in a peer-to-peer network.
Avail Light Client - Peer-To-Peer Network
Each light client shares information it learns about the Avail Network with others, via the light client p2p network. This forms a self-organizing system of light clients checking, verifying and re-creating the Avail blockchain as a distributed hash table. While each light client only has access to part of the blockchain’s data, when they come together, they can recreate a complete picture of the entire network.
An individual light client can sample from both the p2p network, and from the server based network. Avail’s light client is the first and only solution that can verify data availability by sampling from the P2P network.
As more light clients join the network, there’s more verification taking place, creating a self-reinforcing loop. With more verification, it gets increasingly harder to hide data or manipulate the network.
Avail’s P2P Implementation - Kademlia DHT (Kad-DHT)
Kad-DHT is a specific Distributed Hash Table (DHT) variant that organizes nodes and data based on a chord ring—a logical arrangement of nodes ordered by their IDs. Avail employs Kad-DHT to establish a decentralized network for data storage and retrieval. In this structure, each node is tasked with holding a portion of the data. Nodes can directly communicate to access data. Avail utilizes Kad-DHT to store data cells and pinpoint which peer possesses a particular data segment, with matrix data cells uniquely mapped to Peer IDs.
Avail Light Client - Transaction Lifecycle
Remember that light clients are designed to independently check and verify the network so that it can scale efficiently without the need for expensive infrastructure or technical expertise. To understand the Light Client transaction lifecycle, we’ll quickly start by elaborating on exactly what validator nodes do first, as they are the ones that propose and add new blocks to the blockchain. Then we will go into the details of how the light client checks, and verifies this.
- Block producer creates the block body.
- Accumulates transactions (data submissions).
- Orders these transactions into the Avail data matrix (more on this below), which becomes the block body.
- Erasure codes the data matrix for extra redundancy.
- Block producer creates the block header.
- Generates commitments for each row of the matrix.
- Extends those commitments using polynomial interpolation (the generated + extended commitments become the block header).
- Block producer propagates the block (the body + the header).
- Validators and full nodes receive the block.
- Validators and full nodes decode, reconstruct, and verify the block.
- Recreate the data matrix.
- Recreate the commitments.
- Extend the commitments.
- Verify all the data they received matches the commitments they generated.
The Data Matrix is a crucial component of the Avail Network. It’s what validators produce when creating blocks and what light clients check to verify data is available. The data matrix consists of rows and columns of submitted data, with some information about the data.
Avail Data Matrix
Detail
The block producer breaks the block data into chunks and arranges the chunks into n rows and m columns such that it forms a n × m matrix D. It uses the evaluation form to construct a polynomial from each row to obtain φ1(x), . . . , φn(x). It then commits each polynomial to get, C1, . . . , Cn respectively.
For redundancy, it extends C1,...,Cn to C1’,...,Cn’. It puts C1,...,Cn’ inside the block header and broadcasts it. Further reading on KZG Polynomial commitments can be found here.
When creating the data matrix, commitments are generated by block proposers that summarize each row of the data. These commitments are added to the block header, which light clients have access to. Validity proofs are also generated for every individual cell in the data matrix. Light clients can then ask the network for a specific cell’s value, and that cell’s associated validity proof. The light client can check with mathematical certainty that this cell’s value matches the commitment in the block’s header. By doing this a number of times to random cells, the light client increases its confidence that all data is available. The Avail light client can generate full node guarantees with only 8-30 samples.
Light clients also host sampled data for other light clients within the P2P network. The light clients within the P2P network maintain a replica of the Avail blockchain, holding state and the last 256 blocks, the same as Avail Full (non-archival) nodes.
Below, we can see a complete overview of the transaction lifecycle. It starts with data submitted to the Avail network, and Avail validators creating and finalizing blocks. Light clients then sample from the network, and gossip with other light clients to form a robust P2P network.
This architecture guarantees very high availability of data for any blockchain network that can be quickly, efficiently and securely sampled by light clients from almost any device. Light clients can also fetch specific block data and publish data to the Avail network using the Light Client API.
Avail Validator Decentralization Guarantees
Now that we have established how the light clients generate data availability guarantees through validity proofs and data availability sampling, we should also inspect the upstream decentralization guarantees of Avail’s validators.
The Avail Network uses the Nominated Proof of Stake (NPoS) mechanism and Phragmen election algorithm, which is designed to improve validator decentralization outcomes and make it harder for a handful of validators to control the network. NPoS was specifically chosen for the Avail Network because it avoids many of the centralization tendencies that surface on Delegated Proof of Stake (DPoS) based blockchains over time. The Avail Network started with 45-50 validators, it has since doubled (at the time of writing) and is growing to support up to 1,000 active validators. A snapshot of the stake distribution among Avail validators is shown below.
With stake spread evenly across validators, it is much harder to control the network. By comparison, many DPoS based blockchains have fewer than 10 validators controlling at least 33% of the voting power, and less than 15 validators controlling over 50% of the network.
Get Started
We began with an ambitious goal of providing every single user of a decentralized exchange with the same level of certainty that comes from running a full node. Now, with the Avail light client, developers have a path for achieving this at an unprecedented scale. This is not just applicable to a decentralized exchange, but any decentralized use case you can imagine.
It's now possible for every user to be a blockchain verifier. The Avail light client is in production already and can enable fast, trustless data verification at scale. View developer docs.
Pass The Block Part 1.2: Tutorial - How to Set Up & Run A Light Client
If you’d like to speak to the team directly, feel free to reach out: business@availproject.org.
