Horizontal Scaling: 10GB Blocks With Light Clients & Multi-Proofs

In our first post of this series, we discuss how Avail can achieve 10GB blocks and a 600ms block time without centralized infrastructure. Importantly, this is not achieved just by increasing block size alone, but through adapting Avail’s architecture to reach a critical inflection point where scaling bottlenecks are removed, almost entirely. The design leverages the unique characteristics that come with combining cryptographic proofs and Data Availability Sampling (DAS), something which is unique to Avail.

The key phases to achieve this roadmap are:

• Optimize Commitment Generation: Blobs will contain both the data blob and the commitment for that blob.
• Delegate Compute: DA transactions will then become a commitment only, and the mempools receive, verify and gossip commitments only, using DAS to verify DA.
• SNARKify Non-DA Activity: SNARKify all non-DA activity (balance transfers, staking etc.) so that LCs can independently verify them without reliance on full-nodes.

For a more in-depth overview, you can read the first post here.

The Avail Light Client (LC) is a critical part of the roadmap. The LC is operational on Avail today and is used to verify transactions for Space and Time, Skate, Lens Chain and others. Further reading on the Avail LC can also be found here. 

Don’t Trust, Verify.

The Avail LC is designed to run anywhere, including on laptops and smartphones. By running on end-user devices, it can be adapted to enable users to verify a blockchain, including the execution of smart contract functions like crosschain token swaps on a DEX, lending and borrowing agreements, data authenticity for verification and/or AI and more. This not only empowers end-users with cryptographic verifiability on edge devices, but it also fulfills a core promise of blockchain technology - verifiably decentralized networks.

Today, the Avail LC verifies KZG Polynomial commitments and checks for data availability via DAS. The LC also populates a Distributed Hash Table (DHT), collectively held and referenced by other LC nodes. In order to increase the block size and reduce block times on Avail, we will need to optimize the LC p2p network (DHT) to handle the increased load.

When a data blob is submitted to Avail, it’s broken into cells and arranged into rows, with each cell being an element of the polynomial commitment for that row. A simplified illustration of this is below.

The cells and proofs are populated on the DHT. Around 130k cells are populated per block today with 4MB blocks. As we increase block size, this number can go up significantly, into the tens of millions per block.

Don't miss out, get the next update delivered to your inbox.

Removing DHT Bottlenecks

Populating the DHT requires both time and resources, and this important task is the responsibility of the LCs. Not only does all the data need to be populated to the DHT, it needs to be done with sufficient redundancy.

As the block size increases or block time decreases, we begin to place more pressure on LCs and encounter numerous bottlenecks, including:

• The number of connections maintained per light client increases, which is compute bound and will become a limiting factor.
• It increases the number of LOOKUP and PUT requests on the DHT. 
• It increases the overall number of GET requests broadcast on the network, etc.

All of this is to say that we need to reduce the number of DHT entries per block as much as possible to reach our north star of 10GB and 600ms blocks. For this, we introduce multi-proofs.

Multi-Proofs

Put simply, a multi-proof is a collection of cells with a single proof. Multi-proofs enable us to be more efficient at populating the DHT and thus supporting increased scale by generating a single proof for multiple cells against multiple commitments.

Below is a comparison of the current design (left) and multi-proof design (right).

Each grid entry is a red dot 🔴, each commitment is a green dot 🟢, and each cell (unit that we prove in the DHT) is a square 🔲. The proof that some cell corresponds to a given commitment is represented by a purple dot 🟣. 

By increasing the record size in the DHT, we can improve the overall throughput by reducing the number of lookups and network round trips required to retrieve or store data. With larger records, more information can be bundled into a single response, allowing nodes to serve richer results per query. This minimizes protocol overhead and leverages bandwidth more efficiently. As a result, the network can achieve faster resolution of queries and improved scalability under load.

A version of the multi-proof design is already running on devnet with the entire flow tested. We’ll be sharing more about this soon.

Next Steps: Infinity Testnet

The Avail LC today plays a key role in delivering end user verifiability, and tomorrow it will play a key role in facilitating larger blocks as we scale up to 10GB blocks. The unique combination of cryptographic proofs and DAS provide us with a unique advantage over other solutions, as we can achieve both scalability and maintain decentralization. By adapting the LC and having it play a more significant role in Avail’s architecture, we can achieve 10GB blocks with a 600ms block time.

The next step from here is to open up beta access to the infinity testnet which will happen soon. If you are interested in pursuing aspects of this development in more detail, either with regard to research, development or testing, please get in touch via Discord.