The Evolution of Decentralized Cloud Services: From DFINITY IC to Arweave AO

Source: PermaDAO

Decentralized cloud services have always been considered one of the most important applications of blockchain. Web3 + cloud services are not only a very good narrative direction, but also very easy to combine with AI narrative to further enhance the imagination space. DFINITY IC has great technical challenges in achieving instant consensus for computing in a purely asynchronous environment such as cloud services. Arweave AO breaks the shackles of blockchain. It does not deal with the computing itself and the consensus, but ensures the correctness of the computing results through economic models and lazy verification. It can be said that it has taken a different approach.

Basic Introduction

Cloud is an IT environment that can abstract, aggregate and share scalable resources in the entire network. The resources here include computing, storage, network bandwidth, etc. The main technical features include virtualization technology, distributed resource management technology and parallel execution technology. Compared with the traditional IT environment based on physical machines, the cloud has lower costs, better scalability, and is easier to manage.

Cloud services, cloud computing, and cloud storage are all sub-concepts derived from the concept of cloud. Cloud computing refers to providing computing services through the cloud, cloud storage refers to providing storage services through the cloud, and cloud services refer to providing various IT services through the cloud, of course, including computing services and storage services. In particular, cloud computing and cloud services are equated in many places, but they are still distinguished in this article.

The current cloud service market is mainly monopolized by several cloud service giants. For example, Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure in foreign countries occupy more than 60% of the global cloud service market, while Alibaba Cloud, Tencent Cloud, and Huawei Cloud in China also occupy about 60% of the domestic cloud service market. It can be said that the Matthew effect is obvious.

Decentralized cloud services are based on the subversion of the current cloud service model by blockchain technology, relying on market forces to determine the value and allocation of computing power, storage space, and bandwidth, which not only achieves more efficient and direct resource allocation, but more importantly, breaks the monopoly of cloud service giants. Because all the underlying resources are provided by users, once the economic structure of a decentralized cloud service network is no longer applicable, users can choose to exit the network or join other decentralized cloud service networks, which limits the space for decentralized cloud service network providers to "do evil".

The Development Path of Decentralized Cloud Services

Decentralized computing is not mentioned here because the author believes that decentralized computing is a false proposition. First of all, the smart contract of the blockchain itself is responsible for decentralized computing. Secondly, for complex computing tasks, decentralization means a waste of computing power. There is no need to decentralize for the sake of decentralization. Verifiable distributed computing may be a better choice. The author will explain this later.

Compared to instantaneous computing, persistent storage is obviously an excellent application scenario for decentralization, which allows data to be stored and distributed in a decentralized network, which cannot be tampered with and is censorship-resistant. In fact, decentralized storage has become one of the most successful applications of blockchain in non-financial scenarios.

In 2014, Protocol Labs proposed IPFS, which stands for InterPlanetary File System. Just like its name, IPFS also opened the interstellar door of decentralized storage. IPFS implements content-based file addressing. The same files will not be stored repeatedly, which greatly saves storage space. Based on the P2P network, it can download data from multiple nodes concurrently, thereby greatly reducing bandwidth costs. It has become the underlying protocol for a number of decentralized storage projects, among which Filecoin is the most representative.

In July 2017, Protocol Labs, which developed IPFS, announced the establishment of the Filecoin project. Filecoin is an incentive application layer and blockchain public chain system on the IPFS network, which adopts a hybrid consensus mechanism: mainly based on Expected Consensus (EC), supplemented by Proof of Spacetime (PoSt) and Proof of Replication (PoRep). Filecoin's vision is to incentivize a large number of nodes around the world to provide storage and retrieval services to users and promote the widespread use of the IPFS file storage and transfer protocol.

In July 2017, Stroj Labs founded Storj, almost at the same time as Filecoin. Storj focuses on enterprise-level storage services, and its model is more commercialized, directly targeting AWS's S3 service, but its architecture is pseudo-decentralized. Although it began to issue coins in 2018, its metadata management, block generation, rewards and penalties are all completed through satellite nodes, and satellite nodes are currently only maintained by the project party. Although the decentralization of satellite nodes is mentioned in future plans, it is currently difficult, so the current situation is still traditional storage in the guise of blockchain.

In June 2018, the Arweave mainnet was launched. Arweave is not based on the incentive layer of IPFS, but combines data storage with incentives, focusing on achieving permanent storage and access to data. Arweave does not require miners to save all block records, but encourages miners to save as many blocks as possible through Succinct Proof of Random Access (SPoRA), especially blocks with less redundancy, because miners who do so have a greater probability of obtaining mining rewards. It can be said that Arweave ensures that data is copied as much as possible through gaming, thereby improving the reliability of data storage.

In May 2021, the Internet Computer (IC) mainnet developed by the DFINITY Foundation was launched. IC is the first complete decentralized cloud service and is known as the third-generation blockchain architecture. IC can provide unlimited scalability and high-speed transaction processing capabilities through innovative blockchain technology, and can handle HTTP requests to support large-scale decentralized applications, from social media platforms, open source project hosting services to various enterprise-level applications, and even AI large models. In theory, any existing Internet service can be rebuilt as a decentralized version on IC.

In November 2022, the emergence of ChatGPT marked a milestone in strong artificial intelligence, triggering a new round of artificial intelligence craze, and AI start-ups have sprung up like mushrooms after rain. With the iterative upgrade of generative large models, the demand and cost of computing power have increased exponentially, thus giving birth to the decentralized computing power track, which is committed to reducing the training costs of large models of AI start-ups by sharing computing power and weakening the unfair competition generated by cloud service giants using their dominant position.

Well-known projects in the decentralized computing power track include io.net, Render, Akash, Gensyn, etc. Although they can also be classified as decentralized cloud services, their core protocol is to build a computing power market and incentivize computing power providers, which is actually more in line with the definition of DePIN (Decentralized Physical Infrastructure Networks).

In February 2024, Arweave officially launched the hyperparallel computer AO, becoming the second complete decentralized cloud service after DFINITY's IC. The development of decentralized cloud services is yet to be continued.

Arweave AO: Separation of Computation and Consensus

DFINITY IC and Arweave AO are both complete decentralized cloud services, and they are very similar. First of all, they both support the decentralized reconstruction of large-scale Internet services and the introduction of large AI models into smart contracts on the blockchain. Secondly, their architectures are both designed based on the Actor model. Actor is the basic unit of a concurrent computing model in computer science. The Actor model is suitable for building highly concurrent, distributed, and fault-tolerant systems, which is also the origin of the name Arweave AO.

The main differences between the two are in the data storage layer, execution layer, and consensus layer.

• Data storage layer: The smart contract on DFINITY IC is called Canister. Canister has its own exclusive container (similar to Docker). The data of each Canister is encapsulated in its own independent container. The outside world cannot see the details of the data and can only access the internal data through the interface provided by Canister. Arweave AO is based on Arweave, and the data is stored in Arweave and is open to the outside world.

• Execution layer: The virtual machine of DFINITY IC is WASM. The Canister code will be compiled into a WASM module to be deployed on the IC for operation. It only supports standards such as the WebAssembly system interface. Arweave AO is more flexible. As long as it follows the AO protocol standard, any virtual machine can be used, including EVM, WASM, Move VM, etc. The current AOS contract development language officially built by Arweave is Lua.

• Consensus layer: DFINITY IC’s subnets use a variant of the BFT (Byzantine Fault Tolerance) consensus, and subnets are verified through Chain-Key technology; Arweave AO is based on SCP (Storage-based Consensus Paradigm), a storage-based consensus paradigm that emphasizes that consensus occurs at the storage layer and uses Arweave for immutable storage to ensure security and verifiability.

Through the above comparison, we can see that DFINITY IC still follows the standard paradigm of blockchain, while Arweave AO does not seem to be that blockchain-like. After all, there is no consensus mechanism. How can we ensure that different nodes have consistent calculation results?

The answer is that Arweave AO cannot guarantee the consistency of calculation results. Its calculation results do not produce any proof (such as Merkle tree), but Arweave's immutable storage is verifiable. Calculation and consensus are separated, which is the most ingenious design of Arweave AO.

Arweave stores the holographic data of AO and every thread on AO. Anyone can restore AO and any thread on AO through holographic data. This is actually the core idea of ​​SCP, that is, as long as the storage is immutable, the above transactions are traceable, so no matter where the application is calculated, the same result will be obtained.

Once the verifiability problem is solved, the economic model of AO can be used to encourage everyone to provide correct calculation results, similar to the margin mechanism of Chainlink nodes in DON. Nodes need to pledge tokens to join the AO network. When nodes provide correct calculation results, they are incentivized, and when nodes provide incorrect results, they are fined.

Summary

Decentralized cloud services have always been considered one of the most important applications of blockchain. Web3 + cloud services are not only a very good narrative direction, but also very easy to combine with AI narrative to further enhance the imagination space.

Decentralized cloud services have been developed for ten years since IPFS. From a technical point of view, the underlying storage and computing protocols have been perfected. From a market point of view, there is also sufficient demand for resource sharing networks based on computing power. However, there are not many masters now, and they are not satisfactory.

Before the launch of DFINITY IC, it was once considered to be the AWS of Web3. It was the halo of the third-generation blockchain, but unfortunately it suffered the pain of going online, and now it has gone from negative to one. Aside from some market factors, the real-time consensus of computing in a purely asynchronous environment such as cloud services will have extremely high requirements for hardware resources. High hardware resource requirements will affect the degree of decentralization, making the consensus mechanism of blockchain meaningless.

Arweave AO breaks the shackles of blockchain. It does not deal with the calculation itself and the consensus. Instead, it uses economic models and lazy verification to ensure the correctness of the calculation results. It can be said that it has taken a different approach. However, since the supervision and verification of verifiable calculations based on SCP are all carried out off-chain, there is a problem of insufficient off-chain supervision in theory. In the scenario of large-scale computing, it still needs to be tested by the market, but the future prospects are worth looking forward to.