The distributed web seeks to make peer-to-peer (P2P) content distribution the new default. This should help make the web topology more democratic and resilient to nature and political whims.
Lately, I’ve been taken in by the concept of the distributed web and the InterPlanetary File System (IPFS) in particular. On the distributed web, every client is also a server that serves copies of everything the client has recently downloaded. If you think this sounds a lot like how P2P file transfer protocols like BitTorrent works, then you’re absolutely right. It’s the same principle and IPFS shares a lot of the same technology and goals.
The web has grown increasingly centralized and initiatives like Google’s Accelerated Mobile Pages (AMP) outright seeks to move formerly independent websites onto Google’s own servers. The distributed web seeks to go in the opposite direction and make every client participate in content distribution.
Content on IPFS is addressed by cryptographic hashes of the content itself. You query the IPFS’ Distributed Hash Table (DHT) network to discover who else claims to have the file you want, connect to them and request the file. Upon arrival your IPFS client can recalculate the cryptographic hash of the file and thus confirm they sent the right file. This ensures the integrity of the files you access and prevents attacker-in-the-middle (AITM) attackers from modifying the pages you want to access.
There’s no central database that maintains a lit of every file or file hash available on the network. Every client can host a small part of the DHT and share knowledge about how to connect to other clients that may have specific hashes.
The distributed web is as much about democratizing content distribution as its about rearchitecting the web to become more resilient. IPFS can route around service disruptions, natural disasters, shark-bitten undersea cables, packet flooding attacks against major web service providers, human conflicts, commercial interests, as well as government policies and censorship.
You can always get a file assuming that anyone else that you can connect to have the same file that you want. IPFS can work over direct device-to-device links and mesh-networks just as well as over the broader internet.
Imagine a local community being cut off temporarily from the internet by a natural disaster. With the current centralized architecture of the web, that community would lose access to pretty much every resource on the web including their own local newspaper (who’ve likely outsourced web hosting to the likes of Amazon Web Services.) On the distributed web, resources of local interests like the local news, local weather, government information, maps, and other common-interest items would still be accessible from other clients in the neighborhood.
The distributed web also helps content producers cut down on their distribution costs. The more popular a file is on the distributed web, the less of the bandwidth burden and costs must be covered by the content producer directly. As a content producer, you technically only need to maintain a single IPFS node to ensure long-term availability of your content across the open internet.
Distributed web technologies like the InterPlanetary File System currently comes with non-trivial bandwidth, memory, and computational requirements. The technology has to improve considerably, and work its way to become built-in to our web browsers before the decentralized web dream can be realized. Mozilla and Firefox is most definitely interested, with Firefox being the first browser to ship any product changes to support the distributed web.
Let us just say it isn’t quite ready yet for the small battery powered devices we all carry around in our pockets. These devices can still access content stored on IPFS through IPFS-to-HTTPS gateway services which are already offered for free by internet infrastructure behemoth Cloudflare.
There are also dedicated commercial IPFS hosting, or “pinning”, services available like Eternum (discontinued as of 2023) that can help website large and small kickstart its IPFS services. IPFS gateways are IPFS nodes just like any other and can help establish service reliability and content availability as the web begins to transition from centralized to decentralized.
The web seems to be headed into the hands of a few behemoth multinational companies. I prefer to see the web go in the opposite direction and break free from these controlling interests.
I encourage every developer and anyone who produces or publishes anything on the web to learn more about the InterPlanetary File System and the distributed web.