I’ve examined the top one million websites (according to Alexa Internet) to find out how many of them announce availability on the InterPlanetary File System (IPFS) through DNSLink.
IPFS and InterPlanetary Name System (IPNS) reference content on the distributed web using cryptographic hashes, which isn’t very human-friendly. DNSLink is a method for mapping a domain name to an IPFS or IPNS address using the Domain Name System (DNS).
I used DNSLink to discover which websites from Alexa Internet’s Top 1 Million websites list have attempted to set up an IPFS presence for themselves. It took over 4 million DNS queries as some needed to be repeated and I also had to request records for four domain name variants:
- example.com
- www.example.com
- _dnslink.example.com
- _dnslink.www.example.com
I only found 17 websites with DNSLink setup and pointing to either an IPFS or IPNS address. All the discovered websites loaded okay over the World Wide Web, but only 5 (29 %) resolved and loaded over the InterPlanetary File System (IPFS).
Here are some more numbers about the websites I found:
Ten domains contained a hash-based IPNS NURL, four contained a direct IPFS NURL, and three contained another DNSLink-based IPNS address (further DNS requests required).
All of the hash-based NURLs used a CIDv0 type hashes.
Five domains had set up the _dnslink discovery-subdomain, one used both their bare and _dnslink subdomain, and one used their bare and www subdomain.
The dataset is based on crawling the Alexa Internet top one million domains from . The world’s top one million websites don’t appear to be too interested in IPFS as a distribution method yet.
Using DNSLink as a detection method notably failed to detect the other subdomains that people do publish over IPFS. The blog. subdomain seem especially popular among the few websites I’ve found elsewhere that are available over IPFS. Focusing on DNSLink also excluded some popular websites like DTube that are partially built on IPFS but that don’t use DNSLink for their main domain.
Alexa Internet’s rankings doesn’t reflect what content is accessed on IPFS as Alexa Internet doesn’t track content on the IPFS. However, it still interesting to see which popular sites on the web are exploring IPFS distribution.
One of the most interesting findings is the low availability of websites that responded with DNSLink data on IPFS. I don’t know what causes this, but it might suggest a broader problem with maintaining IPFS distribution nodes over time.
To try to rule out DHT or routing issues, I tried resolving the IPNS responses and retrieving the IPFS hashes from my own IPFS node as well as through multiple public IPFS-to-HTTP gateways. To rule out temporary issues, I also repeated the process a few more times over the following days without observing any changes.
The InterPlanetary Name System is poorly documented. IPFS hashes can be cached indefinitely on any IPFS node. IPNS hashes point to IPFS hashes and can be used to link to non-static content. To facilitate changing content, IPNS hashes expire and has to be re-announced into the network periodically. These re-announcements has to come from the IPFS node that created them as they’re required to be signed with that node’s cryptographic keys. IPNS is a centralized point of failure for IPFS website availability, and it seems to have a high failure rate.
This does highlight the need for secondary hosting providers to ensure long-term availability of content on IPFS. So-called “pinning service” like Eternum can store gigabytes of data for pennies per month and act as a fallback in case your primary IPFS nodes goes offline and no one else has a copy of your content.
IPFS hosting services can’t help with IPNS re-announcements without requiring the distributors to relinquishing control over their cryptographic keys. At any rate, I believe that better IPNS documentation could help webmasters (InterPlanetaryMasters?) improve their service availability.