# 🧘♀️ Purpose
This blog entry intends to give an update on what is happening with IPFS development in JS. It’s been a while since we’ve done this, so there’s a lot to cover. This is the first update of more to come. It isn’t a full roadmap, but we aim to give clarity to some of the history of IPFS development in JS, decisions by some maintainers on what to do going forward, and ways you can help.
- 📇 Names and Terms
- ⏳ IPFS JS Development History
- 🔀 Historical IPFS JS Usage Patterns
- 🧑💻 IPFS-in-JS Development the Last 18 Months
- 💡 The Future of IPFS-in-JS in 2022 and 2023
- Go and JS Development are Decoupling
- Seize and Leverage New Browser-Friendly P2P Transports
- Support Fully Speced Delegated Routing Protocols and Endpoints
- PL Delegate and Preload Nodes Will Be Shutting Down
- Release Helia in 2023
- Pause js-ipfs Maintenance Once Helia Is Released
- A New Name Is Coming
- Doc Updates Galore
- 🗺 Timeline
- 🤝 Ways You Can Help
# 📇 Names and Terms
To help with this update, the following names and terms will be used to aid with clarity:
- Kubo (opens new window) – This project was formerly known as go-ipfs. See here (opens new window) for more info.
- js-ipfs (opens new window) - This is the long-standing IPFS implementation written in JS. As described below, we will be deprecating it after Helia is released. We’re currently not planning to rename this implementation like we did with Kubo (opens new window) given its limited lifespan.
- Helia (opens new window) - This is a to-be-created IPFS implementation in JS (opens new window) that is discussed below. The final name is TBD (to be determined), and you can track the naming effort here (opens new window). While it will use many of the underlying libraries of js-ipfs (e.g., js-libp2p (opens new window), js-ipfs-bitswap (opens new window)), it is a separate project with a different API.
- Delegate nodes - These are nodes that expose the
/api/v0/dht/*endpoints of the Kubo RPC API (opens new window) for delegated routing. Because js-ipfs nodes don’t have the DHT enabled by default and wouldn’t make good DHT servers in browsers anyways, they need the help of delegate nodes to resolve DHT queries.
- Preload nodes - These are nodes that expose the
/api/v0/refsendpoint of the Kubo RPC API (opens new window) which can be called so that the remote node will fetch CIDs (but not pin). This is necessary to ensure that blocks that are added in the browser are preloaded onto a long-running IPFS node so that it’s made available to the rest of the network. Preload nodes garbage collect those blocks after a period.
# ⏳ IPFS-in-JS Development History
- ipfs-http-client (opens new window): an implementation that delegates out with RPC HTTP calls, and thus could be used to control and interact with a Kubo daemon or js-ipfs daemon.
This effectively had Kubo and js-ipfs’ development awkwardly interlinked. You would find support for certain functionality in Kubo that wasn’t supported in js-ipfs, and js-ipfs would often be held back in experimenting on a feature because the change wouldn’t get added to the JS Core API without it also being implemented in Kubo.
In addition to interlinking with the API and design choices of Kubo, the top-level js-ipfs project followed a similar project structure to Kubo of being a kitchen sink of interaction options with:
- Daemon to be able to run js-ipfs as a standalone process
- A CLI to the core implementation
- Two RPC client and server implementations:
- RPC-over-HTTP API that matched Kubo
- gRPC-over-websockets API for bidirectional streaming
- HTTP Gateway (opens new window) server (though it lagged behind Kubo and did not support the latest specs for things like subdomain isolation and verifiable block/CAR responses (opens new window))
This “kitchen sink” aspect was also antithetical to the “JS way” of being lean and modular. For example, by importing js-ipfs, you always get MFS, IPNS, etc. even if they don’t use them, which leads to bigger bundles and more dependencies with more attack vectors.
“… You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.“ — Joe Armstrong, creator of Erlang progamming language
This is unlike the more flexible js-libp2p model where you can extend the capabilities of the node by configuring additional modules.
# 🔀 Historical IPFS JS Usage Patterns
# Remote Control of Kubo
With this approach, a long-running Kubo node be controlled by ipfs-http-client (opens new window) via Kubo's RPC API.
Based on npm download metrics, this is by far the most popular approach with ~10x more npm downloads (opens new window).
# 👍 Pros
- Gives access to a full-featured IPFS implementation in Kubo with active maintenance and feature development.
# 👎 Cons
- Requires bringing a long-running Kubo daemon into one’s architecture and potentially a reverse proxy like nginx for TLS termination. The Kubo daemon potentially becomes a black box to debug and tune for a JS development shop.
- Requires proper security to protect which Kubo RPC API endpoints are exposed to the public internet (opens new window).
# js-ipfs in a Node.js Context
With this approach, you either embed ipfs-core (opens new window) directly in your application by importing and instantiating the IPFS object or accessing a shared js-ipfs daemon. With the former approach, the Node.js process of the application would also open IPFS peer-to-peer connections.
# 👍 Pros
# 👎 Cons
- Historically lacking in some functionality that was only more recently added like a working DHT client and server.
- The API is large and hard to grasp. It contains many methods translated directly from the Kubo CLI, designed to be quick to type but often redundant.
# js-ipfs in a Browser Context
Because browsers impose constraints on the number of network connections opened and storage space used by each browser tab, they don’t make for reliable IPFS nodes. For this reason, many of the responsibilities of an IPFS node which include: routing, retrieval, and providing have all been handled in a delegated fashion in the browser.
Unfortunately, the default delegation setup in js-ipfs today is overly complicated and not well-engineered. Specifically, this happens via preload and delegate nodes hosted by Protocol Labs that support secure WebSockets (WSS) that browsers can safely and reliably connect to.
For delegated routing/retrieval, js-ipfs asks the delegate nodes to perform DHT operations on its behalf using the Kubo RPC API (
/v0/dht/*). The delegate nodes query the DHT for peers and content providers, and then they fetch the content, which they provide for the browser nodes that are connected to them.
For delegated providing from the browser, js-ipfs contacts the preload node using Kubo RPC API (
/v0/refs) to preload the entire dag from the browser. The preload node fetches the data from the browser over Bitswap and then advertises it on the DHT.
Note: Preloading could be understood as ephemeral pinning. The CIDs' data is fetched by a preload node for a short period during which it is available to the IPFS network, and after which it is garbage collected.
js-ipfs nodes running in the browser can connect to other js-ipfs nodes via the WebRTC Star transport, but this requires a centralized signaling server (opens new window) to do the initial handshake and cannot be used to connect to Kubo nodes as support for this transport has never materialized.
# 👍 Pros
- It’s useful and works for multi-user apps with small amounts of ephemeral data transmitted in real-time.
- Even though js-ipfs with WebRTC requires a centralized signaling server and is not the ideal long-term solution (opens new window), it enables direct browser-to-browser communication (after the initial SDP manifest (opens new window) is exchanged over signaling server).
# 👎 Cons
- Availability and performance are heavily dependent on non-trivial centralized infrastructure:
- If PL shuts down the hardcoded HTTP endpoints at
preload.ipfs.io(opens new window), js-ipfs delegation is dead, unless someone also runs their own Kubo instance somewhere in the cloud with a TLS cert for HTTPS and sets it up as a preload in js-ipfs config. This is a non-trivial ask and a steep barrier to adoption.
- This was designed as a temporary "hack" anticipating a proper libp2p protocol. Unfortunately, years later we’re still doing the original Kubo RPC
- If PL shuts down the hardcoded HTTP endpoints at
- Inefficient design:
- It doesn’t scale well because certain preload nodes get disproportionate traffic due to hardcoding in old js-ipfs versions.
- Preload via
/v0/refsis very wasteful (opens new window).
- Not specified or adopted by other implementations: the WebRTC Star transport is only implemented in js-libp2p and neither WebRTC Star nor the delegate/preload APIs are specified.
# 🧑💻 IPFS-in-JS Development the Last 18 Months
The last 18 months on the JS front have been spent on:
- General maintenance of the project including mostly maintaining parity between the JS Core API with Kubo.
- Modernizing js-ipfs and js-libp2p to TypeScript and ESM only. 😅
- Improving security and performance of js-libp2p given its criticality for other projects outside of IPFS such as Ethereum Lodestar. For example, a set of DoS and eclipse attack mitigations (opens new window) were added.
- Adding a working DHT client/server implementation to js-libp2p. (Yes, in a NodeJS context you can read from and write to the public IPFS DHT.)
- Expanding connectivity options of IPFS-in-JS implementations. Notably, we introduced a WebTransport transport (opens new window) and are adding a new WebRTC transport (opens new window) that does not require a centralized signaling server to enable both browser-based IPFS nodes to dial Kubo nodes directly and browser-to-browser connectivity. (This is discussed more below.)
# 💡 The Future of IPFS-in-JS in 2022 and 2023
With all the background of IPFS-in-JS over the past covered, this section will transition into our practical plans for 2022 and 2023.
# Go and JS Development are Decoupling
We never had full compatibility between Kubo and js-ipfs, we don’t think we can, and we don’t think it’s worth investing more down this path. At least for the implementations maintained by Protocol Labs’ EngRes (Engineering & Research) group (opens new window), the Go and JS implementations will diverge and develop the APIs that are best for their respective user bases.
In practical terms, this translates to:
- ipfs-http-client will remain the RPC-over-HTTP API for controlling js-ipfs (you can also use ipfs-grpc-client over WebSockets). The current js-ipfs RPC APIs will be maintained until js-ipfs support ceases (discussed below).
- With investment in Helia, a new RPC API for Helia will emerge.
- We won’t test that ipfs-http-client has compatibility with recent versions of Kubo.
- If you want to control a Kubo node via JS, use the Kubo-specific library js-kubo-rpc-client (opens new window) (api).
# Seize and Leverage New Browser-Friendly P2P Transports
We are transitioning to a world where browsers can connect to other libp2p nodes (including other browsers) without Central Authority TLS certs thanks to new transports like WebTransport and WebRTC. See connectivity.libp2p.io (opens new window) for more details.
This means browser nodes have more optionality to which long-running IPFS nodes they delegate routing, retrieval, and providing. For example, making DHT or Bitswap requests to other nodes on the network is now viable.
Note: browser nodes will still want to delegate providing content to a node with more longevity since the new transports won’t stop browser nodes from disappearing from the network when the user closes a tab or puts their laptop to sleep.
We’ll lean into realizing these breakthroughs and remove the more convoluted mechanisms from the past that relied on the Kubo RPC API and preload nodes discussed in js-ipfs in a Browser context.
# Support Fully Speced Delegated Routing Protocols and Endpoints
While it will be possible from a connectivity perspective to make DHT queries from a browser, we expect various applications will want to still delegate out routing. Reframe (opens new window) is a protocol for delegated routing that other IPFS implementations like Kubo have implemented. While it currently uses HTTP as a transport, it is speced and not tied to the Kubo RPC API. If/when there is a speced protocol for ambient discovery of “Limited Delegated Routers” provided by libp2p, we will support that as well.
# PL Delegate and Preload Nodes Will Be Shutting Down
Given the new browser-friendly p2p transports discussed above, we’ll shut down the complicated “song-and-dance” with the legacy delegate/preload nodes and the Kubo RPC API described in js-ipfs in a Browser context. This yields a simpler setup for one’s application and removes centralized infrastructure.
For delegated routing, one can configure Reframe (opens new window) endpoints. When it comes to providing content from a browser node, it will be up to developers to account for user behavior like closing tabs or laptop lids. The general recommendation is to either run your own preload node or upload content explicitly to a pinning service for providing.
# Release Helia in 2023
Helia is the to-be-developed IPFS implementation with all that we’ve learned over the last 8 years while leveraging what is available to us in JS runtime.
Some defining attributes include:
Leaner API not tied to the legacy “core API” concept - Helia will not have API compatibility with js-ipfs. It will expose a more ergonomic JS-developer-first API than what we have with the js-ipfs “core API” that was heavily influenced by Kubo. (One can also create an adapter from the “core API” to Helia’s API if they want to drop Helia to their existing application using js-ipfs’ ipfs-core.)
ESM and TypeScript only - There’s no more debate on the utility of these for JS development. We’ll adopt them from day one.
Leverage existing interplanetary libraries - While we’re moving away from the interface and composition in js-ipfs, we’re not abandoning the underlying layers like js-libp2p, js-bitswap, etc. Those libraries have received a lot of maintenance attention in the last 18 months (including TypeScript and ESM updates) and are battle-tested in production. We will depend on them in Helia as well.
Unified file API - high-level commands that act like a filesystem and return CIDs. For example:
const dirCid = await ipfs.mkdir('/foo') const dirCid2 = await ipfs.touch(dirCid, 'file.txt') const dirCid3 = await ipfs.pipe([ 'file content', ipfs.open(dirCid2, 'file.txt'), ]) // or something const content = await ipfs.cat(dirCid3, 'file.txt') // could be a network request
Expose a block API for low-level IPLD operations.
Focus on the browser use case - We won’t do anything that precludes operating in NodeJS or a cloud service worker, but by default, we will prioritize delivery paths that deliver browser functionality sooner. This is because the browser runtime is the unique runtime the Helia IPFS implementation can enable that other implementations can’t. As a result, it means there aren’t plans to invest in things like a JS implementation of the HTTP Gateway spec. We’ll let other implementations like Kubo, Iroh, etc. pursue that use case.
Enable configurable levels of delegation. With routing, retrieval, and providing there will be varying levels of delegating from none (all handled by the local Helia node) to full (all handled by HTTP Gateways (opens new window) and Pinning Services (opens new window)).
# Pause js-ipfs Maintenance Once Helia Is Released
Shortly after you can add and cat files across the network with Helia, PL EngRes (opens new window) will cease maintenance on js-ipfs. In the absence of an established group with a credible track record to take js-ipfs over, the community is welcome to fork js-ipfs and maintain the fork. (We want to avoid issues that can occur with casually giving away publishing rights.)
As discussed before, we are not ceasing support and development of many of the libraries that js-ipfs depends on like js-libp2p and js-bitswap. These projects will be actively maintained as core dependencies to Helia and other projects.
# A New Name Is Coming
As outlined here (opens new window), Protocol Labs wants to make space for additional IPFS implementations to be made, including in JS. We want to make it clear that js-ipfs is not IPFS and that js-ipfs is not the IPFS implementation in JS. go-ipfs successfully made this transition earlier in 2022 with its minimal rename to Kubo (opens new window). We will certainly not make the same name-squatting mistake with a new implementation like Helia. Details and plans will be shared here (opens new window) and in the IPFS forums (opens new window).
# Doc Updates Galore
From dedicated websites (opens new window), examples (opens new window), to official docs (opens new window), and courses (opens new window), many places will need updating in light of new names and implementations. This is going to be a sizable undertaking that hasn’t been scoped out yet. This will be tracked here (opens new window).
# 🗺 Timeline
The timeline for enacting all of the above is still actively being figured out. We’ll be updating the proposed roadmap (opens new window).
# 🤝 Ways You Can Help
- 🗳 Propose a name for the new “Helia” JS library (opens new window) and cast your votes.
- 🗣 Give feedback on the Helia roadmap (opens new window). Let us know how you’re using js-ipfs now so we can see if/how your use case would be supported with Helia in the future.
- ✋ Contribute - Open source contributors welcome. Have a great idea and need some funding? Consider a grant request (opens new window).
Thank you for reading and being on this journey to make IPFS exceptional in JS runtimes!
Note: An earlier version of this blog post referred to Helia as Pomegranate. The blog post has been updated to reflect the name chosen by the community. (opens new window)