IPFS and Igalia collaborate on dweb in browsers
by Frédéric Wang & Dietrich Ayala on 2021-01-15
- IPFS and Igalia started a collaboration that will continue during 2021.
- Distributed web schemes have been safelisted in Chrome 86’s implementation of custom handlers and registered at IANA.
- Chrome 89 will allow browser extensions to register cross-origin handlers or handlers for schemes with prefix
ext+. Refinement is pending for the permission UI.
- Firefox 84 marks
http://*.localhost/URLs as secure context, which means websites loaded from local subdomain gateway will have access to the same Web APIs as HTTPS version.
- Firefox 84 has improved support for loading locally delivered mixed-resources. Patches have also been submitted to WebKit but are pending on reviews and discussions.
- Work is in progress to improve Chromium’s consistency and specification compliance regarding the notion of secure contexts, including removing non-standard localhost names.
- Miscellaneous other fixes have landed for the Firefox and Chromium’s implementations of custom handlers.
Nowadays, the majority of pages on the Web are coming from central servers controlled by their owners. The IPFS protocol envisions a future Web in which content can be delivered peer-to-peer, meaning directly between individuals or within groups. There have been web platform and browser efforts to reach the goal of a distributed Web.
Nevertheless, having corresponding protocols natively supported in browsers and taken into account in web standards will require coordination with various actors of the Web, including standardization groups (W3C, WHATWG, …) and browser implementers.
With that goal in mind, Protocol Labs started a collaboration with Igalia, an Open Source company with expertise in browser development and the web platform. In this blog post, preliminary results obtained this year for non-native implementations are presented. This is a good first step for users, as well as an opportunity to start discussions and raise awareness of the distributed web.
Distributed web protocols in custom handlers
An existing approach to use IPFS in browsers that don’t natively support this protocol is to rely on an HTTP gateway. Additionally, the redirection of IPFS links can be automatically performed using HTML custom handlers. However, this approach has several limitations:
- Custom handlers are only implemented in Mozilla and Chromium browsers, not the ones based on WebKit.
- Custom handlers only accept schemes with prefix
web+or belonging to a predetermined safe list.
- Custom handlers only allow to register a handler that has same origin as the page registering it.
- Web specifications dealing with URLs may not work well with the redirection to the HTTP gateway.
Regarding the first issue, people have submitted patches to WebKit in the past. Although this may be considered again eventually, for now there is not any consensus within the WebKit community about whether this API should be implemented.
One consequence of this is that in order to safelist more schemes, one must get support from the Mozilla and Chromium implementers. This has turned out to be historically difficult, with many users opening requests for different schemes, that ended up being put on hold due to lack of consensus. A first task has been to re-open past discussions between browser vendors and users in order to unblock the situation.
Focus has then moved to distributed web protocols (
ssb). These (together with several other schemes requested by the community) have been registered at IANA. Corresponding changes to Firefox and the HTML specifications are also ready, but pending on Mozilla. Finally, these schemes for distributed web protocols have been safelisted in the latest Chromium release.
The same-origin limitation is something desired for security reasons when web pages register custom handlers. However, when such a registration happens from browser extensions trusted by users, it makes sense to make an exception. For example, for a long time Firefox has allowed declaring custom handlers for
ipfs and other protocols directly in the WebExtension manifest and without same-origin check.
There were existing requests about this feature in Chromium bug trackers. To tackle this problem, the initial step has been to follow the Chromium project’s process and draft a proposal based on existing use cases and what Firefox implements.
Since there is a strong preference within the Chromium project to rely on Web APIs for new extension features, code owners’ counter-proposal has been to give more power to
registerProtocolHandler in extension context (cross-origin handler and extension-specific schemes). The two patches for these landed recently in Chromium and will be available in version 89!
For now, due to an existing permission UI bug, the protocol registration must happen in an extension tab, but you can already get an idea of the new possibility in this video, (be sure to enable subtitles for a detailed description):
The issue with redirected URLs not behaving like normal URLs is a bit more tricky and can come from several reasons. For local HTTP gateways, such as the one provided by IPFS Desktop, one of the explanations is that browsers used to not treat these local URLs as secure contexts and thus block various web platform features. This was changed three years ago in Chrome and specifications were updated accordingly. More precisely:
- The definition of potentially trustworthy origins includes the ones whose hosts are loopback IPv4 and IPv6 addresses and (optionally)
- This optional behavior is conditioned on the fact that browsers override native DNS when resolving localhost names.
Mozilla has been supportive of this change. The case of Loopback IP addresses has been implemented since Firefox 55, but as this happens with limited development resources and prioritization, the work for localhost names had never been finished. One of the difficulties being that many existing network tests do not assume the above behavior and require some adjustments to keep passing. After several attempts, the main patches landed without being reverted and is released in the latest version of Firefox.
The position within the WebKit community is less clear, some members being skeptical about granting web sites access to local hosts, others thinking the behavior implemented in Chrome and Firefox make sense. Similarly to Firefox, one difficulty is that many tests need to be tweaked. Additionally, implementing the address redirection does not seem to be easily doable at the WebKit level. One can find details on the corresponding bug. In order to get things rolling, patches following a preference-based approach have been submitted for review.
This problem is actually more general than just local resources. The whole notion of “secure context” is not implemented consistently between browsers, or even within each browser. In addition to the one of potentially trustworthy origin mentioned above, the specification has a slightly more general notion of potentially trustworthy URL which used to differentiate from a priori authenticated URL. How these notions should interact with custom handlers is also currently a bit fuzzy.
In Chromium, there were at least 6 implementations of “secure”. Some of these implementations are not very aligned with the current specification. Worse, some of them include historical and non-standard localhost names like
localhost6.localdomain6. Effort is being made to progressively and carefully unify these implementations and follow the specifications. In particular, patches landed to remove non-standard localhost names and make
data: URLs potentially trustworthy.
As usual, performing this kind of effort leads to a lot of side tasks: community and specification discussion, code clean up and refactoring, documentation improvement, and other bug fixes. Here are Chromium bugs fixed that are relevant for custom handlers and interoperability with Firefox:
- Make service workers work with custom handlers
- Do not remove %s token when validating (un)registerProtocolHandler’s URL
- percent-encode U+0020 SPACE when in URLs computed by custom protocol handlers
- percent-encode the delete character when parsing URLs
Finally, for both Chromium and Firefox, the title argument has been removed from registerProtocolHandler() as per Mozilla’s suggestion.
If you are interested in an exhaustive list of contributions related to this work, here it comes:
- [Extensions] Fix broken links in documentation on writing a new API
- [Extensions] Fix typos in chrome.test.sendMessage() doc
- Fix test for RegisterProtocolHandlerDifferentOrigin
- Do not remove %s token when validating (un)registerProtocolHandler’s URL
- percent-encode U+0020 SPACE when using a protocol handler
- percent-encode U+007F in cannot-be-a-base-URL path and fragment states
- Add WPT tests for registerProtocolHandler and ‘web+’ schemes
- 2362802: Introduce common browser/web API for validation of custom handlers
- 2153064: Safelist distributed web schemes for “registerProtocolHandler”
- 2379511: Remove references to ServiceWorkerRequestHandler/ServiceWorkerNavigationLoader
- 2487107: Reland “Make custom protocol handlers work with service workers’ fetch event”
- 2157531: Remove the title argument from registerProtocolHandler()
- 2287304: Add custom security levels for registerProtocolHandler
- 2560305: Add registerProtocolHandler for extension-specific features
- 2560953: Prepare code to improve handling of potentially trustworthy url/origin
- 2563492: Limit about: URLs that are treated as potentially trustworthy
- 2563759: Remove content::IsPotentiallyTrustworthyOrigin
- 2570568: Remove special handling of localhost6 and localhost6.localdomain6
- 2580067: Use network::IsOriginPotentiallyTrustworthy in Insecure Input Tab Helper
- 2563683: Treat data: URLs as potentially trustworthy
- 2563883: Remove blink::network_utils::IsOriginSecure
- 2577688: Remove special handling of localhost.localdomain
- 2587738: Remove SecurityPolicy::IsUrlTrustworthySafelisted()
- 2332675: Restrict protocol handler to potentially trustworthy URLs
- 2595424: Use a standard scheme to test potential trustworthiness
- 2593629: Add tests for SecurityOrigin::IsSecure and network::Is*PotentiallyTrustworthy
- 2610100: Add tests for SecurityOrigin::IsPotentiallyTrustworthy
- 2612947: Remove SecurityPolicy APIs for handling a trustworthy safelist
- 2614784: Remove SchemeRegistry APIs for handling local and secure schemes
- 2615260: Implement SecurityOrigin::IsPotentiallyTrustworthy with network::IsOriginPotentiallyTrustworthy
- 2617709: Rewrite SecurityOrigin::IsSecure() using GURL and url::Origin
- 2617883: Make SecurityOrigin::IsSecure treat localhost and local files as secure
- Improve compatibility of protocol_handlers with registerProtocolHandler
- Safelist cabal, dat, did, dweb, ethereum, hyper, ipfs, ipns, and ssb schemes for registerProtocolHandler().
- Remove the title argument from registerProtocolHandler()
- Do not set network.dns.ipv4OnlyDomains when running XPCShell
- Add a test to ensure loopback host names cannot be overridden
- Hardcode localhost to loopback
- Don’t treat loopback addresses (127.0.0.0/8, ::1⁄128, localhost, .localhost) as mixed content
- Don’t treat loopback IP addresses (127.0.0.0/8, ::1⁄128) as mixed content
- Introduce preference not to treat localhost and .localhost as mixed content
- [GTK] Allow WebKitTestServer to run non-loopback addresses for API test
- Migrate WebKitTestServer to libsoup 2.48 API
- Treat loopback addresses (127.0.0.0/8, ::1⁄128, localhost, .localhost) as potentially trustworthy URL
IPFS and Igalia have made an initial effort to improve support and interoperability of web platform features that would benefit the distributed web, as well as the web community in general. In addition to starting discussions among the different actors, several patches have already landed in browsers. We are looking forward to continuing this work in 2021… Stay tuned! 🚀