go-ipfs 0.8.0, and Remote Pinning, is here!
by Adin Schmahmann on 2021-02-19
What’s new in go-ipfs 0.8.0?
This release is focused on making it easier to work with pins! We have some snazzy new features around being able to ask remote services to pin data for you, and modifying large pin sets is much faster than ever before.
🧷 Remote pinning services
There is now support for asking remote services to pin data for you.
This feature is a redesign of how we’re thinking about pinning and includes some commonly requested features such as:
- Pins can have names (and coming soon metadata)
- Data can be pinned in either the foreground or background
- Pins can be searched for by name, CID, or status
Command-line users benefit from
ipfs pin remote commands, simplifying remote pinning operations. The built-in pinning service API client also executes all necessary remote calls under the hood:
As long as a pinning service supports the vendor-agnostic IPFS Pinning Service API, it can be used directly in go-ipfs. (If you’re a Pinata user, you can already check out their docs for how to set everything up.)
ipfs pin remote service add mysrv https://my-service.example.com/api-endpoint myAccessToken ipfs pin remote service ls --stat # confirm service mysrv is available
ipfs pin remote add /ipfs/bafymydata --service=mysrv --name=myfile # will block until status is pinned ipfs pin remote ls --service=mysrv --name=myfile ipfs pin remote rm --service=mysrv --name=myfile
ipfs pin remote add /ipfs/bafymydata2 --service=mysrv --name=myfile2 --background # queue pin request and finish instantly ipfs pin remote ls --service=mysrv --cid=bafymydata2 --status=queued,pinning,pinned,failed ipfs pin remote rm --service=mysrv --cid=bafymydata2 --status=queued,pinning,pinned,failed
More examples can be found under
ipfs pin remote --help.
A few notes:
- Remote pinning services work with recursive pins. This means commands like
ipfs pin remote lswill not list indirectly pinned CIDs.
- By default, only finished, successful pins are listed. To list or remove pending/failed pins, pass explicit statuses: e.g.
- While pinning service data is stored in the configuration file it cannot be edited directly via the
ipfs configcommands due to the sensitive nature of pinning service API keys. The
ipfs pin remote servicecommands can be used for interacting with remote service settings.
- An OpenAPI ipfs-pinning-service.yaml makes it easy to create or generate a compatible client/server. Anyone can implement it and allow for pin management.
- Additionally, HTTP API users now have access to new commands under
🏠 Remote MFS pinning policy
Every service added via
ipfs pin remote service add can be tasked to update a pin every time the MFS root changes:
$ ipfs config --json Pinning.RemoteServices.mysrv.Policies.MFS.Enable
To avoid flooding the remote service with many updates, go-ipfs will send updates at most once every five minutes. Details about customizing behavior of this feature can be found in the configuration docs.
📌 Faster local pinning and unpinning
The pinning subsystem has been redesigned to be much faster and more flexible in how it tracks pins. For users who are working with many pins this will lead to a big speed increase in listing and modifying the set of pinned items as well as decreased memory usage.
Part of the redesign was set up to account for being able to interact with local pins the same way we can now interact with remote pins (e.g. names, being allowed to pin the same CID multiple times, etc.). Keep posted for more improvements to pinning.
🔒 DNSLink names on https:// subdomains
Previously, DNSLink names would have trouble loading over subdomain gateways with HTTPS support since there is no way to get multi-level wildcard certificates (e.g.
en.wikipedia-on-ipfs.org.ipns.dweb.link cannot be covered by
*.ipns.dweb.link). Therefore, when trying to load DNSLink names over https:// subdomains in go-ipfs, we now forward to an encoded DNS name. Since DNS names cannot contain
. in them they are escaped using
https://en-wikipedia--on--ipfs-org.ipns.dweb.link 👈 a single DNS label, no TLS error
Note: The last redirect is specific to HTTPS, and is triggered only when
X-Forwarded-Proto: https header is present.
Recipes for setting up your own public gateway can be found in configuration docs.
💨 QUIC update
QUIC support has received a number of upgrades, including the ability to take advantage of larger UDP receive buffers for increased performance.
Linux users may notice a logged error on daemon startup if your system needs extra configuration to allow IPFS to increase the buffer size. A helpful link for resolving this is in the log message as well as here.
👋 No more Darwin 386 builds
Go 1.15 (the latest version of Go) no longer supports Darwin 386 and so we are dropping support as well.
For a full list of updates included in this release, you can review the changelog within this release post.
Coming soon …
If you’re an IPFS Desktop or IPFS Web UI fan, you’re in luck. These pinning improvements will land soon in GUI form, too — upcoming releases of Desktop and Web UI will allow you to use any remote pinning service that supports the IPFS Pinning Service API.
Thank you contributors!
A huge thank you to everyone who contributed patches and improvements in this release, all 58 of you! We couldn’t have made this happen without your help and feedback. ❤
Install, upgrade, and join us!
There are many ways to get involved with IPFS based on your skill set, interest, and availability. Please check out our contribution page on GitHub for guidance and next steps.
This is an exciting time for IPFS and the web in general. Join us!