js-ipfs 0.41.0 released

by Alan Shaw on 2020-02-13

🔦 Highlights

One giant leap forward

🦁 Async Await and Async Iterables

🎶 In the jungle, the mighty jungle the lion sleeps tonight!

🎶 async await, async await, async await, async await…

We’ve completed a HUGE refactor to js-ipfs internals 🥳, switching to using Promises and async/await over Callbacks and using async iterables instead of Node.js Streams and Pull Streams. Ok, I tell a lie, it’s actually not just js-ipfs internals, it’s the whole stack, including libp2p, IPLD and multiformats, you might call it a ground up re-write, but, you know, we don’t like to brag.

🚨 Oh, wait, serious note - this release brings big breaking changes to the core API so please consult the “API Changes” section below for all the information.

It’s been a long and emotional voyage but this refactor brings a plethora of incredible immediate and future benefits to consumers, contributors and core developers that make all the work and effort worthwhile. To summarise, we’ve:

Using async/await in JavaScript is gaining a lot of traction in the ecosystem and is rapidly becoming the de facto way of writing idiomatic JS. We want js-ipfs to move with the times and continue to be attractive to contributors by using modern JS features, techniques and practices. The big idea with these changes is for the code to be easier to contribute to, easier to understand, easier to maintain, and be faster and smaller than ever.

This change is so big, and so significant, we wrote a whole blog post about it to explain the motivations behind the changes…in excruciating delightful depth 🤣!

We’ve also compiled some stats on this refactor for your viewing pleasure:

…and a lot of those stats are just for js-ipfs and js-ipfs-http-client - the tip of the iceberg! We saw changes similar to this for between 60-70 dependencies across IPFS, libp2p, IPLD and multiformats.

🌗 UnixFS v1.5

Turns out, it’s really important for package managers to retain file metadata, particularly last modified time (mtime). File mtime allows them to selectively sync only data that has changed. Up until now if you wanted to host a large data set on IPFS, like a package manager’s repository, it would be difficult to update.

“What about the permanent web?” I hear you cry. Well, this absolutely doesn’t prevent a particular snapshot of a package manager’s repository from being permanently available. Metadata just enables diffs to be imported, instead of the whole thing. So, when I say “difficult to update”, like I did up there, I mean slow and/or impossible. When you have Terabytes (or more) of package data and someone publishes a new package, it’s kinda inconvenient to import everything again, when only a little part changed. File mtime is a really good indicator of which things have changed, so you can use it in IPFS now! 🥳

For example, there’s two new options to jsipfs add that allow mode and mtime to be preserved as the file is added to IPFS:

$ jsipfs add -r --preserve-mtime --preserve-mode ~/Desktop/gif
added QmT6WX9McZyx5ZoisRgpsjYKDBWnYpMnBLpfAgjW5kavBA gif/yesthisisdog.jpg
added QmXMrFfZ9zHLZKN7xP2dX76YFFhvBJsQkd4fLnTDkyR31Q gif

Ok, no big changes there aside from the new options, buuut, now when you list directory contents you get Mode and Mtime info:

$ jsipfs ls QmXMrFfZ9zHLZKN7xP2dX76YFFhvBJsQkd4fLnTDkyR31Q -v
Mode       Mtime                           Hash                                           Size  Name
-rw-r--r-- Apr 16, 2018, 12:20:33 PM GMT+1 QmT6WX9McZyx5ZoisRgpsjYKDBWnYpMnBLpfAgjW5kavBA 87779 yesthisisdog.jpg

Rad right!?

Persisting the file mode is also super rad, because it opens up NFS type use cases that weren’t possible before. Imagine your node_modules directory is backed by IPFS and mounted on your file system - the file mode will allow everything in node_modules/.bin to be executable as you’d expect.

The coolest thing about all of this is that it’s completely backwards compatible. The CID for a given file/directory only changes if you opt in to metadata, otherwise the CIDs remain the same. Hooray!

There’s a bunch of changes that add metadata capability to the CLI, HTTP and core API both for inputs and outputs. There’s also a couple of new MFS commands touch and chmod which allow you to change the metadata whenever you like! Magic 🧙‍♂️.

See the API Changes sections below for details of all the new UnixFS v1.5 stuffs.

🏗 API Changes

Core API

There are significant and breaking core API changes in this release. Please see the migration guide.

HTTP API

CLI

Other changes

❤️ Huge thank you to everyone that made this release possible

🙌🏽 Want to contribute?

Would you like to contribute to the IPFS project and don’t know how? Well, there are a few places you can get started:

⁉️ Do you have questions?

The best place to ask your questions about IPFS, how it works and what you can do with it is at discuss.ipfs.io. We are also available at the #ipfs channel on Freenode.

Comments