IPFS Weekly 7
by Richard Littauer, Andrew Chin on 2016-03-17
IPFS is a new hypermedia distribution protocol, addressed by content and identities, aiming to make the web faster, safer, and more open. In these posts, we highlight some of the development that has happened in the past week. For anyone looking to get involved, follow the embedded hyperlinks, search the wealth of information on GitHub or join us on IRC (#ipfs on the Freenode network).
If you would like to get this update as an email, sign up for our weekly newsletter!
Here are some of the highlights for the second week of March:
Orbit, a distributed, peer-to-peer chat application built on IPFS, is back in active development and going through a major code base overhaul. It now uses orbit-db as it’s database layer. orbit-db is a KV-store and Event Log on top of IPFS which allows developers to use IPFS as a database. Last week orbit-db got big performance and stability improvements and is now using CRDTs for eventual consistency.
A nasty bug in our networking code was fixed this week. An issue in yamux (our primary stream multiplexer) would cause code to hang when opening a new stream if there were too many in flight stream opens. As a result of this, large file transfers (
ipfs refs -r,
ipfs get, and so on) would hang.
IPFS 0.4.0 is now very close to shipping, we ran @dignifiedquire’s randor tool quite extensively and are more confident in the repo operations that have changed since 0.3.11. The release now has a somewhat short checklist of things that are blocking the official release.
As a small side project, @whyrusleeping started benchmarking each of our possible datastore implementations with ds-bench. The results will help us improve the performance of our storage moving forward. On that same topic, he started an experiment in new and exciting datastores and wrote a datastore implementation that can be backed by a sql database. @whyrusleeping then spun up a postgresql database (from docker) and ran the benchmarks against it, without any sort of tuning the initial performance metrics showed that it was around three time as fast as our current on-disk datastore. (Note: the metrics are simply testing the performance of writing random 256k binary blobs to each datastore).
The js-multiaddr implementation was upgraded, now that it now matches go-ipfs features. What this means is that js-multiaddr now supports protocols that need to be encoded with varints. A varint is a integer that uses only the number of bytes it needs to be described by using a continuation bit (more details). Before this addition, we were not able to declare http, websockets or even IPFS multiaddrs because these protocol have a code that doesn’t fit in a byte - now we can. You can check the encoding table here.
mafmt, a module that @whyrusleeping wrote and which stands for multiaddr format, filters out different formats to use with certain transports. Thanks to @daviddias, this now does uTP and WebSockets validation, as well.
The new libp2p-swarm has been released; to reiterate last week, the module has been overauled to have a new API, to be agnostic about which transport an application has to use, and also to have more tests. This also exposes more internal processes, which makes error checking and logging more useful. Libp2p will eventually help users traverse IPFS for any node which allows their desired transport, meaning that we can have a more connected and less brittle web.
@jbenet was in Boston this week for consensus research discussions. While in Boston, @jbenet visited the Digital Currency Initiative at the MIT Media Lab, where he spoke about IPFS and its implications for blockchains, for digital publishing, and for access to information.
c-base/ipfs-ringpin is a new set of tools for setting up an IPFS “file pinning ring” between multiple parties. This can be used, for example, for hackerspaces to provide redundancy by pinning each other’s files. pipermerriam/ipfs-persistence-consortium and VictorBjelkholm/pincoop do similar things.
@jbenet gave a talk titled Datastructures In and On IPFS at last year’s QCon SF. The video was released this week, so now you can all see it! The talk includes a typical introduction (skip if you know IPFS well), and then dives into datastructures, including Merkle Links, Mazieres links, how IPNS works, IPRS records, versioning, Keychain (PKI on IPFS), Persona (identity), and more.
The IPFS powered USB deaddrop at c-base was demonstrated at the LoganCIJ 2016 Symposium, in the Investigative Journalism conference. More pictures here. If you want to build your own IPFS USB deaddrops, check out the c-base project here: c-base/ipfs-deaddrop.
Several core IPFS developers will be in NYC from March 28th to April 8. We will be organizing an IPFS Meetup within that time frame. If you would like to come, please follow this github issue for the final details. If you would like to present, post in that issue.
Across the entire IPFS GitHub organization, the following people have committed code, created issues, or made a comment on GitHub between March 7th (noon, GMT) and March 14th. We’re autogenerating this list using this tool and this other tool, so please let us know if your name isn’t here.
- @42 (42 Coffee Cups)
- @anarcat (anarcat)
- @atondwal (Anish Tondwalkar)
- @basilmon92 (Basil M Kuriakose)
- @bergie (Henri Bergius)
- @brailateo (Constantin Teodorescu)
- @candeira (Javier Candeira)
- @chriscool (Christian Couder)
- @christianlundkvist (Christian Lundkvist)
- @clkao (Chia-liang Kao)
- @Crest (Crest)
- @cryptix (Henry)
- @davidar (David A Roberts)
- @DavidBooher (David Booher)
- @deawar (Dean Warren)
- @deb0ch (Thomas de Beauchêne)
- @diasdavid (David Dias)
- @dignifiedquire (Friedel Ziegelmayer)
- @djdv (Dominic Della Valle)
- @doesntgolf (Nate Dobbins)
- @dylanPowers (Dylan Powers)
- @eminence (Andrew Chin)
- @greenkeeperio-bot (Greenkeeper)
- @gritzko (Victor Grishchenko)
- @haadcode (Haad)
- @ianopolous (Ian Preston)
- @jbenet (Juan Benet)
- @klauswuestefeld (Klaus Wuestefeld)
- @knocte (Andres G. Aragoneses)
- @krl (kristoffer)
- @Kubuxu (Jakub Sztandera)
- @kyledrake (Kyle Drake)
- @lgarron (Lucas Garron)
- @lgierth (Lars Gierth)
- @matshenricson (Mats Henricson)
- @micahredding (Micah Redding)
- @MichaelMure (Michael Muré)
- @micxjo (Micxjo Funkcio)
- @mildred (Mildred Ki’Lya)
- @mklarmann (mklarmann)
- @mnp (Mitchell Perilstein)
- @Nadziv (Shahinaz)
- @nginnever (Nathan Ginnever)
- @nicola (Nicola Greco)
- @noffle (Stephen Whitmore)
- @obvio171 (Helder Ribeiro)
- @pietsch (Christian Pietsch)
- @plungu (Paul Lungu)
- @RichardLitt (Richard Littauer)
- @SCBuergel (Sebastian C. Bürgel)
- @sexybiggetje (Martijn de Boer)
- @sivachandran (Sivachandran)
- @Stebalien (Steven Allen)
- @suisha (David Mai)
- @Unode (Renato Alves)
- @wasserfuhr (RainerWasserfuhr)
- @whyrusleeping (Jeromy Johnson)
- @xicombd (Francisco Baio Dias)
- @yncyrydybyl (Yan Minagawa)
- @yuvallanger (Yuval Langer)
This newsletter is also a community effort. If you have cool things to share for the next weekly, drop a comment about it in the weekly sprint issue! The more people mention items they want to see in the weekly there, the easier it is to make this and send it out.
Thanks, and see you next week!
- Richard Littauer and Andrew Chin