IPFS Weekly 9
by Richard Littauer on 2016-04-20
IPFS is a new hypermedia distribution protocol, addressed by content and identities, aiming to make the web faster, safer, and more open. In these posts, we highlight some of the development that has happened in the past week. For anyone looking to get involved, follow the embedded hyperlinks, search the wealth of information on GitHub or join us on IRC (#ipfs on the Freenode network).
If you would like to get this update as an email, sign up for our weekly newsletter!
Here are some of the highlights for the March 21st through March 28th sprint:
Package managers have been a large topic of discussion recently. Mostly, this is because of an issue with an author of some heavily used npm packages unpublishing all of his modules simultaneously. One of these was
left-pad, which was used by thousands of builds globally, all of which broke when the package was removed. A great writeup of what happened is on the npm.js blog here; they took this very seriously, and shortly after changed their unpublish policy as a direct result.
Many people jumped to IPFS as a possible solution to this problem. With a permanent filesystem, unpublishing wouldn’t be possible. Here’s one post titled How to use IPFS to fix npm; here’s an issue on a new GitHub organization, ipmjs, trying to find consensus on how to fix npm using a permanent storage system; here’s an npm module, cowpen that publishes modules directly to IPFS; here’s another decentralized package manager that sprung up using IPFS and Ethereum.
The IPFS community has been thinking about immutable package managers for a long time. IPFS itself began as an immutable package manager, and it is built to make writing them much easier. @diasdavid has a project called
registry-mirror, which allows you to run an npm registry locally that is backed by packages retrieved from IPFS instead of NPM directly. He’s written about a presentation he gave for it, here; the source code is here.
On a similar note, gx, a package manager for Go made by @whyrusleeping, was also mentioned in a lot of the discussions about npm and package managers, especially on Hacker News. In the past two weeks, the project went from 50 to 1000 stars, so people are clearly interested in this now.
We’re using DigitalOcean to provide ipfs.io DNS. On Tuesday, March 24th, DigitalOcean DNS was hit by a severe outage lasting hours, which took the public gateway at ipfs.io down. We switched to DNSimple in an ad-hoc fashion and brought ipfs.io back while DigitalOcean was still down, but this incident obviously hit us on the wrong foot a bit. We’ll be working to never get taken down this way again. It’s HARD not to depend on any single points of failure. Here’s a few things we’ll do:
- codify DNS zones, and tools to upload them to DNS providers
- keep one or two backup DNS providers
- update our monitoring and failover procedures
We’ll post a more detailed post-mortem on our blog in the next few days.
Aye, you might want to check the new js-ipfs Captain.log entry, matey!
Following js-ipfs roadmap, we’re close™ to having a workable js-ipfs version that can be used in the browser and in Node.js. This will mark a very important milestone on the IPFS project and enable a whole set of new distributed web applications to be possible. If you want to be part of this effort, check out our Captain.log entry to get a full update and a list of tasks you can contribute to.
@haadcode has been working on improvements to orbit-db, ipfs-log and Orbit. The message history fetching is now more stable and the UI feedback for loading messages is fixed. All this work will improve the user experience of Orbit.
js-ipfs init works! @noffle finished the remaining pieces this week, including CLI usage. This included a handful of auxiliary PRs that cascaded out of that work. This makes the
js-ipfs init process produce IPFS repos that are compatible with go-ipfs’.
One of the significant contributions made this week was the addition of ‘dictionary’ support for
pako. With this contribution, we are able to have a complete implementation of SPDY 3.1’s framing layer running in the browser, the default stream muxing library used in IPFS. You can find more about this contribution in the following issue and PR discussions:
@whyrusleeping wrote a tool to move content from 0.4.0 to 0.3.11 (see levart-emit). He also discovered a file descriptor leak bug in utp causing connectivity issues, and began work on datastore performance improvements.
jsipfs object cli and http-api endpoints are complete
Now you can use
jsipfs object in the same way you would use
ipfs object. Big thanks to Francisco Dias for leading the last miles of this goal. The complete track of the development can be found at github.com/ipfs/js-ipfs/issues/58.
The infrastructure metrics dashboard didn’t previously have HTTP request/response metrics from nginx’s point of view, but only from IPFS’s and multireq’s point of view. (Multireq is our v04x/v03x multiplexing proxy). Nginx itself provides finegrained metrics only through their commercial subscriptions. We’re now using mtail to parse metrics from nginx access logs and expose them to Prometheus. @lgierth will also contribute the
nginx.mtail program upstream with mtail.
On April 20th, IPFS will host a joint meetup with ConsenSys at MIT in Cambridge, Massachusetts. Sign up here!
First IPFS Meeting in NYC
We had our first IPFS meetup in New York! It went fantastically; expect an upcoming post on the Blog soon.
Meeting with NYC Mesh
@jbenet and @lgierth met with the fine folks of nycmesh.net. For the past two years they’ve been building a community Wifi network in New York City. We had lots of great conversation about wireless mesh networking and IPFS. If you live in NYC, you should come attend their meetups!
Last Monday members of the IPFS comminuty attended a blockchain workshop event held by COALA, “a collaboration between academics, lawyers, technologists and entrepreneurs who have been driving research, policy and infrastructure-building in the blockchain ecosystem for the past three years” at the New York University School of Business. @diasdavid @haad @noffle and @nginnever were in attendance as @jbenet was a part of a protocols panel, speaking on scalability and the future of blockchain technology. A recording of the event should be available on youtube in the future here.
Lisbon Research and Development Meetup
The IPFS Lisbon community had their second “Research & Development Meetup”, hosted by Uniplaces (https://www.uniplaces.com). The focus was “The Distributed Web” (Slides) and “Machine Learning + Artificial Intelligence for Recommender Algorithms”, with talks by David Dias and João Ascensão, respectively. If you are around Lisbon, make sure to join http://www.meetup.com/ipfs-lisbon-meetup to get notified about the next one. Resources for this talk can be found here.
BitCoin news had a discussion on using IPFS and Bitcoin for Decentralised Citizen Journalism. Check it out!
IPFS Meme of the Week
From https://twitter.com/jplur_/status/712670265919086594. Thanks, jplur_!
Content-Type of the Week
Now that we’re using mtail to make better sense of nginx serving the IPFS-to-HTTP gateway, we can graph the frequency of content types served. We’ll showcase interesting content types served from the gateway in the coming weeklies.
The first Content-Type of the week is: application/x-chdr, which signifies a C source header file.
Across the entire IPFS GitHub organization, the following people have committed code, created issues, or made a comment on GitHub between March 21st (noon, GMT) and March 28th. We’re autogenerating this list using this tool and this other tool, so please let us know if your name isn’t here.
- @achin (Andrew Chin)
- @AdamStone (Adam Stone)
- @admiral-Guck (Gucky)
- @alexAubin (Alexandre Aubin)
- @amstocker (Andrew Stocker)
- @anarcat (anarcat)
- @Anthonyzou (Anthony Ou)
- @Azulan (Frank Flores)
- @balupton (Benjamin Lupton)
- @bgrieder (Bruno Grieder)
- @cb0 (Marcus Puchalla)
- @chinshiketsu (Sijie Chen)
- @chriscool (Christian Couder)
- @CMCDragonkai (Roger Qiu)
- @ComSecNinja (Timo Virkkunen)
- @darkstar (Michael D.)
- @davidar (David A Roberts)
- @diasdavid (David Dias)
- @dignifiedquire (Friedel Ziegelmayer)
- @djdv (Dominic Della Valle)
- @ehmry (Emery Hemingway)
- @fomkin (Aleksey Fomkin)
- @gerrickw (Gerrick W)
- @greenkeeperio-bot (Greenkeeper)
- @haadcode (Haad)
- @harlantwood (Harlan T Wood)
- @hshimo (Hiroshi Shimo)
- @ianopolous (Ian Preston)
- @iav (iav)
- @ibnesayeed (Sawood Alam)
- @insanity54 (Chris Grimmett)
- @jbenet (Juan Benet)
- @JGAntunes (João Antunes)
- @josephholsten (Joseph Anthony Pasquale Holsten)
- @julian-smith-code (Julian Smith)
- @jwsher (Justin Sher)
- @kalmi (Tarnay Kálmán)
- @kevina (Kevin Atkinson)
- @knocte (Andres G. Aragoneses)
- @KrishnaPG (Gopalakrishna Palem)
- @Kubuxu (Jakub Sztandera)
- @kumavis (kumavis)
- @kyledrake (Kyle Drake)
- @kyrias (Johannes Löthberg)
- @lgierth (Lars Gierth)
- @lidel (Marcin Rataj)
- @machawk1 (Mat Kelly)
- @matshenricson (Mats Henricson)
- @Mec-iS (Lorenzo)
- @micahredding (Micah Redding)
- @MichaelMure (Michael Muré)
- @micxjo (Micxjo Funkcio)
- @mildred (Mildred Ki’Lya)
- @mitar (Mitar)
- @monoflash (Alex Geer)
- @mpgn (Martial Puygrenier)
- @mrshu (Marek Šuppa)
- @nginnever (Nathan Ginnever)
- @nicola (Nicola Greco)
- @NightRa (Ilan Godik)
- @noffle (Stephen Whitmore)
- @nonchip (Kyra Zimmer)
- @palkeo (palkeo)
- @pguth (Per)
- @polkovnikov-ph (Philip Polkovnikov)
- @puffnfresh (Brian McKenna)
- @RichardLitt (Richard Littauer)
- @se3000 (Steve Ellis)
- @SilverWingedSeraph (Leo Tindall)
- @Stebalien (Steven Allen)
- @tadfisher (Tad Fisher)
- @tinybike (Jack Peterson)
- @voltrevo (Andrew Morris)
- @whyrusleeping (Jeromy Johnson)
- @willglynn (Will Glynn)
- @wking (W. Trevor King)
- @wmhilton (Will Hilton)
- @xicombd (Francisco Baio Dias)
- @xsnk (Sankrant Chaubey)
- @yuvipanda (Yuvi Panda)
This newsletter is also a community effort. If you have cool things to share for the next weekly, drop a comment about it in the next weekly sprint issue! The more people mention items they want to see in the weekly there, the easier it is to make this and send it out.
Thanks, and see you next week!
- Richard Littauer