libp2p in 2020

by The libp2p Team on 2020-06-09

If you’re doing anything in the decentralized, peer-to-peer space, you’ve probably heard of libp2p - a modular networking stack for peer-to-peer applications. libp2p consists of many modular libraries from which p2p network developers can select and reuse just the protocols they need, while making it easy to upgrade and interoperate between applications. This helps web3 developers get up and running faster, build more resilient decentralized applications, and utilize advanced features like decentralized publish-subscribe and a distributed hash table.

What makes libp2p different from the networking stack of today is its focus on transport agnosticism, modularity, and portable encodings (like multiaddr). These properties make libp2p the networking layer of choice for most new dweb projects, blockchains, and peer-to-peer applications. Read more about why projects are choosing to build on libp2p, or watch the recent talk from Tech Lead Raul Kripalani at DevCon5.

2019 Growth

The past year has seen amazing improvements to libp2p with the protocol quickly becoming the de facto web3 networking layer for a huge swath of projects.

Probably the most significant news on that front is the official adoption of libp2p as the networking layer for the Ethereum 2.0 Network Specification. Ethereum 2.0 is the next generation of Ethereum, with a large and thriving network of clients, dapps, and developers. A big focus for libp2p in 2019 was leveling up stability, scalability, and ease of use to meet the requirements of such a large scale blockchain.

libp2p at DevConV

libp2p now has 7 language implementations:

The other major new blockchain building on libp2p is Filecoin which plans to launch their public storage and retrieval network this summer. With over 7.5 PB of storage already live across an active community of testnet miners, a big focus for libp2p in 2019 was landing the performance and hardening improvements to meet Filecoin’s needs.

In addition to Filecoin and Ethereum 2, there are now more than two dozen projects building directly on libp2p, including: Polkadot and Parity Substrate, Nervos, Status.im, Paratii, 0x, Coda, Enigma, Flow, Harmony, Golem, Keep Network and Validity Labs. Groups like Matrix.org are also in the early stages of adding libp2p support to their applications, helping their users benefit from the resilience and openness of p2p networking.

These newcomers to the libp2p ecosystem are a big deal - but they actually still make up less than half of all active libp2p nodes! The vast majority of libp2p end users come from dapps, tools, and projects using libp2p through IPFS, the InterPlanetary File System!

Millions of end users and over 100,000 libp2p nodes participate in the IPFS network every week. Organizations like Opera, Ethereum, Brave, Netflix, Microsoft, and Cloudflare have joined the IPFS ecosystem, and hundreds of nascent community-owned projects like Audius, Fleek, Textile, Qri, Pinata, OpenBazaar, Berty, Infura and 3Box are building their businesses with IPFS.

2019 and early 2020 features

The focus for the year was hardening and evolving the stack - and we blew that out of the water! Here are some of the features the core team and our community have contributed to libp2p:

Releases

Community

We saw huge growth in the libp2p team and community this year, with presentations and workshops at a variety of conferences including EthCC 2, Web3 2019, EthBerlin Zwei, IPFS Camp, ReadyLayerOne, DevCon V (talks: state of libp2p, gossipsub), and more.

The libp2p project also launched a development grants program. The initial kickoff for the program at EthBerlinZwei led to several significant features being implemented by the community, including Noise handshakes in Go and Lua Wireshark dissectors. The bounties program is a great way to get involved in the libp2p project. Check it out on GitHub!

2020 H1 in review

Throughout the first half of the year, our top focus was supporting the needs of current and in-flight adopters, especially IPFS, Eth2, and Filecoin. So far, we have delivered on the following.

Content routing and connectivity

To support IPFS and all our other dapp users out there, we’ve improved libp2p’s content routing and connectivity to make finding and sharing content in distributed networks fast and reliable. This meant we needed to dive deep into major systems like the libp2p DHT, providing subsystem, NAT traversal, and more to optimize and refactor for performance. We’ve been collaborating closely with Protocol Labs’ IPFS and Testground teams to make this a reality. Read more about our progress in the IPFS 0.5 launch post, and our path to get there in the Road to the New DHT post!

Content Routing Speed

Launching Testground to the world

What is Testground you ask? Only the coolest new testing tool for large-scale p2p networks! When starting our network upgrade plans, it was clear that the scope and complexity of changes we’d need to make to core IPFS network functions would be impossible to optimize without powerful testing tools to simulate each change to network performance at scale. We couldn’t find a tool with the scale and configurability we needed, so we built the testing tool we needed, and made it modular and system agnostic just like libp2p so other p2p networks can take advantage of this new OSS infrastructure too!

Launching Testground

Gossipsub security enhancements

For our large blockchain users, the focus of Q1 was stability, scalability, and security hardening. We designed and implemented a new version for the Gossipsub protocol, v1.1. This new P2P PubSub router includes several security extensions that add a protection to Sybil, eclipse, and spam attacks.

This work is of extreme importance as our adopters (e.g., Filecoin and Ethereum 2.0) require a secure messaging layer to distribute their time sensitive and valuable data, without being vulnerable to malicious actors.

You can find the specification and implementation on GitHub. We will be releasing a report that fully describes how we identified the attack vectors, developed the motivations and went through implementing, testing, evaluating under multiple network conditions and got it audited. Stay tuned.

Gossipsub v1.1

Going forward

Both of these workstreams (content routing performance and gossipsub hardening) continue to be top priorities with IPFS having launched go-ipfs 0.5 (their largest network upgrade in years), and Ethereum and Filecoin gearing up for Q3 mainnet launches, as well as other adopters such as Polkadot (who went live recently!), Flow, Coda, 0x, Matrix, Keep Network, Enigma, Harmony, Golem, Validity Labs, etc., reaching new milestones.

Furthermore, in H2 we plan to work on protocol negotiation enhancements, a more efficient connection boostrapping flow, an implementation alignment framework (including a long-awaited conformance test kit), code refactors, and a lot more!

Growing the team

Protocol Labs’s small and scrappy libp2p core team transformed into a true force of nature with some great additions last year. Our doors will continue to be open for great people from both inside and outside of the ecosystem.

In particular, we’re looking for a new Project Lead to help grow and steward the libp2p project and ecosystem to new heights in 2020. Check out the role description for more specific details and information on where you can apply. In particular, we’re looking for candidates with proven leadership experience, a passion for p2p tech, and an interest in scaling the libp2p community as it becomes critical infrastructure for new, large users such as Ethereum (with launch of the Beacon chain) and Filecoin.

Excited about building the networking stack for a more resilient, upgradeable, open network for humanity’s knowledge? We’d love to get in touch! Apply, or just shoot mike@libp2p.io a quick email with your thoughts or questions!

Thank you

2019 was a fantastic, high growth year for the libp2p project and 2020 is already well on track to be even more exciting. Huge thank you to the thousands of community members who contributed to all the new language implementations, feature improvements, performance speedups, and amazing projects bringing libp2p directly to end users.

Thank you for your support! We’re excited to continue working with you to build libp2p.

Appendix

References:

Comments