Bacalhau Beta v1: WASM Support, Simplified UX and Better Reliability

Bacalhau Beta v1: WASM Support, Simplified UX and Better Reliability

Hi all!

Today, we are EXCEEDINGLY pleased to announce the Beta of Project Bacalhau (opens new window). Bacalhau is a distributed compute network for IPFS data, where users can run arbitrary docker containers and Wasm images against data stored in IPFS. With Bacalhau, we plan to take a meaningful step towards unlocking distributed, content-addressed compute for everyone and improving how people interact with the ever-growing amount of information available.

# Background

Since our initial public launch in July, we have heard tons of exciting ways that researchers and entrepreneurs are using Bacalhau. These include:

Based on user feedback, we are introducing a breadth of new features to help people adopt compute over data even faster.

# Introducing Bacalhau Beta v1

We are proud to move the Bacalhau API from alpha to beta. With this change, we are committing to both a more stable API as well as backward compatibility for future versions (alpha jobs will not be supported going forward). For most jobs, this will result in no change, but it will require updating the versions of your jobs from v1alpha1 to v1beta1.

Note: This only applies if you are using a JSON or YAML job format - if you are using the CLI, you should be all set.

Along with this change, we have several additional features that address some big requests:

  • Improvements to network reliability: In particular, by switching the way we are using libp2p gossipsub, you should see significantly reduced network latency.
  • Native Filecoin support: via native Lotus integration as well as Estuary integration via an Estuary API key (Estuary publishes results from the network both into IPFS as well as backing them up in Filecoin)
  • Support for WebAssembly: Bacalhau can now deterministically run WASM code either uploaded via the client or stored on IPFS.
  • Job pipelines from Apache Airflow: Jobs can now be chained together via Airflow, Cron scheduling, and more.

# New Examples

We also have been running Bacalhau through its paces. Below are just a few of the many examples you can already run on Bacalhau against IPFS and/or Filecoin data:

# Roadmap

Our goal is to maintain a quarterly release cadence. By the end of the year, we hope to offer the following:

  • Python SDK and SDKs for our API in some other languages too
  • FIL+ Dashboard
  • Initial simulator framework
  • Improved examples
  • A draft invocation spec in collaboration with the CoD ecosystem
  • Networking design
  • Trusted Execution Environment design
  • An initial game theoretic analysis of the Bacalhau protocol

# Would you like to learn more? Come help!

If you would like to learn more about Bacalhau or let us know how you'd like the project to change and help you, visit any of the following:

Thank you so much!