Bacalhau Beta v1: WASM Support, Simplified UX and Better Reliability

13 December 2022

Share this item:

Hi all!

Today, we are EXCEEDINGLY pleased to announce the Beta of Project Bacalhau (opens new window). Bacalhau is a distributed compute network for IPFS data, where users can run arbitrary docker containers and Wasm images against data stored in IPFS. With Bacalhau, we plan to take a meaningful step towards unlocking distributed, content-addressed compute for everyone and improving how people interact with the ever-growing amount of information available.

# Background

Since our initial public launch in July, we have heard tons of exciting ways that researchers and entrepreneurs are using Bacalhau. These include:

City of Las Vegas & Blocz.io (opens new window): Blocz IO is excited to partner with Bacalhau to process real-time CCTV data for the City of Las Vegas. Our combined services will significantly reduce costs and scale up the existing safety capabilities for its visitors.
Bacalhau Partners with LabDAO to Accelerate Scientific Progress (opens new window): Our goal is to accelerate progress by making scientific tools more accessible - building distributed compute infrastructure is an essential component on the way there. We are excited to be partnering with the Bacalhau team to integrate this piece of the puzzle together.
Bacalhau Case Studies (opens new window):
- Surface Ocean CO₂ Atlas (SOCAT): Youtube Demo (opens new window), Github (opens new window)
- EUREC4A Cloud Masking: Github (opens new window)
- OpenMM Molecular Simulation: Github (opens new window)

Based on user feedback, we are introducing a breadth of new features to help people adopt compute over data even faster.

# Introducing Bacalhau Beta v1

We are proud to move the Bacalhau API from alpha to beta. With this change, we are committing to both a more stable API as well as backward compatibility for future versions (alpha jobs will not be supported going forward). For most jobs, this will result in no change, but it will require updating the versions of your jobs from v1alpha1 to v1beta1.

Note: This only applies if you are using a JSON or YAML job format - if you are using the CLI, you should be all set.

Along with this change, we have several additional features that address some big requests:

Improvements to network reliability: In particular, by switching the way we are using libp2p gossipsub, you should see significantly reduced network latency.
Native Filecoin support: via native Lotus integration as well as Estuary integration via an Estuary API key (Estuary publishes results from the network both into IPFS as well as backing them up in Filecoin)
Support for WebAssembly: Bacalhau can now deterministically run WASM code either uploaded via the client or stored on IPFS.
Job pipelines from Apache Airflow: Jobs can now be chained together via Airflow, Cron scheduling, and more.

# New Examples

We also have been running Bacalhau through its paces. Below are just a few of the many examples you can already run on Bacalhau against IPFS and/or Filecoin data:

# Roadmap

Our goal is to maintain a quarterly release cadence. By the end of the year, we hope to offer the following:

Python SDK and SDKs for our API in some other languages too
FIL+ Dashboard
Initial simulator framework
Improved examples
A draft invocation spec in collaboration with the CoD ecosystem
Networking design
Trusted Execution Environment design
An initial game theoretic analysis of the Bacalhau protocol

# Would you like to learn more? Come help!

If you would like to learn more about Bacalhau or let us know how you'd like the project to change and help you, visit any of the following:

Website: https://bacalhau.org/ (opens new window)
Docs: https://docs.bacalhau.org/ (opens new window)
Mail: https://groups.google.com/g/bacalhau-discuss (opens new window)
Slack: https://filecoin.io/slack (opens new window) (#bacalhau (opens new window) channel)
Github: https://github.com/filecoin-project/bacalhau (opens new window)

Thank you so much!