Objective

Store important open datasets on the Filecoin network, ensuring that they are preserved for many generations to come.

Overview

The Slingshot program started off as a collaborative-competitive community program to accelerate the storage of real, valuable open data on the Filecoin network, rewarding participants along the way. Slingshot v3 is the next significant iteration of the program, with the intent of storing open datasets from a variety of different sources and taking the learnings from Slingshot Restore and Evergreen to emphasize the permanence of the data onboarded.

Slingshot v3 will engage individuals in a new role - “Data Preparers” (referred to as DPs), who are responsible for the downloading, processing (turning raw data into CAR files - see https://spec.filecoin.io/#section-systems.filecoin_files.piece.data-representation for more), and distribution of CAR files to Storage Providers. This is a significant difference from Slingshot v2, where a “Client” had to act as both as the “Data Owner” (the beneficiary of having the data stored in the network) and the “Data Preparer," whereas in Slingshot v3, the “Data Owner” can simply signal high quality and valuable public datasets to the program and benefit from the fact that they are stored in the network, and a separate “Data Preparer” (i.e. anyone in the ecosystem) can be rewarded for facilitating replication of the data to Filecoin.

Deals will be issued by the Slingshot Deal Engine to all qualified Storage Providers (SPs) with access to these CAR files. All deals will be verified deals (Fil+). The program plans to meet all Fil+ requirements and set the standard for future deal distribution engines / brokers.

SPs have to sign-up and go through a lightweight KYC process (familiar for those who are participating in the Slingshot Evergreen program) to be enabled to receive deals from the engine. SPs are expected to maintain a high standard of retrieval success rate to continue being issued new deals from the engine.

Main changes from v2

For those of you that are familiar with Slingshot v2 and may have participated in the program over the last 18 months, here are the key differences in this update:

  • Phases are now named Seasons. Season 1 (v3.1) will last for ~3 months initially.
  • This is a rolling effort to store data in Filecoin. The competition structure from v2 phases has now been replaced with rewards based on replica building, borrowing learnings from Slingshot Restore.
  • Slingshot “clients” have now been replaced with Data Preparers (DPs). DPs pick datasets, download data, process it (divide it up and turn it into CAR files), and upload/distribute CAR files to deal-eligible SPs. DPs are compensated based on how many full replicas of their processed datasets are onboarding to the network.
  • Data preparation and deal making will be consistent across every participant. V2 allowed clients to process data in whichever way they preferred. V3 comes with tooling that makes this easier and more consistent across DPs.
  • Deals are made via a DP-independent deal engine, not by individual clients. DPs coordinate with SPs to get CAR files to them, but deal proposals are issued by the Slingshot Deal Engine. This deal engine is the client and the community shares a single DataCap application.
  • SPs have to be eligible to receive deals from the engine. This includes registering to participate and maintaining a >85% rolling retrieval success rate for free retrievals from a sampling bot (similar to Slingshot v2) and other participants in v3.

How to participate

You can participate in Slingshot v3 as a DP, SP, or both.

Participating as a Data Preparer

DPs are responsible for:

  • Obtaining the data from the source
  • Correctly processing and preparing data for storage (using provided tooling)
  • This includes uploading metadata about CAR files that will be onboarded
  • Hosting/distributing CAR files to SPs to enable data replication on the network.

DPs will be rewarded for their efforts based on the data onboarded and replicated during the course of a season. DPs will be eligible to receive a maximum of 1000 FIL/PiB per dataset. Replicas need to be geographically distributed (see FAQ) and will be stored with eligible SPs.

  • 200 FIL/PiB (prorated for dataset size) for accessible CAR files, uploaded manifests, and first replica onboarded
  • 100 FIL/PiB (prorated for dataset size) for each replica, 2 through 6 (total of 500 FIL/PiB) onboarded within 90d of the first deal on-chain
  • 300 FIL/PiB (prorated for dataset size) for the 7th replica onboarded within 90d of the first deal on-chain

If you would like to participate as a DP:

  • read through the Requirements to understand expectations for DPs
  • Register (top right) to participate and pick up to 3 datasets from the already available list that you’d like to start working on
  • Follow the instructions on the Requirements page and get preparing!

Participating as a Storage Provider

Any SP can participate in the program, including:

  • SPs that are also DPs in the program, and are looking to store data they have prepared
  • SPs that are able to procure the data from the DP or retrieve the data from another SP in the network

SPs participating in the program should be:

  • eligible to win block rewards (have >10TiB of power)
  • set to receive offline-flow (off-network data transfer) deal proposals
  • able to activate deals within 72h of receiving proposal
  • continuously dialable by network clients to remain eligible to receive deal proposals from the deal engine
  • using the index provider

If you are interested in participating as an SP in Slingshot v3, please check out the "Participating as an SP" section on the Requirements page.

FAQ

What are the distribution requirements for replicas? CIDs will be stored up to 10 times, pursuant to the following distribution requirements:

  • Max 1 replica per datacenter/city
  • Max 3 replicas per country
  • Max 4 replicas per continent

CIDs will be eligible for deal proposals for SPs with the following limitations:

  • Max 1 replica per minerID
  • Max 3 replicas per SP organization

For efficient replica building, using just retrievals on Filecoin to distribute data to SPs may not be ideal, so DPs should consider alternative methods of hosting CAR files.

Who’s making the deals? Deal proposals will be issued to eligible SPs that request for them via the Slingshot Deal Engine. This engine is just the Evergreen Deal Engine upgraded with support for multiple deal-issuing tenants (including Slingshot Evergreen, v3, and others). You can see more about the Engine here: https://github.com/filecoin-project/evergreen-dealer.

What happens at the end of each season? Right now, the main function of a season is to serve as a forced check-in point for progress in the program. This includes collecting feedback from participants and making changes to the program design, finalizing and issuing rewards, resetting dataset assignments to participating DPs, and sharing progress with the community. As we move towards building the long term mechanism for permanent open data storage, seasons will likely evolve as a construct.

Dataset assignments can be extended into the following season through coordination with the Slingshot ops team. This will happen in cases where a DP has verifiably finished downloading the dataset, i.e., sharing screenshots and checking uploads of CAR file metadata to Web3.storage, or has already finished preparing it, i.edeals showing up on chain with appropriate CIDs.

For how long should a DP make CAR files available for download? Based on the current design for rewards, DPs are incentivized to get maximum replicas (up to 7) stood up in a 90d window of active dealmaking. Effectively, DPs are likely to use a combination of hosting CAR files and coordinating with SPs to download them directly or retrieve them to parallelize the distribution of their pieces within that 3mo period to get replicas stood up. If a DP decides to host CAR files at http/https endpoints, they should be hosted for either (1) as long as it takes for the target number of replicas to be built or (2) <=90 days. The program design is likely to continue evolving, so this may change in future seasons.

More questions? If you have any questions, please ask them in the #slingshot Filecoin Slack channel. If you run into any issues with the process or the deal engine, please file an issue using the following link.