For some challenges, this hackathon features an automatic benchmarking system. We recommend that at one person from each team starts working on configuring this at the beginning of the hackathon, so you know how well your team is doing during the hackathon.

Page content

• Getting Started
• Benchmarking and Scoreboard
• How Benchmarking Works
  • Overview of Development Flow
  • Compute Limits
  • Testing your Predictor Locally
  • Challenge Repository File Structure
• FAQ
  • My code doesn't get benchmarked - what do I do?

To view the individual scoreboards, see:

• https://biolib.com/biohackathon/optimal-ph-scoreboard/
• https://biolib.com/biohackathon/tcr-pmhc-scoreboard/
• https://biolib.com/biohackathon/protein-language-scoreboard/
• https://biolib.com/biohackathon/antibody-pairing-scoreboard/
• https://biolib.com/biohackathon/structural-solubility-scoreboard/

If your challenge is not in the above list, it does not use the benchmarking system and you can skip this guide 👌

Getting Started

Note: If you run into problems regarding the benchmarking system, please join the #tech-support channel on Slack 💁

1. At the beginning of the hackathon, your team should fork the challenge's repository on GitHub. In this way, you get a new repository where you can collaborate on your submission.
2. After forking the repository, you should have a file called .github/workflows/ci.yml . This is setup to automatically, on every commit to the main branch, push new updates to the benchmarking system.

   1. To activate the workflow script, go to the Actions tab → Enable Workflows

   2. Set the following two environment variables using GitHub Secrets. To do this, go the repository Settings → Secrets → New repository secret:

      1. Make a secret with name BIOLIB_TOKEN and the value should be a valid API token. To generate one, go to https://biolib.com/settings/api-tokens/ and click "Create New +".
      2. Make a secret with name BIOLIB_PROJECT_URI where the value should be the URI of your hackathon submission. If the url of your team page is https://biolib.com/TeamName-abcd/TeamName , set BIOLIB_PROJECT_URI to TeamName-abcd/TeamName.

      Note: Once a repository secret has been added it cannot be viewed again, only replaced.

3. In your GitHub repository, go to submission/README.md and add your team name (make the commit on the main branch). This change will trigger a new build, and if things are setup correctly, you should see your change at https://biolib.com/TeamName-abcd/TeamName 🎉
   • As a bonus you now have a web application that can be used to run predictions easily from the browser! Note: Chrome/Firefox/Edge are supported browsers