BLOOM (BigScience Language Open-science Open-access Multilingual): the BigScience 176 billion parameters model is currently training.

The training started on March 11, 2022 11:42am PST and will last 3-4 months on the 416 A100 GPUs of the Jean Zay public supercomputer

Follow the training at And the tensorboard, detailed model card and intermediate checkpoints at Send questions about the training to bigscience-large-model-training [AT] googlegroups [.] com

More information on the BigScience project: start here BigScience


  1. The model:
  2. The dataset:
  3. The engineering side:
  4. Environmental considerations