Circulars

While Nayomi is splitting the documents to feed into the large language model, I have been searching for which large language model would be best suited to this work. Which has led me to the question, what criteria should we be judging an LLM on?

Is it about…

• Financial cost of the model
• Environmental cost of the model (both in the size and the way it was trained)
• Speed of the model
• Public availability of training data, or inclusivity or the training data (Cohere is designed to understand a range of accents and languages)
• Secureness of the model (PaLM, by google, is designed to specialise in data security)
• Or something else?

When thinking about the project as a whole, should we be…

• Aiming to trial as many models as possible (we are a research project, after all)
• Aiming to use a consistent model type to enable better comparison between digital techniques, not interrupted by the additional confusion of different models
• Aiming to compare different models within the same investigations
• Aiming to reduce how many models we use, to reduce our environmental impact?

The questions go on. Should we use tried and tested tools that we are familiar with (GPT, LLAMA), or do we engage in the newer tools that show promise (Falcon, which is suddenly at the top of Hugging Faces comparison leader board, or BLOOM, which is designed to be more environmentally friendly)? Is it ethical to try both a trusted tool and an experimental one, and compare the results?

Nayomi and I will soon actually be feeding data into a large language model, and so these questions will have to be swept aside in the name of ‘actually getting stuff done.’ But I still think they’re prudent, and welcome any thoughts.

Comms and computer vision

Don’t listen to me – go read Kaspar’s wonderfully detailed discussion of the work, instead: (1) Weaving Communication Objects (notion.so)