GlirOS 0.1.4
The model benchmark: the same test for every model
Desktop app
- Settings > Benchmark runs the same test on any model, local or cloud, and gives a Quality score, a Speed score on this PC and one line per category, from maths and coding to tools, long chats and translation.
- Coding, tools and plans are scored with GlirOS's help and without it, so you see what GlirOS adds for small models. Code answers are run and tested in a sandbox.
- The full run has 161 questions in all 17 categories, including reading pictures, dictation and reading aloud, which use only what is already on this PC.
- The test texts hold nothing of yours and nothing goes into your chats; chatting waits while it runs. A cloud model asks first and shows the estimated tokens, and the result shows what the provider counted.
- Results can be compared side by side, exported as a picture, a file, text or a table, and imported from another PC. Compare models in the chat is now a beta.
Tools
- Trip planner 1.2.1: a Search flights button on flights still to book opens Google Flights with the route and dates, after GlirOS asks.