It all started a little way back with LLaMA and then the outstanding LLaMA 2 the Large Language Model from Google that was released as a platform in open source for others to build on.
Now Google is well and truly back in the opensource LLM world with its new model Gemma!
Gemma has been trained on 6 million tokens compared to LLaMA 2's 4 million.
Gemma has been released with 4 new models.
The Base Models
Firstly a base model with a 7 billion or 2 billion parameters model. variants
This offers raw text processing and all-purpose general capabilities.
The Instruction-Tuned Models
Secondly the 'instruction-tuned' model which also has 7 billion and 2 billion parameters model variants.
This offers the ability to specifically follow instructions and get tasks done effectively.
Google describes these models and versions as a 'family' of lightweight, opensource models built from the Gemini models.
Advantages
So what does this all mean?
Firstly Google says they are light-weight meaning they can run on your own machine, your laptop or on a cloud platform.
Secondly, they are 'open-source' meaning they are available free for development.
Thirdly they are performing well in tests - see sites like Hugging Face.
Next they are pre-trained and tuned meaning different versions can address very specific needs. Customisation can be further addressed with tools like LoRA.
On the downside for the moment
The models are not multimodal meaning they deal with text-in and text-out tasks and currently only operate in the English language.
Google will no doubt continue to improve and go for the next version but this is a big step forward.
If you would like us to help you with your digital solutions journey then contact us!
