Let me guess, you are familiar with Large Language Models or LLMs, or at least you have heard of them. An example is GPT, which is the backbone of ChatGPT. A part of the current AI revolution is coming from large language models. Speaking of LLMs, a critical question is, what is a small language model, or SLM?
A small language model generally refers to a language model with a lot fewer parameters, simpler architecture, or limited training data compared to LLMs. Small language models are faster to train and use less computational resources. SLMs typically have less capability to understand and generate complex language.
Can we create a ChatBot using a small language model? Yes, that is exactly what we are going to do today, we will write a ChatModel using an SLM.
We will be using a very very small language model, a language model so small that it might not even be considered a small language model. We will be using a bigram model to create a ChatBot. The ChatBot we will create will require one word as a prompt, and the bot will generate as many words as you ask for. The quality of the generated text will not be that awesome, but it is fine because today’s entire practice is for clarity of how a basic small language model can be used to build a basic chatbot.
I will use no neural networks, and no complex coding. I will use old-fashioned NLTK for creating the bigrams.
Here is the code for the Chatbot using the SLM: colab.research.google.com/dri...
Here is the code to create the data folder: colab.research.google.com/dri...
Dr. Shahriar Hossain
computing4all.com
24 ноя 2023
