LM Studio (language model studio) is a tool that allows you to exercise large language models (LLMs), such as GPT-x (OpenAI), LLaMA-x (Facebook), and Orca-x (Microsoft) on your local machine instead of via the Cloud. This is useful when you are interested in how different LLMs compare to each other.

First, I opened a browser (Chrome) and went to the home page for LM Studio at http://www.lmstudio.ai. I clicked on the Download LM Studio for Windows button. The download took about five minutes for me. I went to my Downloads folder and double-clicked on the setup.exe file. Installation was very quick.

Next, I launched LM Studio. The UI is clean and fairly intuitive.

Next, I clicked on the magnifying glass search icon in the left margin, and entered “Orca 2” because I wanted to explore the new Microsoft LLM.

Next, on the left side of the screen, I selected the Orca 2, 13 billion parameters family of models (which are in .gguf binary format), and then on the right side, I selected the specific orca-2-13b.Q6_K.gguf model and clicked the Download button. The Q6 in the file name means 6-bit quantization which is a technique to convert floating point numbers to integers for vastly improved performance at the expense of fidelity. The download took about 20 minutes.

Next, on the left side of the screen, I selected the conversation bubble icon to create a Chat Session. At the top, I selected my Orca 2 model (you can have several models on your machine). At the bottom of the screen, I entered my query (see text below) and hit the Enter key and waited for the Orca 2 model to respond.

My query came from the Orca 2 research paper “Orca 2: Teaching Small Language Models How to Reason”. It is:

“John and Mark are in a room with a ball, a basket and a box. John puts the ball in the box, then leaves for work. While John is away, Mark puts the ball in the basket, and then leaves for school. They both come back together later in the day, and they do not know what happened in the room after each of them left the room. Where do they think the ball is?”

This is a difficult reasoning problem.

Orca 2 responded with:

“John thinks the ball is in the box because that’s where he left it before leaving for work. Mark thinks the ball is in the basket because that’s where he left it before leaving for school. They are both unaware of each other’s actions and believe their respective locations are correct.”

I like the LM Studio tool. But it’s not clear to me exactly who created the tool and what connections, if any, they have with other companies such as Hugging Face.