Large Language Models (LLMs) like ChatGPT and Claude, and Artificial Intelligence in general, are rightfully getting a serious amount of attention. Companies are investing billions of dollars in server time and engineering to compete in the “foundation model” space. 

While a lot of media attention is on big investments, the fear of jobs being replaced, and how cool it is to generate art, what about business owners and enterprise leaders who want to make an impact today?

I decided to make a demo to show off some things that LLMs can do, TODAY. The point of the demo is to show some practical applications for how they can be used on your private business data, in a way that can be seen and touched and used by you. It’s easy to see results, since it is a web application; you don’t have to install any complicated software or run any code.

Private data, not generic models

All of the demos use private data. The demos are not searching the internet or relying on facts already embedded in the model. The demos are showing the power of reasoning, question answering, search, and summarization on your own data. I want you to see that you can take private, “messy” data — files, emails, powerpoints – and make sharp insights and decisions much faster than without LLMs.

I loaded file types that you have around the office: PDF’s, excel spreadsheets, word docs.

https://demo.bigcloudcountry.com

The demo shows a few capabilities I want to outline.

Reasoning over private data

Reasoning is the ability to think critically and step-by-step about a problem. In the demo, we show how ChatGPT compares two basketball teams and comes up with a prediction about who would win if they played. The reasoning is laid out and transparent. It uses statistics from the NCAA website, which have been downloaded and saved in a database. So, picture your own databases being used as a source of data, and asking a LLM to reason about that data.

You give it two college basketball teams. It then pulls up a bunch of team statistics like points per game, rebound margin, bench points, and a dozen others. It also pulls individual statistics for 3 or 4 key players from those teams. The individual statistics are things like rebounds, triple doubles, 3-point percentage, free throws, and so on.

The LLM is then presented with the data and a custom prompt, shown below (prompts are very important when using LLMs, and there is a whole field of activity around “prompt engineering”).

The stats along with this prompt are sent to GPT-4, which has the most advanced reasoning. They are also sent with a high “temperature” which is a way of telling the LLM how creative to be.

The results are something like this:

First-round upsets happen from time to time, but GPT-4 thinks Uconn will prevail based on the provided stats. The point is, if you need reasoning and analysis done on a bunch of data in excel spreadsheets or databases, the cost and wait time will rapidly decrease if you use an LLM.

Question answering

This demo allows you to ask a free-form question about March Madness. The context data is Wikipedia articles summarizing the tournaments going back about 20 years. Picture a big pile of Microsoft Word documents or PDFs that normally you’d have to search one by one. In fact, the March Madness articles were saved as PDF and then read by our systems, and finally stored in a special type of database called a vector database. A vector database stores text in big chunks (like full paragraphs or documents or even as small as sentences), but the text is represented as vector arrays – big lists of numbers like [1.23476, -0.43987, 5.98734, …] which is easier for a large language model to understand.

Once you have your PDFs or word documents or excel spreadsheets represented in a vector database, you can do a very effective kind of search called semantic search. Semantic search, as opposed to keyword search, lets you look up source documents based on meaning, not keywords. That means, you can just say what you want, and not worry about keyword matching (we’ve all done awkward Google searches trying to guess what will match).

After you submit a question, your question is converted into a vector. Then the vector database is searched with your question. The text stored in the vector database has all sorts of metadata on it to improve its chances of an accurate match. For example, this chunk of march madness history …

… has this metadata attached to it:

A really important assumption here is that I don’t assume people know exactly what they want to find. So, there are some recommendations on the demo about what kinds of things to ask. These recommendations were actually generated by ChatGPT while the articles were being processed.

Next Steps

Do you want to explore how to use powerful AI to make an impact at your business? Big Cloud Country has starter kits that get you up and running quickly so you see impact fast. Contact me and let’s start building.