Vectors and Vector Databases in Artificial Intelligence

Context

Before I begin discussing vectors, let me explain why you should care. Vectors are a key component of AI, and as AI continues to rise in popularity and permeate all aspects of our business and personal lives, it’s useful to gain a better understanding of what's happening ‘under the hood’.

Currently, the most well-known type of AI in the world is Large Language Models (LLMs) or chatbots. Powerful tools like ChatGPT and Copilot generate large quantities of text for a variety of tasks. But unless you’ve been living under a rock, I doubt I need to explain the capabilities of these tools.

The major limitation of these models is hallucinations: imagining information and presenting it as confidently as if it were the truth. Sometimes these hallucinations are easily recognisable, harmless, even funny. But, in a business setting, misleading information to an employer or potential client can have profound implications.

One method of counteracting this problem is ‘Retrieval Augmented Generation’ (or RAG for short). This is when you connect a chatbot or LLM to your own database, therefore becoming more attuned to your business. In practice, this means the chatbot doesn’t just rely on training data, but can also use up-to-date, domain-specific knowledge to produce more accurate and trustworthy answers.

It would be great if you could simply connect your database to an LLM and be good to go, but unfortunately, it’s not that straightforward. For an LLM to access your database, it first needs to be vectorised...

Vectors and Vector Databases

First things first: what is a vector? A ‘vector’s technical definition is something with both a direction and magnitude, but I find it best to think of it as an arrow.

An arrow can point in any direction and be as long or as short as you like. For example, if I have an arrow pointing north with a length of 5, then ‘north’ is the direction and ‘5’ is the magnitude. If we compare two arrows, where 'arrow A' is longer than 'arrow B', then 'arrow A' has the greater magnitude.

If that makes sense, congratulations, you now understand vectors!

The last thing you need to know is that a vector can also be used to describe a point on a map or graph. As you can see in the example below, to reach the point (7,3), you could go 7 steps across, and 3 steps up, or you can use an arrow (vector) that says ‘head north east for 7.6m’, much in the same way as you’d be given directions on a map.

Vector Database vs Relational Database

So, how does all this fit together? Well, a standard database used in a company is typically a relational database. This means all the data is organised into tables with rows and columns, similar to what you see in an Excel document. This can be simple to use but time-consuming to search, as each row must be checked one at a time.

A vector database works a little differently, where each document is broken down into characteristics that are used to place it somewhere on the graph. You can imagine this like a smart farm full of animals grouped by characteristics. In one place, there are all the animals with whiskers, and in another, all the animals with paws. If you wanted to find a cat, you’d look in the area where whiskers and paws overlap. In the same way, a vector database groups items by similarity, so when you search for something, it looks straight to the area in your data with the closest matches.

Additionally, if you were to search a database of movies using the search term 'Space', the search would be able to return movies like ‘Interstellar’ or ‘The Martian’ even if the word 'Space' isn't in their description because the vector database recognises they’re related to that theme. This enables vector databases to be searched more quickly and pick up on semantic similarity rather than searching for specific words, as in a typical database.

Set Up for Vector Embeddings

To set up a vector database, you would typically take existing data and convert each piece into a list of numbers known as a ‘vector embedding’. If we return to the farm example, each number would be like a score given to each trait of an animal.

So, in this example, if we were to choose ‘Whiskers’, ‘Paws’, ‘Size’, ‘Fur’, and ‘Wings’ to be our traits, then our vector embeddings would be ‘Cat = (8,9,3,7,0)’ and ‘Duck = (0,0,3,2,9)’. Each of these numbers acts as a set of directions to the animal’s location in our database.

Back to AI

Now, looking back to RAG, it should be a bit clearer as to why we need a vector database. Its ability to quickly find up-to-date information means your chatbot can give more relevant and accurate responses without hindering response time.

In practice, this might mean employees can instantly query the company’s HR policy instead of searching through long PDFs, or customers can get quick troubleshooting steps for a product without waiting on hold. Sales teams can find the latest pricing sheets in seconds, and support agents can access previous customer conversations without digging through old tickets. These are just a few examples of how RAG can deliver real business value. There will certainly be others!

Conclusion

Whilst AI can sometimes feel like a black box, I hope I’ve helped to shed some light on how and why it works. If you’re interested in hearing more about what AI and vector databases can do for your company, DSP’s expertise can enable you to turn AI into a strategic advantage. Contact us today to find out more.