DeepSeek is the latest model to emerge in the race for AI supremacy. A brief explainer: DeepSeek is a Large Language Model (LLM)similar to OpenAi’s Chat GPT and Google’s Gemini. Where DeepSeek differs from the former, however, is in its model size.
The mantra in the AI world for the last few years has been ‘bigger is better’. Ai developers have been falling over themselves to create the largest models possible, in an attempt to make the smartest models possible. This approach is costly as larger models mean larger data centres and larger maintenance costs. DeepSeek has managed to subvert this trend by matching the performance of Open Ai’s latest model (GPT 4o) with a model a fraction of the size and at less than 10% of the training cost. This has a knock-on benefit of reducing environmental impact when running this model, due to the lower energy demands.
Excluding the economic disruption the model has caused to the US stock markets, the significance to businesses mostly boils down to one other differentiating factor: it’s open source.
Now it’s not the first open-source LLM (Meta’s Llama is already a popular open-source model), but it is the first to achieve high performance with such a compact model. This is significant because this smaller size means the model can be run locally without requiring powerful GPUs or large data centres. This, in turn, ensures that DeepSeek has no outbound internet connectivity, meaning the model can be used as needed without the risk of your confidential company data being exported, stored and potentially leaked by an LLM provider.
But the real test for any LLM is how well it works in practice. As DSP works predominantly with OCI, I was keen to see how DeepSeek worked in an isolated OCI environment. For the purpose of this demo, I have used the DeepSeek r1 model, however, more powerful r2 and r3 models do currently exist. When running DeepSeek locally, there are a range of available model sizes to choose from. For DeepSeek r1, these range from 1.5b to 671b parameters, with larger models typically providing better results.
To test the model in Oracle Cloud, I created a standard OCI environment with a compute instance running Linux with only 1 OCPU and 6GB of memory. This environment was completely isolated in a test OCI tenancy. I connected to the instance using ssh and began by downloading Ollama, a popular platform for installing and running LLM’s. Once installed, ‘ollama pull deepseek-r1’ allowed me to access the DeepSeek model and ‘ollama run deepseek-r1’ begin a conversation in the command line (running the 7b default model). The model was slow to respond, and a larger quantity of memory and OCPUs would be recommended for a real-world deployment, however, the fact the model was able to run on a machine with such limited power is impressive in its own right.
With the model now running, permissions for outbound internet access were removed from the security list. If DeepSeek were to be quietly transmitting its data outside the VM, then I would have expected the model to stop operating once this connection was removed. Thankfully, the LLM was still able to function as normal, meaning the model could be used safely without concerns of sensitive data being transmitted offsite. Of course, this is based on limited testing, and I would always recommend not just reading any literature around the T&Cs of DeepSeek but also checking internally if running a private LLM contravenes any internal governance or policies.
In summary, running the DeepSeek LLM locally or on OCI provides a unique opportunity to harness the productivity benefits of modern AI without having to compromise on safety or security.
If you would like to speak to one of our experts about AI or OCI, please don't hesitate to contact us.
Subscribe to Eye on OCI Insights for the latest news and updates for all things Oracle Cloud.