Running AI On-Prem for GDPR: A Practical Guide for UK Companies
A practical engineering view, not legal advice. Talk to your DPO before changing how you process personal data.
Every UK company using cloud AI APIs on client or patient data has quietly signed up for the same set of headaches: personal data leaving the building, processor agreements with a chain of AI vendors, international transfer questions, and a straight-faced answer needed for the client who asks "so where exactly does our data go?"
There is a blunt alternative that more firms in healthcare, legal and financial services are choosing: run the model on your own hardware, inside your own walls.
What on-prem inference actually changes
The data never leaves. When an LLM runs on a machine in your office or server room, prompts and documents are processed locally. There is no third-party AI processor for that workload, no data crossing borders, no cloud vendor's sub-processor list to monitor. Your answer to "where does the data go?" becomes "nowhere."
Your DPIA gets simpler. Sensitive-data workflows (patient notes, privileged documents, transaction records) stop depending on another company's security posture and terms. The risk conversation moves from "can we trust their cloud?" to "is our own machine secured?", which is a problem your IT already knows how to handle.
No per-token meter. Cloud AI billing scales with usage. A local machine costs the same whether you run a hundred prompts a day or a hundred thousand. Heavy internal use, summarising case files, drafting, coding assistants, stops being a variable cost.
What it honestly does not change
You still have GDPR obligations: lawful basis, retention, access controls, security of the machine itself. On-prem removes the third-party transfer problem; it does not remove data protection. And open models you run locally (Llama, Qwen, Mistral, DeepSeek distills) are a step behind the frontier cloud models on some tasks. For most document work, summarisation, drafting and internal assistants, current 30B to 70B class open models are genuinely good, but test on your own workload before committing.
What it costs, in real numbers
Less than most compliance officers expect. A GPU that runs 13B-class models locally is under £900; the 48GB card that handles 70B-class models is £1,699.99. A complete, tested, ready-to-run machine built around one is in the low thousands, a one-off purchase that replaces a recurring cloud bill. Run your own spend through our payback calculator; for many firms the hardware pays for itself inside a year, and everything after that is savings.
Where to start
Pick one workflow with sensitive data and a clear cost, put it on one local machine, and measure for a month. We supply the GPUs from UK stock and build complete inference machines to order, benchmarked before dispatch, with a VAT invoice for your books.
If you are weighing this up for your firm, email alex@destellotech.com with your use case and target models. You will get a straight answer from the founder about what fits, what it costs, and what to avoid, including "cloud is fine for your case" when that is the truth.