Use AI without sending everything to the cloud.
Some of your data should not leave the building. Client files, patient records, staff details, legal documents, source code. That is not a reason to sit AI out. It is a reason to decide which parts of it you own.
I work out which AI jobs can run on hardware you control, which genuinely need a frontier model in the cloud, and whether owning any of it is worth the money. Sometimes it is not, and I will say so.
-
"Our lawyers will not let client files go anywhere near a chatbot."
-
"Staff are already pasting confidential documents into ChatGPT and I only found out last week."
-
"Customers are asking where their data goes, and I do not have a good answer."
-
"We have been quoted for a GPU server and I have no way to tell if it is worth it."
-
"The API bill grows every month and nobody can explain why."
-
"If the provider changes their terms or their prices, our feature changes with them."
The uncomfortable version is the second one. Whatever your policy says, if the tool is useful your people are already using it, and the confidential document is already somewhere you did not choose.
The question is not whether AI is safe. It is which of your data you are willing to hand to someone else's computer.
For most businesses that list is short and specific. Everything else can go wherever it likes. Local AI is worth considering when the short list is the part you most want AI to help with.
- Client and customer records
- Medical, health and patient information
- Staff files, payroll and HR matters
- Legal documents and anything under privilege
- Financial records and internal reporting
- Source code and unreleased product work
- Anything covered by a contract that says where the data may be stored
It is not all or nothing, and the interesting answer is almost never "everything local". A sensible setup routes each job to the place that suits it, and most businesses end up with a mix.
Everyday drafting, summarising and rewriting
Your hardware
High volume, not difficult. This is the work that quietly runs up an API bill for no benefit.
Anything containing client, patient, staff or legal data
Your hardware
The data never leaves your network, so there is nothing to disclose, renegotiate or apologise for.
Search and question answering across your own documents
Your hardware
The index is built from the exact files you do not want copied somewhere else.
Bulk processing of a back catalogue
Your hardware
Once you own the machine, cost per request stops being a reason not to do the work.
Hard reasoning, long documents, code generation, anything at the edge
A frontier model in the cloud
Local models are good. They are not the best. Pretending otherwise costs you quality where quality actually matters.
Whatever the frontier model was doing, when it is unavailable
Your hardware, as a fallback
An outage, a rate limit or a price rise stops being an incident and becomes a slightly worse answer.
Local does not mean disconnected. It means the default is a machine you own, and the cloud is something you reach for deliberately, for reasons you could explain to a customer.
Your applications do not need to know which model answered. A gateway makes that decision per request, on cost, capability and whatever your obligations actually are, and the rules can change later without touching the applications.
Feasibility, before you spend anything
I work out which of your AI workloads could genuinely run on hardware you control, what that hardware would cost to buy and to run, and what you would give up in quality. Then I give you a number and a straight recommendation.
This is the same independent-assessment work as the CTO Health Check, applied to one decision.
Private AI infrastructure
If it stacks up, I specify, build and run it: the models, the server, the network boundary, who can reach it, how it is monitored, how it is backed up, and what happens when it fails at 4pm on a Friday.
Hosting, deploy pipelines, monitoring and backups are the same job I already do for every production system I run.
Wiring it into the product you already have
AI that lives in a separate tab is a novelty. Value shows up when it is inside the tool your team already uses, doing one specific job, with an obvious answer to what happens when it gets that job wrong.
I cut product listing time from 20 minutes to 5 on my own ecommerce platform with AI descriptions and image processing.
Map
I list what you actually want AI to do, in your words, and what data each of those jobs touches.
Sort
Each job gets sorted into what must stay in-house, what belongs in the cloud, and what does not need AI at all.
Cost
I price both sides honestly: hardware, power and my time against your current and projected API bill.
Decide
You get a recommendation with a number attached, including the option of doing nothing.
Local AI Feasibility Review
A fixed-price review that answers one question properly: which parts of your AI should run on hardware you own. I map what you want AI to do, work out what data each job touches, price the hardware against your current and projected API bill, and give you a recommendation with a number attached.
You get a written report in plain language and a call to walk through it. The advice is independent, and there is no obligation to continue with me. The report is yours either way, including the version where the answer is no.
Fixed price, quoted before I start.
- Every AI job you want, and the data each one touches
- Which of them a model on your own hardware can actually do well
- What the hardware would cost to buy, run and keep running
- What quality you would give up, stated plainly rather than glossed over
- Your obligations: contracts, privacy, data residency, and what your policy already requires
- A recommendation, including doing nothing, with the reasoning behind it
To do this I need a conversation about what you want AI to do, your last few months of AI spend if you have any, and whatever your contracts or policies say about where data may live. Nothing else.
I make money either way, and I would rather not sell you a server you regret. I will tell you to leave things as they are when:
- Your AI spend is small enough that the hardware would take years to pay for itself.
- Your sensitive data is not really touching AI in the first place.
- A signed data processing agreement with your existing provider covers the obligation you are worried about.
- The quality drop on your particular task would be big enough that people quietly stop using it.
- There is nobody, including me, lined up to keep the thing running in twelve months.
A fixed-price review that ends in "do not do this" is still worth what you paid for it. It is the cheapest of the two possible outcomes.
Two things changed at once. Models small enough to run on a single machine got good enough to be useful, and enough businesses signed AI policies to discover their staff had already ignored them.
If what you actually need is the AI feature itself, and where it runs is not the issue, that is ordinary Software Development.
If the question is bigger than one decision, covering architecture, hiring and whether your current developers are right, start with Fractional CTO.
Not sure which parts you should own?
Tell me what you want AI to do and what data it would touch. I will tell you the honest next step, even if it is not me.