Open-source models and custom, locally hosted AI
“Our data is too sensitive to send to an AI service”, or “our AI bill grows every month and we have no control over the model”.
I build AI for small businesses on open-source models that run on hardware you own.
Your data stays in-house and there are no per-message fees.
I work with open-source models as well as the managed tools from Anthropic and Google.
What it is
AI built on open-source models that runs on hardware you own, so your data stays in-house and there are no per-message fees.
Tools I use
Open-source language models
Many are strictly “open-weight”: the model weights are published, under licences of varying openness.
Open embedding models
Local model serving
Self-hosted search
Local image generation
Open-source machine learning libraries
When it suits a small business
- Your data is sensitive or regulated.
- Your usage would make managed fees high.
- You want control over what the model is and when it changes.
The trade-offs
- There is an upfront hardware cost.
- Someone has to maintain it.
- Open-source models are smaller than the largest managed ones.
What I do with it
- I choose an open-source model and hardware for the job.
- I set up serving, routing, tools and memory.
- I add monitoring and a review routine.
- I compare candidate models on a test set before switching.
Why me for this
The private assistant I built answers every message on hardware I operate, on an open-source model, and is in daily use.
I compare candidate open-source models on a fixed test set, with the decision rule set before the benchmark runs.
I trained a 26-million-parameter model on my own GPU machine with open-source libraries.