All services

Open-source models and custom, locally hosted AI

“Our data is too sensitive to send to an AI service”, or “our AI bill grows every month and we have no control over the model”.

I build AI for small businesses on open-source models that run on hardware you own.

Your data stays in-house and there are no per-message fees.

I work with open-source models as well as the managed tools from Anthropic and Google.

  • Open source

What it is

AI built on open-source models that runs on hardware you own, so your data stays in-house and there are no per-message fees.

Tools I use

  • Open-source language models

    Many are strictly “open-weight”: the model weights are published, under licences of varying openness.

  • Open embedding models

  • Local model serving

  • Self-hosted search

  • Local image generation

  • Open-source machine learning libraries

When it suits a small business

  • Your data is sensitive or regulated.
  • Your usage would make managed fees high.
  • You want control over what the model is and when it changes.

The trade-offs

  • There is an upfront hardware cost.
  • Someone has to maintain it.
  • Open-source models are smaller than the largest managed ones.

What I do with it

  • I choose an open-source model and hardware for the job.
  • I set up serving, routing, tools and memory.
  • I add monitoring and a review routine.
  • I compare candidate models on a test set before switching.

Why me for this

  • The private assistant I built answers every message on hardware I operate, on an open-source model, and is in daily use.

  • I compare candidate open-source models on a fixed test set, with the decision rule set before the benchmark runs.

  • I trained a 26-million-parameter model on my own GPU machine with open-source libraries.