A filterable database of real, sourced self-hosted AI builds: the hardware, the software stack, the models, what works, what doesn't, and how to start. Keep client data on your own machines.
Browse 16 setups Submit yoursA freelancer working on client code ran a fully local stack for 3 months: Ollama + Qwen2.5-Coder-32B via Continue.dev, with Open WebUI for longer chats.
Source: r/LocalLLaMA
A developer runs Ollama on a headless Ubuntu desktop with an RTX 3090 and connects laptops via the Continue VS Code plugin. The 32B model was too heavy in practice, so 7B is used.
Source: Arul Selvam Periyasamy (blog)
A tested guide (Ollama v0.24.0, Continue v1.2.22) for running chat, tab-completion and codebase search on an 8GB NVIDIA GPU.
Source: aifoss.dev
A GitHub issue documents that Continue used ~9.5GB VRAM vs ~6GB for the same model in the Ollama CLI, fixed by lowering contextLength and maxTokens.
Source: GitHub continuedev/continue #6609
A beginner-friendly walkthrough: Ollama runs Llama 3 on an M4 Mac mini, AnythingLLM Desktop provides the UI and local LanceDB vector store.
Source: Benjamin S Powell
AnythingLLM + Ollama (llama3.1:8b) on a Mac mini M4, exposed through a Cloudflare Tunnel protected by Cloudflare Access, queried from Telegram via n8n.
Source: Ovidiu Eftimie (Substack)
A careful workflow for chatting with documents on a single machine, including a hallucination test (ask a question the document can't answer).
Source: localai.computer
IBM's Granite community recipe for running a basic local RAG over a PDF (e.g. a budget document) on your laptop.
Source: IBM Granite Community (GitHub)
A local legal stack guide: a silent Mac mini M4 Pro with 48GB unified memory running Ollama with one AnythingLLM workspace per matter, plus encryption and a written AI policy.
Source: runlocalai.co
A 1-attorney/1-paralegal/2-admin firm asked if a Mac Studio M3 Ultra could handle file sorting and first-pass record review. Replies say hardware is the easy part; PDF/OCR ingestion and guardrails are the real work.
Source: r/LocalLLM
Audio uploaded from a phone PWA is transcribed on a local RTX 3060 (12GB) with faster-whisper int8, sharing the card with an embedding model and Gemma.
Source: DEV Community (hannune)
whisper.cpp (CUDA) with large-v3 on an office PC, behind nginx + n8n for auth, exposing an OpenAI-compatible endpoint used from desktop, iPhone, iPad and Android.
Source: Mr Hoelee (blog)
whisper_api offloads transcription to one GPU server shared by several people; the author suggests the RTX 3060 12GB as the cheapest option for high-quality models.
Source: MayNiklas/whisper_api (GitHub)
Tailscale's Alex Kretzschmar passes an NVIDIA A4000 through to a NixOS VM on Proxmox, runs Ollama + Open WebUI in Docker, and exposes it privately with Tailscale Serve (TLS, no reverse proxy).
Source: Tailscale blog
Hosting provider HOSTKEY built a tech-support assistant for its front-line team: cleaned Markdown docs, Open WebUI RAG (ChromaDB), a custom model with a strict system prompt.
Source: HOSTKEY blog
A small startup asked for the cheapest way to self-host; a top suggestion was to buy used and build around 2x 24GB RTX 3090s for a 32B model.
Source: r/ollama
One email a month with new real-world builds. No spam.