Self-Hosted AI Starter Kit: Run Your Own AI Stack for Free

Every time you send data to ChatGPT, Claude, or any cloud AI service, your business information leaves your control. The self-hosted AI starter kit from n8n changes that — giving you a complete AI stack running on your own hardware, completely free.

No monthly API bills. No data privacy concerns. No vendor lock-in. Just Docker, a few terminal commands, and you’re running local AI workflows that rival what enterprise companies pay thousands for.

Here’s everything you need to know about setting it up, what’s inside, and what you can actually build with it.

What Is the Self-Hosted AI Starter Kit?

The self-hosted AI starter kit is an open-source Docker Compose template created by n8n. It bundles everything you need to run AI workflows locally — a language model, a vector database, a workflow automation platform, and a PostgreSQL database — all in one command.

Think of it as a pre-built AI lab that runs on your laptop or server. Instead of stitching together individual tools and hoping they talk to each other, you get a working stack out of the box. Clone the repo, run docker compose up, and you’re building AI automations in minutes.

It’s designed for experimentation and proof-of-concept work, but the components themselves (n8n, Ollama, Qdrant, PostgreSQL) are production-grade tools used by thousands of companies.

Why Self-Host Your AI Instead of Using Cloud APIs?

Cloud AI APIs are convenient. But they come with real trade-offs that most small business owners don’t think about until the bill shows up or a client asks where their data goes.

• Cost control. API calls add up fast. A busy AI workflow can burn through $50-200/month on GPT-4 alone. Local models like Llama 3 and Mistral run for free once downloaded.
• Data privacy. When you self-host, your customer data, internal documents, and business processes never leave your network. No third-party data processing agreements needed.
• Zero rate limits. Cloud APIs throttle you during peak hours or when you hit usage caps. Your own stack runs as fast as your hardware allows, whenever you need it.
• No vendor lock-in. Switch models, swap components, or customize anything. You’re not locked into one provider’s ecosystem or pricing changes.
• Uptime you control. No waiting for OpenAI to fix their outage. Your local stack stays up as long as your machine does.

The trade-off? Local models aren’t as capable as GPT-4 or Claude for complex reasoning. But for most business automation tasks — classifying emails, summarizing documents, answering FAQs, extracting data — they’re more than good enough.

What’s Included in the Kit

The starter kit bundles four battle-tested open-source tools into a single Docker Compose file. Each one handles a specific job in your AI stack:

• n8n (workflow automation). The brain of the operation. n8n is a low-code automation platform with 400+ integrations. You build AI workflows visually — drag, drop, connect. It handles everything from triggering on new emails to routing AI responses back to your CRM.
• Ollama (local AI models). Ollama makes running large language models dead simple. Download Llama 3, Mistral, Gemma, or dozens of other open models with one command. No GPU rental fees, no API keys.
• Qdrant (vector database). This is what gives your AI memory. Qdrant stores document embeddings so your workflows can search through your business documents, knowledge bases, and historical data using semantic search — not just keyword matching.
• PostgreSQL (relational database). The reliable workhorse for structured data. Stores workflow history, user data, conversation logs, and anything else your automations need to persist.

Together, these four components give you a complete AI development environment. You can build RAG pipelines (retrieval-augmented generation), AI agents, document processors, and chatbots — all running locally.

How to Set Up the Self-Hosted AI Starter Kit

You need two things already installed: Docker and Docker Compose. If you’re on a Mac, download Docker Desktop. On Linux, install Docker Engine. That’s the only prerequisite.

Here’s the setup process, step by step:

1. Clone the repository. Open your terminal and run git clone https://github.com/n8n-io/self-hosted-ai-starter-kit.git then cd self-hosted-ai-starter-kit
2. Copy the environment file. Run cp .env.example .env and update the passwords and secrets inside. Don’t skip this — default passwords are a security risk.
3. Choose your hardware profile. The kit supports three modes depending on your machine.
4. Start the stack. Run the Docker Compose command for your profile (see below).
5. Access n8n. Open http://localhost:5678 in your browser. Create your admin account and you’re in.

The Docker Compose commands by hardware:

# NVIDIA GPU
docker compose --profile gpu-nvidia up

# AMD GPU (Linux only)
docker compose --profile gpu-amd up

# CPU only (no GPU)
docker compose --profile cpu up

Mac users: Docker on Mac can’t access GPUs directly. You have two options — run entirely on CPU (slower but works), or install Ollama natively on your Mac and point the Docker stack at host.docker.internal:11434. The native option uses Apple Silicon’s GPU acceleration, which is actually faster than the Docker CPU route.

The first time you run a workflow that uses an AI model, Ollama will download it automatically. A model like Llama 3 8B is around 4.7 GB. After that first download, it’s cached locally.

What You Can Build With It

The starter kit isn’t just a tech demo. Here are practical workflows small businesses can build from day one:

• Customer support chatbot. Feed your FAQ docs and product manuals into Qdrant, then build an n8n workflow that answers customer questions using RAG. The AI pulls from your actual documentation instead of making things up.
• Email triage and auto-reply. Connect Gmail to n8n, classify incoming emails by intent (support, sales, spam), draft contextual responses, and route urgent ones to your phone. All without your email data touching external servers.
• Document summarizer. Drop PDFs into a shared folder. An n8n workflow picks them up, runs them through Ollama for summarization, and stores the key points in PostgreSQL. Perfect for contracts, reports, or research papers.
• Lead qualification agent. Connect your CRM or form submissions to n8n. The AI scores leads based on your criteria, enriches them with context, and routes hot leads to your sales pipeline automatically.
• Internal knowledge base. Index your SOPs, training materials, and company wiki into Qdrant. Build a Slack bot or web interface that lets your team ask questions and get answers grounded in your actual processes.

These aren’t hypothetical — they’re the exact types of workflows we build for clients at JonJones.AI. The difference is we typically use cloud AI for the heavy reasoning and self-hosted tools for everything that touches sensitive data. If you’re curious about what an AI agent can actually do for a small business, we break that down in detail.

Choosing the Right Model for Your Workflows

Ollama gives you access to dozens of open-source models. Picking the right one depends on your hardware and what you’re building. Here’s a practical guide:

• Llama 3 8B (~4.7 GB). Best all-rounder. Handles classification, summarization, Q&A, and basic reasoning. Runs well on 16 GB RAM machines. Start here.
• Mistral 7B (~4.1 GB). Fast and efficient. Great for tasks that need speed over depth — email triage, simple data extraction, quick classifications.
• Llama 3 70B (~40 GB). Significantly more capable. Approaches GPT-4 quality for many tasks. Needs 64 GB+ RAM or a powerful GPU. Worth it if you have the hardware.
• Nomic Embed (~274 MB). Not a chat model — this creates embeddings for your vector database. Essential for RAG workflows where you need semantic search across your documents.

Download any model by running ollama pull model-name inside the Ollama container, or let n8n trigger the download automatically when a workflow first calls a model.

Who Should (and Shouldn’t) Use This Kit

This kit is for you if:

• You handle sensitive client data and can’t send it to cloud AI providers
• You’re spending $100+/month on AI API bills and want to cut costs
• You want to experiment with AI automation without committing to paid tools
• You’re comfortable running a terminal and basic Docker commands
• You need AI workflows that run 24/7 without usage limits

This kit isn’t for you if:

• You need GPT-4-level reasoning for complex tasks (local models aren’t there yet)
• You don’t have at least 16 GB of RAM available
• You want something that works without any technical setup — try cloud-based AI tools instead
• You need production-grade reliability from day one (this is a starter kit, not a production deployment)

From Starter Kit to Production

The starter kit is exactly that — a starting point. Once you’ve validated your AI workflows locally, here’s how to harden them for real business use:

• Move to a VPS. Run your stack on a cloud server (DigitalOcean, Hetzner, or a dedicated GPU box) so it’s accessible 24/7 and not tied to your laptop being open.
• Add authentication. Put n8n behind a reverse proxy (Nginx, Caddy) with HTTPS and proper auth. The default setup has no SSL.
• Set up backups. Schedule automated backups of your PostgreSQL data and Qdrant vectors. Losing your vector index means re-embedding all your documents.
• Monitor resources. Local AI models are memory-hungry. Watch your RAM and disk usage, especially when running multiple models or processing large documents.
• Consider a hybrid approach. Use local models for routine tasks and data-sensitive operations. Route complex reasoning to cloud APIs (GPT-4, Claude) when accuracy matters more than privacy.

If you want to see how tools like Claude Code fit into a self-hosted workflow, we use it daily for everything from content automation to deployment scripts. Cloud and local AI aren’t competing — they’re complementary.

Get Started Today

The self-hosted AI starter kit removes every barrier to getting started with local AI. No subscriptions. No API keys. No vendor lock-in. Just open-source tools that work together out of the box.

Clone the repo. Run docker compose up. Build your first AI workflow in n8n. Once you see what’s possible with local AI, you’ll wonder why you were paying per token for basic automation tasks.

Need help building AI automations for your business? Check out our consulting services — we’ll design and build the exact workflows your business needs, whether that’s self-hosted, cloud, or a hybrid of both.