Self-host a local AI stack and access it from anywhere
tailscale.com
Use LLMs like Llama 4, Gemma 3, and DeepSeek locally and control your data. Set up your own offline AI lab with an NVIDIA A4000 GPU wired to a NixOS VM via PCIe passthrough, set up Ollama and Open WebUI, and get chatting with local LLMs all entirely within your tailnet.

NOTE: I got to this article by clicking on a banner ad that Tailscale bought in the release notes of OpenWebUI’s GitHub. However, I thought it was still neat and worth sharing. Please discuss and share any alternatives for the paid/proprietary/problematic pieces of this setup:

Proxmox for virtualization, NixOS for repeatability, Docker for packaging, and Tailscale for secure access from anywhere. We’ll wire up an NVIDIA A4000 GPU to a NixOS VM via PCIe passthrough, set up Ollama and Open WebUI, and get chatting with local large language models all entirely within your Tailnet.

  • Lucy :3@feddit.orgEnglish
    0·
    2 days ago

    You mean an A4000? My 1070 can run llama 3.1 comfortably (the low B versions). My 7800XT answers instantly.