• 1 Post
  • 16 Comments
Joined 2 years ago
cake
Cake day: June 23rd, 2023

help-circle





  • ikidd@lemmy.worldtoSelfhosted@lemmy.worldHelp Reviewing My Server Setup?
    link
    fedilink
    English
    arrow-up
    1
    arrow-down
    1
    ·
    16 days ago

    The advantages you gain with running a hypervisor on something like ZFS is immeasurable, for snapshotting, replication, snapshot backups and high availability. You don’t have to quiese machines to back them up and you can do instant COW snapshots before upgrades.

    KVM doesn’t really have overhead, that’s the kernel part. Maybe a bit of RAM, but with LXCs it’s negligible.










  • So, I’m a rabid selfhoster because I’ve spent too many years watching rugpull tactics from every company out there. I’m just going to list what I’ve ended up with, and it’s not perfect, but it is pretty damn robust. I’m running pretty much everything you talk about except much in the way of AI stuff at this point. I wouldn’t call it particularly energy efficient since the equipment isn’t very new. But take a read and see if it provokes any thoughts on your wishlist.


    My Machine 1 is a Proxmox node with ZFS storage backing and machine 2 is mirror image but is a second Proxmox node for HA. Everything, even my OPNsense router runs on Proxmox. My docker/k8s hosts are LXCs or VMs running on the nodes, and the nodes replicate nearly everything between them as a first level, fast recovery backup/high availability failover. I can then live migrate guests around very quickly if I want to upgrade and reboot or otherwise maintain a node. I can also snapshot guests before updates or maintainance that I’m scared will break stuff. Or if I’m experimenting and like to rollback when I fuck up.

    Both nodes are backed up via Proxmox Backup Server for any guests I consider prod, and I take backups every hour and keep probably 200 backups at various intervals and amounts. These dedup in PBS so the space utilization for all these extra backups is quite low. I also backup via PBS to removable USB drives on a longer schedule, and swap those out offsite weekly. Because I bind mount everything in my docker compose stacks, recovering a particular folder at a point in time via folder restore lets me recover a stack quite granularly. Also, since it’s done as a ZFS snapshot backup, it’s internally consistent and I’ve never had a db-file mismatch issue that didn’t just journal out cleanly.

    I also zfs-send critical datasets via syncoid to zfs.rent daily from each proxmox node.

    Overall, this is highly flexible and very, very bulletproof over the last 5 or 6 years. I bought some decade old 1-U dell servers with enough drive bays and dual xeons, so I have plenty of threads and ram and upgraded to IT-mode 12G SAS RAID cards , but it isn’t a powerhouse server or anything, I might be $1000 into each of them. I have considered adding and passing through an external GPU to one node for building an ollama stack on one of the docker guests.

    The PBS server is a little piece of trash i3 with a 8TB sata drive and a GB NIC in it.


  • Just glancing through that guide:

    OPNsense instead of Pfsense, because pfsense is going to rugpull, it’s just a matter of time. I wouldn’t trust the twats that run it farther than I could throw them because they’re pretty silly people. Rossman suggests exactly this in the intro to the router section, he would change if he hadn’t been using it for a decade already. Unfortunately, a lot of this guide is focussed on how to do it via pfsense and if you’re brand new, you’re going to have to figure out how to do it in OPNsense yourself.

    Wireguard/Tailscale instead of openvpn. Faster and way easier to set up. Don’t even try to set up a full LAN routed VPN, just use Tailscale for the services you want. And use it for everything and everyone instead of punching holes in the firewall.

    He’s definitely right about mailcow; if you’re reading that guide for information, you are not a person that should be self-hosting email.


  • Edit: if you’re evaluating this, use a chromium based browser because there’s a pile of things not working if you use FF. That means I’m not going to use it, but you do you.

    The hard to find ARP scan dialog box that is pretty much the starting point for anything here is very special. There’s no dropdown that lists the physical interfaces so you have to hunt around to find the listing for Network Hardware that for some reason is under the About top level menu item. Of course, that lists every virtual docker interface along with the physical network devices so you’d better know what you’re looking for.

    Contrary to the poorly organized docs, the physical interface will rarely be eth0 or eth1, it’ll be something like “enp5s2”. So now you go back to edit the entries to the physical interface but you can’t, all you can do is Remove All. Well, better get your entries letter perfect, because if you make a mistake on a single character, you’re starting from scratch after another Remove All.

    In your docs, you recommend not editing the app.conf file, but that would be way more forgiving than this. At least there you can add VLANs.

    And there’s no way I can see to bulk add new devices to a known and/or trusted state. Go into each device and uncheck the “New Device” box. How do I add a device to My Devices? Who knows.

    Publishers: Nothing to find in the Settings for each of the publishers. Every publishers settings section is blank.

    UI will take to blinking randomly as it gets into a refresh loop. Have to close the window and reopen it to get it usable again. Sometimes all the text in each setting header goes away. No headings on the tables for the Devices, just the set order arrows that would be on each column, not lined up with anything correctly. Oh, now I’ve lost every setting on each setting section, just blank.

    I’m reticent to get much more into the app because if this is the introduction, I’m scared of what other frustration I’m going to find. Not to mention having the arrows for the left menu tree expansion backwards to how nearly everyone else ever does it, but I guess that’s just my OCD.

    Man, I want to like this because it looks like it could be a simple to use version of Nagios, but some of the design is pretty hard to take.