Best practices for setting up a home server?
January 29, 2020 10:17 PM Subscribe
We are thinking of setting up a home server for backups, media, pi-hole, and miscellaneous cronjobs. While we are both professional computer people, neither of us are sysadmins, and we're not looking for new hobbies either. How can we simplify, automate, and future-proof our server?
Our main fear is finding ourselves five years down the line with a bunch of illegible config files, untested backups, and unpatched vulnerabilities.
Second worst would be sinking a bunch of time into tweaking and debugging the server.
Specific questions:
0. Where do people talk about this? I found r/homeserver but it appears to be 90% hardware and the few threads I read made it sound like the r/wallstreetbets of tech.
1. What distribution? We're leaning toward Debian or Ubuntu, but if you're really excited about NixOS or something, go for it.
2. How to manage it? We're used to production environments where containerization/virtualization, config management, etc. are assumed, but we're also used to having people to run it all. Does that stuff make sense for a single box?
3. How to monitor it? Again, we don't know how to scale our expectations down to a single box. But it'd be nice to have some blinkenlights that go red when we fill the disk or fall behind on our patches or something. (I don't even know enough to give good examples.)
Our main fear is finding ourselves five years down the line with a bunch of illegible config files, untested backups, and unpatched vulnerabilities.
Second worst would be sinking a bunch of time into tweaking and debugging the server.
Specific questions:
0. Where do people talk about this? I found r/homeserver but it appears to be 90% hardware and the few threads I read made it sound like the r/wallstreetbets of tech.
1. What distribution? We're leaning toward Debian or Ubuntu, but if you're really excited about NixOS or something, go for it.
2. How to manage it? We're used to production environments where containerization/virtualization, config management, etc. are assumed, but we're also used to having people to run it all. Does that stuff make sense for a single box?
3. How to monitor it? Again, we don't know how to scale our expectations down to a single box. But it'd be nice to have some blinkenlights that go red when we fill the disk or fall behind on our patches or something. (I don't even know enough to give good examples.)
Maybe some info about the cronjobs would help narrow it down?
IANYS, but if I was going to recommend a solution for someone who doesn't want a new hobby, I'd recommend going with appliances, to cover the first three items.
You can buy a plug-and-play pi-hole device. Then a NAS device for mass, fault-protected networked storage, as the home for a digital media library and a target for backups. The fancy models will even do small-scale server tasks like run a Home Media Server and/or deploy agents for automated backups.
If that's all you are really looking for, then you just buy two pieces of off-the shelf hardware, and manage them through going to a web address on your local network. Once you've got them configured the way you want, most troubleshooting will involve turning it off and on again. That's the beauty of a device-based approach.
(If you want something for threat protection and network monitoring, I can recommend something that's not off the shelf, but you could install it on a recycled PC tower in an afternoon, then run it as easily as the above devices.)
Once you get into multiple single-function appliances, it eventually makes sense to virtualize / containerize things, consolidating hardware. But then you've got to learn and manage the virtualization platform, on top of the interface for each appliance/function. And hardware gets pricey. So now you're inevitably headed for hobby territory.
Thus, I'd recommend poking around maybe the higher end NAS options, and seeing if you can just buy a box that looks like Darth Vader's toaster and does most of what you were thinking about.
posted by bartleby at 1:16 AM on January 30, 2020 [2 favorites]
IANYS, but if I was going to recommend a solution for someone who doesn't want a new hobby, I'd recommend going with appliances, to cover the first three items.
You can buy a plug-and-play pi-hole device. Then a NAS device for mass, fault-protected networked storage, as the home for a digital media library and a target for backups. The fancy models will even do small-scale server tasks like run a Home Media Server and/or deploy agents for automated backups.
If that's all you are really looking for, then you just buy two pieces of off-the shelf hardware, and manage them through going to a web address on your local network. Once you've got them configured the way you want, most troubleshooting will involve turning it off and on again. That's the beauty of a device-based approach.
(If you want something for threat protection and network monitoring, I can recommend something that's not off the shelf, but you could install it on a recycled PC tower in an afternoon, then run it as easily as the above devices.)
Once you get into multiple single-function appliances, it eventually makes sense to virtualize / containerize things, consolidating hardware. But then you've got to learn and manage the virtualization platform, on top of the interface for each appliance/function. And hardware gets pricey. So now you're inevitably headed for hobby territory.
Thus, I'd recommend poking around maybe the higher end NAS options, and seeing if you can just buy a box that looks like Darth Vader's toaster and does most of what you were thinking about.
posted by bartleby at 1:16 AM on January 30, 2020 [2 favorites]
For the OS, make sure to get an LTS (long term support) version and note when support ends. Otherwise there is a high likelihood of needing to trash the OS in a year or two once it no longer has security updates.
Make sure to enable automatic security updates ONLY if the update manager supports it.
posted by benzenedream at 1:43 AM on January 30, 2020
Make sure to enable automatic security updates ONLY if the update manager supports it.
posted by benzenedream at 1:43 AM on January 30, 2020
Yes, +1 for buying a Synology NAS (or QNAP or whatever) which will take care of much of the work for you.
You can either run Pi-Hole on the NAS (if you buy one which supports Docker) or on a dedicated Raspberry Pi (the Zero W is fine for this). If you go down the Pi route, stick with Raspbian (essentially, Debian) as it's widely used and supported. As DNS is critical, you might want to run two devices with Pi-hole (i.e. two Pis, or one Pi and one Docker).
The tradeoff is that you're dependent on Synology for keeping your device and data secure. Their record is not unblemished, so I would not expose a NAS to the open internet via opening a port on your router.
A Synology will email you notifications when disk space is low or something needs attention.
posted by matthewr at 2:06 AM on January 30, 2020 [3 favorites]
You can either run Pi-Hole on the NAS (if you buy one which supports Docker) or on a dedicated Raspberry Pi (the Zero W is fine for this). If you go down the Pi route, stick with Raspbian (essentially, Debian) as it's widely used and supported. As DNS is critical, you might want to run two devices with Pi-hole (i.e. two Pis, or one Pi and one Docker).
The tradeoff is that you're dependent on Synology for keeping your device and data secure. Their record is not unblemished, so I would not expose a NAS to the open internet via opening a port on your router.
A Synology will email you notifications when disk space is low or something needs attention.
posted by matthewr at 2:06 AM on January 30, 2020 [3 favorites]
[obDisclosure: I'm a moderator of the FreeNAS forums]
FreeNAS is where the people who actually do want to make networking and storage into a hobby often end up. If you want to avoid the possibility of making this a hobby, I wholeheartedly endorse the idea of a nice little Synology or QNAP unit. While there is a lot of stuff to learn with a vendor NAS such as Syno or QNAP, there is a lot less than if you were to roll your own from FreeBSD or Linux, or a NASware such as FreeNAS.
NONE of these solutions are magic, and you will need to spend some significant time running through all the basics such as getting e-mail notification of problems up and running, figuring out the networking, getting permissions sorted, etc.
Do NOT buy a NAS unit from a fly-by-night vendor. You really need to stick to a name brand such as QNAP or Synology. These companies both specialize in NAS and have a proven track record of releasing firmware for existing devices for years. This is highly important because of security fixes, etc. IoT devices tend to have a problem with ongoing firmware development, and if you bought one of the many off-brand NAS units from a decade ago, where a company momentarily dabbled in NAS before deciding there was little profit, you probably only got one or two firmware updates -- if any. QNAP and Synology sidestep this because they fund firmware development from sales of their newer units. This still isn't a great model, but it solves the problem of paying the developers.
Your second-best option, in terms of drama, is to go with FreeNAS! FreeNAS is a full enterprise-grade managed NAS platform with a web GUI interface, a good manual, and an enthusiastic community. Feel free to visit the FreeNAS forums and check out the hardware guides and community. There are inexpensive options to do it yourself, and there are also prebuilt options available that are ready-to-go (but a little pricey in my opinion). There is a huge upside in that, unlike the Synology or QNAP, you do have the option to easily run a different operating system on the device if you ever become unhappy with FreeNAS, and a FreeNAS host can be scaled to be massively more capable than any Syno or QNAP.
posted by jgreco at 4:15 AM on January 30, 2020 [7 favorites]
FreeNAS is where the people who actually do want to make networking and storage into a hobby often end up. If you want to avoid the possibility of making this a hobby, I wholeheartedly endorse the idea of a nice little Synology or QNAP unit. While there is a lot of stuff to learn with a vendor NAS such as Syno or QNAP, there is a lot less than if you were to roll your own from FreeBSD or Linux, or a NASware such as FreeNAS.
NONE of these solutions are magic, and you will need to spend some significant time running through all the basics such as getting e-mail notification of problems up and running, figuring out the networking, getting permissions sorted, etc.
Do NOT buy a NAS unit from a fly-by-night vendor. You really need to stick to a name brand such as QNAP or Synology. These companies both specialize in NAS and have a proven track record of releasing firmware for existing devices for years. This is highly important because of security fixes, etc. IoT devices tend to have a problem with ongoing firmware development, and if you bought one of the many off-brand NAS units from a decade ago, where a company momentarily dabbled in NAS before deciding there was little profit, you probably only got one or two firmware updates -- if any. QNAP and Synology sidestep this because they fund firmware development from sales of their newer units. This still isn't a great model, but it solves the problem of paying the developers.
Your second-best option, in terms of drama, is to go with FreeNAS! FreeNAS is a full enterprise-grade managed NAS platform with a web GUI interface, a good manual, and an enthusiastic community. Feel free to visit the FreeNAS forums and check out the hardware guides and community. There are inexpensive options to do it yourself, and there are also prebuilt options available that are ready-to-go (but a little pricey in my opinion). There is a huge upside in that, unlike the Synology or QNAP, you do have the option to easily run a different operating system on the device if you ever become unhappy with FreeNAS, and a FreeNAS host can be scaled to be massively more capable than any Syno or QNAP.
posted by jgreco at 4:15 AM on January 30, 2020 [7 favorites]
I've been running home servers for about 20 years, using various flavors of Linux and a variety of services. A few weeks ago I bought a Synology NAS (the DS1019+) and to be honest, it's really nice and I'm likely to use it for much more than just storage. My intent was only to depend on it for storage, and as a backend for Plex.
What gives me pause is "miscellaneous cronjobs" which... could be anything. I'm not sure that what you want to do with misc cronjobs is supported on the Synology. On mine, you can just SSH into the box and get a shell, and it basically seems to be a Debian-derived OS.
That said, the Synology model I have allows running virtual machines and/or Docker containers. So you could spin up a VM or Linux container to run a more free-form application that may not be supported natively on the Synolgoy.
The dings against the Synology I have are 1) It's overkill if you're not looking to store a lot of data, and 2) the company is awful about providing source under its obligations under the GPL and other licenses, and I have some concerns about how the overall roadmap for the OS and apps. If they decide, for example "meh, we're changing the way XYZ works, deal with it" users are sort of in a fix.
But it's easy. To answer the questions directly...
0. I don't know if a forum for home server users in general. I'd look to forums for a specific Linux distribution or r/synology if you go that route.
1. If you go straight-up Linux server, I'd go with CentOS or Ubuntu LTS. Those are the two you're most likely to find answers to any questions on StackExchange or tutorials for doing something on the distro. My personal preference is CentOS, but I'm biased.
2. That stuff can make sense on a single box, but I suspect this starts to drift into the 'new hobby' territory for you.
One thing you could do is to set up a system and run a virtual machine for all your services except storage, and then take snapshots of the VM regularly to roll back if something goes wrong. A CentOS box with Cockpit would be reasonably easy to manage.
I don't have good answers on monitoring. I just make a practice to do updates regularly. My home Plex server gets updated every time I log into the Plex web client and it grumbles about an update. The server that runs my site gets updated about once a week, or when I hear about a CVE that affects RHEL/CentOS.
TBH, a Synology is probably the better answer for you if you're averse to diving into the technologies. I just wish they were better open source citizens and a lot more transparent about development.
posted by jzb at 5:13 AM on January 30, 2020
What gives me pause is "miscellaneous cronjobs" which... could be anything. I'm not sure that what you want to do with misc cronjobs is supported on the Synology. On mine, you can just SSH into the box and get a shell, and it basically seems to be a Debian-derived OS.
That said, the Synology model I have allows running virtual machines and/or Docker containers. So you could spin up a VM or Linux container to run a more free-form application that may not be supported natively on the Synolgoy.
The dings against the Synology I have are 1) It's overkill if you're not looking to store a lot of data, and 2) the company is awful about providing source under its obligations under the GPL and other licenses, and I have some concerns about how the overall roadmap for the OS and apps. If they decide, for example "meh, we're changing the way XYZ works, deal with it" users are sort of in a fix.
But it's easy. To answer the questions directly...
0. I don't know if a forum for home server users in general. I'd look to forums for a specific Linux distribution or r/synology if you go that route.
1. If you go straight-up Linux server, I'd go with CentOS or Ubuntu LTS. Those are the two you're most likely to find answers to any questions on StackExchange or tutorials for doing something on the distro. My personal preference is CentOS, but I'm biased.
2. That stuff can make sense on a single box, but I suspect this starts to drift into the 'new hobby' territory for you.
One thing you could do is to set up a system and run a virtual machine for all your services except storage, and then take snapshots of the VM regularly to roll back if something goes wrong. A CentOS box with Cockpit would be reasonably easy to manage.
I don't have good answers on monitoring. I just make a practice to do updates regularly. My home Plex server gets updated every time I log into the Plex web client and it grumbles about an update. The server that runs my site gets updated about once a week, or when I hear about a CVE that affects RHEL/CentOS.
TBH, a Synology is probably the better answer for you if you're averse to diving into the technologies. I just wish they were better open source citizens and a lot more transparent about development.
posted by jzb at 5:13 AM on January 30, 2020
If you want an actual server instead of just a NAS, run the free vSphere hypervisor as your host layer. I’ve been doing this for years. This gives you the flexibility to run multiple VMs if needed. My current box is a Core2Quad cpu with 8 GB of memory and a 2TB hard drive so you don’t need expensive hardware. I only have a Ubuntu VM and a Windows VM currently but have had more in the past.
posted by LoveHam at 5:42 AM on January 30, 2020
posted by LoveHam at 5:42 AM on January 30, 2020
Response by poster: Oops, yeah, to clarify "miscellaneous cronjobs", our current examples:
1. A Python script that scrapes the local parks and rec dept website for unexpected closures.
2. A small Typescript/React website that renders a table I need for a hobby. Imagine I'm into movie trivia and it lists recent movies vs their actors, and you'll be in the right ballpark.
3. Some Calibre recipes that scrape websites into an ebook periodically.
These seemed eclectic enough that I thought I needed a general-purpose server. I didn't want to end up on some forum asking people, "I'm trying to install npm on my toaster and I'm getting this dependency error..." Maybe that's wrong?
I also have a bit of fear about black-box appliances. I don't know them and I largely don't trust their manufacturers not to decide that my use case isn't profitable enough.
posted by meaty shoe puppet at 7:07 AM on January 30, 2020 [1 favorite]
1. A Python script that scrapes the local parks and rec dept website for unexpected closures.
2. A small Typescript/React website that renders a table I need for a hobby. Imagine I'm into movie trivia and it lists recent movies vs their actors, and you'll be in the right ballpark.
3. Some Calibre recipes that scrape websites into an ebook periodically.
These seemed eclectic enough that I thought I needed a general-purpose server. I didn't want to end up on some forum asking people, "I'm trying to install npm on my toaster and I'm getting this dependency error..." Maybe that's wrong?
I also have a bit of fear about black-box appliances. I don't know them and I largely don't trust their manufacturers not to decide that my use case isn't profitable enough.
posted by meaty shoe puppet at 7:07 AM on January 30, 2020 [1 favorite]
I run a server like this. The "correct" answer to your question is devops; some set of automation scripts that let you deploy all your services reliably on new machines. That's what the big boys and girls do with their cloud computing systems. However it's an enormous amount of work and feels like ludicrous overkill for a single home server.
So instead I continue to just go along hand-editing crontabs, Apache configs, etc. Over time I've slowly taken advantage of modularization where I can. That Python script you have should be set up to run in its own virtual environment, don't rely on the system Python libraries. Make your Typescript website a separate Apache virtual host (if using Apache). Put your Calibre scraping stuff in its own directory with a README and clear management of state files.
Use a simple OS. For me that's Ubuntu 18.04 LTS, soon to be Ubuntu 20.04 LTS. It updates every two years, which is just about the right rhythm for a home server. I would not mess with the NAS systems unless you really want something that's more like an appliance and less like a Unix sedrver.
For bigger stuff like Pi-Hole or Home Assistant I'm starting to experiment with containers like Snap or Docker. Pi-Hole in particular makes a fucking mess if you install it with its own scripts (including absolute garbage like a startup job that rewrites /etc/resolv.conf, a script the uninstaller doesn't disable.) Home Assistant via Docker was completely easy to both install and remove when I decided not to use it. You still have the problem of maintaining the services inside the container, but at least they are hermetically sealed from the rest of the environment.
For monitoring I use good ol' Munin. It's ancient, but the stock Ubuntu install works great and you can add stuff to it easily. It doesn't really alert though, just provides you graphs you should look at.
For backups I still like rsnapshot.
posted by Nelson at 7:47 AM on January 30, 2020
So instead I continue to just go along hand-editing crontabs, Apache configs, etc. Over time I've slowly taken advantage of modularization where I can. That Python script you have should be set up to run in its own virtual environment, don't rely on the system Python libraries. Make your Typescript website a separate Apache virtual host (if using Apache). Put your Calibre scraping stuff in its own directory with a README and clear management of state files.
Use a simple OS. For me that's Ubuntu 18.04 LTS, soon to be Ubuntu 20.04 LTS. It updates every two years, which is just about the right rhythm for a home server. I would not mess with the NAS systems unless you really want something that's more like an appliance and less like a Unix sedrver.
For bigger stuff like Pi-Hole or Home Assistant I'm starting to experiment with containers like Snap or Docker. Pi-Hole in particular makes a fucking mess if you install it with its own scripts (including absolute garbage like a startup job that rewrites /etc/resolv.conf, a script the uninstaller doesn't disable.) Home Assistant via Docker was completely easy to both install and remove when I decided not to use it. You still have the problem of maintaining the services inside the container, but at least they are hermetically sealed from the rest of the environment.
For monitoring I use good ol' Munin. It's ancient, but the stock Ubuntu install works great and you can add stuff to it easily. It doesn't really alert though, just provides you graphs you should look at.
For backups I still like rsnapshot.
posted by Nelson at 7:47 AM on January 30, 2020
I vote, like others, for a dedicated NAS for backups. I'm about to pull the trigger on a Synology myself. I also have a Raspberry Pi 3 that runs a bunch of things:
- pi-hole
- pivpn
- Piaware ADSB flight tracking
- It was a grafana server for awhile when I was messing around with a DIY weather station
- SSH jumpbox for when I don't want/need to VPN back to the home network
The pi is small, low-power, and the bang for the buck in terms of capabilities are unmatched as far as I'm concerned. You could absolutely run your cron jobs on there and still have plenty of overhead to spare. Mine's barely breaking a sweat with 20+ devices and a 700K-deep blocklist.
posted by jquinby at 7:53 AM on January 30, 2020
- pi-hole
- pivpn
- Piaware ADSB flight tracking
- It was a grafana server for awhile when I was messing around with a DIY weather station
- SSH jumpbox for when I don't want/need to VPN back to the home network
The pi is small, low-power, and the bang for the buck in terms of capabilities are unmatched as far as I'm concerned. You could absolutely run your cron jobs on there and still have plenty of overhead to spare. Mine's barely breaking a sweat with 20+ devices and a 700K-deep blocklist.
posted by jquinby at 7:53 AM on January 30, 2020
This was a great resource for me to get my head around docker instances and getting a home server up and running using them.
Ultimately, it's an LTS Ubuntu server, but remote administration is nice and easy. I just used an old laptop since I'm serving media off a beefier PC using Plex already, but it works great for simple things like backups, pi-hole, etc.
posted by hankscorpio83 at 10:42 AM on January 30, 2020
Ultimately, it's an LTS Ubuntu server, but remote administration is nice and easy. I just used an old laptop since I'm serving media off a beefier PC using Plex already, but it works great for simple things like backups, pi-hole, etc.
posted by hankscorpio83 at 10:42 AM on January 30, 2020
I've got a Synology NAS that's been chugging along for 4-5 years and still works great. I'll log into it every week or so to update packages or the OS but otherwise it just works with no fiddling. I use it for backups of my photos and for storing media. The reason I got the particular NAS I did was because I can use an app to log into the server and then have it directly play media to my TV and other devices.
posted by any portmanteau in a storm at 2:17 PM on January 30, 2020
posted by any portmanteau in a storm at 2:17 PM on January 30, 2020
+1 for Synology, that is what I use for storage and always recommend to my less nerdy friends. Have had 3 different models over 15 years and never lost any data or had any major issues.
Synology devices update themselves and are easy enough for non-techies to navigate, yet have pretty advanced capabilities and of course, a Linux shell underneath. They also have nice little GUI apps for running Docker containers which it sounds like you know would allow you to deploy most anything pretty easily without touching the main system.
The reddit you're looking for, although the people here generally have needlessly complicated setups for funsies, is r/homelab
Personally I use Proxmox, which is a Debian based hyper visor for running containers / virtual machines. I do that on a regular server and it uses my Synology for storage. So if you are more looking for the server deployment side rather than the storage side, and just want a nice GUI to manage stuff on top of Debian, Proxmox is worth a look.
posted by bradbane at 2:28 PM on January 30, 2020
Synology devices update themselves and are easy enough for non-techies to navigate, yet have pretty advanced capabilities and of course, a Linux shell underneath. They also have nice little GUI apps for running Docker containers which it sounds like you know would allow you to deploy most anything pretty easily without touching the main system.
The reddit you're looking for, although the people here generally have needlessly complicated setups for funsies, is r/homelab
Personally I use Proxmox, which is a Debian based hyper visor for running containers / virtual machines. I do that on a regular server and it uses my Synology for storage. So if you are more looking for the server deployment side rather than the storage side, and just want a nice GUI to manage stuff on top of Debian, Proxmox is worth a look.
posted by bradbane at 2:28 PM on January 30, 2020
« Older Trying to get over astrology trauma | How to let a romantic relationship grow without... Newer »
This thread is closed to new comments.
Why not just buy a Synology NAS and be done with it?
The pi-hole / ad-filtering ideally needs to be part of your network infrastructure. I use OpenWRT running on my network router.
posted by pharm at 1:06 AM on January 30, 2020 [7 favorites]