where to look for a server migration consultant
January 27, 2024 6:49 AM Subscribe
I need help locating a consultant who can help a small business migrate a critical server
This business needs to create a hot spare server for a live production machine for fail-over and disaster protection. Just had a close call after a power failure and ...oops...this machine can't go down.
I know there's a lot of technical expertise on Mefi, so why not start here.
This box is running Ubuntu server v. 12 with postgres/SQL, with two sets of RAID volumes driven by mdadm (one RAID is mirrored for the OS, second volume is RAID10 for data).
I need an expert who can do a remote analysis, deliver a recipe/documentation for a no-egg-will-fall process, and then be available remotely while we do the thing.
The goal is a hot spare box running 24-7 where the live data is backed up to it regularly so it's always up to date or close to it. Bonus points for also creating a virtual hot spare under a hypervisor.
Where can I start asking for this person or business? What searches might lead me to this business? Really don't want to do Upwork. I'm guessing 10 hours of consulting time as an estimate.
Thanks in advance. Just had a close call and this is on the front burner.
This business needs to create a hot spare server for a live production machine for fail-over and disaster protection. Just had a close call after a power failure and ...oops...this machine can't go down.
I know there's a lot of technical expertise on Mefi, so why not start here.
This box is running Ubuntu server v. 12 with postgres/SQL, with two sets of RAID volumes driven by mdadm (one RAID is mirrored for the OS, second volume is RAID10 for data).
I need an expert who can do a remote analysis, deliver a recipe/documentation for a no-egg-will-fall process, and then be available remotely while we do the thing.
The goal is a hot spare box running 24-7 where the live data is backed up to it regularly so it's always up to date or close to it. Bonus points for also creating a virtual hot spare under a hypervisor.
Where can I start asking for this person or business? What searches might lead me to this business? Really don't want to do Upwork. I'm guessing 10 hours of consulting time as an estimate.
Thanks in advance. Just had a close call and this is on the front burner.
Can the work be done remotely (via VPN) or is there a need to be in a data center, in-person?
posted by wenestvedt at 10:07 AM on January 27
posted by wenestvedt at 10:07 AM on January 27
You might try DPE Systems in Seattle. I worked with them at various employers for over 20 years, though the last time was over a decade ago. They advertise nationwide coverage, and if they're anything like what they were 12 years ago, they're smart, responsive and good to work with.
I think a good search term to use would be "IT Consultants" followed by your city name.
posted by lhauser at 3:58 PM on January 27
I think a good search term to use would be "IT Consultants" followed by your city name.
posted by lhauser at 3:58 PM on January 27
Response by poster: This can all be done remotely via VPN. No need to have someone on site. We have qualified people on staff but this is a bit above their pay grade. I don't need anyone with root or admin on our server, what I need is someone on tap who's done similar projects, knows the territory, and could really speed this up if we have knowledgeable expertise we can tap into.
posted by diode at 4:00 PM on January 27
posted by diode at 4:00 PM on January 27
Best answer: I can tell you how I'd do it (and in fact do it for several clients, coincidentally, starting with Ubuntu 12.04), but I'm not the guy to do the actual job. Since it's an active/passive setup it's pretty simple and the tools will keep you from shooting yourself in the foot unless you try really hard.
The simplest version involves not using VMs and setting up a new server configured exactly like the existing one (preferably with a version of Ubuntu that isn't out of support!) except that you put DRBD between the md device and the filesystem.
I wouldn't do that today, though. Better to set up two new VM servers with DRBD-on-md for your VM storage and creating a new VM server to run your actual application within.
If you have staff familiar with Linux administration, I guarantee they can figure this out. There's a lot of documentation online if you can get past the people saying you need Ceph or whatever. For a simple two node active/passive cluster, you don't need clustered filesystems or CLVM or distributed locking or any of that stuff. Strictly speaking, you don't even need a cluster management stack, so long as it's fine if there's a few minutes of downtime when the active server lets the magic smoke out and you've got someone on staff who can tell the other server to become primary, mount the VM storage, and start the VM on your secondary server.
posted by wierdo at 4:13 PM on January 27
The simplest version involves not using VMs and setting up a new server configured exactly like the existing one (preferably with a version of Ubuntu that isn't out of support!) except that you put DRBD between the md device and the filesystem.
I wouldn't do that today, though. Better to set up two new VM servers with DRBD-on-md for your VM storage and creating a new VM server to run your actual application within.
If you have staff familiar with Linux administration, I guarantee they can figure this out. There's a lot of documentation online if you can get past the people saying you need Ceph or whatever. For a simple two node active/passive cluster, you don't need clustered filesystems or CLVM or distributed locking or any of that stuff. Strictly speaking, you don't even need a cluster management stack, so long as it's fine if there's a few minutes of downtime when the active server lets the magic smoke out and you've got someone on staff who can tell the other server to become primary, mount the VM storage, and start the VM on your secondary server.
posted by wierdo at 4:13 PM on January 27
Best answer: What hypervisor do you expect to be using long term? Find a local expert in migrating to that platform.
If you have the time and budget, I'd suggest something along the lines of what weirdo suggested. Dual production machines running a modern LTS version on hypervisors behind a load balancer, preferably with a DR version at a different site. Decoupling the database server from the app server can also be a good idea.
That gets you a lot closer to not risking going down and makes doing upgrades to prod so much easier.
posted by Candleman at 4:58 PM on January 27
If you have the time and budget, I'd suggest something along the lines of what weirdo suggested. Dual production machines running a modern LTS version on hypervisors behind a load balancer, preferably with a DR version at a different site. Decoupling the database server from the app server can also be a good idea.
That gets you a lot closer to not risking going down and makes doing upgrades to prod so much easier.
posted by Candleman at 4:58 PM on January 27
Response by poster: weirdo, i've just started ringing bells around going to VM's and it's going to take staff time to get their heads wrapped around this. I need to find a experienced admin who can hand hold us, nothing on site, until we are migrated to the dual VM setup you suggest. Any suggestions there? I am significantly paranoid that our current attempts to migrate are (cough...lame) not really helping so I need some Linux firepower to get us pointed in the right direction.
posted by diode at 8:41 PM on January 27
posted by diode at 8:41 PM on January 27
Response by poster: personally, I've been learning proxmox but it's not clear to me that's the right tool for this job as yet.
posted by diode at 8:44 PM on January 27
posted by diode at 8:44 PM on January 27
To be honest (and I hope this doesn't come across as offensive), if your server is really running Ubuntu 12.04, I would suggest doing a company/management self-introspection on why your organization is running a version of Ubuntu that stopped receiving security updates more than 5 years ago. Most small businesses are hanging by a thread, so that is a totally understandable reason. Plus, I'm sure most Fortune 500 companies have outdated software.
You could use this power failure as an opportunity to evaluate if cloud options like AWS Lightsail or Digital Ocean would be worth your while. Just a suggestion. Having on-premise servers can save money, but that's only if you have the staff resources and expertise to do so appropriately.
Cloud companies like AWS have back up power generators, high availability databases with easy back ups, and the option to host servers in different regions/cities for increased resilience. Digital Ocean and Hetzner are other cloud companies. There are also companies like Heroku that aim to make the process of cloud computing simpler than AWS.
Programming meetups on meetup.com is one way to find reputable programmers and Linux enthusiasts. You could try searching to see if there's a Python or Linux meetup in your local area.
posted by mundo at 11:14 PM on January 27
You could use this power failure as an opportunity to evaluate if cloud options like AWS Lightsail or Digital Ocean would be worth your while. Just a suggestion. Having on-premise servers can save money, but that's only if you have the staff resources and expertise to do so appropriately.
Cloud companies like AWS have back up power generators, high availability databases with easy back ups, and the option to host servers in different regions/cities for increased resilience. Digital Ocean and Hetzner are other cloud companies. There are also companies like Heroku that aim to make the process of cloud computing simpler than AWS.
Programming meetups on meetup.com is one way to find reputable programmers and Linux enthusiasts. You could try searching to see if there's a Python or Linux meetup in your local area.
posted by mundo at 11:14 PM on January 27
Best answer: Personally, I don't bother with Proxmox. There's nothing wrong with it, but it's not really useful for my particular use case, which is running one or two VMs that would otherwise be physical servers. I just install virt-manager (or cockpit-machines) and install them like they were physical. I'm not using VM servers so I can consolidate or so I can spin up a bunch of VMs. I just do it to make it so the server or servers aren't dependent on a particular piece of hardware.
If you have a couple of spare desktop PCs sitting around that are new enough to support virtualization (read: less than 15 years old) you can use those to lab a solution. Your production systems should have at least two Ethernet ports, but one is fine for testing purposes while you're getting comfortable with setting up DRBD and installing a VM.
posted by wierdo at 10:08 AM on January 28
If you have a couple of spare desktop PCs sitting around that are new enough to support virtualization (read: less than 15 years old) you can use those to lab a solution. Your production systems should have at least two Ethernet ports, but one is fine for testing purposes while you're getting comfortable with setting up DRBD and installing a VM.
posted by wierdo at 10:08 AM on January 28
Response by poster: As mundo pointed out, running a hardware stack in house (6!!) with one major app per box is an outmoded technical model and a time suck to maintain. We have technical debt regarding one critical server, so that's the really crucial system that needs to be virtualized or migrated in a way we can have a hot spare backup in case of hardware meltdown until we can move to updated services for that function.
We haven't updated from the out of date OS due to lack of available resources in house...it needs quite a bit of development to get our postgres-Access-2010 server into an up to date OS. We're working on it now but in the interim there's that gap where we must have continuity of this one system and no real hot spare fallback other than our backups.
That's why I'm here fishing for insight on this. Next stop is google searches like: consulting, ubuntu, virtualization and see what pops up.
posted by diode at 4:12 PM on January 28
We haven't updated from the out of date OS due to lack of available resources in house...it needs quite a bit of development to get our postgres-Access-2010 server into an up to date OS. We're working on it now but in the interim there's that gap where we must have continuity of this one system and no real hot spare fallback other than our backups.
That's why I'm here fishing for insight on this. Next stop is google searches like: consulting, ubuntu, virtualization and see what pops up.
posted by diode at 4:12 PM on January 28
I'm slammed with an IRL issue today so I can't give a detailed answer but here's a few thoughts.
Is migrating to the cloud viable? It'll cost you more than bare metal but if management isn't putting enough resources staffing, it might be a decent solution.
If that doesn't work, a year ago I'd have told you to go with VMware, just because it's so easy to find consultants for it, but Broadcom has made it pretty clear that the future for small customers is grim after the acquisition.
Assuming that you can shore up the immediate potential disaster, the slower migration I'd consider doing (assuming this is both a database and app server) is build a new database server on a current version of Ubuntu as a VM, test it, put it into prod load balanced with the old one (if they're compatible enough). If that proves stable, create a duplicate VM, remove the old database server from production. Then do the same thing with the app server.
posted by Candleman at 5:23 PM on January 28 [1 favorite]
Is migrating to the cloud viable? It'll cost you more than bare metal but if management isn't putting enough resources staffing, it might be a decent solution.
If that doesn't work, a year ago I'd have told you to go with VMware, just because it's so easy to find consultants for it, but Broadcom has made it pretty clear that the future for small customers is grim after the acquisition.
Assuming that you can shore up the immediate potential disaster, the slower migration I'd consider doing (assuming this is both a database and app server) is build a new database server on a current version of Ubuntu as a VM, test it, put it into prod load balanced with the old one (if they're compatible enough). If that proves stable, create a duplicate VM, remove the old database server from production. Then do the same thing with the app server.
posted by Candleman at 5:23 PM on January 28 [1 favorite]
Response by poster: We cannot migrate to a cloud service because this one server is using postgres to talk to Acess 2010 on workstations on the local network. Doing that via the cloud would be a fail AFAIK. That's a lot of technical debt and moving this to a cloud service is too big a step. We have to have this server running for continuity until it's time to deprecate it. We are running RAID0 (OS) and RAID10 (data) under mdadm utility on this box.
Our current plan is to clone members of the RAID in order to reassemble a clone RAID on another box, thus a hot spare. It seems to me that imaging the vital server to a virtual machine running on our LAN would be safer and perhaps less challenging than what we are attempting to do, thus my post here.
Candleman - we are doing exactly that, we have a test DB server on current Ubuntu in development, it's that with our power supply failure on the critical server it became extremely apparent how critcal this one box is and that the backup isn't going to provide safe continuity of services.
posted by diode at 6:57 AM on January 29
Our current plan is to clone members of the RAID in order to reassemble a clone RAID on another box, thus a hot spare. It seems to me that imaging the vital server to a virtual machine running on our LAN would be safer and perhaps less challenging than what we are attempting to do, thus my post here.
Candleman - we are doing exactly that, we have a test DB server on current Ubuntu in development, it's that with our power supply failure on the critical server it became extremely apparent how critcal this one box is and that the backup isn't going to provide safe continuity of services.
posted by diode at 6:57 AM on January 29
Response by poster: Thanks for all the tips. It's been very helpful as I have a couple of potential solutions now that I can pursue and utilize.
posted by diode at 7:45 PM on January 29 [1 favorite]
posted by diode at 7:45 PM on January 29 [1 favorite]
« Older Languagefilter: “x will need replaced” | "No Mirroring" error on iPhone with Criterion... Newer »
You are not logged in, either login or create an account to post comments
posted by alex1965 at 8:15 AM on January 27 [1 favorite]