Docker to Linux for Windows Containers?
November 14, 2021 1:09 PM Subscribe
My vendor moved to a containers/microservices, which are all Windows and compromise 21 different dockerfiles in a compose. They're all fairly simple: setup IIS, unzip a package, run a few powershell scripts on a package. The developers on the team absolutely do not want to move to docker. There's a lot of non-technical issues here but migrating a giant docker compose file provided by a vendor and updating it as they update is not feasible. I'm hoping that there's a way to run the docker compose on a build server and generate say 80% of what's needed to get the microservices working. Again this is Windows, the vendor is not officially only supporting containers but the non container installs are buggy, out of date and have no documentation. Details within.
1. Team of around ~30 older developers at the end of their career and are not willing to move to Docker. Their current virtual machines they work on are on-prem and do not support nested virtualization e.g., Docker. I solved this by provided Azure VMs so they could use two VMs to transition but was rejected.
2. I was hired specifically to modernize the organization.
3. There's been a LOT of changes in the application regardless of containers (config files to environment variables for example), so there's a lot to absorb outside of learning Docker.
4. My initial thought and setup to reduce change was to keep their current VMs and add cloud VMs that support containers. I have approval for this but none of the developers want to simply learn docker or move.
5. There's no deadlines perse so taking 3 weeks to install a behemoth project manually isn't an issue. Same with fixing errors which would be bad practice like cleaning an environment when switching an environment etc.
6. I have given several workshops on this but even given "the vendor moved to this" isn't a good reason as the vendor will say technically you don't have to, leaving out the huge drawbacks.
7. I understand sunk costs but moving away from containers would lose months of time and have issues.
8. Upper management has my back but it may be too high in the organization. It is a very large company and the CTO simply isn't going to force 30 developers to change. I possibly could but would probably bring different issues such as every small issue with containers would be seen as an issue.
9. Any value brought by containers (easy upgrades, easy plugins, quick installs from scratch... this is not a simple SPA it is is complex) simply don't resonate because they aren't forced on deadlines.
The way the project was upgraded was that custom code would not need containers, but you'd need to read the dockercompose/dockerfiles to figure out how it is setup. Not ideal, but not difficult.
This is an uphill battle, I know, but the project is largely done. I receive no feedback on one on ones and workshops until I realized they just didn't weren't going to utilize the new upgrade with containers. To further illustrate the environment they claim to "automate" which means to them 50% of files are copied, and then there's a 50% manual effort. A large portion of this has been me culturally realizing they're just very, very vintage in the way they think. Before Covid they all had desktops (!) without laptops, did not use Slack/Teams/etc. This is a ~7k person company.
I'm open to any ideas on how to improve this but have been very transparent in what the vendor is requiring, the work I've done and setup READMEs, meetings, etc. My ideal scenario is that I setup a build server somewhere that does something like "Docker to Linux" but I'm having a hard time Googling this for Windows containers. Usually people are moving to containers and not away from it. I can do this manually but it is not within my scope and could take significant time. Also all machines are set up differently so it might be a wasted effort where a virtualized solution or standard cloud VM even without containers would help.
I'm up for any solution, technical or non-technical, but I've already implemented required changes to get the product updated and I think there's a sense I'm doing it for no reason and/or there's an overwhelming amount of change. I noticed their last consultants "cheated" did not really upgrade, kept the same source files, upgraded just the UI to look new and made some database hacks. I'm paying down a lot of technical debt. To be clear I'm doing least possible to upgrade while still actually upgrading, not doing anything for fun and not trying to do a perfect project. They just have a 9 year old product and frankly the software development world has moved quite a bit.
To give an idea of the complexity: installing a plugin used to involve simply uploading a zip file and clicking install. It now requires a separate 3 service docker compose file that must be added to the main project. It makes sense if you know the product and what they're trying to achieve by making upgrades easier and moving towards K8S. I have it down to a simple one line script that needs to be run to bring the entire project up, plugins and all, but even then the level of automation is intimidating when you're used to manually restoring databases, setting up web servers, etc.
Again, there's a lot of things culturally here I'm not trying to change. My question is simply how to get acceptance of containers and new technology required by the vendor. Or possibly some technical solution to use containers but provide a way for those who do not want to use it to do soin a similar fashion of Docker to Linux.
Sorry if this came off as venting I was just trying to explain that this is a uniquely conservative organization and the changes I'm implementing are the "minimal viable product" required by the software vendor.
1. Team of around ~30 older developers at the end of their career and are not willing to move to Docker. Their current virtual machines they work on are on-prem and do not support nested virtualization e.g., Docker. I solved this by provided Azure VMs so they could use two VMs to transition but was rejected.
2. I was hired specifically to modernize the organization.
3. There's been a LOT of changes in the application regardless of containers (config files to environment variables for example), so there's a lot to absorb outside of learning Docker.
4. My initial thought and setup to reduce change was to keep their current VMs and add cloud VMs that support containers. I have approval for this but none of the developers want to simply learn docker or move.
5. There's no deadlines perse so taking 3 weeks to install a behemoth project manually isn't an issue. Same with fixing errors which would be bad practice like cleaning an environment when switching an environment etc.
6. I have given several workshops on this but even given "the vendor moved to this" isn't a good reason as the vendor will say technically you don't have to, leaving out the huge drawbacks.
7. I understand sunk costs but moving away from containers would lose months of time and have issues.
8. Upper management has my back but it may be too high in the organization. It is a very large company and the CTO simply isn't going to force 30 developers to change. I possibly could but would probably bring different issues such as every small issue with containers would be seen as an issue.
9. Any value brought by containers (easy upgrades, easy plugins, quick installs from scratch... this is not a simple SPA it is is complex) simply don't resonate because they aren't forced on deadlines.
The way the project was upgraded was that custom code would not need containers, but you'd need to read the dockercompose/dockerfiles to figure out how it is setup. Not ideal, but not difficult.
This is an uphill battle, I know, but the project is largely done. I receive no feedback on one on ones and workshops until I realized they just didn't weren't going to utilize the new upgrade with containers. To further illustrate the environment they claim to "automate" which means to them 50% of files are copied, and then there's a 50% manual effort. A large portion of this has been me culturally realizing they're just very, very vintage in the way they think. Before Covid they all had desktops (!) without laptops, did not use Slack/Teams/etc. This is a ~7k person company.
I'm open to any ideas on how to improve this but have been very transparent in what the vendor is requiring, the work I've done and setup READMEs, meetings, etc. My ideal scenario is that I setup a build server somewhere that does something like "Docker to Linux" but I'm having a hard time Googling this for Windows containers. Usually people are moving to containers and not away from it. I can do this manually but it is not within my scope and could take significant time. Also all machines are set up differently so it might be a wasted effort where a virtualized solution or standard cloud VM even without containers would help.
I'm up for any solution, technical or non-technical, but I've already implemented required changes to get the product updated and I think there's a sense I'm doing it for no reason and/or there's an overwhelming amount of change. I noticed their last consultants "cheated" did not really upgrade, kept the same source files, upgraded just the UI to look new and made some database hacks. I'm paying down a lot of technical debt. To be clear I'm doing least possible to upgrade while still actually upgrading, not doing anything for fun and not trying to do a perfect project. They just have a 9 year old product and frankly the software development world has moved quite a bit.
To give an idea of the complexity: installing a plugin used to involve simply uploading a zip file and clicking install. It now requires a separate 3 service docker compose file that must be added to the main project. It makes sense if you know the product and what they're trying to achieve by making upgrades easier and moving towards K8S. I have it down to a simple one line script that needs to be run to bring the entire project up, plugins and all, but even then the level of automation is intimidating when you're used to manually restoring databases, setting up web servers, etc.
Again, there's a lot of things culturally here I'm not trying to change. My question is simply how to get acceptance of containers and new technology required by the vendor. Or possibly some technical solution to use containers but provide a way for those who do not want to use it to do soin a similar fashion of Docker to Linux.
Sorry if this came off as venting I was just trying to explain that this is a uniquely conservative organization and the changes I'm implementing are the "minimal viable product" required by the software vendor.
Geoff, that's heroic work. Well done ... but the social side of this tech upgrade will always play against you until it makes their jobs easier or allows this cohort of developers to do other stuff like fixing up the they've forgotten they want to fix up.
I've got some thoughts about the technical side I'll fill in after a night's sleep on my side of the planet. Stay heroic.
posted by k3ninho at 3:18 PM on November 14, 2021 [1 favorite]
I've got some thoughts about the technical side I'll fill in after a night's sleep on my side of the planet. Stay heroic.
posted by k3ninho at 3:18 PM on November 14, 2021 [1 favorite]
Say geoff, other ask.metafilter questions show you want to orchestrate these services using Kubernetes (or compatible) -- do you need resource isolation and on-demand scaling up or ephemeral/disposable scaling down with containers?
I'm pretty certain you want the modularity, but are you sure you can't get standardised deployment with Chocolatey or Octopus? (Or puppet/ansible/chef -- or specifically for K8s, Helm?) Octopus has web front-ends to make it plain to the team what state the system is in -- there's always gains from these changes which benefit observability so as to reduce revovery times when things go wrong.
Can you put these services into Linux-hosted containers running Windows binaries via Wine or running business logic restricted to dotNET Core compiled native to Linux?
posted by k3ninho at 12:28 AM on November 15, 2021 [1 favorite]
I'm pretty certain you want the modularity, but are you sure you can't get standardised deployment with Chocolatey or Octopus? (Or puppet/ansible/chef -- or specifically for K8s, Helm?) Octopus has web front-ends to make it plain to the team what state the system is in -- there's always gains from these changes which benefit observability so as to reduce revovery times when things go wrong.
Can you put these services into Linux-hosted containers running Windows binaries via Wine or running business logic restricted to dotNET Core compiled native to Linux?
posted by k3ninho at 12:28 AM on November 15, 2021 [1 favorite]
Response by poster: @k3ninho that's a great question. Before the product moved to Windows container microservices I did use Terraform and Ansible to build out environments. The problem now as I probably poorly put it is that the product is delivered on Windows container images so I was hoping for some easy to convert that. I also spent a considerable amount of time trying to get it to work with WINE and Linux (as have others) but unfortunately the application predates .NET and makes use of a lot of undocumented Windows features and/or bugs. Even if I got it working with WINE this is a conservative organization and we'd be breaking support on a large enterprise product.
My interest in K8S and Docker was truly driven by this application moving towards it. Not to get off topic of philosophical but this is a beast of an an application that requires 64 GB of RAM now to even run it locally. There's now 13 Windows servers "microservices" and 24 databases. I do not think this is the best solution but the product has been chosen already and the only way around this would be some sort of way to rearchitect the development setup by abstracting what's needed from container images. After thinking about it and working on the product as it has changed and got feature bloat yet still relying on legacy .NET Framework this is probably the best way of going about it, but again this is not my choice but the vendors and will hopefully be the last project I work on it. I could unfortunately tell you way too much about obscure undocumented Windows features, how Docker implements WIndows containers, etc.
In any case whether or not it sucks it is not a custom application and we must go with the chosen architecture. I have simplified it down to one single script to run. It is clear previous contractors did far worse in appeasing the client: it is simply broken which is why I was brought in.
I was hoping there was perhaps a technical solution but it appears not. I'd highly recommend not using Windows containers or even K8S unless there's a good business use case for it and there's buy-in from the team. It works, it is just complex for non-trivial applications and simply requires a different way of thinking.
Apologies for the long-question and long reply but was trying to best to set the background without giving away too much. If I find a way to build out a Windows docker compose solution, then script it automatically to extract what's needed I'll post it here. I'm sure it is possible, just don't know if it is the right way to go if it doesn't already exist.
posted by geoff. at 6:05 AM on November 15, 2021
My interest in K8S and Docker was truly driven by this application moving towards it. Not to get off topic of philosophical but this is a beast of an an application that requires 64 GB of RAM now to even run it locally. There's now 13 Windows servers "microservices" and 24 databases. I do not think this is the best solution but the product has been chosen already and the only way around this would be some sort of way to rearchitect the development setup by abstracting what's needed from container images. After thinking about it and working on the product as it has changed and got feature bloat yet still relying on legacy .NET Framework this is probably the best way of going about it, but again this is not my choice but the vendors and will hopefully be the last project I work on it. I could unfortunately tell you way too much about obscure undocumented Windows features, how Docker implements WIndows containers, etc.
In any case whether or not it sucks it is not a custom application and we must go with the chosen architecture. I have simplified it down to one single script to run. It is clear previous contractors did far worse in appeasing the client: it is simply broken which is why I was brought in.
I was hoping there was perhaps a technical solution but it appears not. I'd highly recommend not using Windows containers or even K8S unless there's a good business use case for it and there's buy-in from the team. It works, it is just complex for non-trivial applications and simply requires a different way of thinking.
Apologies for the long-question and long reply but was trying to best to set the background without giving away too much. If I find a way to build out a Windows docker compose solution, then script it automatically to extract what's needed I'll post it here. I'm sure it is possible, just don't know if it is the right way to go if it doesn't already exist.
posted by geoff. at 6:05 AM on November 15, 2021
Good luck, geoff.!
posted by k3ninho at 11:04 AM on November 15, 2021 [1 favorite]
posted by k3ninho at 11:04 AM on November 15, 2021 [1 favorite]
This thread is closed to new comments.
My suggestion would be to utilize the collective brain power of those 30 devs. In my experience, developer types can be extremely motivated to solve for their own productivity / developer experience issues if empowered to do so. What about an approach thats along the lines of "Hey folks - this is where we are at. Vendor has provided us this solution. We're going to have to adopt it by [insert some date here] unless we can think of something else that addresses concerns X, Y, Z" And then give them some time to figure it out - in general I've found people will be much more accepting of a solution that also addresses their concerns (why did they reject the Azure VMs?). And my guess is these folks are completely capable of solving this problem, probably more so than anyone else.
Your job here is to have very very very buttoned up requirements, because the risk here is they find a solution YOU dont like, but does meet all your communicated requirements. And then you're in a hard spot because "because I said so" wont go over at all as a reason to reject an idea they're pushing if it meets the parameters you set out.
posted by cgg at 3:10 PM on November 14, 2021 [1 favorite]