Minecraft Server + Kubernetes + GCloud + ... multiple preemptible nodes?

July 29, 2018 11:05 AM

I have questions about running Minecraft on Google Cloud using a Kubernetes cluster with multiple nodes.

After futzing about with lots of different examples, scripts, and random commands, I found this clear example that spins up a cluster and downloads a docker image which does all the heavy lifting of setting up the server. It runs great! However, I'm confused about how the cluster was set up.

The command in the shell script creates TWO nodes in the cluster, and marks them as preemptible. As I understand it, that means that either of them could drop at any time, but they'll come back if needed, and I expect it saves me money and they probably won't both go down at the same time.

But does Minecraft actually work that way? I see I have two VM's, but if two users connect to different servers, won't they be isolated from each other? If so, couldn't the game world get corrupted by actions from two machines? Or am I totally misunderstanding how this works? Also, if the nodes are preemptible, do I need to do something extra to make sure that minecraft exits cleanly? Is it even a good idea to use preemptible nodes? Finally, if I need to redo the setup to make it a single non-preemptible node, how do I make sure I don't lose the world data? (I played for way too long last night, I really like this world and don't want to start over!)

(Background info: I am doing things this way as a fun excuse to learn about Google Cloud and Kubernetes. I know I could run my own server at home, buy a Realm, get hosting through a MC-specific company, use some other cloud provider, etc. I've run a small [literally... a Raspberry Pi] server locally so I know a touch about it. I'm only looking to support myself and my kids on the server. I don't need 99% uptime, but I do want reliability when it's up. :-)

posted by rouftop to Computers & Internet (2 answers total)

Best answer: This looks like it creates a single-pod Deployment for the Minecraft server with a persistent disk configured. So there's only one Minecraft server running at a time, and if the node it's running on goes down a replacement pod will come up on the other server, attached to the same persistent storage.

I think the idea is that the nodes are made preemptible to save some money. Looks like two preemptible nodes ends up being about half a price of one non-preemptible. Still, if one goes down, there will be a short amount of downtime while a replacement pod is started on the other node. Not sure what the risk is of both going down at the same time.

Caveat: I know Kubernetes well, but use it on AWS not gcloud, and I don't know anything about Minecraft servers.
posted by borsboom at 12:04 PM on July 29, 2018 [2 favorites]

Best answer: borsboom's explanation is correct, but to elaborate: The deployment.yaml file defines three different "resources", which are pieces of configuration that tell the Kubernetes master to create different kinds of objects.

• The Service object is a pointer to a set of pods -- in this case, all pods with the label "mc-server", which initially don't exist yet. Because the service type is "LoadBalancer", it's instantiated by making an API call to create a Cloud Load Balancer (or an ELB, if you were running on Amazon) which is automatically updated to point to whichever pods are currently running.
• The PersistentVolumeClaim is a request for persistent storage; as long as the claim exists, the storage won't be reused. Kubernetes automatically creates a matching PersistentVolume object (a GCE persistent disk, or an EBS volume on Amazon) with default settings and binds it to the claim object.
• The Deployment is a template that tells Kubernetes how to create "mc-server" pods. It tells the scheduler that there should always be a certain number of healthy matching pods (defaulting to 1), and if it ever sees a different number, it should take action by starting or stopping pods.

These objects are persistent -- they stick around until you update or delete them. Once you define them, the Deployment controller notices that an "mc-server" pod should be running, and starts one. The new pod's template includes a reference to the PersistentVolumeClaim that holds the server data, which tells Kubernetes to attach that volume to the node and mount its filesystem into the server container. When the container starts and Kubernetes decides the pod is "alive", the Service notices that a new pod with the "mc-server" label exists, and updates the load balancer to point to the pod's IP address.

If a node gets preempted, the server process will exit uncleanly, along with the Kubernetes daemons running on that node. After a few seconds or minutes, when the master notices that the node doesn't exist anymore, it will mark the node as dead, and start a new pod using the same persistent volume on the other node.

(Hopefully, Minecraft is designed to not corrupt its data if it happens to get killed in the middle of saving the world. A pull request was merged recently that should make this a bit better: it allows Kubernetes to take advantage of the 30-second warning for preemptible instances, so that the server process would get killed by a SIGTERM, instead of just dying along with the machine.)

If you were to set the "replicas" field of the Deployment to something other than 2, the system would try to start two instances of the server. Because a persistent disk can only be mounted to one node at a time, this might or might not succeed, depending on whether the two instances landed on the same node. If it did succeed, then yes, you'd get two server processes clobbering each others' data, and the load balancer would send each user to one or the other at random.
posted by teraflop at 1:47 PM on July 29, 2018 [1 favorite]