OpenStack – Removing nodes from a cluster
Converse to adding capacity to our OpenStack Object Storage cluster, there may be times where we need to scale back, or remove a failed node for service. You may need to remove a compute node from a cluster for troubleshooting or maintenance reasons.
We can do this by removing nodes from the zones in our cluster. In the following example, we will remove the node 172.16.0.212 in z5, which only has one storage device attached, /dev/sdb1.
Log in to the OpenStack Object Storage Proxy Server. To log on to our OpenStack Object Storage Proxy host that was created using Vagrant, issue the following command:
vagrant ssh swift
How to achieve it…
Carry out the following to remove a storage node from a zone:
- To remove a node from OpenStack Object Storage, we first set its weight to be 0, so that when the rings get rebalanced, data is drained away from this node:
swift-ring-builder account.builder set_weight z5-
swift-ring-builder container.builder set_weight z5-
swift-ring-builder object.builder set_weight z5-
- We then rebalance the rings as follows:
swift-ring-builder account.builder rebalance
swift-ring-builder container.builder rebalance
swift-ring-builder object.builder rebalance
- Once this is done, we can remove the node in this zone from the ring, as follows:
swift-ring-builder account.builder remove z5-
swift-ring-builder container.builder remove z5-
swift-ring-builder object.builder remove z5-
- We then copy the resultant ring.gz, container.ring.gz, and object.ring.gz files over to the rest of the nodes in our cluster. We are now free to decommission this storage node by physically removing this device.
How it works…
Manually removing a node from our OpenStack Object Storage cluster is done in three steps:
- Setting the node’s weight to be 0, so the data isn’t being replicated to it, by using the swift-ring-builder <ring> set-weight command.
- Rebalancing the rings to update the data replication.
Removing the node from OpenStack object storage cluster, using the swift-ring-builder <ring> remove command. Once done, we are then free to decommission that node. We repeat it for each node (or device) in the zone.