Mindmajix

Managing swift cluster capacity – OpenStack

Because Swift is a distributed system that is controlled by software, does not rely on RAID, and writes multiple copies of each object (file), operating a Swift cluster is fundamentally different from operating traditional storage systems such as storage area networks (SAN) or using network-attached storage (NAS) equipment.

We consider, a zone as a group of nodes that is as isolated as possible from other nodes (separate servers, network, power, even geography). A Swift ring functions like a cereal box decoder ring, allowing the swift services to locate each objects. The ring guarantees that every replica is stored in a separate zone. To increase capacity in our environment, we can add an extra zone, to which data will then replicate. In this example, we will add an extra storage node with IP 172.16.0.212, with its second disk, /dev/sdb, used for our OpenStack with IP 172.16.0.212 Storage. This node makes up the only node in this zone.

To add additional capacity to existing zones, we repeat the instructions for each existing zone in our cluster. For example, the following steps assume zone 5 (z5) does not exist, so this gets created when we build the rings. To simply add additional capacity to existing zones, we specify the new servers in the existing zones (zones 1-4). The instructions remain the same throughout.

Getting started

Log in to the OpenStack Object Storage proxy server node as well as a new storage node (that will form the basis of our new zone).

How to achieve it…

To add an extra zone to our OpenStack Object Storage cluster, carry out the following steps:

Proxy Server

  • Add the following entries to the ring where STORAGE_LOCAL_NET_IP is the IP address of our new node and ZONE is our new zone:

Note

Ensure you run these commands while in the /etc/swift directory.

cd /etc/swift
ZONE=5
STORAGE_LOCAL_NET_IP=172.16.0.212
WEIGHT=100
DEVICE=sdb1
swift-ring-builder account.builder add z$ZONE-
$STORAGE_LOCAL_NET_IP:6002/$DEVICE $WEIGHT swift-
ring-builder container.builder add z$ZONE-
$STORAGE_LOCAL_NET_IP:6001/$DEVICE $WEIGHT swift-
ring-builder object.builder add z$ZONE-
$STORAGE_LOCAL_NET_IP:6000/$DEVICE $WEIGHT
  • We need to verify the contents of the rings by issuing the following commands:
swift-ring-builder account.builder 
swift-ring-builder container.builder s
wift-ring-builder object.builder
  • Finally, we rebalance the rings, which could take some time to run:
swift-ring-builder account.builder rebalance 
swift-ring-builder container.builder rebalance 
swift-ring-builder object.builder rebalance
  • Once this has finished, we need to copy ring.gz, container.ring.gz, and object.ring.gz over to our new storage node and all other storage nodes:
scp *.ring.gz $STORAGE_LOCAL_NET_IP:/tmp
# And other scp to other storage nodes

Storage Node

  • We first move the copied ring.gz, container.ring.gz, and object.ring.gz files to the /etc/swift directory and ensure they’re owned by swift:
mv /tmp/*.ring.gz /etc/swift
chown swift:swift /etc/swift/*.ring.gz

Prepare the storage on this node, as described in the section Preparing drives for OpenStack Object Storage.

  • Edit the /etc/swift/swift.conf file, so that the [swift-hash] section matches that of all other nodes, as follows:
[swift-hash]
# Random unique string used on all nodes
 swift_hash_path_suffix = QAxxUPkzb7lP29OJ
  • We now need to create the appropriate the /etc/rsyncd.conf file with the following contents:
uid = swift 
gid = swift
log file = /var/log/rsyncd.log 
pid file = /var/run/rsyncd.pid 
address = 172.16.0.4
[account]
max connections = 2 
path = /srv/node/ 
read only = false
lock file = /var/lock/account.lock
[container]
max connections = 2 
path = /srv/node/ 
read only = false
lock file = /var/lock/container.lock
[object]
max connections = 2 
path = /srv/node/ 
read only = false
lock file = /var/lock/object.lock
  • Enable and start rsync, as follows:
sed -i 's/=false/=true/' /etc/default/rsync service rsync start
  • We need to create the /etc/swift/account-server.conf file with the following contents:
[DEFAULT]
bind_ip = 172.16.0.212 workers = 2
[pipeline:main]
pipeline = account-server
[app:account-server] 
use = egg:swift#account
[account-replicator]
[account-auditor]
[account-reaper]
  • Also create the /etc/swift/container-server.conf file with the following contents:
[DEFAULT]
bind_ip = 172.16.0.212 
workers = 2
[pipeline:main]
pipeline = container-server
[app:container-server] 
use = egg:swift#container
[container-replicator]
[container-updater]
[container-auditor]
  • Finally, create the /etc/swift/object-server.conf file with the following contents:
[DEFAULT]
bind_ip = 172.16.0.212 
workers = 2
[pipeline:main]
pipeline = object-server
[app:object-server] 
use = egg:swift#object
[object-replicator]
[object-updater]
[object-auditor]
  • We can now start this storage node, which we have configured to be in our fifth zone, as follows:
swift-init      all start

How it works…

Adding extra capacity by adding additional nodes or zones is done in the following two steps:

  • Configuring the zones and nodes on the proxy server
  • Configuring the storage node(s)

For each storage node and the devices on those storage nodes, we run the following command, which adds the storage node and device to our new zone:

'swift-ring-builder object.builder add zzone-storage_ip:6000/device weight

Once this has been configured on our proxy node, we rebalance the rings. This updates the object, account, and container rings. We copy the updated gzipped files as well as the swift hash key used within our environment, to all our storage node(s).

On the storage node, we simply run through the following steps:

  • Configure the disk (partition and format with XFS)
  • Configure and start rsyncd
  • Configure the account, container, and object services
  • Start the OpenStack Object Storage services on the storage node(s)

Data is then redistributed within our OpenStack Object Storage environment onto this new zone’s node.

 

 

 


 

 

0 Responses on Managing swift cluster capacity – OpenStack"

Leave a Message

Your email address will not be published. Required fields are marked *

Copy Rights Reserved © Mindmajix.com All rights reserved. Disclaimer.
Course Adviser

Fill your details, course adviser will reach you.