Openstack配置手册-Swift

snow chuai汇总、整理、撰写---2020/2/6


1. 拓扑
                     +-----------------------+     +-----------------------+
                     |    [ Control Node ]   |     |    [  Proxy Node  ]   |
                     |                       |     |                       |
                     |  MariaDB    RabbitMQ  |     |      Swift Proxy      |
                     |  Memcached  httpd     |     |                       |
                     |        Keystone       |     |                       |
                     +-----------------------+     +-----------------------+
                            eth0|192.168.10.11           eth0|192.168.10.12
     ----------------+-----------+-------------+--------------+-------------+-----------
             eth0|192.168.10.13           eth0|192.168.10.14           eth0|192.168.10.15
     +-----------+-----------+    +-----------+-----------+    +-----------+-----------+
     |   [ Storage Node#1 ]  |    |   [ Storage Node#2 ]  |    |   [ Storage Node#3 ]  |
     |                       |    |                       |    |                       |
     |     Swift-Account     |    |     Swift-Account     |    |     Swift-Account     |
     |    Swift-Container    |    |    Swift-Container    |    |    Swift-Container    |
     |     Swift-Object      |    |     Swift-Object      |    |     Swift-Object      |
     +-----------------------+    +-----------------------+    +-----------------------+
2. 在Control节点上配置Swift
[root@node1 ~(keystone)]# openstack user create --domain default --project service --password servicepassword swift
+---------------------+----------------------------------+
| Field               | Value                            |
+---------------------+----------------------------------+
| default_project_id  | 1be75e85ec9445ab9ff7dd7ec2f02b71 |
| domain_id           | default                          |
| enabled             | True                             |
| id                  | 7e25e413af67478c8ddcdab6f3c20fb0 |
| name                | swift                            |
| options             | {}                               |
| password_expires_at | None                             |
+---------------------+----------------------------------+
[root@node1 ~(keystone)]# openstack role add --project service --user swift admin
[root@node1 ~(keystone)]# openstack service create --name swift --description "OpenStack Object Storage" object-store +-------------+----------------------------------+ | Field | Value | +-------------+----------------------------------+ | description | OpenStack Object Storage | | enabled | True | | id | 551af505e6644b50af636f6b767bcc86 | | name | swift | | type | object-store | +-------------+----------------------------------+
[root@node1 ~(keystone)]# openstack endpoint create --region RegionOne object-store public http://192.168.10.12:8080/v1/AUTH_%\(tenant_id\)s +--------------+-------------------------------------------------+ | Field | Value | +--------------+-------------------------------------------------+ | enabled | True | | id | b1a63b5c297940cca52446bae28137a2 | | interface | public | | region | RegionOne | | region_id | RegionOne | | service_id | 551af505e6644b50af636f6b767bcc86 | | service_name | swift | | service_type | object-store | | url | http://192.168.10.12:8080/v1/AUTH_%(tenant_id)s | +--------------+-------------------------------------------------+
[root@node1 ~(keystone)]# openstack endpoint create --region RegionOne object-store internal http://192.168.10.12:8080/v1/AUTH_%\(tenant_id\)s +--------------+-------------------------------------------------+ | Field | Value | +--------------+-------------------------------------------------+ | enabled | True | | id | 6f3d4740c88d4373b6b0251e6caff2e7 | | interface | internal | | region | RegionOne | | region_id | RegionOne | | service_id | 551af505e6644b50af636f6b767bcc86 | | service_name | swift | | service_type | object-store | | url | http://192.168.10.12:8080/v1/AUTH_%(tenant_id)s | +--------------+-------------------------------------------------+
[root@node1 ~(keystone)]# openstack endpoint create --region RegionOne object-store admin http://192.168.10.12:8080/v1 +--------------+----------------------------------+ | Field | Value | +--------------+----------------------------------+ | enabled | True | | id | 3e5c2799b763455f946f4e671cf0421b | | interface | admin | | region | RegionOne | | region_id | RegionOne | | service_id | 551af505e6644b50af636f6b767bcc86 | | service_name | swift | | service_type | object-store | | url | http://192.168.10.12:8080/v1 | +--------------+----------------------------------+
3. 在Swift Proxy上设置Swift
1) 安装Swift-proxy等软件
[root@node2 ~]# yum --enablerepo=centos-openstack-queens,epel install openstack-swift-proxy python-memcached openssh-clients -y
2) 配置Swift-proxy root@node2 ~]# vim /etc/swift/proxy-server.conf [filter:cache] use = egg:swift#memcache
# 42行,指定Memcache Server的地址 memcache_servers = 192.168.10.11:11211 # 需改[filter:authtoken]区段信息,如下 [filter:authtoken] paste.filter_factory = keystonemiddleware.auth_token:filter_factory
# 注释91-97行 #admin_tenant_name = %SERVICE_TENANT_NAME% #admin_user = %SERVICE_USER% #admin_password = %SERVICE_PASSWORD% #auth_host = 127.0.0.1 #auth_port = 35357 #auth_protocol = http #signing_dir = /tmp/keystone-signing-swift # 于文档最后追加如下内容 www_authenticate_uri = http://192.168.10.11:5000 auth_url = http://192.168.10.11:5000 memcached_servers = 192.168.10.11:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = swift password = servicepassword delay_auth_decision = true
2) 设定Swift共享密码 [root@node2 ~]# vim /etc/swift/swift.conf [swift-hash] swift_hash_path_suffix = qyy_cloud swift_hash_path_prefix = qyy_cloud
3) 生成Swift Ring [root@node2 ~]# swift-ring-builder /etc/swift/account.builder create 12 3 1 [root@node2 ~]# swift-ring-builder /etc/swift/container.builder create 12 3 1 [root@node2 ~]# swift-ring-builder /etc/swift/object.builder create 12 3 1
[root@node2 ~]# swift-ring-builder /etc/swift/account.builder add r0z0-192.168.10.13:6202/device0 100 Device d0r0z0-192.168.10.13:6202R192.168.10.13:6202/device0_"" with 100.0 weight got id 0 [root@node2 ~]# swift-ring-builder /etc/swift/container.builder add r0z0-192.168.10.13:6201/device0 100 Device d0r0z0-192.168.10.13:6201R192.168.10.13:6201/device0_"" with 100.0 weight got id 0 [root@node2 ~]# swift-ring-builder /etc/swift/object.builder add r0z0-192.168.10.13:6200/device0 100 Device d0r0z0-192.168.10.13:6200R192.168.10.13:6200/device0_"" with 100.0 weight got id 0

[root@node2 ~]# swift-ring-builder /etc/swift/account.builder add r1z1-192.168.10.14:6202/device1 100 Device d1r1z1-192.168.10.14:6202R192.168.10.14:6202/device1_"" with 100.0 weight got id 1 [root@node2 ~]# swift-ring-builder /etc/swift/container.builder add r1z1-192.168.10.14:6201/device1 100 Device d1r1z1-192.168.10.14:6201R192.168.10.14:6201/device1_"" with 100.0 weight got id 1 [root@node2 ~]# swift-ring-builder /etc/swift/object.builder add r1z1-192.168.10.14:6200/device1 100 Device d1r1z1-192.168.10.14:6200R192.168.10.14:6200/device1_"" with 100.0 weight got id 1

[root@node2 ~]# swift-ring-builder /etc/swift/account.builder add r2z2-192.168.10.15:6202/device2 100 Device d2r2z2-192.168.10.15:6202R192.168.10.15:6202/device2_"" with 100.0 weight got id 2 [root@node2 ~]# swift-ring-builder /etc/swift/container.builder add r2z2-192.168.10.15:6201/device2 100 Device d2r2z2-192.168.10.15:6201R192.168.10.15:6201/device2_"" with 100.0 weight got id 2 [root@node2 ~]# swift-ring-builder /etc/swift/object.builder add r2z2-192.168.10.15:6200/device2 100 Device d2r2z2-192.168.10.15:6200R192.168.10.15:6200/device2_"" with 100.0 weight got id 2

[root@node2 ~]# swift-ring-builder /etc/swift/account.builder rebalance Reassigned 12288 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 [root@node2 ~]# swift-ring-builder /etc/swift/container.builder rebalance Reassigned 12288 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00 [root@node2 ~]# swift-ring-builder /etc/swift/object.builder rebalance Reassigned 12288 (300.00%) partitions. Balance is now 0.00. Dispersion is now 0.00

[root@node2 ~]# chown swift. /etc/swift/*.gz [root@node2 ~]# systemctl enable --now openstack-swift-proxy
4) 防火墙设定 [root@node2 ~]# firewall-cmd --add-port=8080/tcp --permanent success [root@node2 ~]# firewall-cmd --reload success
4. 设置Swift存储节点
1) 为所有存储节点安装Swift-Account, Swift-Container, Swift-Object等软件
[root@node2 ~]# pssh -h host-list.txt -i ' \
yum --enablerepo=centos-openstack-queens,epel install \
openstack-swift-account openstack-swift-container openstack-swift-object xfsprogs rsync openssh-clients -y'
[1] 19:55:30 [SUCCESS] root@192.168.10.13
[2] 19:55:30 [SUCCESS] root@192.168.10.14
[3] 19:55:30 [SUCCESS] root@192.168.10.15
2) 为所有存储节点分区、格式化并挂载 [root@node2 ~]# vim fdisk-vdc1.sh #! /bin/bash fdisk /dev/vdc << EOF n # 回车 # 回车 # 回车 # 回车 w EOF
[root@node2 ~]# pscp.pssh -h host-list.txt fdisk-vdc1.sh /root/ [1] 20:06:35 [SUCCESS] root@192.168.10.14 [2] 20:06:35 [SUCCESS] root@192.168.10.15 [3] 20:06:35 [SUCCESS] root@192.168.10.13
[root@node2 ~]# pssh -h host-list.txt -i 'fdisk /dev/vdc < /root/fdisk-vdc1.sh'
[root@node2 ~]# pssh -h host-list.txt -i 'lsblk | grep vdc' [1] 20:08:21 [SUCCESS] root@192.168.10.14 vdc 253:32 0 20G 0 disk └─vdc1 253:33 0 20G 0 part [2] 20:08:21 [SUCCESS] root@192.168.10.15 vdc 253:32 0 20G 0 disk └─vdc1 253:33 0 20G 0 part [3] 20:08:21 [SUCCESS] root@192.168.10.13 vdc 253:32 0 20G 0 disk └─vdc1 253:33 0 20G 0 part
[root@node2 ~]# pssh -h host-list.txt -i 'mkfs.xfs -i size=1024 -s size=4096 /dev/vdc1' [1] 20:09:04 [SUCCESS] root@192.168.10.13 meta-data=/dev/vdc1 isize=1024 agcount=4, agsize=1310656 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=0, sparse=0 data = bsize=4096 blocks=5242624, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [2] 20:09:04 [SUCCESS] root@192.168.10.14 meta-data=/dev/vdc1 isize=1024 agcount=4, agsize=1310656 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=0, sparse=0 data = bsize=4096 blocks=5242624, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [3] 20:09:04 [SUCCESS] root@192.168.10.15 meta-data=/dev/vdc1 isize=1024 agcount=4, agsize=1310656 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=0, sparse=0 data = bsize=4096 blocks=5242624, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0
# 建立各节点的挂载点并挂载 [root@node3 ~]# mkdir -p /srv/node/device0 [root@node3 ~]# mount -o noatime,nodiratime,nobarrier /dev/vdc1 /srv/node/device0 [root@node3 ~]# chown -R swift. /srv/node [root@node4 ~]# mkdir -p /srv/node/device1 [root@node4 ~]# mount -o noatime,nodiratime,nobarrier /dev/vdc1 /srv/node/device1 [root@node4 ~]# chown -R swift. /srv/node [root@node5 ~]# mkdir -p /srv/node/device2 [root@node5 ~]# mount -o noatime,nodiratime,nobarrier /dev/vdc1 /srv/node/device2 [root@node5 ~]# chown -R swift. /srv/node
# 添加fstab [root@node3 ~]# echo '/dev/vdc1 /srv/node/device0 xfs noatime,nodiratime,nobarrier 0 0' >> /etc/fstab [root@node4 ~]# echo '/dev/vdc1 /srv/node/device1 xfs noatime,nodiratime,nobarrier 0 0' >> /etc/fstab [root@node3 ~]# echo '/dev/vdc1 /srv/node/device2 xfs noatime,nodiratime,nobarrier 0 0' >> /etc/fstab
3) 复制Ring文件到所有节点并更改属主等信息 [root@node2 ~]# pscp.pssh -h host-list.txt /etc/swift/*.gz /etc/swift/ [1] 20:14:00 [SUCCESS] root@192.168.10.14 [2] 20:14:00 [SUCCESS] root@192.168.10.15 [3] 20:14:00 [SUCCESS] root@192.168.10.13
[root@node2 ~]# pssh -h host-list.txt -i 'chown swift. /etc/swift/*.gz' [1] 20:15:56 [SUCCESS] root@192.168.10.14 [2] 20:15:56 [SUCCESS] root@192.168.10.15 [3] 20:15:56 [SUCCESS] root@192.168.10.13
[root@node2 ~]# pscp.pssh -h host-list.txt /etc/swift/swift.conf /etc/swift/ [1] 20:16:51 [SUCCESS] root@192.168.10.14 [2] 20:16:51 [SUCCESS] root@192.168.10.13 [3] 20:16:51 [SUCCESS] root@192.168.10.15
4) 设置所有存储节点上的account、container、object配置文件 [root@node3 ~]# cd /etc/swift/ [root@node3 swift]# vim account-server.conf container-server.conf object-server.conf # 将三个文件的第6行,改为0.0.0.0 account-server.conf文件 ...... bind_ip = 0.0.0.0 bind_port = 6202 ...... container-server.conf文件 ...... bind_ip = 0.0.0.0 bind_port = 6201 ...... objecct-server.conf文件 ...... bind_ip = 0.0.0.0 bind_port = 6200 ......
5) 在所有的节点上配置rsync [root@node3 ~]# vim /etc/rsyncd.conf ...... ...... ...... ...... ...... ......
# 于最后添加如下内容 pid file = /var/run/rsyncd.pid log file = /var/log/rsyncd.log uid = swift gid = swift # 指定本机IP地址 address = 192.168.10.13
[account] path = /srv/node read only = false write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 25 lock file = /var/lock/account.lock
[container] path = /srv/node read only = false write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 25 lock file = /var/lock/container.lock
[object] path = /srv/node read only = false write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 25 lock file = /var/lock/object.lock
[swift_server] path = /etc/swift read only = true write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 5 lock file = /var/lock/swift_server.lock

[root@node4 ~]# vim /etc/rsyncd.conf ...... ...... ...... ...... ...... ......
# 于最后添加如下内容 pid file = /var/run/rsyncd.pid log file = /var/log/rsyncd.log uid = swift gid = swift # 指定本机IP地址 address = 192.168.10.14
[account] path = /srv/node read only = false write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 25 lock file = /var/lock/account.lock
[container] path = /srv/node read only = false write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 25 lock file = /var/lock/container.lock
[object] path = /srv/node read only = false write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 25 lock file = /var/lock/object.lock
[swift_server] path = /etc/swift read only = true write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 5 lock file = /var/lock/swift_server.lock

[root@node5 ~]# vim /etc/rsyncd.conf ...... ...... ...... ...... ...... ......
# 于最后添加如下内容 pid file = /var/run/rsyncd.pid log file = /var/log/rsyncd.log uid = swift gid = swift # 指定本机IP地址 address = 192.168.10.13
[account] path = /srv/node read only = false write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 25 lock file = /var/lock/account.lock
[container] path = /srv/node read only = false write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 25 lock file = /var/lock/container.lock
[object] path = /srv/node read only = false write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 25 lock file = /var/lock/object.lock
[swift_server] path = /etc/swift read only = true write only = no list = yes incoming chmod = 0644 outgoing chmod = 0644 max connections = 5 lock file = /var/lock/swift_server.lock

6) 在所有节点设定SELinux [root@node3 ~]# semanage fcontext -a -t swift_data_t /srv/node/device0 [root@node3 ~]# restorecon /srv/node/device0
[root@node4 ~]# semanage fcontext -a -t swift_data_t /srv/node/device1 [root@node4 ~]# restorecon /srv/node/device0
[root@node5 ~]# semanage fcontext -a -t swift_data_t /srv/node/device2 [root@node5 ~]# restorecon /srv/node/device0
7) 在所有节点设定防火墙 [root@node3 ~]# firewall-cmd --add-port={873/tcp,6200/tcp,6201/tcp,6202/tcp} --permanent success [root@node3 ~]# firewall-cmd --reload success
[root@node4 ~]# firewall-cmd --add-port={873/tcp,6200/tcp,6201/tcp,6202/tcp} --permanent success [root@node4 ~]# firewall-cmd --reload success
[root@node5 ~]# firewall-cmd --add-port={873/tcp,6200/tcp,6201/tcp,6202/tcp} --permanent success [root@node5 ~]# firewall-cmd --reload success
8) 在所有节点上启动服务 [root@node3 ~]# systemctl enable --now rsyncd [root@node3 ~]# for ringtype in account container object; do systemctl start openstack-swift-$ringtype systemctl enable openstack-swift-$ringtype for service in replicator updater auditor; do if [ $ringtype != 'account' ] || [ $service != 'updater' ]; then systemctl start openstack-swift-$ringtype-$service systemctl enable openstack-swift-$ringtype-$service fi done done
5. 使用Swift
1) 于keystone处添加swift服务
[root@node1 ~(keystone)]# openstack project create --domain default --description "Swift Service Project" swiftservice
+-------------+----------------------------------+
| Field       | Value                            |
+-------------+----------------------------------+
| description | Swift Service Project            |
| domain_id   | default                          |
| enabled     | True                             |
| id          | f1ca0edd1073414787a9e160d2d9d768 |
| is_domain   | False                            |
| name        | swiftservice                     |
| parent_id   | default                          |
| tags        | []                               |
+-------------+----------------------------------+
# 创建role [root@node1 ~(keystone)]# openstack role create SwiftOperator +-----------+----------------------------------+ | Field | Value | +-----------+----------------------------------+ | domain_id | None | | id | 5930ea3cf9ef42329f5a26f0c039983b | | name | SwiftOperator | +-----------+----------------------------------+
# 创建swift账户 [root@node1 ~(keystone)]# openstack user create --domain default --project swiftservice --password userpassword snowchuai +---------------------+----------------------------------+ | Field | Value | +---------------------+----------------------------------+ | default_project_id | f1ca0edd1073414787a9e160d2d9d768 | | domain_id | default | | enabled | True | | id | e024d58abf064644b0fb992f741159b7 | | name | snowchuai | | options | {} | | password_expires_at | None | +---------------------+----------------------------------+
# 将账户加入至Swift role中 [root@node1 ~(keystone)]# openstack role add --project swiftservice --user snowchuai SwiftOperator
2) 客户端操作 (1) 配置客户端环境 [root@client ~]# yum --enablerepo=centos-openstack-queens,epel install python-openstackclient python-keystoneclient python-swiftclient -y
[root@client ~]# vim ~/keystonerc_swift export OS_PROJECT_DOMAIN_NAME=default export OS_USER_DOMAIN_NAME=default export OS_PROJECT_NAME=swiftservice export OS_USERNAME=snowchuai export OS_PASSWORD=userpassword export OS_AUTH_URL=http://192.168.10.11:5000/v3 export OS_IDENTITY_API_VERSION=3 export PS1='[\u@\h \W(swift)]\$ '
[root@client ~]# chmod 600 ~/keystonerc_swift [root@client ~]# source ~/keystonerc_swift [root@client ~(swift)]# echo "source ~/keystonerc_swift " >> ~/.bash_profile
(2) 查看swift状态 [root@client ~(swift)]# swift stat Account: AUTH_f1ca0edd1073414787a9e160d2d9d768 Containers: 0 Objects: 0 Bytes: 0 X-Put-Timestamp: 1580993634.87513 X-Timestamp: 1580993634.87513 X-Trans-Id: tx0647b32bb36b4f5b8046c-005e3c0c62 Content-Type: text/plain; charset=utf-8 X-Openstack-Request-Id: tx0647b32bb36b4f5b8046c-005e3c0c62
(3) 创建Container [root@client ~(swift)]# openstack container create test_container +---------------------------------------+----------------+------------------------------------+ | account | container | x-trans-id | +---------------------------------------+----------------+------------------------------------+ | AUTH_f1ca0edd1073414787a9e160d2d9d768 | test_container | tx353a03e89a4b45fe9e458-005e3c0f87 | +---------------------------------------+----------------+------------------------------------+
[root@client ~(swift)]# openstack container list +----------------+ | Name | +----------------+ | test_container | +----------------+
(4) 上传文件至Container [root@client ~(swift)]# touch test.txt [root@client ~(swift)]# openstack object create test_container test.txt +----------+----------------+----------------------------------+ | object | container | etag | +----------+----------------+----------------------------------+ | test.txt | test_container | d41d8cd98f00b204e9800998ecf8427e | +----------+----------------+----------------------------------+
[root@client ~(swift)]# openstack object list test_container +----------+ | Name | +----------+ | test.txt | +----------+ [root@client ~(swift)]#
(5) 从container中下载文件 [root@client ~(swift)]# rm -rf test.txt [root@client ~(swift)]# openstack object save test_container test.txt [root@client ~(swift)]# ls -l test.txt -rw-r--r-- 1 root root 0 Feb 6 21:11 test.txt
(6) 从container中删除文件 [root@client ~(swift)]# openstack object list test_container +----------+ | Name | +----------+ | test.txt | +----------+
[root@client ~(swift)]# openstack object delete test_container test.txt [root@client ~(swift)]# openstack object list test_container
[root@client ~(swift)]#
(6) 删除container [root@client ~(swift)]# openstack container list +----------------+ | Name | +----------------+ | test_container | +----------------+
[root@client ~(swift)]# openstack container delete test_container [root@client ~(swift)]# openstack container list
[root@client ~(swift)]#

 

如对您有帮助,请随缘打个赏。^-^

gold