Overview
In this blog I wanted to cover the 2 different ways of configuring Ceph as the unified storage for DevStack:
- hook-based approach (deprecated & to be removed soon, but good to know)
- plugin-based approach (newly introduced and complies with devstack plugin policy)
Hook-based approach
This is the classic way of configuring DevStack to use Ceph.
In this approach, all the hook scripts that install/configure ceph and openstack services to use ceph as the storage backend, are present in DevStack repo itself.
These hook scripts are called by DevStack (as part of stack.sh) at appropriate times to install & initialize a ceph cluster, create openstack service specific ceph pools, and finally configure the openstack services to use the service specific respective ceph pool as the storage backend.
Thus, at the end of stack.sh you get a fully working local ceph cluster, with openstack
service specific ceph pools acting as the storage for respective openstack services.
Following is an example of my localrc for this approach:
# Clone git repos, connect to internet if needed
RECLONE=True
OFFLINE=False
# Passwords for my setup
DATABASE_PASSWORD=abc123
RABBIT_PASSWORD=abc123
SERVICE_TOKEN=abc123
SERVICE_PASSWORD=abc123
ADMIN_PASSWORD=abc123
# I don't need a GUI/Dashboard
disable_service horizon
# Currently, not interested in Heat
disable_service heat
disable_service h-eng
disable_service h-api
disable_service h-api-cfn
disable_service h-api-cw
# Disable nova-network and tempest
disable_service n-net
disable_service tempest
# Enable neutron and its associated services
enable_service neutron
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
# Enable ceph service, ask Cinder to use ceph backend
ENABLED_SERVICES+=,ceph
CINDER_ENABLED_BACKENDS=ceph
ENABLED_SERVICES causes DevStack's ceph hooks script to install ceph, create a ceph cluster and openstack service specific ceph pools.
CINDER_ENABLED_BACKENDS tells DevStack to invoke Cinder's ceph backend script to configure ceph for Cinder
Nova and Glance doesn't have a backend specific scripts, so they are configured directly by DevStack's ceph hook script
With the above localrc, running stack.sh should get you a basic openstack setup,
that uses Ceph as the storage backend for Nova, Glance, & Cinder services
Plugin-based approach
This is a new way of configuring DevStack to use ceph
In this approach, all the plugin scripts that install/configure ceph and openstack services to use ceph as the storage backend, are present outside of the DevStack repo, in a plugin-specific repo.
For ceph, the repo is
https://github.com/openstack/devstack-plugin-ceph
Like hook scripts, the plugin scripts are called by DevStack (as part of stack.sh) at
appropriate times to install & initialize a ceph cluster, create openstack service specific
ceph pools, and finally configure the openstack services to use the service specific ceph
pool as the storage backend.
Thus, at the end of `stack.sh you get a fully working local ceph cluster, with openstack
service specific ceph pools acting as the storage for respective openstack services.
This is better than hook-based approach because:
- Plugin/Backend changes are not carried by DevStack repo, so DevStack becomes lean & better manageable
- Provides an API abstraction / contract between DevStack and plugin repo, thus providing a modular and supportable plugin model
- Changes to plugin/backend can happen independently of DevStack, thus both can evolve independently & as long as the API contract is maintained, its guaranteed to work
- Changes to the plugin repo can be CI'ed (CI = Continuous Integration) at the plugin repo itself (instead of DevStack) thus ensuring that the plugin change doesn't harm / break DevStack for that plugin/backend. Inversely, this also means that changes to DevStack doesn't have to worry about different plugins as the changes to them are gated by the plugin's respective CI job(s)
Following is an example of my localrc for this approach:
# Clone git repos, connect to internet if needed
RECLONE=True
OFFLINE=False
# Passwords for my setup
DATABASE_PASSWORD=abc123
RABBIT_PASSWORD=abc123
SERVICE_TOKEN=abc123
SERVICE_PASSWORD=abc123
ADMIN_PASSWORD=abc123
# I don't need a GUI/Dashboard
disable_service horizon
# Currently, not interested in Heat
disable_service heat
disable_service h-eng
disable_service h-api
disable_service h-api-cfn
disable_service h-api-cw
# Disable nova-network and tempest
disable_service n-net
disable_service tempest
# Enable neutron and its associated services
enable_service neutron
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
# Enable ceph DevStack plugin
enable_plugin devstack-plugin-ceph git://git.openstack.org/openstack/devstack-plugin-ceph
Unlike hook approach, all that is needed in localrc is the enable_plugin ... line,
no need to set/override any DevStack env vars!
Looks a lot more logical, neat and modular, isn't it ? :)
With the above localrc, running stack.sh should get you a basic openstack setup,
that uses Ceph as the storage backend for Nova, Glance, & Cinder services
Summary
This blog was aimed at providing steps required to quickly get a DevStack setup with ceph as the unified storage backend. I hope to have done justice to that :)
Feel free to provide comments / questions and I will do my best to answer
A more detailed writeup on how devstack plugin actually works, its structure, the hooks it has, is out of scope of this blog, maybe I will write another blog for it :)
