How to do a Ceph cluster maintenance/shutdown

Last week someone asked on the ceph-users ML how to shutdown a Ceph cluster and I would like to summarize the steps that are neccessary to do that.

  1. Stop the clients from using your Cluster (this step is only neccessary if you want to shutdown your whole cluster)

  2. Important - Make sure that your cluster is in a healthy state before proceeding

  3. Now you have to set some OSD flags:

    # ceph osd set noout
    # ceph osd set nobackfill
    # ceph osd set norecover
    Those flags should be totally suffiecient to safely powerdown your cluster but you
    could also set the following flags on top if you would like to pause your cluster completely::
    # ceph osd norebalance
    # ceph osd nodown
    # ceph osd pause
    ## Pausing the cluster means that you can't see when OSDs come
    back up again and no map update will happen
  4. Shutdown your service nodes one by one

  5. Shutdown your OSD nodes one by one

  6. Shutdown your monitor nodes one by one

  7. Shutdown your admin node

After maintenance just do everything mentioned above in reverse order.

Ceph Manager Dashboard v2

The original Ceph Manager Dashboard that was introduced in Ceph "Luminous" started out as a simple, read-only view into various run-time information and performance data of a Ceph cluster, without authentication or any administrative functionality.

However, as it turns out, there is a growing demand for adding more web-based management capabilities, to make it easier for administrators that prefer a WebUI to manage Ceph over the command line. Sage Weil also touched upon this topic in the Ceph Developer monthly call in December and created an etherpad with some ideas for improvement.


A preliminary screen shot of the Ceph health dashboard

After learning about this, we approached Sage and John Spray from the Ceph project and offered our help to implement the missing functionality. Based on our experiences in developing the Ceph support in openATTIC, we think we have a lot to offer in the form of code and experience in creating a Ceph administration and monitoring UI.

Read more…

How to create a vagrant VM from a libvirt vm/image

It cost's me some nerves and time to figure out how to create a vagrant image from a libvirt kvm vm and how to modify an existing one. Thanks to pl_rock from stackexchange for the awesome start.

  • First of all you have to install a new vm as usual. I've installed a new vm with Ubuntu 16.04 LTS. I'm not sure if it's really neccessary but set the root password to "vagrant", just to be sure.
  • Connect to your VM via ssh or terminal and do the following steps.

Read more…

openATTIC 3.6.1 has been released

It is our great pleasure to announce Version 3.6.1 of openATTIC.

3.6.1 is a bugfix release for the 3.6 stable branch, containing fixes for multiple issues that were reported by users.

In addition to that, it contains several usability enhancements and security improvements.

Behind the scenes, we continued with converting the WebUI code into Angular components in preparation for moving to Angular 2 in the near future.

Note that this release depends on a DeepSea version 0.8.2 or higher, as one improvement to the iSCSI target management (OP-2926) required making changes to both openATTIC and DeepSea.

Read more…

The demo is back!

We're happy to announce that our demo is up and running again.

The demo will be reinstalled every day at midnight. The login credentials are openattic/openattic. With the next release we'll include the demo mode which allows us to show this information at the top of the demo.

So please take the chance to visit

Conference Report: SALTCONF17


Last week SALTCONF17 took place at Salt Lake City and I was one of the attendees of this great event which included 2 days of pre-conference training and 2 days of conference.

Salt Palace Convention Center was the chosen place for this conference that included 60 keynotes and breakout sessions with dozens of SaltStack use cases presented by SaltStack users and in-depth tech talks from the SaltStack developers and experts.

The SaltStack Certified Engineer exam was available to attendees that took the pre-conference training (learn more about the SSCE certification and exam here).

Read more…

openATTIC 3.6.0 has been released

We're happy to announce version 3.6.0 of openATTIC.

Given the fact that openATTIC 3.5.3 was only a bug fix release this 3.6.0 release includes all the improvements and changes since 3.5.2. We cleaned up and removed a lot of unnecessary things and also made some usability improvements to the UI.

The most visible change in openATTIC 3.6.0 is the unification of Grafana and our "old" widgets into one single dashboard page.

We now have one unified dashboard and you don't have to switch between the "old" and the new Grafana dashboard. We included the functionality of the old status widget into our default dashboard. If your cluster is in an error or warning state, this will be shown at the top of your dashboard including the failure message from the 'ceph health' command.

Read more…