Red Hat Ceph Storage

Release Notes8.0 Release NotesBefore You BeginCompatibility GuideRed Hat Ceph Storage and Its Compatibility With Other ProductsHardware GuideHardware selection recommendations for Red Hat Ceph StorageArchitecture GuideGuide on Red Hat Ceph Storage ArchitectureData Security and Hardening GuideRed Hat Ceph Storage Data Security and Hardening GuideConfiguration GuideConfiguration settings for Red Hat Ceph StorageInstallingInstallation GuideInstalling Red Hat Ceph Storage on Red Hat Enterprise LinuxEdge GuideGuide on Edge Clusters for Red Hat Ceph StorageUpgradingUpgrade GuideUpgrading a Red Hat Ceph Storage ClusterGetting StartedGetting Started GuideGuide on getting started with Red Hat Ceph StorageCeph Clients and SolutionsFile System GuideConfiguring and Mounting Ceph File SystemsStorage AdministrationOperations GuideOperational tasks for Red Hat Ceph StorageAdministration GuideAdministration of Red Hat Ceph StorageStorage Strategies GuideCreating storage strategies for Red Hat Ceph Storage clustersMonitoringDashboard GuideMonitoring Ceph Cluster with Ceph DashboardTroubleshootingTroubleshooting GuideTroubleshooting Red Hat Ceph StorageAPI and Resource ReferenceDeveloper GuideUsing the various application programming interfaces for Red Hat Ceph StorageObject Gateway GuideDeploying, configuring, and administering a Ceph Object GatewayBlock Device GuideManaging, creating, configuring, and using Red Hat Ceph Storage Block DevicesBlock Device to OpenStack GuideConfiguring Ceph, QEMU, libvirt and OpenStack to use Ceph as a back end for OpenStack.

The [global] section of the configuration file. Deployment tools usually generate the fsid and store it in the monitor map, so the value may not appear in a configuration file. The fsid makes it possible to run daemons for multiple clusters on the same hardware. Do not set this value if you use a deployment tool that does it for you. 3.9. Ceph Monitor data store Ceph provides a default path where Ceph monitors store data. Red Hat recommends running Ceph monitors on separate drives from Ceph OSDs for optimal performance in a production Red Hat Ceph Storage cluster. A dedicated /var/lib/ceph partition should be used for the MON database with a size between 50 and 100 GB. Ceph monitors call the fsync() function often, which can interfere with Ceph OSD workloads. Ceph monitors store their data as key-value pairs. Using a data store prevents recovering Ceph monitors from running corrupted versions through Paxos, and it enables multiple modification operations in one single atomic batch, among other advantages. Red Hat does not recommend changing the default data location. If you modify the default location, make it uniform across Ceph monitors by setting it in the [mon] section of the configuration file. 3.10. Ceph storage capacity When a Red Hat Ceph Storage cluster gets close to its maximum capacity (specifies by the mon_osd_full_ratio parameter), Ceph prevents you from writing to or reading from Ceph OSDs as a safety measure to prevent data loss. Therefore, letting a production Red Hat Ceph Storage cluster approach its full ratio is not a good practice, because it sacrifices high availability. The default full ratio is .95, or 95% of capacity. This a very aggressive setting for a test cluster with a small number of OSDs. When monitoring a cluster, be alert to warnings related to the nearfull ratio. This means that a failure of some OSDs could result in a temporary service disruption if one or more OSDs fails. Consider adding more OSDs to increase storage capacity. A common scenario for test clusters involves a system administrator removing a Ceph OSD from the Red Hat Ceph Storage cluster to watch the cluster re-balance. Then, removing another Ceph OSD, and so on until the Red Hat Ceph Storage cluster eventually reaches the full ratio and locks up. Red Hat recommends a bit of capacity planning even with a test cluster. Planning enables you to gauge how much spare capacity you will need in to maintain high availability. Ideally, you want to plan for a series of Ceph OSD failures where the cluster can recover to an active + clean state without replacing those Ceph OSDs immediately. You can run a cluster in an active + degraded state, but this is not ideal for normal operating conditions. The following diagram depicts a simplistic Red Hat Ceph Storage cluster containing 33 Ceph Nodes with one Ceph OSD per host, each Ceph OSD Daemon reading from and writing to a 3TB drive. So this exemplary Red Hat Ceph Storage

Red Hat Ceph Storage 8Configuration settings for Red Hat Ceph StorageAbstract This document provides instructions for configuring Red Hat Ceph Storage at boot time and run time. It also provides configuration reference information. Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright's message. Chapter 1. The basics of Ceph configuration As a storage administrator, you need to have a basic understanding of how to view the Ceph configuration, and how to set the Ceph configuration options for the Red Hat Ceph Storage cluster. You can view and set the Ceph configuration options at runtime. Prerequisites Installation of the Red Hat Ceph Storage software. 1.1. Ceph configuration All Red Hat Ceph Storage clusters have a configuration, which defines: Cluster Identity Authentication settings Ceph daemons Network configuration Node names and addresses Paths to keyrings Paths to OSD log files Other runtime options A deployment tool, such as cephadm, will typically create an initial Ceph configuration file for you. However, you can create one yourself if you prefer to bootstrap a Red Hat Ceph Storage cluster without using a deployment tool. 1.2. The Ceph configuration database The Ceph Monitor manages a configuration database of Ceph options that centralize configuration management by storing configuration options for the entire storage cluster. By centralizing the Ceph configuration in a database, this simplifies storage cluster administration. The priority order that Ceph uses to set options is: Compiled-in default values Ceph cluster configuration database Local ceph.conf file Runtime override, using the ceph daemon DAEMON-NAME config set or ceph tell DAEMON-NAME injectargs commands There are still a few Ceph options that can be defined in the local Ceph configuration file, which is /etc/ceph/ceph.conf by default. However, ceph.conf has been deprecated for Red Hat Ceph Storage 8. cephadm uses a basic ceph.conf file that only contains a minimal set of options for connecting to Ceph Monitors, authenticating, and fetching configuration information. In most cases, cephadm uses only the mon_host option. To avoid using ceph.conf only for the mon_host option, use DNS SRV records to perform operations with Monitors. Red Hat recommends that you use the assimilate-conf administrative command to move valid options into the configuration database from the ceph.conf file. For more information about assimilate-conf, see Administrative Commands. Ceph allows you to make changes to the configuration of a daemon at runtime. This capability can be useful for increasing or decreasing the logging output, by enabling or disabling debug settings, and can even be used for runtime optimization. When the same option exists in the configuration database and the Ceph configuration file, the configuration database option has a lower priority than what is set in the Ceph configuration file. Sections and Masks Just as you can configure Ceph options globally, per daemon type, or by a specific

Red Hat Ceph Storage 1.2.3Architecture GuideAbstract This document is an architecture guide for Red Hat Ceph Storage. Preface Red Hat Ceph is a distributed data object store designed to provide excellent performance, reliability and scalability. Distributed object stores are the future of storage, because they accommodate unstructured data, and because clients can use modern object interfaces and legacy interfaces simultaneously. For example: Native language binding interfaces (C/C++, Java, Python) RESTful interfaces (S3/Swift) Block device interfaces Filesystem interfaces The power of Red Hat Ceph can transform your organization’s IT infrastructure and your ability to manage vast amounts of data, especially for cloud computing platforms like RHEL OSP. Red Hat Ceph delivers extraordinary scalability–thousands of clients accessing petabytes to exabytes of data and beyond. At the heart of every Ceph deployment is the Ceph Storage Cluster. It consists of two types of daemons: Ceph OSD Daemon: Ceph OSDs store data on behalf of Ceph clients. Additionally, Ceph OSDs utilize the CPU and memory of Ceph nodes to perform data replication, rebalancing, recovery, monitoring and reporting functions. Ceph Monitor: A Ceph monitor maintains a master copy of the Ceph storage cluster map with the current state of the storage cluster. Ceph client interfaces read data from and write data to the Ceph storage cluster. Clients need the following data to communicate with the Ceph storage cluster: The Ceph configuration file, or the cluster name (usually ceph) and monitor address The pool name The user name and the path to the secret key. Ceph