encryption and compression services. And QAT driver in kernel space have to
be loaded to drive the hardware.
-The driver package can be downloaded from `Intel Quickassist Technology`_.
+The out-of-tree QAT driver package can be downloaded from `Intel Quickassist
+Technology`_.
-2. The implementation for QAT based encryption is directly base on QAT API which
- is included the driver package. But QAT support for compression depends on
- QATzip project, which is a user space library which builds on top of the QAT
- API. Currently, QATzip speeds up gzip compression and decompression at the
- time of writing.
+The QATlib can be downloaded from `qatlib`_, which is used for the in-tree QAT
+driver.
+
+ .. note::
+ The out-of-tree QAT driver is gradually turning to intree driver+QATlib.
+
+2. The implementation of QAT-based encryption is directly based on the QAT API,
+ which is included the driver package. However, QAT support for compression
+ depends on the QATzip project, which is a userspace library that builds on
+ top of the QAT API. At the time of writing (July 2024), QATzip speeds up
+ gzip compression and decompression.
See `QATzip`_.
1. QAT based Encryption for RGW
`OpenSSL support for RGW encryption`_ has been merged into Ceph, and Intel also
-provides one `QAT Engine`_ for OpenSSL. So, theoretically speaking, QAT based
-encryption in Ceph can be directly supported through OpenSSl+QAT Engine.
+provides one `QAT Engine`_ for OpenSSL. Theoretically, QAT-based encryption in
+Ceph can be directly supported through OpenSSl+QAT Engine.
-But the QAT Engine for OpenSSL currently supports chained operations only, and
-so Ceph will not be able to utilize QAT hardware feature for crypto operations
-based on OpenSSL crypto plugin. As a result, one QAT plugin based on native
-QAT API is added into crypto framework.
+However, the QAT Engine for OpenSSL currently supports only chained operations,
+which means that Ceph will not be able to utilize QAT hardware features for
+crypto operations based on the OpenSSL crypto plugin. As a result, one QAT plugin
+based on native QAT API is added into the crypto framework.
2. QAT Support for Compression
-As mentioned above, QAT support for compression is based on QATzip library in
-user space, which is designed to take full advantage of the performance provided
-by QuickAssist Technology. Unlike QAT based encryption, QAT based compression
-is supported through a tool class for QAT acceleration rather than a compressor
-plugin. The common tool class can transparently accelerate the existing compression
-types, but only zlib compressor can be supported at the time of writing. So
-user is allowed to use it to speed up zlib compressor as long as the QAT
-hardware is available and QAT is capable to handle it.
+As mentioned above, QAT support for compression is based on the QATzip library
+in user space, which is designed to take full advantage of the performance that
+QuickAssist Technology provides. Unlike QAT-based encryption, QAT-based
+compression is supported through a tool class for QAT acceleration rather than
+a compressor plugin. This common tool class can transparently accelerate the
+existing compression types, but only zlib compressor is supported at the
+time of writing. This means that the user can use this tool class to speed up
+zlib compressor if the QAT hardware is available and if QAT is capable of
+handling it.
Configuration
=============
#. Prerequisites
- Make sure the QAT driver with version v1.7.L.4.14.0 or higher has been installed.
- Remember to set an environment variable "ICP_ROOT" for your QAT driver package
- root directory.
+ **For out-of-tree QAT**
- To enable the QAT based encryption and compression, user needs to modify the QAT
- configuration files. For example, for Intel QuickAssist Adapter 8970 product, revise
- c6xx_dev0/1/2.conf in the directory ``/etc/`` and keep them the same, e.g.:
+ Make sure the out-of-tree QAT driver with version v1.7.L.4.14.0 or higher
+ has been installed. Remember to set an environment variable ``ICP_ROOT``
+ for your QAT driver package root directory.
+
+ To enable the QAT based encryption and compression, the user must modify the
+ QAT configuration files. For example, for the Intel QuickAssist Adapter 8970
+ product, revise ``c6xx_dev0/1/2.conf`` in the directory ``/etc/`` and keep them
+ the same. For example:
.. code-block:: ini
# List of core affinities
Dc0CoreAffinity = 0
-#. QAT based Encryption for RGW
+ **For in-tree QAT**
+
+ There are some prerequisites for using QATlib. Make sure that your system
+ meets the `QATlib System Requirements`_ .
+
+ * To properly use the QATlib library, the Intel VT-d and SR-IOV parameters
+ must be enabled in the platform BIOS.
+ * Some qatlib features require a recent kernel driver or firmware version.
+ See `QATlib Kernel Driver Releases`_.
+ * The supported platform contains a 4xxx Intel Communications device or
+ newer.
+ * The ``intel_iommu`` parameter must be enabled. Verify that this setting is
+ enabled by running the following commands:
+
+ .. prompt:: bash $
- The CMake option ``WITH_QAT=ON`` must be configured. If you build Ceph from
+ cat /proc/cmdline | grep intel_iommu=on
+ sudo sh -c 'echo "@qat - memlock 204800" >> /etc/security/limits.conf'
+ sudo su -l $USER
+
+ For configuration and Tuning see `QATlib Configuration and Tuning`_.
+
+#. QAT-based Encryption for RGW
+
+ The CMake option ``WITH_QATDRV=ON`` must be set. If you build Ceph from
source code (see: :ref:`build-ceph`), navigate to your cloned Ceph repository
and execute the following:
.. prompt:: bash $
cd ceph
- ./do_cmake.sh -DWITH_QAT=ON
+ ./do_cmake.sh -DWITH_QATDRV=ON
cd build
ininja
- .. note::
- The section name of the QAT configuration files must be ``CEPH`` since
- the section name is set as "CEPH" in Ceph crypto source code.
+ .. note:: The section name of the QAT configuration files must be ``CEPH``,
+ because the section name is set as ``CEPH`` in the Ceph crypto source code.
- Then, edit the Ceph configuration file to make use of QAT based crypto plugin::
+ Edit the Ceph configuration file (usually ``ceph.conf``) to make use of the
+ QAT-based crypto plugin::
plugin crypto accelerator = crypto_qat
#. QAT Support for Compression
- Before starting, make sure both QAT driver and `QATzip`_ have been installed. Besides
- "ICP_ROOT", remember to set the environment variable "QZ_ROOT" for the root directory
- of your QATzip source tree.
+ **For out-of-tree QAT**
- The following CMake options have to be configured to trigger QAT based compression
- when building Ceph:
+ For the out-of-tree QAT driver package, before starting ensure that both QAT
+ driver and `QATzip`_ have been installed. Besides ``ICP_ROOT``, remember to
+ set the environment variable ``QZ_ROOT`` to the root directory of your QATzip
+ source tree.
+
+ The following CMake options have to be configured to trigger QAT-based
+ compression when building Ceph:
.. prompt:: bash $
- ./do_cmake.sh -DWITH_QAT=ON -DWITH_QATZIP=ON
+ ./do_cmake.sh -DWITH_QATDRV=ON -DWITH_QATZIP=ON -DWITH_SYSTEM_QATZIP=ON -DWITH_QATLIB=OFF
- Then, set an environment variable to clarify the section name of User Process Instance
- Section in QAT configuration files, e.g.:
+ Set an environment variable to clarify the section name of User Process
+ Instance Section in the QAT configuration files. For example:
.. prompt:: bash $
export QAT_SECTION_NAME=CEPH
- Next, edit the Ceph configuration file to enable QAT support for compression::
+ **For in-tree QAT**
+
+ For in-tree QAT, make sure that your system meets the `QATlib System
+ Requirements`_. QATlib can be installed from pre-built packages or from
+ source code. See `QATlib Installation`_ . After QATlib is installed, you
+ can run ``cpa_sample_code`` to check if the QAT environment is OK.
+
+ If you are using QATlib source code, Ceph cmake build has the compilation
+ options of qatlib and qatzip enabled by default. Our normal compilation
+ already includes QAT-compressor-related code.
+
+ .. prompt:: bash $
+
+ ./do_cmake.sh
+
+ If you are using pre-built packages installed on the system, the following
+ CMake options have to be configured when building Ceph:
+
+ .. prompt:: bash $
+
+ ./do_cmake.sh -DWITH_SYSTEM_QATLIB=ON -DWITH_SYSTEM_QATZIP=ON
+
+
+ **For both out-of-tree QAT and in-tree QAT**
+
+ Edit the Ceph configuration file (usually ``ceph.conf``) to enable QAT
+ support for *zlib* compression::
qat compressor enabled=true
+ Set the RGW compression method:
+
+ .. prompt:: bash $
+
+ # for storage class(STANDARD)
+ radosgw-admin zone placement modify --rgw-zone=default --placement-id=default-placement --compression=zlib
+ # or create a new storage class(COLD) and define data pool(default.rgw.cold.data)
+ radosgw-admin zonegroup placement add --rgw-zonegroup default --placement-id default-placement --storage-class COLD
+ radosgw-admin zone placement add --rgw-zone default --placement-id default-placement --storage-class COLD --compression zlib --data-pool default.rgw.cold.data
+
+CONFIG REFERENCE
+================
+The following QAT-related settings can be added to the Ceph configuration file
+(usually `ceph.conf`) under the ``[client.rgw.{instance-name}]`` section.
+
+.. confval:: qat_compressor_session_max_number
+.. confval:: qat_compressor_busy_polling
+
+
.. _QAT Support for Compression: https://github.com/ceph/ceph/pull/19714
.. _QAT based Encryption for RGW: https://github.com/ceph/ceph/pull/19386
.. _QATzip: https://github.com/intel/QATzip
.. _OpenSSL support for RGW encryption: https://github.com/ceph/ceph/pull/15168
.. _QAT Engine: https://github.com/intel/QAT_Engine
+.. _qatlib: https://github.com/intel/qatlib
+.. _QATlib User's Guide: https://intel.github.io/quickassist/qatlib/index.html
+.. _QATlib System Requirements: https://intel.github.io/quickassist/qatlib/requirements.html
+.. _QATlib Installation: https://intel.github.io/quickassist/qatlib/install.html
+.. _QATlib Configuration and Tuning: https://intel.github.io/quickassist/qatlib/configuration.html
+.. _QATlib Kernel Driver Releases: https://intel.github.io/quickassist/RN/In-Tree/in_tree_firmware_RN.html#qat-kernel-driver-releases-features
projects / ceph.git / commitdiff