From 0870ecd8a52432538b3b64c9ed0b913ee53936ee Mon Sep 17 00:00:00 2001 From: John Mulligan Date: Tue, 21 Mar 2023 16:47:35 -0400 Subject: [PATCH] doc/cephadm: document haproxy protocol options There are now two ways to run NFS with ingress using haxproxy. This makes three distinct modes for ingress. Document these. Signed-off-by: John Mulligan (cherry picked from commit 92ce0ae60039be899ae04e14504badd9b0e8939e) Conflicts: doc/mgr/nfs.rst --- doc/cephadm/services/nfs.rst | 47 ++++++++++++++++++++++++++++++++++++ doc/mgr/nfs.rst | 39 ++++++++++++++++++------------ 2 files changed, 70 insertions(+), 16 deletions(-) diff --git a/doc/cephadm/services/nfs.rst b/doc/cephadm/services/nfs.rst index 0e263275d13bc..2f12c591631f6 100644 --- a/doc/cephadm/services/nfs.rst +++ b/doc/cephadm/services/nfs.rst @@ -161,6 +161,53 @@ that will tell it to bind to that specific IP. Note that in these setups, one should make sure to include ``count: 1`` in the nfs placement, as it's only possible for one nfs daemon to bind to the virtual IP. +NFS with HAProxy Protocol Support +---------------------------------- + +Cephadm supports deploying NFS in High-Availability mode with additional +HAProxy protocol support. This works just like High-availability NFS but also +supports client IP level configuration on NFS Exports. This feature requires +`NFS-Ganesha v5.0`_ or later. + +.. _NFS-Ganesha v5.0: https://github.com/nfs-ganesha/nfs-ganesha/wiki/ReleaseNotes_5 + +To use this mode, you'll either want to set up the service using the nfs module +(see :ref:`nfs-module-cluster-create`) or manually create services with the +extra parameter ``enable_haproxy_protocol`` set to true. Both NFS Service and +Ingress service must have ``enable_haproxy_protocol`` set to the same value. +For example: + +.. code-block:: yaml + + service_type: ingress + service_id: nfs.foo + placement: + count: 1 + hosts: + - host1 + - host2 + - host3 + spec: + backend_service: nfs.foo + monitor_port: 9049 + virtual_ip: 192.168.122.100/24 + enable_haproxy_protocol: true + +.. code-block:: yaml + + service_type: nfs + service_id: foo + placement: + count: 1 + hosts: + - host1 + - host2 + - host3 + spec: + port: 2049 + enable_haproxy_protocol: true + + Further Reading =============== diff --git a/doc/mgr/nfs.rst b/doc/mgr/nfs.rst index c25410fcdf5e2..7e6637684faa7 100644 --- a/doc/mgr/nfs.rst +++ b/doc/mgr/nfs.rst @@ -107,22 +107,29 @@ of the details of NFS redirecting traffic on the virtual IP to the appropriate backend NFS servers, and redeploying NFS servers when they fail. -If a user additionally supplies ``--ingress-mode keepalive-only`` a -partial *ingress* service will be deployed that still provides a virtual -IP, but has nfs directly binding to that virtual IP and leaves out any -sort of load balancing or traffic redirection. This setup will restrict -users to deploying only 1 nfs daemon as multiple cannot bind to the same -port on the virtual IP. - -Instead providing ``--ingress-mode default`` will result in the same setup -as not providing the ``--ingress-mode`` flag. In this setup keepalived will be -deployed to handle forming the virtual IP and haproxy will be deployed -to handle load balancing and traffic redirection. - -Enabling ingress via the ``ceph nfs cluster create`` command deploys a -simple ingress configuration with the most common configuration -options. Ingress can also be added to an existing NFS service (e.g., -one created without the ``--ingress`` flag), and the basic NFS service can +An optional ``--ingress-mode`` parameter can be provided to choose +how the *ingress* service is configured: + +- Setting ``--ingress-mode keepalive-only`` deploys a simplified *ingress* + service that provides a virtual IP with the nfs server directly binding to + that virtual IP and leaves out any sort of load balancing or traffic + redirection. This setup will restrict users to deploying only 1 nfs daemon + as multiple cannot bind to the same port on the virtual IP. +- Setting ``--ingress-mode haproxy-standard`` deploys a full *ingress* service + to provide load balancing and high-availability using HAProxy and keepalived. + Client IP addresses are not visible to the back-end NFS server and IP level + restrictions on NFS exports will not function. +- Setting ``--ingress-mode haproxy-protocol`` deploys a full *ingress* service + to provide load balancing and high-availability using HAProxy and keepalived. + Client IP addresses are visible to the back-end NFS server and IP level + restrictions on NFS exports are usable. This mode requires NFS Ganesha version + 5.0 or later. +- Setting ``--ingress-mode default`` is equivalent to not providing any other + ingress mode by name. When no other ingress mode is specified by name + the default ingress mode used is ``haproxy-standard``. + +Ingress can be added to an existing NFS service (e.g., one initially created +without the ``--ingress`` flag), and the basic NFS service can also be modified after the fact to include non-default options, by modifying the services directly. For more information, see :ref:`cephadm-ha-nfs`. -- 2.39.5