Create NFS Ganesha Cluster
--------------------------
-.. code:: bash
+.. prompt:: bash #
-
$ ceph nfs cluster create <cluster_id> [<placement>] [--ingress] [--virtual_ip <value>] [--ingress-mode {default|keepalive-only|haproxy-standard|haproxy-protocol}] [--port <int>]
+ ceph nfs cluster create <cluster_id> [<placement>] [--ingress] [--virtual_ip <value>] [--ingress-mode {default|keepalive-only|haproxy-standard|haproxy-protocol}] [--port <int>]
This creates a common recovery pool for all NFS Ganesha daemons, new user based on
``cluster_id``, and a common NFS Ganesha config RADOS object.
daemon containers running on them and, optionally, the total number of NFS
Ganesha daemons on the cluster (should you want to have more than one NFS Ganesha
daemon running per node). For example, the following placement string means
-"deploy NFS Ganesha daemons on nodes host1 and host2 (one daemon per host)::
+"deploy NFS Ganesha daemons on nodes host1 and host2" (one daemon per host)::
"host1,host2"
wish to check that these services do successfully start and stay running.
When using cephadm orchestration, these commands check service status:
-.. code:: bash
- $ ceph orch ls --service_name=nfs.<cluster_id>
- $ ceph orch ls --service_name=ingress.nfs.<cluster_id>
+.. prompt:: bash #
+
+ ceph orch ls --service_name=nfs.<cluster_id>
+ ceph orch ls --service_name=ingress.nfs.<cluster_id>
Ingress
To examine an NFS cluster's IP endpoints, including the IPs for the individual NFS
daemons, and the virtual IP (if any) for the ingress service,
-.. code:: bash
+.. prompt:: bash #
- $ ceph nfs cluster info [<cluster_id>]
+ ceph nfs cluster info [<cluster_id>]
-.. note:: This will not work with the rook backend. Instead, expose the port with
- the kubectl patch command and fetch the port details with kubectl get services
- command::
+.. note:: This will not work with the rook backend. Instead, expose the port
+ with the kubectl patch command and fetch the port details with kubectl get
+ services command:
- $ kubectl patch service -n rook-ceph -p '{"spec":{"type": "NodePort"}}' rook-ceph-nfs-<cluster-name>-<node-id>
- $ kubectl get services -n rook-ceph rook-ceph-nfs-<cluster-name>-<node-id>
+ .. prompt:: bash #
+
+ kubectl patch service -n rook-ceph -p '{"spec":{"type": "NodePort"}}' rook-ceph-nfs-<cluster-name>-<node-id>
+ kubectl get services -n rook-ceph rook-ceph-nfs-<cluster-name>-<node-id>
Delete NFS Ganesha Cluster
--------------------------
-.. code:: bash
+.. prompt:: bash #
- $ ceph nfs cluster rm <cluster_id>
+ ceph nfs cluster rm <cluster_id>
This deletes the deployed cluster.
wish to check that these services are no longer reported. When using cephadm
orchestration, these commands check service status:
-.. code:: bash
+.. prompt:: bash #
- $ ceph orch ls --service_name=nfs.<cluster_id>
- $ ceph orch ls --service_name=ingress.nfs.<cluster_id>
+ ceph orch ls --service_name=nfs.<cluster_id>
+ ceph orch ls --service_name=ingress.nfs.<cluster_id>
Updating an NFS Cluster
-----------------------
-In order to modify cluster parameters (like the port or placement), you need to
-use the orchestrator interface to update the NFS service spec. The safest way to do
-that is to export the current spec, modify it, and then re-apply it. For example,
-to modify the ``nfs.foo`` service,
+In order to modify cluster parameters (like the port or placement), you need
+to use the orchestrator interface to update the NFS service spec. The safest
+way to do that is to export the current spec, modify it, and then re-apply it.
+For example, to modify the ``nfs.foo`` service,
-.. code:: bash
+.. prompt:: bash #
- $ ceph orch ls --service-name nfs.foo --export > nfs.foo.yaml
- $ vi nfs.foo.yaml
- $ ceph orch apply -i nfs.foo.yaml
+ ceph orch ls --service-name nfs.foo --export > nfs.foo.yaml
+ vi nfs.foo.yaml
+ ceph orch apply -i nfs.foo.yaml
For more information about the NFS service spec, see :ref:`deploy-cephadm-nfs-ganesha`.
List NFS Ganesha Clusters
-------------------------
-.. code:: bash
+.. prompt:: bash #
- $ ceph nfs cluster ls
+ ceph nfs cluster ls
This lists deployed clusters.
Set Customized NFS Ganesha Configuration
----------------------------------------
-.. code:: bash
+.. prompt:: bash #
- $ ceph nfs cluster config set <cluster_id> -i <config_file>
+ ceph nfs cluster config set <cluster_id> -i <config_file>
With this the nfs cluster will use the specified config and it will have
precedence over default config blocks.
Example use cases include:
-#. Changing log level. The logging level can be adjusted with the following config
- fragment::
+#. Changing log level. The logging level can be adjusted with the following
+ config fragment::
LOG {
COMPONENTS {
#. Adding custom export block.
The following sample block creates a single export. This export will not be
- managed by `ceph nfs export` interface::
+ managed by the ``ceph nfs export`` interface::
EXPORT {
Export_Id = 100;
.. note:: User specified in FSAL block should have proper caps for NFS-Ganesha
daemons to access ceph cluster. User can be created in following way using
- `auth get-or-create`::
+ `auth get-or-create`:
- # ceph auth get-or-create client.<user_id> mon 'allow r' osd 'allow rw pool=.nfs namespace=<nfs_cluster_name>, allow rw tag cephfs data=<fs_name>' mds 'allow rw path=<export_path>'
+ .. prompt:: bash #
+
+ ceph auth get-or-create client.<user_id> mon 'allow r' osd 'allow rw pool=.nfs namespace=<nfs_cluster_name>, allow rw tag cephfs data=<fs_name>' mds 'allow rw path=<export_path>'
View Customized NFS Ganesha Configuration
-----------------------------------------
-.. code:: bash
+.. prompt:: bash #
- $ ceph nfs cluster config get <cluster_id>
+ ceph nfs cluster config get <cluster_id>
This will output the user defined configuration (if any).
Reset NFS Ganesha Configuration
-------------------------------
-.. code:: bash
+.. prompt:: bash #
- $ ceph nfs cluster config reset <cluster_id>
+ ceph nfs cluster config reset <cluster_id>
This removes the user defined configuration.
Create CephFS Export
--------------------
-.. code:: bash
+.. prompt:: bash #
-
$ ceph nfs export create cephfs --cluster-id <cluster_id> --pseudo-path <pseudo_path> --fsname <fsname> [--readonly] [--path=/path/in/cephfs] [--client_addr <value>...] [--squash <value>] [--sectype <value>...] [--cmount_path <value>]
+ ceph nfs export create cephfs --cluster-id <cluster_id> --pseudo-path <pseudo_path> --fsname <fsname> [--readonly] [--path=/path/in/cephfs] [--client_addr <value>...] [--squash <value>] [--sectype <value>...] [--cmount_path <value>]
This creates export RADOS objects containing the export block, where
``<path>`` is the path within cephfs. Valid path should be given and default
path is '/'. It need not be unique. Subvolume path can be fetched using:
-.. code::
+.. prompt:: bash #
- $ ceph fs subvolume getpath <vol_name> <subvol_name> [--group_name <subvol_group_name>]
+ ceph fs subvolume getpath <vol_name> <subvol_name> [--group_name <subvol_group_name>]
``<client_addr>`` is the list of client address for which these export
permissions will be applicable. By default all clients can access the export
for permissible values.
``<squash>`` defines the kind of user id squashing to be performed. The default
-value is `no_root_squash`. See the `NFS-Ganesha Export Sample`_ for
+value is ``no_root_squash``. See the `NFS-Ganesha Export Sample`_ for
permissible values.
``<sectype>`` specifies which authentication methods will be used when
To export a *bucket*:
-.. code::
+.. prompt:: bash #
-
$ ceph nfs export create rgw --cluster-id <cluster_id> --pseudo-path <pseudo_path> --bucket <bucket_name> [--user-id <user-id>] [--readonly] [--client_addr <value>...] [--squash <value>] [--sectype <value>...]
+ ceph nfs export create rgw --cluster-id <cluster_id> --pseudo-path <pseudo_path> --bucket <bucket_name> [--user-id <user-id>] [--readonly] [--client_addr <value>...] [--squash <value>] [--sectype <value>...]
-For example, to export *mybucket* via NFS cluster *mynfs* at the pseudo-path */bucketdata* to any host in the ``192.168.10.0/24`` network
+For example, to export ``mybucket`` via NFS cluster ``mynfs`` at the
+pseudo-path ``/bucketdata`` to any host in the ``192.168.10.0/24`` network
-.. code::
+.. prompt:: bash #
- $ ceph nfs export create rgw --cluster-id mynfs --pseudo-path /bucketdata --bucket mybucket --client_addr 192.168.10.0/24
+ ceph nfs export create rgw --cluster-id mynfs --pseudo-path /bucketdata --bucket mybucket --client_addr 192.168.10.0/24
-.. note:: Export creation is supported only for NFS Ganesha clusters deployed using nfs interface.
+.. note:: Export creation is supported only for NFS Ganesha clusters deployed
+ using nfs interface.
``<cluster_id>`` is the NFS Ganesha cluster ID.
-``<pseudo_path>`` is the export position within the NFS v4 Pseudo Filesystem where the export will be available on the server. It must be an absolute path and unique.
+``<pseudo_path>`` is the export position within the NFS v4 Pseudo Filesystem
+where the export will be available on the server. It must be an absolute path
+and unique.
``<bucket_name>`` is the name of the bucket that will be exported.
-``<user_id>`` is optional, and specifies which RGW user will be used for read and write
-operations to the bucket. If it is not specified, the user who owns the bucket will be
-used.
+``<user_id>`` is optional, and specifies which RGW user will be used for read
+and write operations to the bucket. If it is not specified, the user who owns
+the bucket will be used.
-.. note:: Currently, if multi-site RGW is enabled, Ceph can only export RGW buckets in the default realm.
+.. note:: Currently, if multi-site RGW is enabled, Ceph can only export RGW
+ buckets in the default realm.
``<client_addr>`` is the list of client address for which these export
permissions will be applicable. By default all clients can access the export
-according to specified export permissions. See the `NFS-Ganesha Export Sample`_
-for permissible values.
+according to specified export permissions. See the `NFS-Ganesha Export
+Sample`_ for permissible values.
-``<squash>`` defines the kind of user id squashing to be performed. The default
-value is `no_root_squash`. See the `NFS-Ganesha Export Sample`_ for
+``<squash>`` defines the kind of user id squashing to be performed. The
+default value is ``no_root_squash``. See the `NFS-Ganesha Export Sample`_ for
permissible values.
``<sectype>`` specifies which authentication methods will be used when
-connecting to the export. Valid values include "krb5p", "krb5i", "krb5", "sys",
-and "none". More than one value can be supplied. The flag may be specified
-multiple times (example: ``--sectype=krb5p --sectype=krb5i``) or multiple
-values may be separated by a comma (example: ``--sectype krb5p,krb5i``). The
-server will negotatiate a supported security type with the client preferring
-the supplied methods left-to-right.
+connecting to the export. Valid values include "krb5p", "krb5i", "krb5",
+"sys", and "none". More than one value can be supplied. The flag may be
+specified multiple times (example: ``--sectype=krb5p --sectype=krb5i``) or
+multiple values may be separated by a comma (example: ``--sectype
+krb5p,krb5i``). The server will negotatiate a supported security type with the
+client preferring the supplied methods left-to-right.
-.. note:: Specifying values for sectype that require Kerberos will only function on servers
- that are configured to support Kerberos. Setting up NFS-Ganesha to support Kerberos
- is outside the scope of this document.
+.. note:: Specifying values for sectype that require Kerberos will only
+ function on servers that are configured to support Kerberos. Setting up
+ NFS-Ganesha to support Kerberos is outside the scope of this document.
RGW user export
^^^^^^^^^^^^^^^
To export an RGW *user*:
-.. code::
+.. prompt:: bash #
-
$ ceph nfs export create rgw --cluster-id <cluster_id> --pseudo-path <pseudo_path> --user-id <user-id> [--readonly] [--client_addr <value>...] [--squash <value>]
+ ceph nfs export create rgw --cluster-id <cluster_id> --pseudo-path <pseudo_path> --user-id <user-id> [--readonly] [--client_addr <value>...] [--squash <value>]
For example, to export *myuser* via NFS cluster *mynfs* at the pseudo-path */myuser* to any host in the ``192.168.10.0/24`` network
-.. code::
+.. prompt:: bash #
- $ ceph nfs export create rgw --cluster-id mynfs --pseudo-path /bucketdata --user-id myuser --client_addr 192.168.10.0/24
+ ceph nfs export create rgw --cluster-id mynfs --pseudo-path /bucketdata --user-id myuser --client_addr 192.168.10.0/24
Delete Export
-------------
-.. code:: bash
+.. prompt:: bash #
-
$ ceph nfs export rm <cluster_id> <pseudo_path>
+ ceph nfs export rm <cluster_id> <pseudo_path>
This deletes an export in an NFS Ganesha cluster, where:
List Exports
------------
-.. code:: bash
+.. prompt:: bash #
- $ ceph nfs export ls <cluster_id> [--detailed]
+ ceph nfs export ls <cluster_id> [--detailed]
It lists exports for a cluster, where:
Get Export
----------
-.. code:: bash
+.. prompt:: bash #
-
$ ceph nfs export info <cluster_id> <pseudo_path>
+ ceph nfs export info <cluster_id> <pseudo_path>
-This displays export block for a cluster based on pseudo root name,
-where:
+This displays export block for a cluster based on pseudo root name, where:
``<cluster_id>`` is the NFS Ganesha cluster ID.
.. prompt:: bash #
- ceph nfs export apply *<cluster_id>* -i <json_file>
+ ceph nfs export apply *<cluster_id>* -i <json_file>
+
+For example:
-For example,::
+.. prompt:: bash #
+
+ ceph nfs export info mynfs /cephfs > update_cephfs_export.json
+ cat update_cephfs_export.json
+
+::
- $ ceph nfs export info mynfs /cephfs > update_cephfs_export.json
- $ cat update_cephfs_export.json
{
"export_id": 1,
"path": "/",
"clients": []
}
-The imported JSON can be a single dict describing a single export, or a JSON list
-containing multiple export dicts.
+The imported JSON can be a single dict describing a single export, or a JSON
+list containing multiple export dicts.
The exported JSON can be modified and then reapplied. Below, *pseudo*
and *access_type* are modified. When modifying an export, the
authentication credentials, which will be carried over from the
previous state of the export where possible.
-!! NOTE: The ``user_id`` in the ``fsal`` block should not be modified or mentioned in the JSON file as it is auto-generated for CephFS exports.
-It's auto-generated in the format ``nfs.<cluster_id>.<fs_name>.<hash_id>``.
+.. note:: The ``user_id`` in the ``fsal`` block should not be modified or
+ mentioned in the JSON file as it is auto-generated for CephFS exports. It
+ is auto-generated in the format ``nfs.<cluster_id>.<fs_name>.<hash_id>``.
+
+.. prompt:: bash #
+
+ ceph nfs export apply mynfs -i update_cephfs_export.json
+ cat update_cephfs_export.json
::
- $ ceph nfs export apply mynfs -i update_cephfs_export.json
- $ cat update_cephfs_export.json
{
"export_id": 1,
"path": "/",
"clients": []
}
-An export can also be created or updated by injecting a Ganesha NFS EXPORT config
-fragment. For example,::
+An export can also be created or updated by injecting a Ganesha NFS EXPORT
+config fragment. For example:
+
+.. prompt:: bash #
+
+ ceph nfs export apply mynfs -i update_cephfs_export.conf
+ cat update_cephfs_export.conf
+
+::
- $ ceph nfs export apply mynfs -i update_cephfs_export.conf
- $ cat update_cephfs_export.conf
EXPORT {
FSAL {
name = "CEPH";
After the exports are successfully created and NFS Ganesha daemons are
deployed, exports can be mounted with:
-.. code:: bash
+.. prompt:: bash #
-
$ mount -t nfs <ganesha-host-name>:<pseudo_path> <mount-point>
+ mount -t nfs <ganesha-host-name>:<pseudo_path> <mount-point>
-For example, if the NFS cluster was created with ``--ingress --virtual-ip 192.168.10.10``
-and the export's pseudo-path was ``/foo``, the export can be mounted at ``/mnt`` with:
+For example, if the NFS cluster was created with ``--ingress --virtual-ip
+192.168.10.10`` and the export's pseudo-path was ``/foo``, the export can be
+mounted at ``/mnt`` by running the following command:
-.. code:: bash
+.. prompt:: bash #
- $ mount -t nfs 192.168.10.10:/foo /mnt
+ mount -t nfs 192.168.10.10:/foo /mnt
If the NFS service is running on a non-standard port number:
-.. code:: bash
+.. prompt:: bash #
- $ mount -t nfs -o port=<ganesha-port> <ganesha-host-name>:<ganesha-pseudo_path> <mount-point>
+ mount -t nfs -o port=<ganesha-port> <ganesha-host-name>:<ganesha-pseudo_path> <mount-point>
.. note:: Only NFS v4.0+ is supported.
Troubleshooting
===============
-Checking NFS-Ganesha logs with
+There are two methds for examining NFS-Ganesha logs:
-1) ``cephadm``: The NFS daemons can be listed with:
+#. ``cephadm``: List the NFS daemons by running the following command:
- .. code:: bash
+ .. prompt:: bash #
- $ ceph orch ps --daemon-type nfs
+ ceph orch ps --daemon-type nfs
- You can via the logs for a specific daemon (e.g., ``nfs.mynfs.0.0.myhost.xkfzal``) on
- the relevant host with:
+ You can search via the logs for a specific daemon (e.g.,
+ ``nfs.mynfs.0.0.myhost.xkfzal``) on the relevant host with:
- .. code:: bash
+ .. prompt:: bash #
- # cephadm logs --fsid <fsid> --name nfs.mynfs.0.0.myhost.xkfzal
+ cephadm logs --fsid <fsid> --name nfs.mynfs.0.0.myhost.xkfzal
-2) ``rook``:
+#. ``rook``:
- .. code:: bash
+ .. prompt:: bash #
- $ kubectl logs -n rook-ceph rook-ceph-nfs-<cluster_id>-<node_id> nfs-ganesha
+ kubectl logs -n rook-ceph rook-ceph-nfs-<cluster_id>-<node_id> nfs-ganesha
-The NFS log level can be adjusted using `nfs cluster config set` command (see :ref:`nfs-cluster-set`).
+The NFS log level can be adjusted using the ``nfs cluster config set`` command
+(see :ref:`nfs-cluster-set`).
.. _nfs-ganesha-config:
projects / ceph.git / commitdiff