mgr/nfs: remove unused 'realm' arg for 'nfs export create rgw'

This argument is not used.  The rgw exports currently only work on the
default realm.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2db3cfabc3d5154036ccf3168252f26a75ef0935)

doc/mgr/rook: update title

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9aeefbc666e2d373f95aca20deb15a9e9a0b6f86)

doc/mgr/nfs: reference customizing ingress

Link to the cephadm docs on modifying the service directly.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7d3443412ce8feadac4258f852c44d6947dd1389)

doc/mgr/nfs: add section for manual ganesha config; reframe

This documentation is incomplete because this mode of operation is not
tested/validated.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b5a32d632d2acb77664df9b25d98fb6489b673fc)

doc/mgr/nfs: document ingress in more detail

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8d9db910f797557d615b5e5e3e612bed7dd00f22)

doc/mgr/nfs: typo

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0448a8d39de71e5135061873a40f9302ad2c5dce)

doc/mgr/nfs: add note about incomplete ingress

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 684c6a2dbaf5c6c2d8d2abe24243f6b973d14c58)

qa/suites/orch/cephadm: add rgw nfs export test

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit cd089ee74eb01bb9d2208400296e61e9636df5ab)

mgr/cephadm: ingress: tolerate no daemons

This doesn't normally happen, but did before the daemon inventory breakage
(see previous patches) was fixed.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit bae406f74607b16c0da1382537d9bab49d45061e)

mgr/nfs: add --squash option to 'nfs export create rgw ...'

The cephfs variant already has it.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ad5fe4452d8ae5226f0d9bda918b048078b548e0)

mgr/nfs: use bucket owner creds for rgw bucket export

The bucket owner can always read/write to the bucket, so use those creds
for the export.  This is less complicated than setting up a dedicated
user anyway.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7d2f7efe3fd36e88e5ba9469cc918c4e915b7e82)

mgr/cephadm: use new CEPH_IMAGE_TYPES for all daemons using ceph container image

We were using CEPH_TYPES + GATEWAY_TYPES, but that isn't really accurate.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit deaff0c42cb362b21a03dc76cff551741c761561)

qa/tasks/python: simple task to run python code

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 45737fe95ac3a2b53dda466179e426c2ce574fb6)

doc/mgr/nfs: revisions

- clean up language
- move config hierarchy to the bottom (this is an implementation detail
that is only useful if managing ganesha externally)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8b96d9c22bf05c34b86e0f440edff5f72449246f)

mgr/nfs/export: nicer exceptions on cap update

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b73f73074941ca0f34d7e26a1705b86453549bfc)

doc/mgr/nfs: fix 'export apply', pool name

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 62e4ea5ff195d103d1e800964ff1df66e4eef9cc)

PendingReleaseNotes: document workaround for NFS storage change

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c100812bff699af4d96c2861fe13bd23b937810b)

qa/tasks/mgr/test_orchestrator_cli: fix test

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a1ee80fcf149ba1c6df1d0d21a7bc196d5e8a2fa)

qa/suites/orch/cephadm/mgr-nfs-upgrade: add test for nfs migration

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 52f6989fff4809e7acdd8cfe8f0d2d04298cb868)

Conflicts:
  qa/suites/orch/cephadm/mgr-nfs-upgrade/0-centos_8.2_kubic_stable.yaml

mgr/cephadm: migrate nfs grace file

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 93bc85fc231a115a8c194617b5f3cef0f732fecd)

mgr/nfs: migrate pre-pacific nfs.ganesha-foo clusters to nfs.foo

In octopus, the service is nfs.ganesha-$id instead of nfs.$id

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit bd85c678a574a61d249f78034cf5f83a4668ae7d)

doc/cephfs/fs-nfs-exports: document new export apply capabilities

- import list
- import ganesha conf

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3cfd779e5b5ce1307baece4267d3b94f1004786b)

qa/tasks/cephfs/test_nfs: define NFS_POOL_NAME

We can't import from mgr_module.py from here, sadly.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d41b60404d4098ef6ca965be89d500499d83319c)

mgr/nfs: use NFS_POOL_NAME in test_nfs.py

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8026eb5e8eecd136046a2b8174a233e10875507f)

mgr/nfs: test export apply on JSON list

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit bfd5ab3124ec73a7b4d28feba3d50780b1a0e8ca)

mgr/nfs: add test for ganesha conf apply/import

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7760cb2aedf52e247d9908b65d30250e2a049df3)

qa/tasks/cephfs/test_nfs: retry mount a few times

It may take a moment for a ganesha to (re)configure itself with a new
export.  If a mount fails, retry a couple times.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1e6fd912f646c77d22bf38f28d50d31f9679fe4c)

mgr/cephadm: migrate all legacy nfs exports to new .nfs pool

Migrate all past NFS pools, whether they were created by mgr/nfs or by
the dashboard, to the new mgr/nfs .nfs pool.

Since this migrations relies on RADOS being available, we have to be a bit
careful here: we only attempt the migration from serve(), not during
module init.

After the exports are re-imported, we destroy existing ganesha daemons so
that new ones will get recreated.  This ensures the (new) daemons have
cephx keys to access the new pool.

Note that no attempt is made to clean up the old NFS pools.  This is out
of paranoia: if something goes wrong, the old NFS configuration data will
still be there.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c183bccdc6728e8321eec615b1acd88216acabca)

mgr/nfs: adjust cephfs export caps if necessary

If we are importing an old export, we may find that the cephx user
existed but with the wrong caps.  Adjust caps in that case!

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 379a1fc4d794405edd4c34d9ac44c2c5621cfe58)

python-common: don't accept pool/ns for NFSServiceSpec

All users are now gone.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d0d95db00d462cc8101311d591bda26dc452581f)

mgr/orchestrator: drop rados_config_location ServiceDescription property

I'm not sure what this was intended to be used for...

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 75970ff80652e10bd79234b97de63cb0f39d9d90)

mgr/cephadm: move rados_config_location() out of NFSServiceSpec

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7cbd1fbce82c567a2f8608b051dd16bbb8c5ae07)

mgr/nfs: change nfs pool to .nfs

This is a new pool that we can migrate all past NFS configuration to,
simplifying the migration process (and also allowing us to pick a
.-prefixed name).

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 82e939d89c3b6e2ce6faad9a5315c5372fef341d)

Conflicts:
src/pybind/mgr/rook/rook_cluster.py

mgr/nfs/export: accept a JSON or ganesha EXPORT config

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6b2ccb2049484d2325bd2189410165093217a766)

mgr/nfs: allow 'nfs export apply' to take a list of exports

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1d5f63ecc5e047eac548f85b819d3ea3f5b2215c)

python-common: remove pool + namespace from NFSServiceSpec

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9d38ee24cf6a2dd79d1fbce50ff4f83dac79f436)

mgr/nfs: used fixed pool + ns

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3ee6775c3ca96b9f408d4e5977e9d15142b5f1e5)

mgr/rook: used fixed pool + ns

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d799f9b5bdf39b2c3226cd160ac5f470951eea67)

Conflicts:
src/pybind/mgr/rook/rook_cluster.py

mgr/dashboard: use fixed pool + ns

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ea7c8910ea8ed383749e0c15f95d6f61796b98ff)

mgr/cephadm: always use fixed pool and namespace

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 568c630760a35c953e945fe56d9524fee66b803e)

mgr/nfs: adjust test to match pool name

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 362e121e468f7f2b16eb457ba24d9af8b74c3f9c)

mgr/nfs: always create ganesha pool with well-defined name

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 48de07886a6ebbed71b2b85babf56fae6ab0eceb)

doc/mgr/nfs: update about RGW exports

This patch just moves the RGW exports created using nfs module to mgr/nfs
document. The RGW requirements will be updated in a different PR.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 387369373cba9f569b5fd84e64fffb0fe8de6821)

Conflicts:
doc/mgr/nfs.rst

doc/cephfs/nfs: update about nfs module

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 65489e8d3e7f40f19e1df5ed6871d7116e4a5023)

Conflicts:
    doc/cephfs/nfs.rst

doc/mgr/nfs: update cephfs export create command about client and squash arguments

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 5db2b48f14cddf673323d72f303ed1cfedd877f9)

doc/mgr/nfs: update nfs links

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit ce7fef695b82d798ed38728ccc032990b1ad9dd9)

Conflicts:
.github/labeler.yml
doc/releases/pacific.rst

doc/mgr/nfs: add missing cluster_id to export info command

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 29f28563ec94d06eaebf1a014645f259e93f8246)

doc/cephfs: move nfs doc under mgr docs

Fixes: https://tracker.ceph.com/issues/51428
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit c79b797465bda1a30a085ea25bc1b3f82e5665de)

qa/tasks/vstart_runner: add LocalCluster.run

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1bfa812f5d976e5c9558a0d52ca160566f38c9d3)

qa/tasks/cephfs/test_nfs: fiddle with sudo

- no sudo for 'ceph' commands
- explicit sudo for _sys_cmd (things like 'rados' don't need sudo!)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8788af5663c52f54bede3076b52b8fda11b4c908)

mgr/nfs/export: some cleanup, minor refactoring

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d1c20f800397f15878147bff4a642e695b25a345)

mgr/nfs/cluster: remove unused @cluster_setter

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 51bb1703f1f44fc1182cc15154f7f5d8476ee805)

nfs/mgr: fix help message case

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 14cf8c7174d15c164c609a3d4a91e06f2a6a3ce9)

doc/cephfs/fs-nfs-export: add note about export update behavior

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit fc304f2d56c739633ae4a4f118d48c85a3affee0)

mgr/nfs: move user create/delete into helper

- Do user create or delete via a helper
- Defer until after we have validated the Export (on create or update)
- Support updates to user_id, which is needed to keep the naming consistent
and to also support changing the bucket, since the user_id is derived
from that.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 10786a63806d91d655b3914af7b98d059ae05464)

mgr/nfs: refactor _delete_user helper

Single caller, trivial functions, and caller is also short.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 165e02561c439f459a6b0e515e5e3fbe65bfcacc)

mgr/nfs: refactor create_export_from_dict() helper

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 636c1575474f593e96bf2c149e8ac531f1ca70da)

mgr/nfs: keep 'nfs export get' around for backward-compat

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f307c0dccdc999aa9cbef0e510e4c0ff9dc1147e)

mgr/nfs: rename method

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 317143d8bfd4a0721bdd32006f13a1aecca7b109)

qa/tasks/cephfs/test_nfs: test new export via apply

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6fb9f0a391138440aab605434608a399f3c21316)

doc/cephfs/fs-nfs-export: be consistent with cluster_id and _ vs -

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 87c98df2cd2aa27806fe7f4f8cd4e5ec0a4fd74b)

mgr/nfs: addr -> client_addr for 'nfs export create ...'

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 729f2f6adf7ea6207d61cf9f3f3ccb90560733fc)

mgr/nfs: fix tests

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a2679b38845eb652f31130c1335f071d2f4f9d89)

mgr/nfs: 'nfs export get' -> 'nfs export info'

This is a breaking change, but aligns the CLI verbs with the rest of ceph.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2400937ce25289f5a8e003244f8a14654735d0c2)

mgr/nfs: binding -> pseudo_path

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5a2382a5a85eeba80bdd7f6192d4bc7a4af094e5)

mgr/nfs: more revisions based on review

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b276adbd4780702fc98303f05b78d946cb677182)

mgr/nfs: adjust NFSExceptoin errno arg

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9cf2ae5e8d84d60d63d67643596eca7b32082cf7)

doc/cephfs: update 'nfs export {get,apply}' docs

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f0798cf0681325bb780d2737db3bf86c199c0aa1)

mgr/nfs: merge FSExport back into ExportMgr

This class captures no data, and the current methods cover both cephfs
and rgw exports.  There is little value to making the class separate from
ExportMgr.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1529f78ecf43ddc878ae028e6a56baa87daeed3b)

doc/radosgw/nfs: document mgr/nfs way to add/remove rgw exports

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0907658f21b643dd767abdafcbbbf173213a6558)

mgr/nfs: merge 'nfs export {update,import}' -> 'nfs export apply'

The only thing we lose is a strict 'update' that errors out if the
export didn't already exist, and we don't have any known need for that.

Fixes: https://tracker.ceph.com/issues/51265
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2cb2abb385f97714cbe59371aa788394799db925)

mgr/nfs: test export creation and list

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 69681f04ba78ad377219d113593759218e5bf4e4)

mgr/nfs: test export_update (+ fixes)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit cfee205826d6a3ce1a9cb0fc5a66088f67e49186)

mgr/nfs: test Export.validate(); several fixes

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 01a8d0ba3d5aaf09b26de3acee33772c43ed9cac)

mgr/nfs: test that export <-> block+dict conversions go both ways

Fixes: https://tracker.ceph.com/issues/50449
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 16c5cb62f797b0154362678aeed0bf7aebe12f40)

mgr/nfs: clean up test a bit

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 59587fd0b4f27e6322935cc1d3773b057f6cb184)

mgr/nfs/export: fix export validation

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit fdd47c476e4b6232461a62f956837a049faf6ca0)

mgr/nfs/export: fix tests

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 473097a36f92d4e6694d0be7b5cac123d723a8e6)

mgr/nfs: handle option addr/client block in create_export()

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d6d9e59ef76f33f0f9611acfb6406d849b9e29a8)

mgr/nfs: allow multiple addrs for new exports

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit aaa57afd2930b6dafcf4dde4e2d939c428d290f4)

mgr/nfs: fix/finish rgw export

- create the rgw keys for the export
- cleanup the rgw user on export deletion

Fixes: https://tracker.ceph.com/issues/47172
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 119555b84d4ae460b59ec74c665a6b215e2ba628)

mgr/nfs/module: clusterid -> cluster_id

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b54998106330cc8e4754a35b872e470a95568ba9)

mgr/nfs/export: fix export_update_1 to type check

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6771de220435d5a8f6ebc7601f0aa451d8e3c68b)

mgr/nfs/cluster: fix type error

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f9d291b5755520feca393c7fca964de999a6e90a)

mgr/nfs/export: wrap long lines

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1cb98a04974da044af81b9ab6c998c6f746c6473)

mgr/nfs: ExportMgr._delete_export only works for cephfs for now

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 8ba3b5bf2e0492c73c0982fa1389cb46edbf2475)

mgr/nfs: Remove pool_ns from NFSCluster

Cause this is a singleton instance. Better to explictly pass it along.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e6db0c810187fccbd3d5b77576727c63e3fc1cc3)

mgr/nfs: Remove ExportMgr.rados_namespace

Cause this is a singleton instance. Better to explictly pass it along.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e0a9d9dccf4474253aec43290d54da8f9cd3742d)

mgr/nfs: flake8

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit a24e912099851a7c77d453afa84154858233e7d8)

Conflicts:
src/pybind/mgr/tox.ini

mgr/nfs: Add type checking

Also introduce RawBlock class

Fixes: https://tracker.ceph.com/issues/50816
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 42d01c5ed3da749d2af6d3debffd3e117d18856f)

Conflicts:
src/mypy.ini

mgr/nfs: Add __eq__ method to Export

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit b3ddff838461be1805612184a6cd8ccbedb03ae5)

mgr/nfs: Add some compatibility to mgr/dashboard

provide default values for `squash` and `security_label`

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 42a8d383e6b6da46cc4ec011f491f9221b48ec4d)

mgr/nfs: Fix whitespace handling

keep whitespace in lines starting with %url

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit e1928d57060ca3e180d6b8fc7fe5b8ee8c4aefad)

mgr/nfs: Copy unit tests from mgr/dashboard

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit a30328c49677d887c35697aa89619ca1bf6159f4)

mgr/nfs: partially implement rgw export support

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 377756c9dff1bf690a5be700e99c22d63d0ebad2)

mgr/nfs: abstract FSAL; add RGWFSAL

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5c33431ce132ef7fc5db72d0c06cb89c5524c3a1)

mgr/nfs: refactor to merge 'update' and 'import' code

- update must be an existing export
- import can be a new or existing export, and we will either update or
  create as appropriate.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 097e4f9aa3b9ed663116381a13eadf279f77b8a2)

mgr/nfs: add 'nfs export import' command

Allow import of JSON to create a new export.  Similar to 'update', which
allows an import of JSON to update an existing export.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 778674a9c728b2b04f1b6922005798303475d352)

mgr/nfs: refactor 'nfs export update' and export validation

Move validation to the Export class, so it can check any object in
place.

Refactor update code to extract allowed changes checks from the old
validation class.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2b93356fac49a5bd50a9e4b97314db87ade51666)

mgr/nfs: fix _fetch_export to distinguish between clusters

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0eea3d772667a6d9ba4d42427404e8e6730e5df5)