qa/tasks/ceph_manager: fix assertion

The osd may be 0.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit e0a926a2c18d76225fd4d4051bc19b9a1917b932)

osd: re-cache peer_bytes on every peering state activate

peer_bytes is used for backfill reservation request and may be
reset if backfill is interrupted, and we want it set back before
continuing backfill and re-sending the reservation request.

Fixes: https://tracker.ceph.com/issues/52448
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit bdfdf96d2f6c3cf7e5595ae5b8238fd4c0b3c6bc)

Merge pull request #43348 from cfsnyder/wip-52350-pacific

pacific: rgw: fix sts memory leak

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42643 from cfsnyder/wip-51803-pacific

pacific: rgw/notifications: send correct size in case of delete marker creation

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #43235 from MrFreezeex/wip-51839-pacific

pacific: ceph.spec: selinux scripts respect CEPH_AUTO_RESTART_ON_UPGRADE

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Dan van der Ster <daniel.vanderster@cern.ch>

Merge pull request #43264 from cfsnyder/wip-52332-pacific

pacific: cmake: s/Python_EXECUTABLE/Python3_EXECUTABLE/

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #43099 from cfsnyder/wip-51952-pacific

pacific: osd: fix to recover adjacent clone when set_chunk is called

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #43306 from myoungwon/pacific-backport-52322

pacific: osd: fix to allow inc manifest leaked

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #43238 from rhcs-dashboard/wip-52685-pacific

pacific: mgr/dashboard: Fix failing config dashboard e2e check

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

rgw: fix sts memory leak

fix https://tracker.ceph.com/issues/52290

Signed-off-by: yuliyang_yewu <yuliyang_yewu@cmss.chinamobile.com>
(cherry picked from commit ef921bcdaa78d33ed0611a60ec58826d8e6ccb45)

rgw/notifications: send correct size in case of delete marker creation

Fixes: https://tracker.ceph.com/issues/51681
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit d81e27faa1033c5290cfd0b4cf27cdaf98d34bc4)

Conflicts:
src/rgw/rgw_op.cc
src/test/rgw/bucket_notification/test_bn.py

Cherry-pick notes:
- src/test/rgw/bucket_notification/test_bn.py changes manually applied to src/test/rgw/rgw_multi/tests_ps.py for Pacific
- conflicts in rgw_op.cc due to rename of RGWObject to Object after Pacific

Merge pull request #43185 from rhcs-dashboard/wip-52617-pacific

pacific: mgr/dashboard: Incorrect MTU mismatch warning

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #43113 from idryomov/wip-rbd-validate-pool-async-pacific

pacific: librbd: fix pool validation lockup

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

test: allowing >= the real refcount to avoid false alarm

Current dedup allow to contain multiple same sources using
multiset, which results in inconsistent situation as follow
(during set_chunk, but not confined in set_chunk).

1. User issues set_chunk
2. OSD receives the set_chunk, and sends increment message
to an object in the low tier (INPROGRESS).
3. OSD map is changed (841 → 843)
3.5. on_change() is called
4. the set_chunk op is reenqueued by requeue_op()
5. OSD handles the duplicated set_chunk, but it is not able to
 know the set_chunk is duplicated because it does not log on the disk yet.
6. OSD issues increment message again to the object
in the low tier. (increment operation is executed twice)

To fix this, this commit allows >= the real refcount in test cases

fixes: https://tracker.ceph.com/issues/51000

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 126df96)

Merge pull request #41731 from sseshasa/wip-51117-pacific

pacific: osd: Run osd bench test to override default max osd capacity for mclock

Reviewed-by: Neha Ojha <nojha@redhat.com>

tools/setup-virtualenv: verify the `--python` param

when given an invalid or empty (PYTHON_BINARY)
fixup for 494fea50ced7

Fixes: https://tracker.ceph.com/issues/52304
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit c8d4c131adfe41739fd557091dc2931c1b7f15c5)

cmake: s/Python_EXECUTABLE/Python3_EXECUTABLE/

pass the python3 exec when creating the ceph-volume build venv
fixup for 5fc657b40dc7

Fixes: https://tracker.ceph.com/issues/52304
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 7db830598507d90d1c9e1f4468f818bebce58037)

Merge pull request #43242 from ceph/wip-yuriw-p2p-pacific

qa/tests: advanced version to reflect the latest 16.2.6 release

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests: advanced version to reflect the latest 16.2.6 release

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #42949 from yuvalif/wip-51597-pacific

pacific: rgw/notifications: cache object size to avoid accessing invalid memory

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42947 from yuvalif/wip-51511-pacific

pacific: rgw/notifications: support v4 auth for topics and notifications

Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #42946 from yuvalif/wip-51350-pacific

pacific: rgw/notification: make notifications agnostic of bucket reshard

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #42945 from yuvalif/wip-51045-pacific

pacific: rgw: fix spelling of eTag in S3 message structure

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #43075 from sebastian-philipp/pacific-backport-41510-41509-41964-41574-42278-42073-42406-42463-42627-42680-

pacific: mgr/nfs: nfs-rgw batch backport

Reviewed-by: Sage Weil <sage@newdream.net>

doc/cephfs/nfs: Remove missing target

Fixes: `Unknown target name: "nfs-ganesha nfs server".`
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>

Merge pull request #43188 from rhcs-dashboard/wip-51274-pacific

pacific: mgr/dashboard: deprecated variable usage in Grafana dashboards

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: p-se <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

mgr/dashboard: Fix failing config dashboard e2e check

Recently a new osd config has been added in
6ca32bde2e1d0dd58df168126582a570ac09aad6 and this is getting Modified.
So on our dashboard e2e config check which checks for the Modified
filter, this is also coming on the entry. So we need to increase the
count.

Fixes: https://tracker.ceph.com/issues/52649
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 53f78d330c98f803c09a2314397d48f253db0866)

Merge pull request #43190 from rhcs-dashboard/wip-52251-pacific

pacific: mgr/dashboard: cephfs MDS Workload to use rate for counter type metric

Merge pull request #42678 from callithea/wip-52021-pacific

pacific: mgr/dashboard: Visual regression tests for ceph dashboard

Reviewed-by: aaryanporwal <NOT@FOUND>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Yuri Weinstein <yweins@redhat.com>

ceph.spec: selinux scripts respect CEPH_AUTO_RESTART_ON_UPGRADE

In /etc/sysconfig/ceph we allow operators to define if ceph daemons
should be restarted on upgrade: CEPH_AUTO_RESTART_ON_UPGRADE.

But the post selinux scripts will stop ceph.target regardless if this
is set to `no`, leading to operators adding various hacks to prevent
these unexpected or inconvenient daemon restarts. By now, if users
are using rpms directly, they are likely orchestrating their own
daemon restarts so should not rely on the rpm itself to do this.

Fixes: https://tracker.ceph.com/issues/21672
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 092a6e3e83e9ef8e37cb6f1033c345dcb5224cfc)

Merge pull request #43002 from ktdreyer/pacific-52472-rm-virtualenv

pacific: *: s/virtualenv/python -m venv/

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

16.2.6

[mgr/dashboard] cephfs metrics in MDS Workload panels to use rate because of counter type metric

Fixes: https://tracker.ceph.com/issues/51954
Signed-off-by: Jan Horacek <jan.horacek@livesport.eu>
(cherry picked from commit 5bf516dcc7f59a970c6369ab075cbca7d81ac7d5)

mgr/dashboard: deprecated variable usage in Grafana dashboards

Fixes: https://tracker.ceph.com/issues/50059
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit a709abf8bf5a6b25c21db100e87af3a6c2cf382d)

mgr/dashboard: Incorrect MTU mismatch warning

The MTU mismatch warning was being fired for those NIC's as well that are in down state. This PR intends to fix this issue

Fixes:https://tracker.ceph.com/issues/52028
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 58d635455d1f59921d5ad821168f31b6f937588a)

Merge pull request #43168 from linuxbox2/pacific-perl-semicolon

workunits/rgw: semicolon terminates perl statements

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

workunits/rgw: semicolon terminates perl statements

Fixes a lexical error in one line of code added in
90e9307ab0a52da260bc1ebb50329cd8ff942eb9, removing the dependency
on lsb_release, on 8/16/2021.

Fixes: https://tracker.ceph.com/issues/52613
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 221fdb858be083a981697cdb59c19f0659be3f1d)

Merge pull request #43151 from ceph/pacific-systemd

pacific: debian/control: dh-systemd is part of debhelper now

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #43131 from ceph/remove-cython

pacific: debian/control: remove cython from Build-Depends

Reviewed-by: Kefu Chai <kchai@redhat.com>

pacific: debian/control: dh-systemd is part of debhelper now

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=822670

Signed-off-by: David Galloway <dgallowa@redhat.com>
(cherry picked from commit 4f21d41b73e30b71c9222841751dcebf898613cc)

qa/tasks: Set default caps for 'osd' type in generate_caps()

Assign the default caps for osds to be the same as what the AuthMonitor
sets for a new osd. See AuthMonitor::validate_osd_new() which sets the
following caps for a new osd:

 mon='allow profile osd'
 mgr='allow profile osd'
 osd=''allow *'

When an actual real world cluster is deployed, the above caps are applied.
Unless the user modifies the defaults, a cluster will operate with the
above caps. Therefore, it makes sense to use the defaults when testing
Ceph so that issues if any due to the default settings may be caught and
fixed.

Therefore, the caps for the 'osd' type is reset to the default in
generate_caps(). The caps for 'mgr' already reflects the system defaults.
The caps for 'mds' type is not changed in this commit and will be
investigated and changed if necessary later.

Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 4b0dba28b62bcc89c46e1f5ff0dab8583af4397c)

mon/MonCap: Update osd profile to allow cmd to set iops capacity on mon db

The default mon caps for osds is set to "allow profile osd", which allows
only "rw" capability. Osds with mclock scheduler enabled store their max
iops capacity on the mon config store. This can be achieved by executing
the "config set" command. However, since the osd(s) by default do not have
the execute permission, the command fails with "Permission denied" error.

Therefore, modify the default osd profile to allow running the "config set"
command with restriction to only set keys with name matching either (regex)
"osd_mclock_max_capacity_iops_hdd" or "osd_mclock_max_capacity_iops_ssd"
so that the osd has the permission to update the mon config store with the
desired information.

Fixes: https://tracker.ceph.com/issues/52329
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 2cdbe81d7dd34b65e5c3c51005df5214a1e3a577)

doc: Update mclock-config-ref doc steps to override osd max iops capacity.

Update the steps in the mclock config reference document to manually
override an OSDs max IOPS capacity. Provide information on the alternative
ways to override the osd_mclock_max_capacity_iops_[hdd,ssd] options for
an OSD.

Fixes: https://tracker.ceph.com/issues/52025
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 7c25511a41e452bcc5806f192e90aac6cd113f9e)

osd: Add config option to skip running the OSD benchmark on start-up.

Introduce a new dev config option "osd_mclock_skip_benchmark" that
when set skips running the OSD benchmark on start-up. By default
this option is disabled. This is useful in the following scenarios:

 - Dev/CI testing,
 - Configurations that don't need QoS.

If the option is enabled, the default OSD iops capacity is read from
osd_mclock_max_capacity_iops_[hdd,ssd].

Fixes: https://tracker.ceph.com/issues/52025
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 6ca32bde2e1d0dd58df168126582a570ac09aad6)

 Conflicts:
src/common/options/osd.yaml.in
 - Removed non-existent file: src/common/options/osd.yaml.in since the
   switch to yaml for config options is not available in pacific yet.

debian/control: remove cython from Build-Depends

as cython3 is enough. and we've dropped the python2 support.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit bcd7f3ac0f299855516df871a56ef26c1ee21bc9)

Merge pull request #43100 from ifed01/wip-ifed-fix-migrate-pac

pacific: os/bluestore: fix bluefs migrate command

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #42976 from idryomov/wip-51419-pacific

pacific: common/buffer: fix SIGABRT in  rebuild_aligned_size_and_memory

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #43116 from guits/wip-52560-pacific

pacific: ceph-volume: fix lvm activate arguments

Merge pull request #43110 from guits/wip-52554-pacific

pacific: ceph-volume: fix lvm migrate without args

doc/mgr/nfs: add section on updating an nfs cluster

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7cc4c91dce2a1f1bcb6a02efb231bdbda0929500)

mgr/cephadm: drop daemon_id arg to CephadmService.config()

Unused (and nonsensical since this is *service* config).

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e9dabdab6540d949bd3401fef66ec583368b33de)

mgr/nfs: add --port to 'nfs cluster create' and port to 'nfs cluster info'

Fixes: https://tracker.ceph.com/issues/51787
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8ebe341198d95da1a0fbfe4d5ff5bbbec7aa155e)

qa/suites/orch/cephadm/smoke-roleless: test taking ganeshas offline

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a5e8227a83c229698af462a18f9548c03c559408)

qa/tasks/vip: exec with bash -ex

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3c1e086be013bf875bfe32b5ba9a297fc27410a3)

qa/suites/orch/cephadm: separate test_nfs from test_orch_cli

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0ae020979d964b50f550cab6366ca019133e1dd1)

src/pybind/mgr/nfs/tests: pass cluster_id to from_export_block()

As from_export_block() requires cluster id.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 13c0525619ab928d664e6896489029aa7548e159)

src/pybind/mgr/nfs: remove `tag` option

'tag' is required for NFSv3 mounts. We don't support v3, so let's remove it.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit ab80ab4675d70c3c9fbf5f5f0c3ec6680f055b17)

src/pybind/mgr/nfs: remove per daemon config test

This is not required as config object is per cluster.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 399edcd321e95338ff96fa1f795485b5ed97bbcb)

src/pybind/mgr/nfs: directly use cluster_id and remove daemon related stuff

Fixes: https://tracker.ceph.com/issues/51614
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 3c51638d4ee5e116e55238a4836d0824ca6d9bf3)

.github/labeler: add nfs dev doc

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 482a13077a079b88a1dcf7ed2582e345f568f38c)

doc/dev/vstart-ganesha: update about RGW export

Fixes: https://tracker.ceph.com/issues/51683
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 265561bef6b31e446ec2fd00d508c0eed1796c7a)

src/vstart: update ganesha pid dir location

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit accac3beb9f9f06e8ec7776af4b455421585ed34)

src/vstart: create rgw export for nfs

Fixes: https://tracker.ceph.com/issues/51800
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 8a77e017fd2110ec9d7fbc0a33de0fb85f14cbca)

Conflicts:
src/vstart.sh

src/pybind/mgr/test_orchestrator: remove nfs pool and namespace

'orch apply nfs' no longer requires pool and namespace.

Fixes: https://tracker.ceph.com/issues/51794
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit da5aa88844531c7b9ef0420cf274a72d5b4e308e)

src/vstart: update nfs-ganesha pool name

Recently, the default pool name for nfs was changed from 'nfs-ganesha' to
'.nfs'. Change the pool name in vstart too.

Fixes: https://tracker.ceph.com/issues/51795
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit ff33b4f23b9263ae7e189cffef18ec2128e52a35)

mgr/nfs: remove unused 'realm' arg for 'nfs export create rgw'

This argument is not used.  The rgw exports currently only work on the
default realm.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 2db3cfabc3d5154036ccf3168252f26a75ef0935)

doc/mgr/rook: update title

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 9aeefbc666e2d373f95aca20deb15a9e9a0b6f86)

doc/mgr/nfs: reference customizing ingress

Link to the cephadm docs on modifying the service directly.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7d3443412ce8feadac4258f852c44d6947dd1389)

doc/mgr/nfs: add section for manual ganesha config; reframe

This documentation is incomplete because this mode of operation is not
tested/validated.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b5a32d632d2acb77664df9b25d98fb6489b673fc)

doc/mgr/nfs: document ingress in more detail

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8d9db910f797557d615b5e5e3e612bed7dd00f22)

doc/mgr/nfs: typo

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 0448a8d39de71e5135061873a40f9302ad2c5dce)

doc/mgr/nfs: add note about incomplete ingress

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 684c6a2dbaf5c6c2d8d2abe24243f6b973d14c58)

qa/suites/orch/cephadm: add rgw nfs export test

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit cd089ee74eb01bb9d2208400296e61e9636df5ab)

mgr/cephadm: ingress: tolerate no daemons

This doesn't normally happen, but did before the daemon inventory breakage
(see previous patches) was fixed.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit bae406f74607b16c0da1382537d9bab49d45061e)

mgr/nfs: add --squash option to 'nfs export create rgw ...'

The cephfs variant already has it.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ad5fe4452d8ae5226f0d9bda918b048078b548e0)

mgr/nfs: use bucket owner creds for rgw bucket export

The bucket owner can always read/write to the bucket, so use those creds
for the export.  This is less complicated than setting up a dedicated
user anyway.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7d2f7efe3fd36e88e5ba9469cc918c4e915b7e82)

mgr/cephadm: use new CEPH_IMAGE_TYPES for all daemons using ceph container image

We were using CEPH_TYPES + GATEWAY_TYPES, but that isn't really accurate.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit deaff0c42cb362b21a03dc76cff551741c761561)

qa/tasks/python: simple task to run python code

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 45737fe95ac3a2b53dda466179e426c2ce574fb6)

doc/mgr/nfs: revisions

- clean up language
- move config hierarchy to the bottom (this is an implementation detail
that is only useful if managing ganesha externally)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8b96d9c22bf05c34b86e0f440edff5f72449246f)

mgr/nfs/export: nicer exceptions on cap update

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b73f73074941ca0f34d7e26a1705b86453549bfc)

doc/mgr/nfs: fix 'export apply', pool name

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 62e4ea5ff195d103d1e800964ff1df66e4eef9cc)

PendingReleaseNotes: document workaround for NFS storage change

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c100812bff699af4d96c2861fe13bd23b937810b)

ceph-volume: fix lvm activate arguments

When using the `lvm activate` subcommand without any osd ID or osd FSID
then ceph-volume fails.

Currently we can either activate with:
 - both osd ID and osd FSID
 - only the osd FSID (because it's unique)

The remaining scenarios aren't covered and generate a stack trace:
 - only the osd ID
 - no osd ID nor osd FSID

This ends up with an error like:

-->  UnboundLocalError: local variable 'tags' referenced before assignment

Fixes: https://tracker.ceph.com/issues/50665
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit b77ed5f99d3cb14a46bb36bb74e4136c22edf48a)

qa/tasks/mgr/test_orchestrator_cli: fix test

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit a1ee80fcf149ba1c6df1d0d21a7bc196d5e8a2fa)

qa/suites/orch/cephadm/mgr-nfs-upgrade: add test for nfs migration

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 52f6989fff4809e7acdd8cfe8f0d2d04298cb868)

Conflicts:
  qa/suites/orch/cephadm/mgr-nfs-upgrade/0-centos_8.2_kubic_stable.yaml

mgr/cephadm: migrate nfs grace file

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 93bc85fc231a115a8c194617b5f3cef0f732fecd)

mgr/nfs: migrate pre-pacific nfs.ganesha-foo clusters to nfs.foo

In octopus, the service is nfs.ganesha-$id instead of nfs.$id

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit bd85c678a574a61d249f78034cf5f83a4668ae7d)

doc/cephfs/fs-nfs-exports: document new export apply capabilities

- import list
- import ganesha conf

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3cfd779e5b5ce1307baece4267d3b94f1004786b)

qa/tasks/cephfs/test_nfs: define NFS_POOL_NAME

We can't import from mgr_module.py from here, sadly.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d41b60404d4098ef6ca965be89d500499d83319c)

mgr/nfs: use NFS_POOL_NAME in test_nfs.py

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 8026eb5e8eecd136046a2b8174a233e10875507f)

mgr/nfs: test export apply on JSON list

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit bfd5ab3124ec73a7b4d28feba3d50780b1a0e8ca)

mgr/nfs: add test for ganesha conf apply/import

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7760cb2aedf52e247d9908b65d30250e2a049df3)

qa/tasks/cephfs/test_nfs: retry mount a few times

It may take a moment for a ganesha to (re)configure itself with a new
export.  If a mount fails, retry a couple times.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1e6fd912f646c77d22bf38f28d50d31f9679fe4c)

mgr/cephadm: migrate all legacy nfs exports to new .nfs pool

Migrate all past NFS pools, whether they were created by mgr/nfs or by
the dashboard, to the new mgr/nfs .nfs pool.

Since this migrations relies on RADOS being available, we have to be a bit
careful here: we only attempt the migration from serve(), not during
module init.

After the exports are re-imported, we destroy existing ganesha daemons so
that new ones will get recreated.  This ensures the (new) daemons have
cephx keys to access the new pool.

Note that no attempt is made to clean up the old NFS pools.  This is out
of paranoia: if something goes wrong, the old NFS configuration data will
still be there.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit c183bccdc6728e8321eec615b1acd88216acabca)

mgr/nfs: adjust cephfs export caps if necessary

If we are importing an old export, we may find that the cephx user
existed but with the wrong caps.  Adjust caps in that case!

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 379a1fc4d794405edd4c34d9ac44c2c5621cfe58)

python-common: don't accept pool/ns for NFSServiceSpec

All users are now gone.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d0d95db00d462cc8101311d591bda26dc452581f)

mgr/orchestrator: drop rados_config_location ServiceDescription property

I'm not sure what this was intended to be used for...

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 75970ff80652e10bd79234b97de63cb0f39d9d90)

mgr/cephadm: move rados_config_location() out of NFSServiceSpec

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 7cbd1fbce82c567a2f8608b051dd16bbb8c5ae07)

mgr/nfs: change nfs pool to .nfs

This is a new pool that we can migrate all past NFS configuration to,
simplifying the migration process (and also allowing us to pick a
.-prefixed name).

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 82e939d89c3b6e2ce6faad9a5315c5372fef341d)

Conflicts:
src/pybind/mgr/rook/rook_cluster.py

mgr/nfs/export: accept a JSON or ganesha EXPORT config

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 6b2ccb2049484d2325bd2189410165093217a766)