doc/cephfs/add-remove-mds: added cephadm note, refined "Adding an MDS"

Description: 1) Add a note about using cephadm for setting up the
                cluster and mds(s), also mention the use of ceph
                orchestrator if one needs to setup mds(s) manually.
     2) Changed the term `data point` to `directory` in
                point 1 under "Adding an MDS" section for better
                clarity.

Fixes: https://tracker.ceph.com/issues/54551
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 9e8e1a250e1192cdb1b86650596543d42a2f0401)

Merge pull request #41528 from rhcs-dashboard/wip-50968-pacific

pacific: mgr/dashboard: show RGW tenant user id correctly in 'NFS create export' form

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge PR #41053 into pacific

* refs/pull/41053/head:
cephfs-top: set the cursor to be invisible
cephfs-top: self-adapt the display according the window size
cephfs-top: use the default window object from curses.wrapper()
cephfs-top: improve the output

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41389 into pacific

* refs/pull/41389/head:
qa/tasks/nfs: add test to check if cmds fail on not passing required arguments
mgr/nfs: fix flake8 missing whitespace around parameter equals error
mgr/nfs: annotate _cmd_nfs_* methods return value
doc/cephfs/nfs: add section about ganesha logs
doc/cephfs/nfs: Replace volume/nfs with nfs
doc/cephfs/nfs: add note about export management with volume/nfs interface only
spec: add nfs to spec file
mgr/nfs: Don't enable nfs module by default
mgr/nfs: check for invalid chars in cluster id
mgr/nfs: Use CLICommand wrapper
mgr/nfs: reorg nfs files
mgr/nfs: Check if transport or protocol are list instance
mgr/nfs: reorg cluster class and common helper methods
mgr/nfs: move common export helper methods to ExportMgr class
mgr/nfs: move validate methods into new ValidateExport class
mgr/nfs: add custom exception module
mgr/nfs: create new module for export utils
mgr/nfs: rename fs dir to export
mgr/volumes/nfs: Move nfs code out of volumes plugin

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40634 into pacific

* refs/pull/40634/head:
qa: check mounts attribute in ctx

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #41169 into pacific

* refs/pull/41169/head:
pacific: qa: drop the distro~HEAD directory from the fs suite.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #41629 from ideepika/wip-pr-41514

pacific: qa/upgrade: conditionally disable update_features tests

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

qa/upgrade: conditionally disable update_features tests

with the recent support for async rbd operations from pacific+ when an
    older client(non async support) goes on upgrade, and simultaneously
    interacts with a newer client which expects the requests to be async,
    experiences hang; considering the return code for request completion to
    be acknowledgement for async request, which then keeps waiting for
    another acknowledgement of request completion.

    this if happens should be a rare only when lockowner is an old client
    and should be deferred if compatibility issues arises.

see also: 541230475d3b25ab18c4eb9bc5011060462594a6(octopus)

Signed-off-by: Deepika <dupadhya@redhat.com>
(cherry picked from commit 9c0b239d70cdeb0c5705fa89fa5aac111152671b)

Merge pull request #41428 from rhcs-dashboard/wip-50774-pacific

pacific: mgr/dashboard: ingress service creation follow-up

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41427 from rhcs-dashboard/wip-50882-pacific

pacific: mgr/dashboard: fix ESOCKETTIMEDOUT E2E failure

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #41505 from rhcs-dashboard/wip-50959-pacific

pacific: mgr/dashboard: drop container image name and id from services list

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41507 from rhcs-dashboard/wip-50962-pacific

pacific: mgr/dashboard: fix API docs link

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41440 from rhcs-dashboard/wip-50864-pacific

pacific: mgr/dashboard: Fix for query params resetting on change-password

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

mgr/dashboard: show RGW tenant user id correctly in 'NFS create export' form.

Fixes: https://tracker.ceph.com/issues/50909
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 440d6d643c172252790791ec38ce27364ccc183e)

mgr/dashboard: fix API docs link

Fixes: https://tracker.ceph.com/issues/50890
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit e79c9d3f51970fbe275087e92e468205f70e6f12)

Merge pull request #41494 from aaSharma14/wip-50902-pacific

pacific: mgr/dashboard: provide the service events when showing a service in the UI

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: drop container image name and id from services list

Fixes: https://tracker.ceph.com/issues/50889
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Dropping container image name and id columns from services list.

(cherry picked from commit ee98d73a231ce99e9a5a74ed602724c2111ee93d)

mgr/dashboard: provide the service events when showing a service in the UI

When service deployment failures occur, events are generated and associated with the service. This PR intends to add these events as Daemon Logs and Service Logs to the UI for troubleshooting and doing diagnostics for the same.

Fixes: https://tracker.ceph.com/issues/49262
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit 00ce7c90ef19f02beea9431ab360d23744efde2d)

Merge PR #41463 into pacific

* refs/pull/41463/head:
cephadm: write config files as utf-8
cephadm: reduce `noqa` usage
cephadm: apply autopep8 changes
cephadm: introduce autopep8
cephadm: remove `verify-prereqs` subcommand
cephadm: add HAProxy sysctl settings
cephadm: move keepalived sysctl settings
cephadm: apply osd sysctl settings
mgr/cephadm: warn users if they try to pass a daemon to 'orch rm'
qa/suites/orch/rook: disable centos for now
qa/suites/orch/rook/smoke: initial smoke suite
qa/tasks/rook: ROOK_HOSTPATH_REQUIRES_PRIVILEGED=true on centos
qa/tasks/rook: simplify shutdown
qa/tasks/rook: archive logs
qa/tasks/rook: more orderly cluster teardown
qa/tasks/rook: deploy ceph via rook on top of kubernetes
qa/tasks/kubeadm: install kubernetes with kubeadm
qa/suites: move rados/cephadm -> orch/cephadm; symlink
qa/tasks/cephadm: add whitespace between functions
qa/tasks/cephadm: clean up ctx.manager setup
mgr/cephadm: pin haproxy image to 2.3.z
mgr/cephadm: fix issue with missing prometheus alerts
mgr/cephadm: check version in upgrade check
mgr/cephadm: Fix OSD replacement in hosts with FQDN host name
mgr/cephadm: fix prometheus jinja template
doc/cephadm: rewrite "disabling monitoring"
mgr/cephadm: induce retune of osd memory on osd creation
qa/tasks/cephadm.conf: autotune osd memory by default
mgr/cephadm: do not autotune when _no_autotune_memory label is present
mgr/cephadm: autotune osd memory
common/options: add osd_memory_target_autotune
mgr/cephadm: report memory usage, request (limit) in 'orch ps'
doc/cephadm/host-management: document _admin group
mgr/orchestrator: fix help formatting
mgr/cephadm/schedule: fix filter message
mgr/cephadm: Use `cephfs-mirror` caps profile

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #41281 from idryomov/wip-rbd-pwl-status-pacific

pacific: rbd: don't attempt to interpret image cache state json

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

Merge pull request #41244 from idryomov/wip-rbd-pwl-reopen-pacific

pacific: librbd/cache/pwl: fix parsing of cache_type in create_image_cache_state()

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

Merge pull request #41195 from idryomov/wip-rbd-qemu-iotests-focal-pacific

pacific: qa/workunits/rbd: use bionic version of qemu-iotests for focal

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

cephadm: write config files as utf-8

If we don't specify the encoding, python may use something like 'latin-1',
which cannot handle unicode characters.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit fa2cf2987cccea9f152c04862fc684e146c4692e)

cephadm: reduce `noqa` usage

- E722 do not use bare 'except'
- W504 line break occurred after a binary operator

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit fa75096e68110e8673d3c1cfede92c50af97a676)

cephadm: apply autopep8 changes

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 6eaf23231d6048e6e583eff850395666b370c631)

cephadm: introduce autopep8

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 0efb4b546c6c24d8a5d109a31d859c6ada77556e)

cephadm: remove `verify-prereqs` subcommand

sysctl settings are now managed per-host, rather than per-daemon

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit bd308816746431331a047ab5f39d47272125685a)

cephadm: add HAProxy sysctl settings

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 465e7ef243f85db27642b2cd626841e9e4be848e)

cephadm: move keepalived sysctl settings

from container pre-start to sysctl file

Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 04127a0d2226c0499af53503a936134aa744a2b3)

cephadm: apply osd sysctl settings

These were added to ceph-salt (ceph/ceph-salt@800dbb2)
.. but let's make them available to all consumers of cephadm!

Fixes: https://tracker.ceph.com/issues/47873
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 141d0cc5d3a0aedecd1f5755015581d8b63030ca)

mgr/cephadm: warn users if they try to pass a daemon to 'orch rm'

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit b07efcb4192ad5988baf3bb92df42cea877a8351)

qa/suites/orch/rook: disable centos for now

We are hitting an selinux denial:

type=AVC msg=audit(1621353663.595:6016): avc: denied { write } for pid=28812 comm="cephcsi" name="pids.max" dev="cgroup" ino=2933 scontext=system_u:system_r:container_t:s0:c43,c177 tcontext=system_u:object_r:cgroup_t:s0 tclass=file permissive=1

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ad8096e0a0493795a57c44493581feb98b014cdd)

qa/suites/orch/rook/smoke: initial smoke suite

- os: 18.04, 20.04, centos, centos+kubic
- nodes: 1 or 3
- rook: 1.6.2 or master
- k8s: 1.21 always (for now)
- net: calico always (flannel + centos is wonky)
- workload: none, rados bench

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ab33ca365567eeed7881e034bbc78e7cd66432d4)

qa/tasks/rook: ROOK_HOSTPATH_REQUIRES_PRIVILEGED=true on centos

This fixes OSD creation on centos.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f765677b8138ab5b57cd4d4113f402d89c3d9dfd)

qa/tasks/rook: simplify shutdown

For some reason deleting common.yaml sometimes fails.  Not really
sure why, but since we will tear down kubernetes anyway this
cleanup isn't really needed.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 929d6ad8ca7f319a4ebcefde0202ca5f993d626b)

qa/tasks/rook: archive logs

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e5148b3f59022068498334d056756de03c350fef)

qa/tasks/rook: more orderly cluster teardown

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 539cd362dde182bc78f750b62750d3fac82e8748)

qa/tasks/rook: deploy ceph via rook on top of kubernetes

This assumes that k8s is installed and kubectl works.

The ceph container to use is selected the same way the cephadm
task does it.

All scratch devices are consumed as OSDs.

A ceph.conf and client.admin keyring are deployed on all test
nodes, so normal tasks should work (if/when packages are installed).

Fixes: https://tracker.ceph.com/issues/47507
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4574ed70f42002028e2d30e8aa04fbff95daf728)

qa/tasks/kubeadm: install kubernetes with kubeadm

- install k8s with kubeadm
- initial support for flannel only
- remove taint from bootstrap/master node
- create PVs for all scratch_devs + a 'scratch' SC
- kubeadm.kubectl task

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ebf841a82f3939441fa118fa689c8e290ef31cfe)

qa/suites: move rados/cephadm -> orch/cephadm; symlink

Move cephadm under orch/ top-level suite.  Symlink so that we
still include it in a rados run.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 5b25f8a2e51ccdea1f5409a4b8e55f8e625c8193)

# Conflicts:
# qa/suites/rados/cephadm
 - included thrash-old-clients, which is different in master

qa/tasks/cephadm: add whitespace between functions

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b6d1536a67428b4ffa0df3791e9eff667736c765)

qa/tasks/cephadm: clean up ctx.manager setup

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 267cefc0401d078e247d387d93a60840f6b53978)

mgr/cephadm: pin haproxy image to 2.3.z

The 2.4 version fails for reasons that aren't immediate clear:

May 17 15:28:42 dael conmon[3007013]: [NOTICE]   (8) : haproxy version is 2.4.0-6cbbecf
May 17 15:28:42 dael conmon[3007013]: [NOTICE]   (8) : path to executable is /usr/local/sbin/haproxy
May 17 15:28:42 dael conmon[3007013]: [ALERT]    (8) : Cannot open configuration file/directory /var/lib/haproxy/haproxy.cfg : Permission denied

Even with mode 777 on those files, there is an error about the chroot to
/var/lib/haproxy.

For now, just stick with a working version.

Fixes: https://tracker.ceph.com/issues/50830
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit b94c8dedd8aa77aeb0b4957e88197ac89370ffb4)

mgr/cephadm: fix issue with missing prometheus alerts

Files passed as configuration to the cephadm binary had not been created
and mapped to the container, if those files weren't included in the
required files section inside cephadm. This prevented optional file
includes in the configuration.

The configuration file for the Prometheus default alerts is not
mandatory and hence wasn't included in the required files section, still it
needs to be added to the container by cephadm.

This change enables optional files to be included in the configuration
for monitoring components, so that those files are created and mapped
within the container.

Note that a `required_files` variable has been removed at one position
in these changes, though it wasn't used to ensure that required files
were included in the configuration at that point anyway. The test which
ensures that all required files are passed is somewhere else.

Fixes: https://tracker.ceph.com/issues/49856
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 38f9846d13ae2c5f12bde3397d2826b1cf43759b)

mgr/cephadm: check version in upgrade check

Fixes: https://tracker.ceph.com/issues/50784
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit e1979fb5bda09d7028d53a64b36cca5206e9e3a4)

mgr/cephadm: Fix OSD replacement in hosts with FQDN host name

Resolves: https://tracker.ceph.com/issues/50805

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
(cherry picked from commit b1feb876d3bbe8798be062c61d523dde09d281d7)

mgr/cephadm: fix prometheus jinja template

The prometheus yaml file shouldn't use "tab" character but only spaces.

Fixes: https://tracker.ceph.com/issues/50717
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 2d8995e1ded22e680f778c4ef73ddd86592df21c)

doc/cephadm: rewrite "disabling monitoring"

* Simplified the text.
* Correctly formatted the prompts.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit b67b6112ac2b2b21360fca2ffd7ed0136a5d4929)

mgr/cephadm: induce retune of osd memory on osd creation

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 821a57ceabc70e8f4dce11fd04ec20ffead3789b)

qa/tasks/cephadm.conf: autotune osd memory by default

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit e5c88e2c49266447334aa42b90a3f8004ea31eab)

mgr/cephadm: do not autotune when _no_autotune_memory label is present

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1e4d20a592aa6ca111201b0a3edc4dd619885f93)

mgr/cephadm: autotune osd memory

- set osd_memory_target_autotune=true to enable
- tuning is periodic (check every 10m by default)
- tuned values are reflected by osd_memory_target config options scoped
  to the host
- only make a change if it appears that we will affect at least 1 of the
  relevant OSDs
- attempt to clean out conflicting options.  (This is imperfect, since any
  manner of weirdly-scoped config options could be responsible; we only
  attempt to clean out one scoped directly to the osd name.)

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 85ea078787d864bd8ae199f9517a9d2dbaf33c7c)

common/options: add osd_memory_target_autotune

This does not cherry-pick 6029a0e256cb10a8c35b3bbdc9a22ad679e6fe03
because options.cc is autogenerated in master branch.

Signed-off-by: Sage Weil <sage@newdream.net>

mgr/cephadm: report memory usage, request (limit) in 'orch ps'

Fill in from {osd,mon}_memory_target if no container limit is set.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 4a8182a60658bfbd7034d8eb03e54dc1b154b165)

doc/cephadm/host-management: document _admin group

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit eb3c38577a4424080e4f297f79374d7be5aed9fa)

mgr/orchestrator: fix help formatting

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3ad3f06d2cdb5b700f8642814c32a3680bbd0dd5)

mgr/cephadm/schedule: fix filter message

This is now only used when scheduling mons.  (Units now enable the kernel
features needed instead of checking for them during placement.)  Move the
message to the filter itself.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d5aba1ec65a0ca49c6dbb1bb8975e5209cb020b3)

mgr/cephadm: Use `cephfs-mirror` caps profile

The profile is required to establish a peer
connection, but was added after this got merged

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit f95ded3f902d383c166dfde9bd178f5bc9563036)

Merge pull request #41122 from idryomov/wip-rbd-nuke-source-spec-file-pacific

pacific: rbd: --source-spec-file should be --source-spec-path

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Sunny Kumar <sunkumar@redhat.com>

mgr/dashboard: Fix for query params resetting on change-password

Fixes: https://tracker.ceph.com/issues/50857
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 09307ab3bdd9ce27f746c6c1fa8018f807388182)

qa/tasks/nfs: add test to check if cmds fail on not passing required arguments

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 94644d1e5f68a0dc02686cbe826d34751b005f93)

mgr/nfs: fix flake8 missing whitespace around parameter equals error

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 57dba10325b23e52524b2a2c82381b1b5a020964)

mgr/nfs: annotate _cmd_nfs_* methods return value

Fixes: 7c57d1634
Fixes: https://tracker.ceph.com/issues/50783
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 1ec90c804319b32d86212f7402cac93fb14cb1d2)

doc/cephfs/nfs: add section about ganesha logs

Fixes: https://tracker.ceph.com/issues/50008
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 6ff3c1da87d8a4fe3750065a554e348e1882f240)

doc/cephfs/nfs: Replace volume/nfs with nfs

Recently, nfs related code was moved out of volumes plugin[1]. So using the
name volume/nfs for the interface is not appropriate.

[1] https://github.com/ceph/ceph/pull/40526

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit ff11cdb4bc7cfc0a51ca047e6b7c8c686075b2ec)

doc/cephfs/nfs: add note about export management with volume/nfs interface only

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 923e689e0828ac5186123a2ffc57db01465cbdd0)

spec: add nfs to spec file

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 8ab49501719f4d3d997a3b4e0f500ac985e9558f)

mgr/nfs: Don't enable nfs module by default

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 2a99da961adac780ac45dd9e1073d5e695c0c1f5)

mgr/nfs: check for invalid chars in cluster id

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 07e7263936191f6be07d7e03be2de16aa8bffd58)

mgr/nfs: Use CLICommand wrapper

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 7c57d1634049484b3037d297e0955b14dcac9654)

mgr/nfs: reorg nfs files

This patch does the following:
- removed export dir and moved its files to nfs dir
- renamed nfs.py to export.py

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 6045c4690b75b8be8969c67d5a5d5caf6196abea)

mgr/nfs: Check if transport or protocol are list instance

This is to fix "int object is not iterable” error.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit ef4d7d7f2c6faa4d755d2da9d5f0fd88c9f81c09)

mgr/nfs: reorg cluster class and common helper methods

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit f8adac3e709f85707910d09acb8667fd861a2fa9)

mgr/nfs: move common export helper methods to ExportMgr class

This reorganization is to support RGW exports in future PR.

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit c6288a69a02434c5419d5b92c8e79d8206fad350)

mgr/nfs: move validate methods into new ValidateExport class

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit be22f0592e3123bf121b572cbb22fc8d5d329d4f)

mgr/nfs: add custom exception module

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 3a08139452f6ebeff9eda6d2743ac27741d07380)

mgr/nfs: create new module for export utils

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit d064306dd3afcbab620c5489654359bb2f911c3d)

mgr/nfs: rename fs dir to export

Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 23c63ccb7b5cdff834bed0ab7302ed3eb51ea1c8)

mgr/volumes/nfs: Move nfs code out of volumes plugin

This PR is to support integration of dashboard with volumes/nfs plugin.

Fixes: https://tracker.ceph.com/issues/50080
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit af9336316f978a9f67faafad2eb3e608ed2616b1)

mgr/dashboard: ingress service creation follow-up

Fixes: https://tracker.ceph.com/issues/50568
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Pre-populating the service id(read-only) with the value same as backend service.

(cherry picked from commit 549daa7579c7f3e317e2f7de31010fa2d577416d)

mgr/dashboard: fix ESOCKETTIMEDOUT E2E failure

Fixes: https://tracker.ceph.com/issues/49828
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit bf4894f4953e9ec0e8c01ee95726c2bd92972982)

Merge pull request #41005 from mgfritch/pacific-backport-40411

pacific: mgr/volumes/nfs: drop type param during cluster create

Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #40996 from batrick/i50258

pacific: qa: convert some legacy Filesystem.rados calls

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #40855 from batrick/i50254

pacific: mds: standby-replay only trims cache when it reaches the end of the replay log

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #40685 from singuliere/wip-50183-pacific

pacific: client: fix the opened inodes counter increasing

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #41402 from rhcs-dashboard/wip-50881-pacific

pacific: mgr/dashboard: update frontend deps due to security vulnerabilities

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: update frontend deps due to security vulnerabilities.

Fixes: https://tracker.ceph.com/issues/50800
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 7ee75880c1c600a25f05f98c280be87758579901)

Conflicts:
  - src/pybind/mgr/dashboard/frontend/package-lock.json
    Resolved conflicts.

mgr/dashboard: get SASS vars values in TS through CssHelper.

Now SASS vendor variables' values can be retrieved by var name via CssHelper.

We were relying on the angular webpack config that has changed in the latest v10.x and v11.
See: https://github.com/angular/angular-cli/issues/19622#issuecomment-744395101

Fixes: https://tracker.ceph.com/issues/50800
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit 7310e15a9bb704c3b214b9b5373807b1167df009)

Merge pull request #41324 from rhcs-dashboard/wip-50772-pacific

pacific: mgr/dashboard: fix OSDs Host details/overview grafana graphs

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: wornet-mwo <NOT@FOUND>

Merge pull request #41200 from votdev/wip-50662-pacific

pacific: mgr/dashboard: disable NFSv3 support in dashboard

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #41298 from rhcs-dashboard/wip-50543-pacific

pacific: mgr/dashboard: fix HAProxy (now called ingress)

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41296 from rhcs-dashboard/wip-50769-pacific

pacific:  mgr/dashboard: RGW buckets async validator performance enhancement and name constraints

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #41136 from ideepika/wip-pacific-40572

pacific: qa/config/rados: add dispatch delay testing params

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/dashboard: add Services e2e tests

Fixes: https://tracker.ceph.com/issues/50567
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Introducing e2e tests for service creation for Ingress and RGW service types.

qa: check mounts attribute in ctx

check mounts attribute in ctx before unmounting in bark()

Fixes: https://tracker.ceph.com/issues/49898
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit b4b932fe1cf947a487af2cc307be9b0289eca2dc)

mgr/dashboard: disable NFSv3 support in dashboard

Fixes: https://tracker.ceph.com/issues/49718
Related to: https://github.com/ceph/ceph/pull/40154

Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 864a362389e988bc9aab23f66da608b4ceab74c0)

Merge remote-tracking branch 'origin/pacific-saved' into pacific

16.2.4

mgr/dashboard: fix cookie injection issue

Fixes: CVE-2021-3509
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit b39922818bc57cde1b016e9ad41908b18063b93b)

systemd: remove `ProtectClock=true` for `ceph-osd@.service`

Ceph 16.2.0 Pacific by https://github.com/ceph/ceph/commit/9a84d5a introduce following new systemd restriction:

    ProtectClock=true
    ProtectHostname=true
    ProtectKernelLogs=true
    RestrictSUIDSGID=true

BTW, `ceph-osd@.service` failed with `ProtectClock=true` unexpectly, also see:

  - <https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/TNBGGNN6STGDKARAQTQCIPTU4KLIVJQV/>
  - <https://serverfault.com/questions/1059317/bluestore-var-lib-ceph-osd-ceph-2-block-read-bdev-label-failed-to-open-var-l>

This PR intruduce:

  - Remove `ProtectClock=true` for our systemd service templates

Fixes: https://tracker.ceph.com/issues/50347
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
(cherry picked from commit 85bc551b179d940a50cbdfd0c20848e3187c70a6)