client: fix incorrectly showing the .snap size for stat

We should set the 'stat->size' to the real number of snapshots for
snapdirs.

Fixes: https://tracker.ceph.com/issues/57344
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit aa918d2e383eb9da2f6f837d731cc18bbad21e44)

Merge pull request #48404 from zdover23/wip-doc-2022-10-10-pacific-releases-cleanup

pacific: doc/releases: update pacific release notes

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/releases: update pacific release notes

This commit updates the Pacific release notes, which were
not present until now.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48403 from zdover23/wip-doc-2022-10-09-backport-48401-to-pacific

pacific: doc/various: update link to CRUSH pdf

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/various: update link to CRUSH pdf

This commit updates link to the research paper that announces and
explains the CRUSH algorithm. This link was broken in the migration from
the old Ceph website to ceph.io.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 6877b592a289cb473f16daa79d5112f5b2d8006c)

Merge pull request #48392 from zdover23/wip-doc-2022-10-02-backport-44433-to-pacific

pacific: docs/start: fixes typo and empty headline in hardware recommendation …

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

docs/start: fixes typo and empty headline in hardware recommendation docs

fixes typo and empty headline in hardware recommendation docs

Signed-off-by: Sebastian Schmid <mail@sebastian-schmid.de>
(cherry picked from commit 48421403119c87d4bfc62c76de7c5598ccabb0a2)

Merge pull request #48031 from idryomov/wip-50378-pacific

pacific: test: bump DecayCounter.steady acceptable error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #48165 from kotreshhr/wip-57555-pacific

pacific: qa: ignore disk quota exceeded failure in test

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48142 from rhcs-dashboard/wip-57582-pacific

pacific: mgr/dashboard: fix Expected to find element: `cd-modal .badge but never found it

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Francesco Torchia <francesco.torchia@suse.com>

Merge pull request #48112 from NitzanMordhai/wip-57544-pacific

pacific: pybind/rados: notify callback reconnect

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #47112 from nmshelke/wip-56468-pacific

pacific: mgr/volumes: A few dependent mgr volumes PRs

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR khiremat@redhat.com
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #47721 from rhcs-dashboard/wip-55468-pacific

pacific: mgr/dashboard: grafana frontend e2e testing and update cypress

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #48381 from zdover23/wip-doc-2022-10-06-backport-48379-to-pacific

pacific: doc/install: clone-source.rst s/master/main

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/install: clone-source.rst s/master/main

s/master/main in doc/install/clone-source.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 5c17509b0a3402ca4a3ee574ed7e73949d51c72a)

Merge pull request #48371 from zdover23/wip-doc-2022-10-05-backport-44564-to-pacific

pacific: doc/mgr: name data source in "Man Install & Config"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/mgr: name data source in "Man Install & Config"

This PR specifies that the data source must be set to
be "Dashboard1" when you configure Grafana and Prometheus
manually.

This is a fixup of another PR which was created by Dr
Jake Grimmett. This is that PR:

Credit goes to Dr Jake Grimmett of Cambridge.

https://github.com/ceph/ceph/pull/44150/

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 987713da331e941c05d114f1e167c21d1e7896eb)

Merge pull request #48366 from zdover23/wip-doc-2022-10-05-backport-48362-to-pacific

pacific: doc/rados: refine English in crush-map-edits.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: refine English in crush-map-edits.rst

This commit makes several refinements to the English in
rados/operations/crush-map-edits.rst, which refinements were suggested
by Cole Mitchell and Anthony D'Atri in the discussion of PR#48085.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit a70b720032876386d7fbebfb649849fa34e9da5f)

Merge pull request #48364 from zdover23/wip-doc-2022-10-05-backport-48085-to-pacific

pacific: doc/rados: update prompts in crush-map-edits.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/rados: update prompts in crush-map-edits.rst

This PR updates the prompts in crush-map-edits.rst
to make them unselectable.

There remains no good known way to render parts of
a file if the first line of that file begins with a
hash (#). Sphinx italicizes such a first line, which
is not what I want. Two examples of this are present
in the file crush-maps-rst under the section called
"CRUSH Map Bucket Types". I set this down here for
my own records, in case it is helpful in finding a
way to make these sections render as I would prefer.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 7484b7c10a3f762e6649fb304fd0f7a5da5e418f)

Merge pull request #48048 from aaSharma14/wip-57498-pacific

pacific: mgr/dashboard: fix snapshot creation with duplicate name

Merge pull request #48358 from zdover23/wip-doc-2022-10-04-backport-48346-to-pacific-3

pacific: doc/glossary.rst: remove duplicates

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/glossary.rst: remove duplicates

This commit removes similar but distinct entries for the following:
   * CephFS
   * Ceph Client

Removal of a glossary term that is referred to in the body of the
documentation suite requires the alteration of the text string
that refers to the glossary term. Alterations of this kind have
been made to doc/architecture.rst and doc/rados/api/index.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 0692b223540143cdb4a60b54595a7069284b5481)

Merge pull request #48339 from zdover23/wip-doc-2022-10-02-backport-48333-to-pacific

pacific: doc/glossary.rst: alphabetize glossary terms

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/glossary.rst: alphabetize glossary terms

This commit (finally) alphabetizes the terms in the glossary.
This is not a grammar-correcting or usage-correcting commit.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48335 from zdover23/wip-doc-2022-10-02-backport-48332-to-pacific

pacific: doc/rados: fix prompts in erasure-code.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/rados: fix prompts in erasure-code.rst

This commit adds unselectable prompts to doc/rados/
erasure-code.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48324 from zdover23/wip-doc-2022-10-01-backport-48322-to-pacific

pacific: doc/rados: rewrite EC intro

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/rados: rewrite EC intro

This PR rewrites the front matter in the "Erasure Code"
section of the RADOS documentation. Previously, the information
in this section was syntactically confused. I have also fleshed
out the distinction between erasure coding and replication.

Co-Author: Cole Mitchell <cole.mitchell@gmail.com>
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48276 from torchiaf/wip-57681-pacific

pacific: mgr/dashboard: Add text to empty life expectancy column

Merge pull request #48106 from pdvian/wip-57313-pacific

pacific: osd, mds: fix the "heap" admin cmd printing always to error stream

mgr/dashboard: Add text to empty life expectancy column

- Add life_expectancy_enabled field to /api/osd/{svc_id}/devices
- Add new value 'n/a' for 'Life Expectancy' column

Fixes: https://tracker.ceph.com/issues/43116
Signed-off-by: Francesco Torchia <francesco.torchia@suse.com>
(cherry picked from commit 811bba7e083919cf3b21ed01ac29a107b5bd006d)

Merge pull request #48056 from adk3798/pacific-staggered-version-pacific

pacific: doc/cephadm: correct version staggered upgrade got in pacific

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #48069 from adk3798/pacific-docs-eca-mounts

pacific: doc/cephadm: document recommended syntax for mounting files with ECA

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #48109 from lxbsz/wip-57393

pacific: client: abort the client if we couldn't invalidate dentry caches

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #47868 from MrFreezeex/pacific-ceph-mixin-backports

Pacific ceph mixin backports

Merge pull request #48108 from lxbsz/wip-57395

pacific: client: stop the remount_finisher thread in the Client::unmount()

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #47998 from neesingh-rh/wip-57282-pacific

pacific: cephfs-top: adding filesystem menu option

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48020 from rzarzynski/wip-doc-dups-trimming-relnotes-pacific

pacific: PendingReleaseNotes: document online and offline trimming of PG Log's…

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #48236 from zdover23/wip-doc-2022-09-12-backport-48235-to-pacific

pacific: doc/dev: improve presentation of note (git remote)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/dev: improve presentation of note (git remote)

This commit corrects the formatting of a note that previously
did not make console output and commands as clear as this commit
makes them.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48176 from torchiaf/wip-57584-pacific

pacific: mgr/dashboard: Add details to the modal which displays the `safe-to-d…

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

PendingReleaseNotes: document online and offline trimming of PG Log's dups

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit dbfb50453c2f7fe5b0a06d846dd38693cceb63fe)

Conflicts:
    PendingReleaseNotes
- Moved the note under the new ">=16.2.11" section

Merge pull request #47764 from rzarzynski/wip-get_or_fail-debug-louder-pacific

pacific: msg: Log at higher level when Throttle::get_or_fail() fails

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>

Merge pull request #48189 from rhcs-dashboard/wip-57224-pacific

pacific: mgr/dashboard: fix rgw connect when using ssl

Reviewed-by: Pegonzal <pegonzal@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #47701 from rzarzynski/wip-dup-trimming-test2-pacific

pacific: osd, tools, kv: non-aggressive, on-line trimming of accumulated dups

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #47303 from petrutlucian94/wip-56729-pacific

pacific: msg: Fix Windows IPv6 support

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #47939 from idryomov/wip-56703-pacific

pacific: librbd/cache/pwl: narrow the scope of m_lock in write_image_cache_state()

Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #47990 from guits/wip-57381-pacific

pacific: ceph-volume: add a retry in util.disk.remove_partition

Merge pull request #48158 from zdover23/wip-doc-2022-09-18-backport-48157-to-pacific

pacific: doc/rados: update prompts in network-config-ref

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

ceph-mixin: fix CephNodeNetworkPacket alerts

Signed-off-by: Aswin Toni <aswin.toni@cern.ch>
(cherry picked from commit 351e1ac63950164ea5f08a6bfc7c14af586bb208)

ceph-mixin: fix config inheritance

Signed-off-by: Aswin Toni <aswin.toni@cern.ch>
(cherry picked from commit 35183140f60fe445de8d256fa08639b288b6e768)

ceph-mixin: Remove jsonnet building

Signed-off-by: Aswin Toni <aswin.toni@cern.ch>
(cherry picked from commit 2e0e684fc20cbf6c2e48215b431419c8573b3863)

cmake: set $PATH for tests using jsonnet tools

otherwise they would not able to find executables installed into
${CMAKE_CURRENT_BINARY_DIR}.

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
(cherry picked from commit 4a3afcf27769a512fba7e0194d2e3b6a55e06bca)

prometheus: add multicluster support to alerts

Signed-off-by: Aswin Toni <aswin.toni@cern.ch>
(cherry picked from commit 5cdc1c62c5de52a1f777f3d83fc85c3fc144db38)

monitoring/ceph-mixin: clean up prometheus_alerts.yml

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 9b6597446814ebced6ee6d963af18ce1a915e0bf)

Merge pull request #48099 from adk3798/wip-57376-pacific

pacific: mgr/dashboard: enable addition of custom Prometheus alerts

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #48101 from adk3798/wip-57398-pacific

pacific: qa/cephadm: remove fsid dir before bootstrap in test_cephadm.sh

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #48097 from adk3798/wip-57503-pacific

pacific: qa/tasks/kubeadm: set up tigera resources via kubectl create

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #48096 from adk3798/wip-57528-pacific

pacific: mgr/cephadm: reconfig iscsi daemons if trusted_ip_list changes

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

mgr/dashboard: enable addition custom Prometheus alerts

Fixes: https://tracker.ceph.com/issues/57294
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit 3551d7f8b36d883a72b85f0bd5568a33ac00e62c)

Conflicts:
doc/cephadm/services/monitoring.rst
src/pybind/mgr/cephadm/services/monitoring.py
src/pybind/mgr/cephadm/tests/test_services.py

Merge pull request #48098 from adk3798/wip-57424-pacific

pacific: cephadm: Fix disk size calculation

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #48100 from adk3798/wip-57427-pacific

pacific: mgr/cephadm: allow setting prometheus retention time

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #48102 from adk3798/wip-57379-pacific

pacific: cephadm: return nonzero exit code when applying spec fails in bootstrap

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #48103 from adk3798/wip-57384-pacific

pacific: mgr/cephadm: Adding logic to store grafana cert/key per node

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

mgr/dashboard: fix rgw connect when using ssl

connecting with the hostname instead of the ip address when using ssl

Fixes: http://tracker.ceph.com/issues/56970
Signed-off-by: Henry Hirsch <henry.hirsch@1und1.de>
(cherry picked from commit ca8f77d6a7fa361328a942e3f37c81dd9577d4de)

Merge pull request #47888 from rhcs-dashboard/wip-57357-pacific

pacific: mgr/dashboard: ensure limit 0 returns 0 images

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #47636 from rhcs-dashboard/wip-57143-pacific

pacific: mgr/dashboard: fix _rbd_image_refs caching

Reviewed-by: Nizamudeen A <nia@redhat.com>

doc/cephfs-top: adding doc for menu options

Fixes: https://tracker.ceph.com/issues/54978
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
(cherry picked from commit e9d43d783829397bed054047d16ffee9f316f4fc)

cephfs-top: adding filesystem menu

Cephfs-top now contains two options 'm' for filesystem
selection and 'q' to go back.The home screen displays
the clients belonging to a particular filesystem as a group.

Fixes: https://tracker.ceph.com/issues/54978
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
(cherry picked from commit 527737d8c3aa40eacac460d702c9359fc5545c4d)

cephfs-top: fixed flake8 errors

Fixes: https://tracker.ceph.com/issues/54978
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
(cherry picked from commit 65b03d658ce033683717786418b621335e9e6587)

Merge pull request #47978 from neesingh-rh/wip-57439-pacific

pacific: cephfs-top: display average read/write/metadata latency

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #48045 from rhcs-dashboard/wip-57493-pacific

pacific: mgr/dashboard: fix openapi-check

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #47995 from idryomov/wip-52810-pacific

pacific: librbd: retry ENOENT in V2_REFRESH_PARENT as well

Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #47866 from neesingh-rh/wip-57274-pacific

pacific: mgr/stats: missing clients in perf stats command output.

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com

Merge pull request #47769 from neesingh-rh/wip-57263-pacific

pacific: mgr/volumes: Add volume info command

Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #47647 from joscollin/wip-57155-pacific

pacific: cephfs-top: fix the rsp/wsp display

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>

mgr/dashboard: Add details to the modal which displays the `safe-to-destroy` result

- Add warnings type information in the case of the OSDs are not safe to destroy
- Add info type information in the case of the OSDs are safe to destroy

Fixes: https://tracker.ceph.com/issues/37327
Signed-off-by: Francesco Torchia <francesco.torchia@suse.com>
(cherry picked from commit 0d6100bbf99ffa8da0e099343ede050f1cca509c)

Merge pull request #47386 from s0nea/wip-56990-pacific

pacific: monitoring/ceph-mixin: OSD overview typo fix

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

qa: ignore disk quota exceeded failure in test

Quota enforcement takes some time. It is documented.
Test should ignore 'Disk Quota Exceeded' failure.

Fixes: https://tracker.ceph.com/issues/56632
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
(cherry picked from commit 41b0cb70e0c1b3d46ed94862b3065ed1d6e80097)

doc/rados: update prompts in network-config-ref

This PR adds unselectable prompts to network-config-
ref.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit e26c1d8574a27e97cbc2ef5e65d6b679b25cd710)

mgr/dashboard: fix Expected to find element: `cd-modal .badge but never found it

This looks related to https://github.com/ceph/ceph/pull/46323, but i am
not sure why it started failing recently in the teuthology.

The error is fixed by deselecting the row manually, so that it won't
accidentally get deselected when we actually want it to select it.

Fixes: https://tracker.ceph.com/issues/57207
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit dcbf179106273b85361dd6067239c4e826d1f3bb)

Merge pull request #47528 from lxbsz/wip-57083

pacific: qa/import-legacy: install python3 package for nautilus ceph

Reviewed-by: Kotresh HR khiremat@redhat.com

qa/cephfs: omit_sudo must be passed to underlying method...

so that it can have it's intended effect.

Fixes: https://tracker.ceph.com/issues/55572
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 02f0a3f136f85e472f3657a4af2b94e8af33c46b)

Conflicts:
   qa/tasks/cephfs/mount.py: timeout change wasn't backported

qa: Validate cleaning of the stale snapshot metadata

Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit aece3b9b28fccb6cad77e81fd3e4b84c333f1609)

mgr/volumes: Remove stale snapshot user metadata

This patch adds the capability to remove the stale snapshot user
metadata while loading the subvolume if it is present. It can't
be done in 'SubvolumeBase.discover' since v1 and v2 snapshot paths
are different. This is done just after the discover before returning
the specific version object.

Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 65af2d123a1f1ef9c4b370e908ece588eec19a1f)

mgr/volumes: Allow forceful snapshot removal on osd full

When the osd is full, if the snapshot has metadata set, it
can't be removed as user metadata can't be removed when osd
is full. This patch provides a way to remove the snapshot
with 'force' option while keeping the corresponding metadata
which gets removed on subvolume discover when it finds space.

Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 0687f78650dd348619b06e20c299f82f2a0c1bf5)

qa: Add subvolume clone and snapshot rm tests when osd is full

Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit a64f049614454e98920e2abcb685ca61fa49a148)

mgr/volumes: Better handle config file on osd full scenario

The 'metadata_mgr.flush()' used to truncate the config file
before flushing the new config data. This could lead to an
empty config file when there is no space to write new config
data. This patch handles this scenario by writing it to
temporary file and rename it to config file. This would
retain the config file without truncating it.

Also, there are bunch of places which wasn't handling
'MetadataMgrException' because of this. Fixed those.

Fixes: https://tracker.ceph.com/issues/55976
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit ec57215d508e6cb5b3a4d84fd6a3a5b0c9b96c71)

mgr/volumes: drop pre-python 3.2 version checks

Based on other conversations we believe that there is no need to support
python versions lower than Python 3.6 for pacific and later.  This means
it is safe to drop the remaining version checks for python
3.2.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 47162e4996a511d399809516a4890d72e6f84a44)

mgr/volumes: a lock to guard against races reading/writing config

Fixes: https://tracker.ceph.com/issues/55583
Use a python threading lock to avoid race conditions where the
config file is being both read and written to at the same time.

Before this change, the content of the config file being parsed could be
'corrupted' by the MetadataManager racing with itself.  Along with the
previous two patches, additional logging was added to the mgr code to
produce the simplified version of the mgr log below:

```
[volumes INFO volumes.fs.operations.versions.metadata_manager] READ: b'[GLOBAL]\nversion = 2\ntype = clone\npath = /volumes/Park/babydino2/c9f773af-5221-49c6-846c-d65c0920ae3f\nstate = pending\n\n[source]\nvolume = cephfs\ngroup = Park\nsubvolume = Jurrasic\nsnapshot = dinodna0\n\n'
[volumes INFO volumes.fs.operations.versions.metadata_manager] READ: b''
[volumes INFO volumes.fs.operations.versions.metadata_manager] READ: b'[GLOBAL]\nversion = 2\ntype = clone\npath = /volumes/Park/babydino2/c9f773af-5221-49c6-846c-d65c0920ae3f\nstate = pending\n\n[source]\nvolume = cephfs\ngroup = Park\nsubvolume = Jurrasic\nsnapshot = dinodna0\n\n'
[volumes INFO volumes.fs.operations.versions.metadata_manager] wrote 203 bytes to config b'/volumes/Park/babydino2/.meta'
[volumes INFO volumes.fs.operations.versions.metadata_manager] READ: b'a0\n\n'
[volumes INFO volumes.fs.operations.versions.metadata_manager] READ: b''
[volumes ERROR volumes.module] Failed _cmd_fs_clone_cancel(clone_name:babydino2, format:json, group_name:Park, prefix:fs clone cancel, vol_name:cephfs) < "":
Traceback (most recent call last):
  ...
  File "/usr/lib64/python3.6/configparser.py", line 1111, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: b'/volumes/Park/babydino2/.meta'
[line 13]: 'a0\n'
```

Looking at the above you can see that the log indicates a write to the
config file (of 203 bytes). This happens before the file has finished
reading and thus instead of getting an empty string indicating EOF, it
gets that last four bytes of the new content of the file. The lock
prevents the MetadataManager from both reading and writing the config
file at the same time.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 2fe83d72d94d1e0939d390554a833cd93b4dd965)

mgr/volumes: write volume metadata with shim class

Add a class that works a bit like a python file object so that we
can simplify the flush function. Providing a file-like object to
the ConfigParser's write function avoids unnecessary copies to
a StringIO object and makes the code easier to read.

With no more uses of StringIO, the StringIO imports are removed.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d06d8a1f5255b5da9106cd70acc0de99f89d2b96)

mgr/volumes: read volume metadata file using read_string

The read_string method, available in Python 3.2 (we assume Python 3.6 as
our current minimum python versino), supports parsing a provided string
for ini-style configuration parameters. Refactoring the reading of the
config file from cephfs into a simple iterator function and then
providing it to the ConfigParser as a single string, allows us to avoid
using StringIO and  simplifies the refresh function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit dc1b4aac90beb82fd994ad7fa8ccf3b72b9936ac)

qa: display in-progress clones for a snapshot

If any clone is in pending or in-progress state then
show these clones in 'fs subvolume snapshot info'
command output.

Fixes: https://tracker.ceph.com/issues/55041
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
(cherry picked from commit f4c6bdb3c6418efbf261bdce6e7f1b5753a61d7c)

docs: display in-progress clones for a snapshot

If any clone is in pending or in-progress state then
show these clones in 'fs subvolume snapshot info'
command output. This field only exists if clones are
in pending or in progress state.

Fixes: https://tracker.ceph.com/issues/55041
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
(cherry picked from commit a8b819da71804868d83bd9775c41ede39b1b65a7)

mgr/volumes: display in-progress clones for a snapshot

If any clone is in pending or in-progress state then
show these clones in 'fs subvolume snapshot info'
command output.

Fixes: https://tracker.ceph.com/issues/55041
Signed-off-by: Nikhilkumar Shelke <nshelke@redhat.com>
(cherry picked from commit 099efb424977f86597826e3f56734b3deddfd0dc)

doc: include read, write, metadata average latencies in doc/man.

Also, the sample cephfs-top image in the doc is outdated. Update that!

Fixes: http://tracker.ceph.com/issues/48619
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit aa04f3faedb6edcb0897e802a8390904deb6f936)

cephfs-top: display latency in milliseconds

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit bf261f2a07111361ec8db36a7a4b13b54ff5d891)

cephfs-top: switch to displaying average latencies and stdev

Do away with cumulative latencies -- those are not much useful.
However, these types need to be maintained since `perf stats`
command (via mgr/stats plugin) includes them. So, maintain a
legacy metrics list which is ignored when choosing metrics to
display.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 60f33a8ca3055ec5ae5c8d67fd03f571bcec8892)