mgr/cephadm: move _check_host to serve.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d6914694003cc4de3c01d8ec7d4b05c14e8bc67f)

mgr/cephadm: move _refresh_hosts_and_daemons to serve.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit d73dc54e4ee9150e750afe779546d0242ad176c2)

mgr/cephadm: move _serve_sleep to serve.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 776fd78f67b1a01c641de4c93f9a1c67e870295d)

mgr/cephadm: move serve() to serve.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 2228d59229bff1aef4ef7f8e18b9d5f6c395a51d)

mgr/cephadm: add serve.py

This module contains functions that are executed in the
serve() thread.
Meaning they don't block the CLI

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit acd6854bbc8985f8b2f6f2bfaa84f7e4c31d6660)

cephadm: Fix error setting 'mgr/cephadm/container_init' config

Fixes: https://tracker.ceph.com/issues/47501
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit cce937a599ebaf37fc0b35de19ed44afa80d1f06)

docs: <orch osd spec> is no longer present

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit b38bc14525460e0bec7829774d2385b62d6c8742)

mgr/cephadm: do not escape HTML for custom templates

Fixes: https://tracker.ceph.com/issues/47648
Signed-off-by: Patrick Seidensal <pseidensal@suse.com>
(cherry picked from commit ae9cab58e24389ded885561a763e9826cfa5c645)

mgr/cephadm: continue trying all hosts for service deployment

One failure should not stop the attempt to start all instances.

Fixes: https://tracker.ceph.com/issues/46665
Signed-off-by: Dan Mick <dmick@redhat.com>
(cherry picked from commit dd7bd4807a1668e1fb91581c69bb549ae62611db)

mgr/cephadm: Make type annotnations for cephadm.services mandatory

* Found a wrong default for `OSDService.create_single_host`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit fa034736b1871e637e65392921626aab049c032c)

Conflicts:
src/mypy.ini
src/pybind/mgr/cephadm/services/nfs.py

mgr/cephadm: get rbd-mirror daemon-id when checking for strays

Currently, list_servers() gets the rbd-mirror service-id instead
of the daemon-id so the daemon is marked as stray. This PR uses
that service-id to find the daemon-id and uses that to check if
the daemon is stray.

Fixes: https://tracker.ceph.com/issues/47639
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 83d878f3d94a31c03ce94788236bd584c878a89c)

mgr/cephadm: upgrade_state can be literally "null"

Fixes: https://tracker.ceph.com/issues/47580
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 155fd9800211cf4c90cecae59b5f332ecdd665e8)

Merge pull request #38045 from dsavineau/wip-48184-octopus

octopus: ceph-volume: fix lvm batch auto with full SSDs

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

ceph-volume: add a unit tests to lvm batch

This commit adds unit tests in order to cover `_sort_rotational_disks()`
call when deploying with full hdd/ssd or mixed hdd/sdd scenarios.

Fixes: https://tracker.ceph.com/issues/48150
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
Co-authored-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 13514a24cfdc32d67cfbc1201aa427168a926978)

ceph-volume: fix lvm batch auto with full SSDs

The ceph-volume lvm batch --auto introduced by [1] breaks the backward
compatibility when using non rotational devices only (SSD and/or NVMe).
Those devices are reaffected as bluestore db or filestore journal
devices while we want them as data devices.

Fixes: https://tracker.ceph.com/issues/48106
[1] https://github.com/ceph/ceph/pull/34740

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 2a854ca373fadef099a1d037930eb241e757b2c3)

Merge pull request #37553 from Vicente-Cheng/wip-47747-octopus

octopus: mon: set session_timeout when adding to session_map

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #37885 from bk201/wip-47944-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Varsha Rao <varao@redhat.com>

Merge pull request #37962 from votdev/custom_container_docs

octopus: doc/mgr/orchestrator: Add hints related to custom containers to the docs

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #37857 from smithfarm/wip-47940-octopus

octopus: mon/MDSMonitor: divide mds identifier and mds real name with dot

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37856 from smithfarm/wip-47936-octopus

octopus: mds: account for closing sessions in hit_session

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37855 from smithfarm/wip-47891-octopus

octopus: mgr/volumes/nfs: Fix wrong error message for pseudo path

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37854 from smithfarm/wip-46959-octopus

octopus: cephfs-journal-tool: fix incorrect read_offset when finding missing objects

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37841 from smithfarm/wip-47991-octopus

octopus: qa/cephfs: add session_timeout option support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37839 from smithfarm/wip-47989-octopus

octopus: cephfs: client: fix inode ll_ref reference count leak

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #37837 from smithfarm/wip-47954-octopus

octopus: vstart.sh: fix fs set max_mds bug

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37724 from rishabh-d-dave/wip-46610-octopus

octopus: pybind/cephfs: add special values for not reading conffile

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37671 from Vicente-Cheng/wip-47824-octopus

octopus: mgr/volumes: Make number of cloner threads configurable

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37620 from rhcs-dashboard/wip-47811-octopus

octopus: mgr/dashboard: get rgw daemon zonegroup name from mgr

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #37858 from smithfarm/wip-47958-octopus

octopus: mon/MDSMonitor do not ignore mds's down:dne request

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37853 from smithfarm/wip-47826-octopus

octopus: osd/osd-rep-recov-eio.sh: TEST_rados_repair_warning:  return 1

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #37835 from smithfarm/wip-47934-octopus

octopus: tools/rados: flush formatter periodically during json output of "rados ls"

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #37819 from smithfarm/wip-47994-octopus

octopus: test/store_test: use 'threadsafe' style for death tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #37817 from smithfarm/wip-47987-octopus

octopus: mon/MonMap: fix unconditional failure for init_with_hosts

Reviewed-by: Wido den Hollander <wido@widodh.nl>

Merge pull request #37784 from bk201/wip-47657-octopus

octopus: mgr/dashboard: display devices' health information within a tabset

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

doc/mgr/orchestrator: Add hints related to custom containers to the docs

Fixes: https://tracker.ceph.com/issues/48113
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 1927809b0b58243dbe84756b9cec7c29bd0a7494)

Merge pull request #37239 from rhcs-dashboard/read_only_backport

octopus: mgr/dashboard: Disabling the form inputs for the read_only modals

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #37530 from batrick/i47734-octopus

octopus: osdc: add timeout configs for mons/osds

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37256 from batrick/i47249

octopus: mon: deleting a CephFS and its pools causes MONs to crash

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37852 from smithfarm/wip-47889-octopus

octopus: rbd: librbd: ignore -ENOENT error when disabling object-map

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #37851 from smithfarm/wip-47888-octopus

octopus: rbd: librbd: update AioCompletion return value before evaluating pending count

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #37850 from smithfarm/wip-47886-octopus

octopus: rbd: journal: possible race condition between flush and append callback

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #37849 from smithfarm/wip-48003-octopus

octopus: rgw: fix: S3 API KeyCount incorrect return.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37847 from smithfarm/wip-47956-octopus

octopus: rgw/gc: fix for incrementing the perf counter 'gc_retire_object'

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37846 from smithfarm/wip-47955-octopus

octopus: rgw/gc: fixing the condition when marker for a queue is

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37845 from smithfarm/wip-47896-octopus

octopus: rgw: use yum rather than dnf for teuthology testing of rgw-orphan-list

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #37812 from smithfarm/wip-48007-octopus

octopus: rbd: rbd-nbd: don't ignore namespace when unmapping by image spec

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #37809 from smithfarm/wip-47962-octopus

octopus: rgw: Add request timeout to beast

Reviewed-by: Friedmann <ofriedma@redhat.com>

Merge pull request #37807 from smithfarm/wip-47960-octopus

octopus: rgw: fix expiration header returned even if there is only one tag in the object the same as the rule

Reviewed-by: Friedmann <ofriedma@redhat.com>

Merge pull request #37803 from smithfarm/wip-47819-octopus

octopus: rgw: radosgw-admin should paginate internally when listing bucket

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37800 from smithfarm/wip-47817-octopus

octopus: rgw: allow rgw-orphan-list to note when rados objects are in namespace

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #37779 from smithfarm/wip-47037-octopus

octopus: rgw: fix user stats iterative increment

Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/cephadm: do not configure Dashboard Ganesha settings

The Dashboard can get cluster information from the Orchestrator.
For settings that are set by previous revisions, the Dashboard will
check them and ask user to remove them.

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit 99e421065748c03da1fc468b2a09bf22f7bc31b0)

Conflicts:
      src/pybind/mgr/cephadm/services/nfs.py

doc/dashboard: add information for Orchestrator NFS-Ganesha clusters

Fixes: https://tracker.ceph.com/issues/46492
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit a5aaaa69cc82af2a6f4b7f0bd60ce2da5015c8c2)

mgr/dashboard: support Orchestrator and user-defined Ganesha clusters

This change make the Dashboard support two types of Ganesha clusters:

- Orchestrator clusters (Since Octopus)
  - Deployed by the Orchestrator.
  - The Dashboard gets the pool/namespace that stores Ganesha
    configuration objects from the Orchestrator.
  - The Dashboard gets the daemons in a cluster from the Orchestrator.

- User-defined clusters (Since Nautilus)
  - Clusters defined by using `ceph dashboard
    set-ganesha-clusters-rados-pool-namespace` command is treated as
    user-defined clusters.
  - Each daemon has its own RADOS configuration objects. The
    Dashboard uses these objects to deduce daemons.

Fixes: https://tracker.ceph.com/issues/46492
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit a9accaeccf88e1b0ee4688ef2ae9ddbd3bd3dc5e)

Conflicts:
      src/pybind/mgr/dashboard/openapi.yaml
          - We don't have openapi-check feature in the Octopus. The file
            is removed in the backport.
      src/pybind/mgr/dashboard/services/ganesha.py
      src/pybind/mgr/dashboard/tests/test_ganesha.py
          - The conflicts are mainly caused by code re-format in the
    master.

mgr/dashboard: refator orchestrator service and daemon APIs

- Allow listing services by service_type.
- Allow listing daemons by daemon_type.

Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit b88638873bd738af1ce258549abb6c25e0683907)

Merge pull request #37551 from Vicente-Cheng/wip-47736-octopus

octopus: rgw: rgw_file: avoid long-ish delay on shutdown

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #37688 from bk201/wip-47822-octopus

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #37597 from rhcs-dashboard/wip-47792-octopus

octopus: mgr/dashboard: Add short descriptions to the telemetry report preview

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

doc/PendingReleaseNotes: clean up for 15.2.6

This commit drops release notes that have already been published and
organizes the remaining release notes under a heading so it is clear
they are targeting the 15.2.6 release.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mon/MonMap: fix unconditional failure for init_with_hosts

This bug prevents setting mon_host to a DNS name.

Fixes: https://tracker.ceph.com/issues/47951
Fixes: 7a1f02acfe6b5d8a760efd16bb594a0656b39eac
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 21d9f8333b8c76979bbe90d60a8ad6ebb5e36c76)

test/mon: add tests for mon_host build by hostname

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 4022c1f1fb4c37e63bf884e36a2b533735c56f94)

Conflicts:
    src/test/mon/MonMap.cc
- do not attempt to introduce boost::intrusive_ptr into Nautilus
- monmap.build_initial takes bare cct in nautilus (master: cct.get())

test/mon: fix compiler errors in MonMap unittest

The code atrophied. Clean this up.

The tests are disabled because they SIGSEGV during SetUp.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 06f44cbf12c20ce8f1862111340f3b3f132577d0)

Conflicts:
    src/test/mon/MonMap.cc
- do not attempt to introduce boost::intrusive_ptr into nautilus
- monmap.build_initial takes bare cct in nautilus (master: cct.get())

Merge pull request #37691 from smithfarm/wip-47877-octopus

octopus: doc: cephfs: improve documentation of "ceph nfs cluster create" and "ceph fs volume create" commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

strtol: Add parse/consume for string_view friendly interface

Also these don't have the stringstream overhead.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
(cherry picked from commit a29695e82ec8a93b000322773949f30694abf3d3)

mon/MDSMonitor: do not ignore mds's down:dne request

Fixes: https://tracker.ceph.com/issues/47881
Signed-off-by: chencan <chen.can2@zte.com.cn>
(cherry picked from commit 768d7fc4e8b74c88ea2a623ee4d21ac1f20d8c7a)

mon/MDSMonitor: divide mds identifier and mds real name with dot

Fixes: https://tracker.ceph.com/issues/47806
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
(cherry picked from commit 4400d70c15e8fb4f013ca22db9fd5fe60c99dc32)

mds: account for closing sessions in hit_session

While stopping an mds we can reply to a request while all client
sessions are closing. We shouldn't assert in this case.

Fixes: https://tracker.ceph.com/issues/47833
Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 6823d8fb619c07b4e749ae564df565eadc59c187)

mgr/volumes/nfs: Fix wrong error message for pseudo path

Fixes: https://tracker.ceph.com/issues/47783
Signed-off-by: Varsha Rao <varao@redhat.com>
(cherry picked from commit 1552f7239c0e2dc4f661cd80f17369422c919c50)

cephfs-journal-tool: fix wrong read_offset when get missing objects

Fixes: https://tracker.ceph.com/issues/45575
Signed-off-by: Xue Yantao <jhonxue@tencent.com>
(cherry picked from commit bfa63666bb40c7939aa4da3c2c8f43a7022a78e8)

test: Avoid races by waiting for PGs go clean before query

Fixes: https://tracker.ceph.com/issues/46405
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 3ba7ebd3e299587b3828a7f794f070d3d01da4c4)

test: Inconsequential change to get object names as desired

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit b20a277f0546b951df8c29650d1d699afd43e498)

librbd: ignore -ENOENT error when disabling object-map

Fixes: https://tracker.ceph.com/issues/47840
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8e88224d8d1e7602392a81ed4da8139a79600d8f)

librbd: update AioCompletion return value before evaluating pending count

If the pending count is decremented before the return value is updated,
there is a possibility of two ASIO threads concurrently decrementing the
pending count down from 2 -> 1 -> 0. In the second thread (the one that
performs the final decrement from 1 -> 0), it can finalize the completion
before the first thread has had a chance to update the return value.

Fixes: https://tracker.ceph.com/issues/47847
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 94f3bce53c39017028ce44a80697f55af2a82e68)

journal: possible race condition between flush and append callback

When notifying the journal recorder of an overflow or if the object
close request has completed due to no more in-flight IO, it was
possible for a race between a flush request and the processing of
an append completion to attempt to kick off duplicate notifications.
Since the overflowed and closed callbacks are properly protected from
duplicates, use a counter instead of a boolean to track possible
in-flight handler callbacks.

Fixes: https://tracker.ceph.com/issues/47880
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 458ab997fe77ea78803a34c6c9715225aa3413ba)

rgw: fix: S3 API KeyCount incorrect return.

KeyCount should return object count + common prefix count.
see S3 example: https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html#API_ListObjectsV2_Example_5

Related: https://github.com/docker/distribution/issues/3200

Signed-off-by: 胡玮文 <huww98@outlook.com>
(cherry picked from commit f96a6fdad16da2a7093f538ee577248dfbc65ca1)

rgw/gc: fix for incrementing the perf counter 'gc_retire_object'
in the new gc queue code for omap offload, when gc objects from queue
are deleted. This was missed out initially.

Fixes: https://tracker.ceph.com/issues/47908
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit bde4c5bf9123bfa03189accd064b813a0d3179b9)

rgw/gc: fixing the condition when marker for a queue is
always reset to empty which causes RGWGC::list to get stuck in
a loop, which ultimately is broken out of when the queue's truncated
flag is false.

1. Check for entries size also while evaluating whether objects cache for
a gc object should be marked as 'transitioned' in case of cls_rgw_gc_list.
When there are no entries, we get back a return value of 0, and the
object cache is not marked as 'transitioned'.

2. Also for the last gc object, we need to check whether the queue is still
under process and set the correct flag.

Missing the two conditions above causes the GC::list to loop continously
over the same gc object.

Fixes: https://tracker.ceph.com/issues/47909
Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
(cherry picked from commit bf3f3ba675d092f48e403826fc0813e23c07045d)

rgw: use yum rather than dnf for testing rgw-orphan-list

The teuthology testing for rgw-orphan-list needs to install
`s3cmd`. Switch from using dnf to yum to work on a wider variety of
platforms.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 11a08a5bb867e05d033f126d9de7c370770ee63f)

qa/cephfs: add session_timeout option support

When the mds revoking the Fwbl caps, the clients need to flush
the dirty data back to the OSDs, but the flush may make the OSDs
to be overloaded and slow, which may take more than 60 seconds to
finish. Then the MDS daemons will report the WRN messages.

For the teuthology test cases, let's just increase the timeout
value to make it work.

Fixes: https://tracker.ceph.com/issues/47565
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 0422673b6150df851a4ea1662637a77585cde52d)

qa/cephfs: move the cephfs's opertions setting to create()

Fixes: https://tracker.ceph.com/issues/47565
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit cb8081ce7f4e0897cb2047d409ac2865afb3227c)

qa/cephfs: add 'cephfs:' section support

Fixes: https://tracker.ceph.com/issues/47565
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 3b5303482ff77667d48b85f4a3bc54e3ff6a60a6)

client: fix inode ll_ref reference count leak

Fixes: https://tracker.ceph.com/issues/47918
Signed-off-by: sepia-liu <liuwei_coder@163.com>
(cherry picked from commit 019ba52c8f3ba8263b67b4d1a3bfd6d20e98eeda)

vstart.sh: fix fs set max_mds bug

Fix a bug where the name used when creating a volume and the name used when setting max_mds were different.
Fixes: https://tracker.ceph.com/issues/47946
Signed-off-by: Jinmyeong Lee <jinmyeong.lee@linecorp.com>
(cherry picked from commit 6a9445c2cbe6c0c7045bfaed007cc1920ad132ed)

tools/rados: flush formatter periodically during json output of `rados ls`

While `rados ls` is emitting object info through a json formatter,
flush the formatter after there are at least 4096 bytes are buffered
for output.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 1548ef7a97559f17023f17842dab51d47cef89df)

test/store_test: use 'threadsafe' style for death tests.

Hopefully Fixes: https://tracker.ceph.com/issues/47328
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 99ac34cbfeb98c36ffcc3e1b5b65174930273c4c)

test/mon: build MonMap unittest

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 8408b63e908e3f7145b2df5520f28ac12d615967)

rbd-nbd: don't ignore namespace when unmapping by image spec

Fixes: https://tracker.ceph.com/issues/47665
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit b360186eb6463811ce19f21e8d24ab7c44dc7279)

qa/workunits/rbd: yet another attempt to improve rbd-nbd unmap

Previously it still could race when unmap_device returned success
because the device was not found in `rbd-nbd list-mapped` (the nbd
device was removed) but the test failed because the process was still
found in the ps table.

Fixes: https://tracker.ceph.com/issues/47394
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit f0c69761c8036a57319ead5cdf97cebb0f0fb5d7)

Conflicts:
qa/workunits/rbd/rbd-nbd.sh
- omit changes in tests that are not in octopus

rgw: Add request timeout beast

Add request timeout beast

The beast frontend will use the same parameter as the civetweb one, request_timeout_ms and will be configured to 65 seconds by default

Fixes: https://tracker.ceph.com/issues/45431
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit 5d5f9a0d41721f08b19f8425149fd13f4ef13696)

rgw: radosgw-admin should paginate internally when listing bucket

Currently `radosgw-admin bucket list ...`, when listing a bucket, asks
for the value of "--max-entries" internally. To list a large bucket
entirely the user would have to set "--max-entries" to a large value
(e.g., 10000000). Internally this doesn't paginate, so it will try to
produce the entire list at once. This can consume a lot of memory, and
there are known cases where this induces an out-of-memory crash.

So now we'll set a maximum pagination size of 10,000. So even with
large values of "--max-entries" it will still be able to produce the
full listing without stressing memory, because it will ask for at most
10,000 entries at a time.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 6d033061bf9eaebf3dab37b9ed45de22ce6fa6b7)

Conflicts:
src/rgw/rgw_admin.cc
- formatter does not have get() method in octopus

rgw: fix expiration header returned even if there is only one tag in the object the same as the rule

Expiration header returned even if there is only one tag in the object the same as the rule

Signed-off-by: Or Friedmann <ofriedma@redhat.com>
Reported-by: Avi Mor <avmor@redhat.com>
Fixes: https://tracker.ceph.com/issues/46614
(cherry picked from commit bf7c7e59f390afb53cb1e30a440ab26bb093c11c)

rgw: rgw-orphan-list should use "plain" formatted `rados ls` output

The previous version that used "json-pretty" output for `rados ls`
added complications due to json's escaping of special characters. So
this version returns to the "plain" output for `rados ls` but deals
with entries (oids) that might have namespaces and/or locators as
well.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 5b994f90594208dab81045732099a03686819b30)

rgw: allow rgw-orphan-list to note when rados objects are in namespace

Currently namespaces and locators are ignored when `rados ls` is run
by rgw-orphan-list to record RADOS's known objects.

However there have been cases where RADOS objects have a locator, and
when one is included in the listing, the script does not handle it
correctly. Now when objects have locators, we will prevent their
output from entering the .intermediate file.

Additionally we do not expect RGW data objects to be in RADOS
namespaces, so when a namespaced object is detected, we'll error out
with a message.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit ddf52016fa03ba192f242ad641a5c8e5a95035a1)

mgr/dashboard: display devices' health information within a tabset

Wrap all devices' health information within a tabset
instead of displaying them from top to bottom.

Add more guard in the HTML template to prevent referencing undefined
variables.

Fixes: https://tracker.ceph.com/issues/47494
Fixes: https://tracker.ceph.com/issues/43177
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
(cherry picked from commit ba3350c7e8c755d5c84c1a027a3a173191cb898d)

Conflicts:
      src/pybind/mgr/dashboard/frontend/src/app/ceph/shared/smart-list/smart-list.component.html
      src/pybind/mgr/dashboard/frontend/src/app/ceph/shared/smart-list/smart-list.component.ts
      - Use ngx-bootstrap tabset for tabs.

rgw: fix user stats iterative increment

The RGWBucketCtl::sync_user_stats() function can increment or reset the
stats [1][2]

[1]https://github.com/ceph/ceph/blob/master/src/rgw/rgw_bucket.cc#L3745
[2]https://github.com/ceph/ceph/blob/master/src/rgw/services/svc_bi_rados.cc#L379-L381

fixes: https://tracker.ceph.com/issues/46400

Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 21e877ca67db7840026b1768751b167e2c0a53da)

Conflicts:
src/rgw/rgw_sal.cc
- master's owner->get_id() becomes user.info.user_id in octopus

Merge pull request #37722 from rishabh-d-dave/wip-47845-octopus

octopus: ceph-volume: add no-systemd argument to zap

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #37705 from smithfarm/wip-47898-octopus

octopus: mon: have 'mon stat' output json as well

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #37697 from s0nea/wip-47602-octopus

octopus: Enable per-RBD image monitoring

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #37674 from Vicente-Cheng/wip-47706-octopus

octopus: osdc/ObjectCacher: overwrite might cause stray read request callbacks

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #37686 from bk201/wip-47832-octopus

octopus: mgr/dashboard: fix the error when exporting CephFS path "/" in NFS exports

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>