doc/rados: format "initial troubleshooting"

Format the steps in the "Initial Troubleshooting" section of
doc/rados/troubleshooting/troubleshooting-mon.rst. A near-future PR (not
this one) will add context to this section and explain that the steps
described here are the first steps that you should undertake when you
determine that you have an unresponsive or down Monitor. This PR is
merely for formatting.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge PR #54338 into main

* refs/pull/54338/head:
script: enable checkout of specific sha1 as base

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #54457 from zdover23/wip-doc-2023-11-12-rados-config-ceph-conf-config-ceph

doc/config: edit "ceph-conf.rst"

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>

Merge pull request #54458 from zdover23/wip-doc-2023-11-12-rados-troubleshooting-mon-parallel-structure

doc/rados: parallelize t-mon headings

doc/rados: parallelize t-mon headings

Give parallel structure to the questions in the Q&A section of the "The
Cluster Has Quorum But At Least One Monitor Is Down" subsection of the
"Most Common Monitor Issues" section of
doc/rados/troubleshooting/troubleshooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/config: edit "ceph-conf.rst"

Edit the first section of doc/rados/configuration/ceph-conf.rst.
Initially I just wanted to change "series" to "set", but once I got my
hands dirty I ended up simplifying some sentences.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54452 from zdover23/wip-doc-2023-11-11-rados-troubleshooting-mon-common-issues-5-of-x

doc/rados: edit t-mon "common issues" (5 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmai.com>

doc/rados: edit t-mon "common issues" (5 of x)

Edit the fifth part of the section "Most Common Monitor Issues" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Follows https://github.com/ceph/ceph/pull/54440.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #49914 from adk3798/monitoring-teuth-test

qa/cephadm: basic test for monitoring stack

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #54398 from phlogistonjohn/jjm-cephadm-reorg-ctr-binds

cephadm: reorganize container binds functions

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #54046 from adk3798/warn-hosts-drain-spec

mgr/cephadm: warn when draining host explicitly listed in placement

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #54450 from cbodley/wip-test-rgw-rados-includes

test/rgw: clean up unused include dirs

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #54448 from ceph/wip-make-down-dependencies-explicit

rgw: make subclass dependencies explicit

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

script: enable checkout of specific sha1 as base

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

test/rgw: clean up unused include dirs

store/rados was renamed to driver/rados, so these include directories
aren't required

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: make subclass dependencies explicit

As part of the Zipper project generic back-end code is being teased
apart from rados-specific back-end code. This is a work in progress,
so currently generic code and other subclasses of StoreDriver (and
related high-level classes) depend on the rados-specific declarations.
Some of these dependencies are not always obvious since
src/rgw/driver/rados was put on the include path. That is now removed,
so any includes needing files from that subclass have to give a more
fully specified path.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #50419 from myoungwon/wip-rbm-partial-overwrite

crimson/os/seastore: introduce delta-based overwrite

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #54379 from cbodley/wip-rgw-sal-bucket-owner

rgw/sal: Bucket owner as rgw_user

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #54440 from zdover23/wip-doc-2023-11-10-rados-troubleshooting-mon-common-issues-4-of-x

doc/rados: edit t-mon "common issues" (4 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #54415 from jincreator/cls-fmt-ftbfs

cls: fix unable to link with system-installed fmt

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc/rados: edit t-mon "common issues" (4 of x)

Edit the fourth part of the section "Most Common Monitor Issues" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Follows https://github.com/ceph/ceph/pull/54432.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

cephadm: convert get_container_binds to use class based approach

Since all types affected by get_container_binds now have the common
customize_container_binds, use a generic class-based approach by
creating an instance of ContainerDaemonForm and calling the method.
All other classes have a customize_container_binds that is a no-op.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: only call get_container_binds on types that have binds

This is a step towards not calling get_container_binds in get_container.
A future commit will replace uses of get_container_binds with direct
uses of common class methods.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: move bind mounts assignment to a variable

Move the call to get_container_binds out of the function call.
This will aid with the next refactoring steps, so that the uses
of get_container_binds can be brought into the get_container call
directly.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: add customize_container_binds method to custom container

Add a customize_container_binds function to the CustomContainer daemon
type class and use it from the common get_container_binds function.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: covert nvmeof type to provide a customize_container_binds method

Convert the CephNvmeof type's get_container_binds to
customize_container_binds and use it in the generic method.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: covert iscsi type to provide a customize_container_binds method

Convert the CephIscsi type's get_container_binds to
customize_container_binds and use it in the generic method.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: ensure the iscsi test can create iscsi daemon class

Ensure that the iscsi test can call CephIscsi.create inside other
functions by passing a required, but fake, gateway config file.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: always pass ctx to customize_container_{binds,mounts}

These functions often derive the binds and/or mounts from the context
variable. Thus we should have the base class method accept the context.
Not all subclassess will use it but it will be there for those that do.
Also, fix the type for customize_container_mounts - it should be a dict
not a list.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

qa/cephadm: adjust host drain test to handle explicit placement warning

Since we're adding a warning if any host is listed explicitly
in the placement of any service when removing the host,
we need to adjust the host drain test that removes a host
without the --force flag to not have the explicit hostname
in the placement for the mon service.

Signed-off-by: Adam King <adking@redhat.com>

mgr/cephadm: warn when draining host explicitly listed in placement

In the case you apply a spec like

```
service_type: node-exporter
placement:
  hosts:
  - host3
```

and then you run `ceph orch host drain host3`, cephadm will remove
the daemon from that host and the placement would now match nothing.

This is definitely an issue that should be able to be bypassed as
it generally isn't serious, but it would be good to let users
know they have the host listed explicitly in placements like this
when they want to drain it.

Fixes: https://tracker.ceph.com/issues/63220
Signed-off-by: Adam King <adking@redhat.com>

Merge pull request #54387 from phlogistonjohn/jjm-cephadm-isolate-mounts-code

cephadm: isolate mounts code in methods

Reviewed-by: Adam King <adking@redhat.com>

rgw/sal: Bucket owner as rgw_user

stop maintaining a User pointer for the bucket owner, and use the
stored rgw_user owner directly

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54432 from zdover23/wip-doc-2023-11-09-rados-troubleshooting-mon-common-issues-3-of-x

doc/rados: edit t-mon "common issues" (3 of x)

crimson/os/seastore/object_data_handler: prevent from being overwritten if cloned

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

test/crimsons/seastore/object_data_handler: add overwrite test cases

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

crimson/os/seastore: store data as a delta using mutable extent when overwriting

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

doc/rados: edit t-mon "common issues" (3 of x)

Edit the second part of the section "Most Common Monitor Issues" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Follows https://github.com/ceph/ceph/pull/54417.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54392 from guits/cv-revert-52429

ceph-volume: Revert "ceph-volume: fix raw list for lvm devices"

Merge pull request #54412 from zdover23/wip-doc-2023-11-08-rados-troubleshooting-mon-top-matter

doc/rados: edit t-shooting-mon.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

rgw/sal: remove Bucket::get_acl_owner()

was only used under the line by rados and dbstores

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #50599 from cbodley/wip-rgw-sal-forward

rgw: move forward_request_to_master() out of sal

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

ceph-volume: fix raw list for non-existent device

ceph-volume should not crash when given a device which doesn't exist.

Fixes: https://tracker.ceph.com/issues/63391
Signed-off-by: Matthew Booth <mbooth@redhat.com>
(cherry picked from commit e009b1a319c834a504947423ae1dd55387809235)

cls: fix unable to link with system-installed fmt

Signed-off-by: Jinkyu Yi <jincreator@jincreator.net>

Merge pull request #50212 from rishabh-d-dave/fs-swap-subcmd

cephfs: add command "ceph fs swap"

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-volume: Revert "ceph-volume: fix raw list for lvm devices"

This reverts commit e5e429617c1c27dcd631171f65d30571e32f7266.
This commit introduced a regression, see linked tracker for details.

Fixes: https://tracker.ceph.com/issues/63391
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #54417 from zdover23/wip-doc-2023-11-08-rados-troubleshooting-mon-common-issues-2-of-x

doc/rados: edit t-mon "common issues" (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #54397 from phlogistonjohn/jjm-buildpy-latin1-fix

cephadm: work around pip failure on some envs

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

doc/rados: edit t-shooting-mon.rst

Improve the front matter of the section "Troubleshooting Monitors" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54413 from zdover23/wip-doc-2023-11-08-rados-troubleshooting-mon-common-issues-1-of-x

doc/rados: edit t-mon "common issues" (1 of x)

doc/rados: edit t-mon "common issues" (2 of x)

Edit the second part of the section "Most Common Monitor Issues" in
doc/rados/troubleshooting/troubleshooting-mon.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/rados: edit t-mon "common issues" (1 of x)

Edit the first part of the section "Most Common Monitor Issues" in
doc/rados/troubleshooting/troublehsooting-mon.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54402 from athanatos/sjust/wip-fix-tmdriver

crimson/tools/store_nbd/tm_driver: use TransactionManager::remove

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

crimson/tools/store_nbd/tm_driver: use TransactionManager::remove

231c7ac removed TransactionManager::dec_ref, adjust store_nbd/tm_driver
to use TransactionManager::remove instead.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #54158 from michaelalang/master

[cephadm](BUG): honor --skip-firewalld in section Open ports explicitly required for the daemon

Reviewed-by: Adam King <adking@redhat.com>

cephadm: work around pip failure on some envs

Work around an encoding/locale issue when the dashboard tests are run
(ubuntu 20.04).
The build.py changes brought in a9d1c62ca86 were validated for package
builds, teuthology, and other CI jobs but a different error was masking
this failure in the dashboard ci job.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #54178 from leonid-s-usov/balance-after-stop

mds/MDBalancer: ignore queued callbacks if MDS is not active

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #54149 from mihalicyn/ceph_mds_request_head_legacy_fallback_fix

MClientRequest: properly handle ceph_mds_request_head_legacy for ext_num_retry, ext_num_fwd, owner_uid, owner_gid

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53912 from joscollin/wip-B63099-fix-prev-snapshot-none

client: avoid getvxattr op for ceph.mirror* attrs

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53887 from lxbsz/wip-fix-check-access

client: two fix for checking mds auth access in client side

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #53899 from rishabh-d-dave/before-fs-rename-fail-fs

 cephfs,mon: fs rename must require FS to be offline and refuse_client_session to be set

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rgw/sal: rename Bucket::remove_bucket() to remove()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: remove forward-to-master functions from Driver

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add free function forward_iam_request_to_master()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add free function rgw_forward_request_to_master()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: forward_request takes const req_info

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: remove ZoneGroup::get_endpoint()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: move User::create_bucket() to Bucket::create()

the CreateBucket operation always has to deal with existing buckets, so
we have to load the Bucket handle first anyway

also moves the related placement and forward_request_to_master() logic
out of sal entirely

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54364 from rkachach/fix_issue_github_template

github: adding command for rook e2e jenkins job

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #54391 from rhcs-dashboard/fix-63469-main

mgr/dashboard: fix rgw multi-site import form helper

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #54101 from rhcs-dashboard/dashboard-v3-unit-test

mgr/dashboard: dashboard area chart unit test

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix rgw multi-site import form helper

Before : To obtain the token, generate it from your primary Ceph cluster. This token includes encoded information about the secondary cluster's endpoint, access key, and secret key.
Fix: To obtain the token, generate it from your primary Ceph cluster. This token includes encoded information about the primary cluster's endpoint, access key, and secret key.

Fixes: https://tracker.ceph.com/issues/63469
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #54372 from linuxbox2/wip-fix-deletemakers

rgwlc: fix delete marker regression in currentversion lc expiration

Merge pull request #54384 from cityofships/main

doc/cephadm: fix typo in set ssh key command

Reviewed-by: Zac Dover <zac.dover@proton.me>

cephadm: move code setting up mounts for monitoring daemon types

Move the code used to set up mounts for monitoring daemon types to
a method on the Monitoring class.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: move code setting mounts for ceph daemon types

Move the code in an if-block for ceph daemon types within get_container_mounts
to a new method of the Ceph type. The logic is moved to a staticmethod
so that it can be reused within the CephExporter type. This can be fixed
up in the future, but currently the staticmethod handles both Ceph and
CephExporter. Both types gain a get_container_mounts for common usage.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: remove unused get_container_mounts CephExporter method

The get_container_mounts method on CephExporter is never called.
Remove it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

cephadm: move where _get_container_mounts_for_type is called

The _get_container_mounts_for_type method is only useful for ceph based
daemon types. There's no need to call this method in a universal
context.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

doc/cephadm: fix typo in set ssh key command

Signed-off-by: Piotr Parczewski <piotr@stackhpc.com>

Merge pull request #54196 from ivancich/wip-log-bi-list-errors

rgw: log errors during "bi list"

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>

Merge pull request #54257 from ivancich/wip-fix-restore-bucket-index

rgw: handle instances w/ leading hyphens correctly

Reviewed-by: Michael Kidd <linuxkidd@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53537 from cyx1231st/wip-crimson-osd-fix-complete-exit

crimson/osd: decouple cross-core pg submission out of the OrderedExclusivePhase

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #54197 from ivancich/wip-bucket-stats-add-gen

rgw: add layout generation info to "bucket stats"

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54365 from cbodley/wip-63455

common: admin socket catches exception by reference

Reviewed-by: Leonid Usov <leonid.usov@ibm.com>

rgwlc: fix delete marker regression in currentversion lc expiration

Fixes a regression introduced in https://github.com/ceph/ceph/pull/50680,
which attempted to re-use an already created sal object handle as
an optimiztion.  This change suppressed creation of delete markers in
current version expiration when buckets are versioned.

Fixes: https://tracker.ceph.com/issues/63458
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge PR #53568 into main

* refs/pull/53568/head:
common: resolve config proxy deadlock using refcounted pointers
common: add missing locks in config_proxy methods
common/ceph_mutex: note whether mutex debug methods are usable
qa: add reproducer for obs removal deadlock
qa: narrow search to debug_asok

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #54134 from phlogistonjohn/jjm-cephadm-podman-props-2

cephadm: move more container engine specific logic

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #54264 from thotz/remove_realm_name_period_config

rgw/rgw_period : remove realm name from period config

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #54342 from anthonyeleven/anthonyeleven/improvements

doc/radosgw: Improve dynamicresharding.rst

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #54147 from AliMasarweh/wip-alimasa-perf-counters

RGW: perf counters for persistent topic stats

reviewed-by: yuvalif, alimaredia, cbodley

common: admin socket catches exception by reference

probably not a real bug, but resolves a compiler warning:

src/common/admin_socket.cc: In member function ‘int RaiseHook::parse_signal(std::string&&, ceph::Formatter*, std::ostream&)’:
src/common/admin_socket.cc:869:21: warning: catching polymorphic type ‘class std::invalid_argument’ by value [-Wcatch-value=]
  869 |       } catch (std::invalid_argument) {
      |                     ^~~~~~~~~~~~~~~~

Fixes: https://tracker.ceph.com/issues/63455
Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add free function rgw::get_zonegroup_endpoint()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: use load_bucket()'s rgw_bucket overload

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: add rgw_bucket() ctor for tenant+name

this avoids the need to construct a temporary rgw_bucket_key just to
construct a rgw_bucket without an instance id

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: remove load_bucket() overload with tenant/name

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/sal: rename loading get_bucket()s to load_bucket()

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/services: make set_attrs const correct

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: remove legacy bucket placement and 'radosgw-admin pool' commands

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: pass obj_lock_enabled into RGWRados::create_bucket

there's a retry loop here that may end up overwriting RGWBucketInfo, so
pass obj_lock_enabled and use it to update info.flags on each iteration

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: read existing placement and swift_ver_location outside of sal

Signed-off-by: Casey Bodley <cbodley@redhat.com>