tools/cephfs/first-damage: repair specific first=CEPH_NOSNAP damage

first==CEPH_NOSNAP is never valid. Apparently somehow the MDS may set
this in some code path yet to be discovered, even without snapshots.

Note, we probably should not necessarily automate this fix via the MDS
because we do not know if this dentry (visible in HEAD) should also be
visible before NEXT_SNAP.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

tools/cephfs/first-damage.py: load CEPH_CONF

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #49500 from BryceCao/wip-fix-result-when-users-mfas-is-null

rgw: fix result when users mfas is null

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #49637 from zdover23/wip-doc-2023-01-05-top-bar-scroll-margin-top-repair

doc/css: add scroll-margin-top to dt elements

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/css: add scroll-margin-top to dt elements

add "scroll-margin-top: em3;" to custom.css so that the header bar
doesn't obscure the text of headwords in glossary.rst. Note that this
applies only to elements in the documentation that are rendered into
HTML with the dt (which stands for "description term" or "description
list") tag. Other modifications will be necessary in order to ensure
that the anchor points of non-dt elements are not obscured by the header
bar.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #48730 from dang/wip-dang-zipper-chown

Fix chown to unlink

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #49618 from petrutlucian94/rbd_py_test2

qa: retry test_rbd_wnbd.py fs operations

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

qa: retry test_rbd_wnbd.py fs operations

Certain FS related operations can fail, especially under load
(e.g. initializing partitions, volume formatting, etc).

For this reason, we're going to introduce some retries.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #49620 from bradfitz/bradfitz/doc

doc: fix a couple grammatical things

Reviewed-by: Zac Dover <zac.dover@gmail.com>

doc: fix a couple grammatical things

Signed-off-by: Brad Fitzpatrick <brad@danga.com>

Merge pull request #49613 from zdover23/wip-doc-2023-01-02-start-documenting-ceph-49609-followup

doc/start: add Anthony D'Atri's suggestions

doc/start: add Anthony D'Atri's suggestions

Add the suggestions made by Anthony D'Atri in
https://github.com/ceph/ceph/pull/49609.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49609 from zdover23/wip-doc-2023-01-02-start-documenting-ceph-cleanup

doc/start: refine "Quirks of RST"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/start: refine "Quirks of RST"

Refine the language that was added yesterday, language that explains how
certain aspects of RST work.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49606 from zdover23/wip-doc-2023-01-01-start-documenting-ceph-quirks-of-rst

doc/start: add link-related metadocumentation

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/start: add link-related metadocumentation

Add two kinds of link-related metadocumentation (documentation about how
to write documentation) to the "Documenting Ceph" section of the "Intro
to Ceph" document: 1. metadocumentation about external links, and 2.
metadocumentation about internal links.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49602 from zdover23/wip-doc-2022-12-31-glossary-DAS-capitalization

doc/glossary: capitalize "DAS" correctly

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/glossary: capitalize "DAS" correctly

Correctly capitalize "Direct-Attached Storage" in the glossary. (And
test the "Quincy" branch, which seems lately not to have picked up any
docs backports.)

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49599 from zdover23/wip-doc-2022-12-30-glossary-releases

doc/glossary: collate "releases" entries

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/glossary: collate "releases" entries

Collect the "Releases"-related entries together under the "Releases"
headword, in order to give readers a sense at a glance of how the
different kinds of releases relate to one another.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49488 from zdover23/wip-doc-2022-12-18-rbd-rbd-exclusive-locks-refine

doc/rbd: refine rbd-exclusive-locks.rst

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

doc/rbd: refine rbd-exclusive-locks.rst

Refine grammar (mostly semantics) in rbd-exclusive-locks.rst.

Co-authored-by: Ilya Dryomov <idryomov@redhat.com>
Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49593 from zdover23/wip-doc-2022-12-29-glossary-osd-id-each

doc/glossary: s/an/each/ where it's needed

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/glossary: s/an/each/ where it's needed

s/an/each/ in accordance with the suggestion made by Anthony D'Atri
here: https://github.com/ceph/ceph/pull/49590/files#r1058390357

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49584 from zdover23/wip-doc-2022-12-27-glossary-cleanup

doc/glossary: clean OSD id-related entries

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #49587 from zdover23/wip-doc-2022-12-27-rbd-wuold-typo

doc/rbd: s/wuold/would/ in rados-rbd-cmds.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #49583 from RanjiniMandyam/RMN-2151232-syntax

doc/dev/cephadm: fix host maintenance enter/exit syntax -

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rbd: s/wuold/would/ in rados-rbd-cmds.rst

s/wuold/would/ in rados-rbd-cmds.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/glossary: clean OSD id-related entries

Tidy up the sentences under the headwords "OSD fsid", "OSD id", and "OSD
uuid".

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/dev/cephadm: fix host maintenance enter/exit syntax -

Signed-off-by: Ranjini Mandyam Narasiodeyar <rmandyam@rmandyam.remote.csb>

Merge pull request #48893 from weixinwei/master

librbd: avoid EUCLEAN error after "rbd rm" is interrupted

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #49261 from ajarr/fix-57726

mgr/rbd_support: remove localized schedule option during module startup

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #49573 from zdover23/wip-doc-2022-12-26-glossary-cleanup

doc/glossary: disambiguate clauses

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/glossary: disambiguate clauses

Disambiguate various clauses, most of which contain forms of the verb
"to require".

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49569 from zdover23/wip-doc-2022-12-26-start-documenting-ceph-backport-fix

doc/start: update documenting-ceph.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

Merge pull request #49567 from zdover23/wip-doc-2022-12-25-glossary-cleanup

doc/glossary: improve glossary entries

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/start: update documenting-ceph.rst

Add an article to documenting-ceph.rst in order to troubleshoot a
backporting problem.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/glossary: improve glossary entries

Improve multiple glossary entries.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49564 from zdover23/wip-doc-2022-12-25-start-documenting-ceph

doc/start: improve documenting-ceph.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/start: improve documenting-ceph.rst

Improve the grammar and the pragmatics of the "TOC and Hyperlinks"
section of start/documenting-ceph.rst.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49558 from zdover23/wip-doc-2022-12-24-rados-api-librados-intro-prompt-removal

doc/rados: remove prompt from php.ini line

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

Merge pull request #49557 from zdover23/wip-doc-2022-12-24-rados-api-python-typo

doc/rados: correct typo in python.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/rados: remove prompt from php.ini line

Remove an unselectable prompt from the beginning of a line meant to be
included in the "php.ini" file.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/rados: correct typo in python.rst

s/you Ceph configuration file/your Ceph configuration file/

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49532 from petrutlucian94/rbd_py_test2

qa: fix test_rbd_wnbd.py, properly retrieving the drive letter

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

qa: add some retries to test_rbd_wnbd.py

The following operations may fail right after a block device
is attached:

* retrieving the disk number (can return -1)
* opening the disk
* setting the disk online or writable

For this reason, we'll need to add some retries. For convenience,
we're moving the existing retry logic to a separate decorator.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

qa: fix test_rbd_wnbd.py, properly retrieving the drive letter

Instead of trying to use the first partiton which may be reserved
by Windows, we'll fetch the first non-empty drive letter from
the disk that we've just mounted.

While at it, we're ensuring that the drive letter is actually a
letter and not a null character, which the Powershell command
returns in case of empty drive letters.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #49547 from idryomov/wip-rbd-qemu-xfstests-curl

qa: switch to curl for qemu-xfstests

Reviewed-by: Christopher Hoffman <choffman@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #49550 from zdover23/wip-doc-2022-12-23-rados-api-librados-intro-prompts

doc/rados: add prompts to librados-intro.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: add prompts to librados-intro.rst

Add unselectable prompts to doc/rados/api/librados-intro.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge PR #49406 into main

* refs/pull/49406/head:
mds: fix scan_stray_dir not reset next.frag on each run of stray inode

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #49172 into main

* refs/pull/49172/head:
qa/test_nfs: added testcase test_non_existent_cluster
mgr/nfs: handle bad cluster name during info command

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #48907 into main

* refs/pull/48907/head:
test: verify that snapshot items are accounted when deciding to split/merge a directory
mds: account for snapshot items when deciding to split or merge a directory

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>

Merge pull request #49259 from kamoltat/wip-ksirivad-fix-58155

mon/Elector.cc Added additional prank >= ranks_size sanity check
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

qa: switch to curl for qemu-xfstests

This is a follow-up for commit 631899ffeb84 ("qa: switch back to git
protocol for qemu-xfstests"), needed for the same "ancient execution
environment" reason.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge PR #48281 into main

* refs/pull/48281/head:
mds: md_log_replay thread blocks waiting to be woken up

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #48822 into main

* refs/pull/48822/head:
mds: switch submit_mutex to fair mutex for MDLog

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

test: verify that snapshot items are accounted when deciding to split/merge a directory

Signed-off-by: Venky Shankar <vshankar@redhat.com>

mds: account for snapshot items when deciding to split or merge a directory

Its easy to "overload" a directory object with large number of omap entries by
doing the following (one shot or over and over again):

      - touch dir/file{0..11000} ; create 11000 files (> mds_bal_split_size)
      - mkdir dir/.snap/snap_a
      - rm -f dir/file{0..11000}

End result - the directory object would have 11000 omap entries since the MDS
does not fragment directory snapshots. If the number of such entries exceed
`osd_deep_scrub_large_omap_object_key_threshold` (default: 200000), a cluster
health warning is generated:

       Large Omap objects found in pool...

CDir::should_merge() does not take into account COW'd inodes and the frags
get merged.

Fixes: http://tracker.ceph.com/issues/55215
Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge PR #49098 into main

* refs/pull/49098/head:
client/fuse: add support for 3.12 API of libfuse
client/fuse: set max_idle_threads to the correct value

Reviewed-by: Venky Shankar <vshankar@redhat.com>

qa/test_nfs: added testcase test_non_existent_cluster

Fixes: https://tracker.ceph.com/issues/58138
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

mgr/nfs: handle bad cluster name during info command

It should say the cluster does not exist

Fixes: https://tracker.ceph.com/issues/58138
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

Merge PR #49423 into main

* refs/pull/49423/head:
qa: remove qa/suites/marginal and restart task

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #49534 from idryomov/wip-rbd-qemu-xfstests-patchup

qa: switch back to git protocol for qemu-xfstests

Reviewed-by: Christopher Hoffman <choffman@redhat.com>

Merge pull request #48794 from dang/wip-dang-bucket-policy

RGW - Make sure PostObj set bucket on s->object

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>

qa: switch back to git protocol for qemu-xfstests

As noted in commit 89177d65988c ("qa: switch to https protocol for
repos' server"), git.ceph.com mirror doesn't make git:// available
anymore.  However, run_xfstests-obsolete.sh has "obsolete" in its
name for a reason -- due to an ancient execution environment, git://
is the only viable option:

  $ git clone https://git.ceph.com/xfstests-dev.git
  Cloning into 'xfstests-dev'...
  error: gnutls_handshake() failed: A TLS fatal alert has been received. while accessing https://git.ceph.com/xfstests-dev.git/info/refs
  fatal: HTTP request failed

Ditch git.ceph.com mirror for now.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #49529 from zdover23/wip-doc-2022-12-21-rados-bluestore-config-ref-prompts

doc/rados: add prompts to bluestore-config-ref.rst

Merge pull request #49403 from tchaikov/cmake-boost

cmake: pass cflags using compileflags in BuildBoost.cmake

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #49305 from petrutlucian94/dokan_subsys

dokan: use the right logging subsystem

Merge pull request #48929 from petrutlucian94/rbd_py_test

qa: fix test_rbd_wnbd.py

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/rados: add prompts to bluestore-config-ref.rst

Add unselectable prompts to
doc/rados/configuration/bluestore-config-ref.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49497 from BryceCao/wip-fix-mfa.rst-spelling-mistake

doc/radosgw: fix spelling mistake in mfa.rst

rgw: fix result when user's mfas is null

Fixes: https://tracker.ceph.com/issues/58308
Signed-off-by: caolei <cao.leilc@inspur.com>

doc/radosgw: fix spelling mistake in mfa.rst

Signed-off-by: caolei <cao.leilc@inspur.com>

Merge pull request #49402 from BryceCao/wip-fix-usage-trim-cant-be-control-by-remove-all

rgw: fix problem that usage is trimmed when the remove-all parameter …

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #49390 from cbodley/wip-qa-rgw-inline

qa/rgw: rgw/verify suite randomizes --placement-inline-data

Reviewed-by: Cory Snyder <csnyder@iland.com>

Merge pull request #49387 from cfsnyder/wip-cfsnyder-58243

rgw: fix dereference of empty std::optional in RGWDeleteMultiObj::execute()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #49362 from cbodley/wip-58231

rgw/sal: get_notification() takes optional_yield

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #49324 from BryceCao/wip-add-num-limit-for-user-policies

rgw : add num limit for IAM user policies

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #49298 from BryceCao/wip-fix-the-inheriting-class-member-is-passed-to-base-ctor

rgw : fix the inheriting class member is passed to base ctor

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #49194 from joke-lee/list-delimiter-encode

rgw: list with Delimiter should url encode when encoding-type=url

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #49426 from BryceCao/wip-fix-ListOpenIDConnectProviders-XML-format

rgw: fix ListOpenIDConnectProviders XML format

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #49016 into main

* refs/pull/49016/head:
doc/dev: remove release notes for major release branch
doc: hard-code latest release link
doc: remove rtd main checkout

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

mgr/rbd_support: remove localized schedule option during module startup

The localized schedule option was removed during every load of a
scheduler handler. This occured multiple times during a mgr process's
lifespan. Instead, remove the localized schedule module option when the
rbd_support module starts up.

Fixes: https://tracker.ceph.com/issues/57726
Signed-off-by: Ramana Raja <rraja@redhat.com>

Merge PR #48743 into main

* refs/pull/48743/head:
mds: do not acquire xlock in xlockdone state

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #48843 from rhcs-dashboard/expose_slow_ops

mgr/prometheus: expose daemon health metrics

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #47276 from rhcs-dashboard/mirror-pool-health

mgr/dashboard: display real health in rbd mirroring pools

Reviewed-by: Pegonzal <pegonzal@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

mgr/prometheus: expose daemon health metrics

Until now daemon health metrics were stored without being used. One of
the most helpful metrics there is SLOW_OPS with respect to OSDs and MONs
which this commit tries to expose to bring fine grained metrics to find
troublesome OSDs instead of having a lone healthcheck of slow ops in the
whole cluster.

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #49496 from cyx1231st/wip-seastore-fix-denc

crimson/os/seastore: fix denc device_spec_t, device_type_t and segment_type_t

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #49509 from rhcs-dashboard/fix-prometheus-module-blank-line

mgr/prometheus: Fix make check flake8 blank line missing

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #49514 from zdover23/wip-doc-2022-12-20-glossary-cephx-config-ref-link

doc/glossary: link to CephX Config ref

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #49513 from zdover23/wip-doc-2022-12-20-rados-auth-config-ref-prompts

doc/rados: add prompts to auth-config-ref.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/glossary: link to CephX Config ref

Link to the CephX Configuration Reference in the "CephX" entry in the
glossary.

Signed-off-by: Zac Dover <zac.dover@gmail.com>

doc/rados: add prompts to auth-config-ref.rst

Add unselectable prompts to doc/rados/configuration/auth-config-ref.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #49510 from zdover23/wip-doc-2022-12-20-rados-config-msgr2-prompts

doc/rados: add prompts to msgr2.rst

Reviewed-by: Cole Mitchell <cole.mitchell@gmail.com>

doc/rados: add prompts to msgr2.rst

Add unselectable prompts to doc/rados/configuration/msgr2.rst.

https://tracker.ceph.com/issues/57108

Signed-off-by: Zac Dover <zac.dover@gmail.com>

mgr/prometheus: add blank like previously wrongly deleted that was causing issues on makecheck due to flake8

Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge PR #48473 into main

* refs/pull/48473/head:
qa: fix _run_mount_cmd() and _get_mount_cmd() args in vstart_runner

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>

Merge pull request #49490 from tchaikov/wip-pybind-mgr-tox-fixes

pybind/mgr: tox and test fixes

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #49049 from rhcs-dashboard/add-osd-blocklist-metric

mgr/dashboard: Add metric relative to osd blocklist