mds: switch to rank number instead

Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #40453 from tchaikov/wip-doc-confval

doc/rados/configuration: format confval with "confval" directive

Reviewed-by: Zac Dover <zac.dover@gmail.com>

Merge pull request #40462 from cyx1231st/wip-seastore-fix-onode-tree-eagain

crimson/onode-staged-tree: allow non-empty DeltaRecorder to be destructed

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #40472 into master

* refs/pull/40472/head:
mgr/cephadm/upgrade: ignore deployed_by until mgr is upgraded

Reviewed-by: Adam King <adking@redhat.com>

Merge PR #40443 into master

* refs/pull/40443/head:
doc/rbd: missed blacklist -> blocklist

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40390 into master

* refs/pull/40390/head:
cstart.sh: some improvements
cpatch: allow python 3.6 or 3.8
cpatch: behave with ninja (instead of make)

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #40409 into master

* refs/pull/40409/head:
qa/workunits/cephadm/test_cephadm: specify image separately

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #40428 into master

* refs/pull/40428/head:
win32*.sh: disable libcephsqlite when targeting Windows

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/cephadm/upgrade: ignore deployed_by until mgr is upgraded

Until we upgrade the mgr itself, we will never be able to make our
deployed daemons have a deployed_by == target_digests.  Ignore those
daemons until the mgr is the right version.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #40470 from idryomov/wip-msgr1-extra-tickets

auth/CephxClientHandler: explain why skipping extra_tickets is needed

Reviewed-by: Sage Weil <sage@redhat.com>

crimson/onode-staged-tree: allow non-empty DeltaRecorder to be destructed

Fixes: http://tracker.ceph.com/issues/50028
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/.../transaction_manager: add log for eagain

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

auth/CephxClientHandler: explain why skipping extra_tickets is needed

Since nautilus, CEPHX_GET_AUTH_SESSION_KEY shares both the auth ticket
and the service tickets, sparing an extra round-trip to get the service
tickets via CEPHX_GET_PRINCIPAL_SESSION_KEY.  This applies to msgr1 as
well, but we don't take advantage of it on the client side.

However, fixing CephxClientHandler to do the right thing breaks msgr1.
Since msgr1 is on its way out, rather than also fixing MonClient just
document the bug and the current behaviour.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #40466 from inspur-wyq/patch-2

blk/spdk/NVMEDevice.cc:  remove unused variables

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40188 from rhcs-dashboard/fix-49869-master

mgr/dashboard: fix broken feature toggles

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38679 from ceph/dependabot/npm_and_yarn/src/pybind/mgr/dashboard/frontend/node-notifier-8.0.1

build(deps): bump node-notifier from 8.0.0 to 8.0.1 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #39908 from rhcs-dashboard/fix-49645-master

mgr/dashboard: Remove username, password fields from Manager Modules/dashboard,influx

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #40023 from rhcs-dashboard/fix-49354-master

mgr/dashboard: Device health status is not getting listed under hosts section

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #40332 from rhcs-dashboard/prometheus-alert-notification-fix

mgr/dashboard: Fix for alert notification message being undefined

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #39917 from rhcs-dashboard/fix-43058-master

mgr/dashboard: warn password expiration in User Management

blk/spdk/NVMEDevice.cc:  remove unused variables

Signed-off-by: wangyunqing <wangyunqing@inspur.com>

Merge pull request #40240 from rhcs-dashboard/clear-cookies-on-logout

mgr/dashboard: Unable to login to ceph dashboard until clearing cookies manually

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #40458 from lxbsz/typo

client: fix typo in the comments

crimson/onode-staged-tree: fix typo in compare_full_key()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

client: fix typo in the comments

Signed-off-by: Xiubo Li <xiubli@redhat.com>

doc/rados/configuration: format confval with "confval" directive

for better formatting

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/dev/cephadm: quote command with backquotes

Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: add a new directive "confval"

* so we can define configuration in a more structured way
* so the confvals can be cross referenced

see also https://www.sphinx-doc.org/en/master/extdev/appapi.html#sphinx.application.Sphinx.add_object_type

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40445 from liewegas/cephadm-upgrade-progress-version

mgr/cephadm: make upgrade progress bar mention target version, not repo digest

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #40447 from Daniel-Pivonka/cephadmdocapplyrgwflags

mgr/orchestrator: fix rgw realm and zone flags

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #40379 from adk3798/no-overwrite

mgr/cephadm: --no-overwite flag for apply command

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #39969 from batrick/i49684

qa: wait for daemons to come up via cephadm

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #40396 from maydaycc/wip-49968

mgr: attr_list reassigned before use in diskprediction_local module

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40421 from badone/wip-fix-typo-get_device_id

common/blkdev: Remove trailing single quote

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40450 from smithfarm/wip-50007-alt

rpm: drop extraneous explicit sqlite-libs runtime dependency

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

rpm: drop extraneous explicit sqlite-libs runtime dependency

Commit 75980798f19b8c11efd75ba4aae3e491d4c99f98 introduced a new package,
libcephsqlite, with a hard RPM dependency on a package "sqlite-libs" which
does not exist in openSUSE.

Since the runtime library dependencies of libcephsqlite are handled by RPM
transparently, this line is not needed.

Fixes: https://tracker.ceph.com/issues/50007
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #40442 from Thingee/update-foundation-page

doc: Update Foundation page

Reviewed-by: Sage Weil <sage@redhat.com>

doc: Updating Foundation membership page

Signed-off-by: Mike Perez <thingee@gmail.com>

Merge pull request #40408 from varshar16/wip-doc-update-ninja

doc/install: update about ninja

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40345 from rzarzynski/wip-crimson-doc-waitstates-follow-up

doc/crimson: improve wording about the pipeline / wait states.

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #38732 into master

* refs/pull/38732/head:
client: fire the finish_cap_snap() after buffer being flushed
client: simplify the iterating code
client: remove unused _flushed_cap_snap
client: clean up the code

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40326 into master

* refs/pull/40326/head:
client: rebuild bl to avoid too many vector(> IOV_MAX)

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge PR #40394 into master

* refs/pull/40394/head:
osd: handle MOSDPGInfo
osd/PeeringState: init last_require_osd_release in ctor

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc/cephadm: fix rgw realm and zone flags

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>

Merge pull request #40415 from sseshasa/wip-qa-wait-for-clean-check

qa/tasks: Add wait_for_clean() check prior to initiating scrubbing.

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr/cephadm: make upgrade progress bar mention target version, not repo digest

The repo digest is super long and meaningless for a human user.  Instead,
use the target version (as soon as we know what it is--until then, use
the target image name).

Signed-off-by: Sage Weil <sage@newdream.net>

doc/rbd: missed blacklist -> blocklist

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>

Merge pull request #40438 from idryomov/wip-crimson-reopen-session

crimson/monc: discard active/pending connections when reopening

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/monc: discard active/pending connections when reopening

Otherwise pending_conns vector just keeps growing with redundant
connections all trying to reach the same set of monitors.  When one
of the attempts finally succeeds, _finish_auth() will pick the first
connection with a matching entity_addr_t, designate it as active and
close all others.  The match is very likely to be wrong and hence the
actual authenticated connection gets closed, leaving the OSD with
a bogus active_con and no monitor session.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge PR #40404 into master

* refs/pull/40404/head:
auth: require CEPHX_V2 by default

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #40321 into master

* refs/pull/40321/head:
cephadm: fix parsing of keepalived version (drop leading 'v')
cephadm: keepalived needs --cap-add=NET_RAW
cephadm: fix --cap-add=NET_ADMIN
cephadm: fix quoting for keepalived env var
mgr/cephadm: ha-rgw: use correct port

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #40015 into master

* refs/pull/40015/head:
cephadm: validate fsid during cephadm shell command

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>

Merge PR #40341 into master

* refs/pull/40341/head:
os/bluestore: separate omap per-pool vs per-pg alerts

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #40429 from varshar16/wip-fix-fs-top-error

src/tools/cephfs-top: fix flake8 line too long error

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #40401 from lnussel/master

ceph.spec: prepare openSUSE usrmerge (boo#1029961)

src/tools/cephfs-top: fix flake8 line too long error

Fixes: https://tracker.ceph.com/issues/50005
Signed-off-by: Varsha Rao <varao@redhat.com>

mgr/dashboard: Unable to login to ceph dashboard until clearing cookies

Fixes: https://tracker.ceph.com/issues/49897
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Clears the cookie for token after logout.

win32*.sh: disable libcephsqlite when targeting Windows

The Windows build is failing as sqlite can't be retrieved:
http://paste.openstack.org/raw/803940/

We don't support libcephsqlite on Windows for the time being,
so we'll just pass WITH_LIBCEPHSQLITE=OFF.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #40170 from Rethan/fix-sync-pause-max-lat

os/FileStore: fix sync_pause_max_lat which should not be inced

Reviewed-by: Kefu Chai <kchai@redhat.com>

README.md: remove extra 'the' and format para related to ninja

Signed-off-by: Varsha Rao <varao@redhat.com>

doc/install: replace do_cmake note with link to ceph README

Ceph README.md doc already has the do_cmake note and other build related info.

Signed-off-by: Varsha Rao <varao@redhat.com>

doc/install: Update about ninja

'ninja' replaced 'make' in this PR[1]. This patch updates the doc about it.

[1] https://github.com/ceph/ceph/pull/39826

Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #40334 from dvanders/pick

common/ipaddr: skip loopback interfaces named 'lo' and test it

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40292 from tchaikov/wip-39191

debian,cmake,cephsqlite: hide non-public symbols

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #40171 from swegener/bluefs-assert-always-true

bluefs: fix assert always being true

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #40304 from tchaikov/wip-telegraph-ann

pybind/mgr/telegraf: add type annotations

Reviewed-by: Wido den Hollander <wido@widodh.nl>

Merge pull request #40403 from vshankar/wip-cephfs-top-add-metrics

cephfs-top: include additional metrics reported by `fs perf stats`

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #40167 from singuliere/wip-49781

common/mempool: only fail tests if sharding is very bad

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

common/blkdev: Remove trailing single quote

Example output:

e1 collect_metadata :  no unique device id for : fallback method has no
model nor serial'

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #40331 from athanatos/sjust/wip-seatore-gc-refactor

crimson/seastore: refactor gc into concurrent task

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #40336 from tchaikov/wip-cmake-fio

cmake: pass compile options by fio interface library

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #40323 from aclamk/wip-onode-resiliant-to-split-cache

os/bluestore: Make Onode::put/get resiliant to split_cache

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

mgr/volumes: do not overwrite existant mds specs

Signed-off-by: Adam King <adking@redhat.com>

crimson/os/seastore/segment_cleaner: bump reclaim_ratio limits

GC behavior still needs a ton of work, but this adjustment should
reduce gc activity for now until we have time to refine the params
and introduce the ability to write cold extents to a segment other
than the current journal segment.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore/.../btree_range_pin: use ceph_assert

There's a performance overhead, but seastore is quite immature and
detecting these logic errors saves a ton of debugging time.

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/os/seastore/.../block.cc: add error message on alloc error to segment_write

Signed-off-by: Samuel Just <sjust@redhat.com>

mgr/cephadm: no-overwite flag for apply command

no-overwrite flag makes it so cephadm will only apply the spec
if the spec does not already exist.

Signed-off-by: Adam King <adking@redhat.com>

mgr/dashboard: warn user's password expiration in user management
Warn about user's incoming password expiration on the User Management with an icon.
Fixes: https://tracker.ceph.com/issues/43058
Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>

qa/tasks: Add wait_for_clean() check prior to initiating scrubbing.

Fixes: https://tracker.ceph.com/issues/49983
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>

Merge pull request #39776 from alimaredia/wip-rgw-log-req-latency

rgw: add latency to the request summary of an op

cephadm: validate fsid during cephadm shell command

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>

Merge PR #40366 into master

* refs/pull/40366/head:
mgr/cephadm: Fix dashboard gateway configuration when using IPV6

Reviewed-by: Sage Weil <sage@redhat.com>

qa/workunits/cephadm/test_cephadm: specify image separately

On master, this will be IMAGE_MASTER.  On pacific branch, we'll make it
IMAGE_PACIFIC so that we don't hit the release mismatch error.

Signed-off-by: Sage Weil <sage@newdream.net>

Merge pull request #40395 from rhcs-dashboard/fix-49952-master

mgr/dashboard:Simplify some complex calculations in test_alerts.yml

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

auth: require CEPHX_V2 by default

It's been almost three years and support is present in all relevant
clients.

From the security perspective, roughly the same could be achieved
with "ceph osd set-require-min-compat-client nautilus", but this is
more user friendly as the client gets ENOTSUP instead of spinning on
"feature set mismatch" faults.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

cephfs-top: include additional metrics reported by `fs perf stats`.

Without this, `cephfs-top` hits an exception since the additional
metrics keys were not configured.

Also, include a validation suring selftest that checks if `ceph
fs perf stats` metrics match what cephfs-top is configured to
report.

Fixes: http://tracker.ceph.com/issues/49974
Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #40400 from tchaikov/wip-ctest-jenkins

run-make-check.sh: let ctest generate XML output

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #40327 from racpatel/wip-cephfs-top-refresh-interval

cephfs-top: allow configurable stats refresh interval

Merge pull request #40210 from joscollin/wip-B49736-missing-client-metadata

cephfs-top: be resilient to missing client metadata keys

ceph.spec: prepare openSUSE usrmerge (boo#1029961)

The compat symlink in /sbin is no longer required and actually in
the way in the usrmerge case.

Signed-off-by: Ludwig Nussel <ludwig.nussel@suse.de>

run-make-check.sh: let ctest generate XML output

to enable XUnit plugin of jenkins to consume the ctest output and
publish it in the dashboard, we need to

* let ctest generate XML output instead of plain text output
* do not fail the test if any test case fails. this allows the publisher
  to do its job by checking the XML output.
* prevent ctest from compressing the output. see
  https://issues.jenkins.io/browse/JENKINS-21737

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard:Simplify some complex calculations in test_alerts.yml

run-promtool-unittests is failing with difference in floating point values in some complex calculations. This PR intends to simplify those calculations and fix this issue.

Fixes: https://tracker.ceph.com/issues/49952
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

osd: handle MOSDPGInfo

this change partially reverts b37e959238e75690d38873744515fa54413f4393,
which introduced a regression where we fail to handle MOSDPGInfo
messages sent by osd before aba13c7661bf8a62f2879513512053dc9857c003
and/or f7130db7a9fa85e475f14e4feed0097bf2d0b964.

Fixes: https://tracker.ceph.com/issues/49963
Signed-off-by: Kefu Chai <kchai@redhat.com>

osd/PeeringState: init last_require_osd_release in ctor

so we can use the latest osdmap to tell what is the required osd release in
*current* cluster before consuming a new osdmap which is fed to
PeeringState only if we advance to a new map.

a osd shares its pg_info using MOSDPGInfo2 or MOSDPGInfo according to
PeeringState::last_require_osd_release, this happens before a new osdmap
is fully consumed.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr: attr_list reassigned before use in diskprediction_local module

According to the document description, attr_list should be intersection
set of all disk days.

Fixes: https://tracker.ceph.com/issues/49968
Signed-off-by: wencong wan <wanwc@chinatelecom.cn>

Merge pull request #40377 from tchaikov/wip-doc-fixes

doc/man: fix formatting

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #38044 from dvanders/dvanders_buffered

common/options: bluefs_buffered_io=true by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #40380 from adamemerson/wip-num-rados-handles-die-die-die

Die, `num_rados_handles`, die.

Reviewed-by: Ali Maredia <amaredia@redhat.com>

cstart.sh: some improvements

- keep using the same port over multiple incantations (like we do the fsid)
- fix the image inspect line to work with (sudo) podman
- pass args on to cephadm bootstrap
- disable use_repo_digest so that we don't do a pull from
  cephadm's convert_tags_to_repo_digest

Signed-off-by: Sage Weil <sage@newdream.net>

cephadm: fix parsing of keepalived version (drop leading 'v')

Signed-off-by: Sage Weil <sage@newdream.net>