CODEOWNERS: update cephadm paths

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #34324 from rzarzynski/wip-crimson-returnvec

crimson/osd: OP_CALL does support RETURNVEC now.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34290 from sebastian-philipp/doc-cephadm-service-spec

doc/mgr/orchestrator: Add "Service Specification"

Reviewed-by: Joshua Schmid <jschmid@suse.de>

Merge pull request #34322 from cbodley/wip-44857

rgw: pubsub sync module ignores ERR_USER_EXIST

Reviewed-by: Yuval Lifshitz <yuvalif@yahoo.com>

Merge pull request #34134 from zhangdaolong/fix_incorrect_log_info

pybind/rbd: fix no lockers are obtained, ImageNotFound exception will be output

Merge PR #34320 into master

* refs/pull/34320/head:
cephadm: ceph-volume: disallow concurrent execution

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #34096 from smanjara/wip-dynamic-resharding

rgw: groundwork for supporting dynamic resharding in multisite environment

Reviewed-by: Casey Bodley <cbodley@redhat.com>

crimson/osd: OP_CALL does support RETURNVEC now.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #34345 from tchaikov/wip-crimson-less-chatty

crimson: do not warn() under expected circumstances

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: do not use "warn()" when handling non-error

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: do not print error log at seeing unknown connection

it happens if a client or an peer osd drops the connection, so it's not
an error and hence we should not print this error message using
"error()".

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34341 from guits/update_cv_deploy

ceph-volume: update functional testing deploy.yml playbook

Merge pull request #34331 from zealoussnow/wip-fix-clone-error

test/common/unittest_blkdev: delete unused test file

Reviewed-by: Kefu Chai <kchai@redhat.com>

unittest_blkdev: delete unused test file

Signed-off-by: Leo Zhang <nguzcf@gmail.com>

doc/mgr/orchestrator: Update `Placement by pattern matching`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/cephadm/drivegroup: Don't introduce DriveGroups

We already have a name for it: a Service Specification of
type `osd`. We don't need to introduce a new name for it.

Well, they are "DriveGroups", but users don't need to know it.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/cephadm/drivegroups: Update to new yaml

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/mgr/orchestrator: use yaml syntax highlighting

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/mgr/orchestrator: Add "Service Specification"

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

ceph-volume: update functional testing deploy.yml playbook

This commit adds a call to `ceph-facts` role in the first play of this
playbook. This is needed so `ceph-validate` won't fail because of
following error:

```
fatal: [osd0]: FAILED! => {}

MSG:

'osd_pool_default_size' is undefined
```

`osd_pool_default_size` is set in ceph-facts.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

pybind/rbd: fix no lockers are obtained, ImageNotFound exception will be output

No lockers are obtained, ImageNotFound exception will be output,
but tht image is always exist.when lockers number is zero,
Should not output any exceptions。

Fixes: https://tracker.ceph.com/issues/44613
Signed-off-by: zhangdaolong <zhangdaolong@fiberhome.com>

Merge pull request #34232 from bk201/wip-44743

qa/tasks/mgr/dashboard/test_rbd: wait longer when purging

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

modified get_target_shard_id() to take bucket_index_normal_layout directly

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw: pubsub sync module ignores ERR_USER_EXIST

otherwise, this error gets returned by RGWPSDataSyncModule::start_sync()
and data sync fails to start

Fixes: https://tracker.ceph.com/issues/44857
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34291 from dillaman/wip-44814

librbd: fix client backwards compatibility issues

Merge pull request #34295 from sebastian-philipp/raise-for-invalid-dg

python-common: raise on empty drive selections

Reviewed-by: Joshua Schmid <jschmid@suse.de>

cephadm: ceph-volume: disallow concurrent execution

Fixes: https://tracker.ceph.com/issues/44820
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #34026 from votdev/issue_44539_shorten_container_id

mgr/dashboard: shorten `Container ID` and `Container image ID` in Services page

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

qa/tasks/mgr/dashboard/test_rbd: wait longer when purging

Fixes: https://tracker.ceph.com/issues/44743
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>
Co-authored-by: Kefu Chai <kefu@redhat.com>
Co-authored-by: Volker Theile <vtheile@suse.com>

Removed redundant encoding of layout.* and adjusted the corresponding decoding

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

Moved ostream operator overloading to BucketIndexType() to rgw_bucket_layout header file

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

mgr/dashboard: shorten `Container ID` and `Container image ID` in Services page

Fixes: https://tracker.ceph.com/issues/44539
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #34259 from rzarzynski/wip-crimson-fix-bad-methods

crimson/osd: fix method's flag handling on CEPH_OSD_OP_CALL.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34305 from tchaikov/wip-crimson-stddev

test/crimson: increase variance of stdev to 0.20

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #34167 from javacruft/32bit-fixes-march

mds: resolve type mismatches on 32 bit archs

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

test/crimson: increase variance of stdev to 0.20

per Mark Nelson,

> yeah, 5% variation is way too low
> Sometimes we can stay within 5%, but especially if we are pushing the
> system hard we can see closer to 10-20% sometimes.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34112 from trociny/wip-44610

pybind/rbd: ensure image is open before permitting operations

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34019 from changchengx/blockguard

librbd: refine BlockGuard debug output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #34150 into master

* refs/pull/34150/head:
cephadm: rm-cluster clean up /etc/ceph

Reviewed-by: Sage Weil <sage@redhat.com>

python-common: raise on emtpy drive groups

adds a test for https://tracker.ceph.com/issues/44758

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

librbd: re-add support for nautilus clients talking to jewel clusters

We want to support N - 3 client backward compatibility (special case
to support Jewel since it was a LTS release). The "get_snapshot_timestamp"
cls method does not exist in Jewel clusters so librbd should fallback
to excluding the op if it fails.

Note that this N - 3 also needs to apply for downstream releases as well,
which implies we still need Jewel for the time being.

Fixes: http://tracker.ceph.com/issues/39450
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c644121820b83c97e68f9896393a45cd34787672)

Conflicts:
src/test/librbd/image/test_mock_RefreshRequest.cc: tweaked to support pool configs

librbd: ignore -EOPNOTSUPP errors when retrieving image group membership

The Luminous release did not support adding images to a group (it only
included the bare-minimum support for creating groups). Commit f76df32666b
incorrectly dropped support for ignoring this possible failure. This
prevents Nautilus-release clients from opening images contained within
a Luminous-release cluster.

Fixes: http://tracker.ceph.com/issues/38834
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6f29dc69a0db3b6d982c95ab9d3b9b00a7029e37)

Merge pull request #34253 from cbodley/wip-qa-rgw-too-many-shards

qa/rgw: remove 1999-shard tests from rgw/verify

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #33557 from yuvalif/add_2pc_cls_queue

cls/queue: add 2-phase-commit queue implementation

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #33933 from cbodley/wip-rgw-doc-reshard-v2

doc/rgw: update multisite reshard design for cross-datalog-shard coordination

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge PR #34260 into master

* refs/pull/34260/head:
Revert "Merge PR #34091 into master"

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #34248 into master

* refs/pull/34248/head:
qa/tasks/cephadm: no need to explicitly reconfig
qa/tasks/cephadm: fetch final ceph.conf
qa/tasks/cephadm: distribute ceph.conf and admin keyring to all nodes

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge PR #34266 into master

* refs/pull/34266/head:
mgr/DaemonServer: fetch metadata for new daemons (e.g., mons)

Reviewed-by: Neha Ojha <nojha@redhat.com>

rgw: replace RGWBucketInfo's sharding-related fields with BucketLayout

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

rgw: add BucketLayout to RGWBucketInfo

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: use qualified encode/decode in putobj processor

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34279 from votdev/issue_44805

mgr/dashboard: 'Prometheus / All Alerts' page shows progress bar

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: 'Prometheus / All Alerts' page shows progress bar

Fixes: https://tracker.ceph.com/issues/44805
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #34276 from badone/wip-fix-argument-ordering-docker-test

tests: Reinstate "opts" as final argument

Reviewed-by: Kefu Chai <kchai@redhat.com>

tests: Reinstate "opts" as final argument

a1e8f61cb72 broke the "opts" argument.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #34252 from JackSlateur/doc-os-recommendations2

doc/start/os-recommendations.rst: recommend bluestore

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34267 from JackSlateur/doc-man-monmaptool

doc/man/8/monmaptool.rst: update synopsis etc with the latest options

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/man/8/monmaptool.rst: update the page with the latest options

Signed-off-by: Alexandre Bruyelles <jack@jack.fr.eu.org>

Merge pull request #34224 from markhpc/wip-bluefs-disable-buffered-io

common/options: Disable bluefs_buffered_io by default again.

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #33052 from ifed01/wip-ifed-fix-unused2

os/bluestore: fix 'unused' calculation

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #34203 from majianpeng/bluefs-remove-unused-func

os/bluestore: remove unused func.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #34061 into master

* refs/pull/34061/head:
mgr/orch: Add `ceph orch ls --export`
mgr/dashboard: adapt to new `ServiceDescription.to_json()`
python-common: add `service_name` to `ServiceSpec.to_json`
python-common: make ServiceSpec and ServiceDescription compatible
src/ceph.in: add yaml to known formats
mgr/orch: add yaml to `orch ls`
mgr/orch: remove `orch spec dump`
python-common: reorder RGWSpec arguments
python-common: prevent ServiceSpec of wrong type
pybind/mgr: tox.ini: omit cov report
mgr/cephadm: test describe_service
mgr/orch: ServiceDescription: change json representation
mgr/orch: ServiceDescription: Make spec a requirement

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #34215 into master

* refs/pull/34215/head:
cephadm: Fix trucated output of `mgr dump`

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #34268 from poyea/docs-options-1

doc: Fix inconsistency in logging settings

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: Fix inconsistency in logging settings

This patch fixes inconsistency in logging settings with options, namely `log_flush_on_exit` and `log_to_stderr`. This patch also adds `log_to_file` to the section.

Signed-off-by: John Law <johnlaw.po@gmail.com>

mgr/DaemonServer: fetch metadata for new daemons (e.g., mons)

We fetch metadata for mon/osd/mds daemons on startup.  If a new one comes
along *after* we start up, we need to fetch it on demand.  Otherwise,
notably, we will ignore any new mons added to the cluster:

- mon.a starts
- mgr starts, fetchs a's metadata
- mon.b added
- mon.b tries top open mgr connection, and is rejected each time

This can lead to follow-on badness when the mon tries to proxy mgr
commands or do other things.

Fix by fetching metadata on demand, like we already do in the report
path.

Fixes: https://tracker.ceph.com/issues/44798
Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #34062 into master

* refs/pull/34062/head:
cephadm: update check-host() to return all problems

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #33943 from xxhdx1985126/wip-crimson-alienstore-bugs

crimson: fix bugs that come up when osds go through down/up

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/osd: fix method's flag handling on CEPH_OSD_OP_CALL.

This patch fixes a regression introduced in d8d0ab1a and
makes `ClsHello.BadMethods` green again.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

cephadm: Fix trucated output of `mgr dump`

In case the process has terminated, we
have to make sure to read till EOF, otherwise
we might skip some bytes.

Fixes: https://tracker.ceph.com/issues/44642
Co-authored-by: Tim Serong <tserong@suse.com>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

qa/rgw: remove 1999-shard tests from rgw/verify

causing test failures because the host runs out of disk space

Signed-off-by: Casey Bodley <cbodley@redhat.com>

doc/start/os-recommendations.rst: recommend bluestore

Signed-off-by: Alexandre Bruyelles <jack@jack.fr.eu.org>

Merge PR #34247 into master

* refs/pull/34247/head:
mgr/dashboard: do not fail on user creation

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #33279 from rishabh-d-dave/fs-move-run_shell-and-related-methods

qa/cephfs: move run_shell to mount.py and add methods for negative testing

Merge pull request #34021 from lxbsz/deleg_XXX

test/libcephfs: retry and wait for caps when setting the RD deleg

Merge pull request #33494 from lxbsz/mount

mount.ceph: stop printing warning message for unrecognized mount options

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #33740 from lxbsz/qa_test_timeout_failed

qa/test_config_session_timeout: fix options conflicting

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #34024 from vshankar/wip-ignore-mgrmap-when-laggy

mds: do not defer incoming mgrmap when mds is laggy

Reviewed-by: "Yan, Zheng" <zyan@redhat.com>

Revert "Merge PR #34091 into master"

This reverts commit f865f3e0a0f3a646b093b3571ea76713eca1916c, reversing
changes made to 7ef5458e26ec7c0565509a7882fa31fa064eb49d.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/cephadm: no need to explicitly reconfig

cephadm does this magically now.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/cephadm: fetch final ceph.conf

Otherwise we'll distribute the initial ceph.conf, which only has the
bootstrap mon in it.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tasks/cephadm: distribute ceph.conf and admin keyring to all nodes

Revert part of 96220c0c0574eb5b896023e1552f528bef9e1ca5 so that we still
distribute a *final* ceph.conf and admin keyring to all nodes, right after
all of the mons are up.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: do not fail on user creation

Accordingly to other Ceph commands do not fail on user
creation with a non-zero error code if the user already
exists.
Instead succeed and return the message 'User <username>
already exists'.

Fixes: https://tracker.ceph.com/issues/44502
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

cephadm: rm-cluster clean up /etc/ceph

rm-cluster command will delete config, keyring, and pub key files from /etc/ceph

Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>

Merge pull request #33992 from rhcs-dashboard/44626-fix-notifications-e2e-tests

mgr/dashboard: fix notifications E2E tests

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34209 from smithfarm/wip-doc-multi-cluster-follow-on-fix

doc: fix cephadm link

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #34235 from toabctl/add-sle-15-sp2

qa/distros: add SLE-15-SP2

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #32749 from james58899/fix-capacity

monitoring: Fix pool capacity incorrect

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #34233 from tchaikov/wip-qa-py3

qa/tasks: change to be py3 compatible and cleanups

Reviewed-by: Thomas Bechtold <tbechtold@suse.com>

qa/distros: add SLE-15-SP2

Ceph octopus is known to run on SLE-15-SP2 so add it to
qa/distros/all.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>

cmake: add import-tasks to run-tox-qa test

to ensure that qa suites are python3 compatible

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/mgr: use relative import

for better readability, and to ease the pain of developer to track back
to the top level python package for referencing a submodule

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/tasks/cephfs: fix print() to be py3 compatible

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #33949 from toabctl/import-tasks-with-py3

qa: Add a tox env that can test importing files

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34141 from tchaikov/wip-test-threadsafe

test/objectstore: set "threadsafe" flag for ASSERT_DEATH tests

Reviewed-by: Neha Ojha <nojha@redhat.com>

crimson: fix pglog read_log_and_missing issue

Currently pglog's read_log_and_missing_crimson's implementation
would cause null pointer exception, since at the time, we don't the
key to start from

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson: add necessary FuturizedStore APIs for data recovery

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

crimson: make sure pg meta is valid when loading pg

currently, PGMeta instances are all local variables in method,
which would lead to daggling reference/pointer problems when using
seastar::futures.

Signed-off-by: Xuehan Xu <xxhdx1985126@163.com>

common/options: Disable bluefs_buffered_io by default again.

Signed-off-by: Mark Nelson <mnelson@redhat.com>

Merge PR #32093 into master

* refs/pull/32093/head:
doc/start: Update hardware recommendations.

Reviewed-by: Sage Weil <sage@redhat.com>