]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
3 years agorgw: improve bucket index listing cls logging 45087/head
J. Eric Ivancich [Wed, 25 Aug 2021 13:52:31 +0000 (09:52 -0400)]
rgw: improve bucket index listing cls logging

Adding more level 20 logging for bucket index listing.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
3 years agorgw: clean-up logging of function entering to make thorough and consistent
J. Eric Ivancich [Mon, 10 May 2021 21:36:49 +0000 (17:36 -0400)]
rgw: clean-up logging of function entering to make thorough and consistent

This provides more thorough and consistent function tracing in CLS/RGW
when logging is set to 10 or higher.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
3 years agorgw: fix bucket index listing count bug
J. Eric Ivancich [Wed, 25 Aug 2021 13:50:29 +0000 (09:50 -0400)]
rgw: fix bucket index listing count bug

Fix bugs surrounding calculation of number of entries returned and
whether the end of a listing range has been reached.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
3 years agotest/rgw: fix and add to rgw/cls bi_list tests
J. Eric Ivancich [Tue, 17 Aug 2021 21:35:19 +0000 (17:35 -0400)]
test/rgw: fix and add to rgw/cls bi_list tests

Make sure marker is cleared. Put end-of-list check inside the
conditional with the rest of the test. Add some additional testing.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 1bf058199a13783e227c0e87ea6f3244f3560cfd)

3 years agorgw: fix bucket index list minor calculation bug
J. Eric Ivancich [Wed, 12 Jan 2022 18:41:42 +0000 (13:41 -0500)]
rgw: fix bucket index list minor calculation bug

When "bucket index list" traverses the different regions in the bucket
index assembling the output, it miscalculates how many entries to ask
for at one point. This fixes that.

This fixes previous "rgw: bucket index list can produce I/O errors".

Credit for finding this bug goes to Soumya Koduri <skoduri@redhat.com>.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit aa7605151f0a5f467d50f13f27c7aef42a40cc39)

3 years agorgw: allow ordered bucket listing to work when many filtered out entries
J. Eric Ivancich [Mon, 19 Jul 2021 18:24:11 +0000 (14:24 -0400)]
rgw: allow ordered bucket listing to work when many filtered out entries

A previous PR moved the much of the filtering that's part of bucket
listing to the CLS layer. One unanticipated result was that it is now
possible for a call to return 0 entries. In such a case we want to
retry the call with the marker moved forward (i.e., advanced),
repeatedly if necessary, in order to either retrieve some entries or
to hit the end of the entries. This PR adds that functionality.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 423c18308a35f0f2d6c7697539301cf7001d6329)

3 years agorgw: allow CLSRGWConcurrentIO to handle "advancing" retries
J. Eric Ivancich [Mon, 19 Jul 2021 18:23:42 +0000 (14:23 -0400)]
rgw: allow CLSRGWConcurrentIO to handle "advancing" retries

When doing an asynchronous/concurrent bucket index operation against
multiple bucket index shards, a special error code is set aside to
indicate that an "advancing" retry of a/some shard(s) is necessary. In
that case another asynchronous call is made on the indicated shard(s)
from the client (i.e., CLSRGWConcurrentIO).  It is up to the subclass
of CLSRGWConcurrentIO to handle the retry such that it "advances" and
simply doesn't get stuck, looping forever.

The retry functionality only works when the "need_multiple_rounds"
functionality is not in use.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 5d283074750dc6bd458877bd42921037b5bb7f4b)

3 years agorgw: de-conflate shard_id and request_id in CLSRGWConcurrentIO
J. Eric Ivancich [Fri, 16 Jul 2021 19:31:35 +0000 (15:31 -0400)]
rgw: de-conflate shard_id and request_id in CLSRGWConcurrentIO

When using asynchronous (concurrent) IO for bucket index requests,
there are two int ids that are used that need to be kept separate --
shard id and request id. In many cases they're the same -- shard 0
gets request 0, and so forth.

But in preparation for re-requests, those ids can diverge, where
request 13 maps to shard 2. The existing code maintained the OIDs that
went with each request. This PR also maintains the shard id as
well. Documentation has been beefed up to help future developers
navigate this.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 9606346592dfd6261aa2daa4cbec56f9a72c65fc)

3 years agorgw: bucket index list produces incorrect result when non-ascii entries
J. Eric Ivancich [Tue, 13 Jul 2021 19:36:53 +0000 (15:36 -0400)]
rgw: bucket index list produces incorrect result when non-ascii entries

A recent PR that helped address the issue of non-ascii plain entries
didn't cover all the bases, allowing I/O errors to be produced in some
circumstances during a bucket index list (i.e., `radosgw-admin bi list
...`).

This fixes those issue and does some additional clean-up.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit e714f0dbaf251472d41ccbd682c4d92c32ea5fac)
(cherry picked from commit 3473bf325508a145d65ed013c119370bfae4f940)

3 years agoMerge pull request #45002 from idryomov/wip-krbd-rxbounce-option-pacific
Ilya Dryomov [Mon, 28 Feb 2022 11:20:50 +0000 (12:20 +0100)]
Merge pull request #45002 from idryomov/wip-krbd-rxbounce-option-pacific

pacific: rbd: recognize rxbounce map option

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #45003 from idryomov/wip-54128-pacific
Ilya Dryomov [Mon, 28 Feb 2022 11:19:00 +0000 (12:19 +0100)]
Merge pull request #45003 from idryomov/wip-54128-pacific

pacific: krbd: return error when no initial monitor address found

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #44897 from aclamk/wip-53480-pacific
Yuri Weinstein [Fri, 25 Feb 2022 22:57:15 +0000 (14:57 -0800)]
Merge pull request #44897 from aclamk/wip-53480-pacific

pacific: crush: Fix segfault in update_from_hook

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44847 from kamoltat/wip-ksirivad-pacific-backport-44241
Yuri Weinstein [Fri, 25 Feb 2022 22:56:47 +0000 (14:56 -0800)]
Merge pull request #44847 from kamoltat/wip-ksirivad-pacific-backport-44241

pacific: mon: osd pool create <pool-name> with --bulk flag

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44613 from ifed01/wip-ifed-fix-ram-gridy-fsck-pac
Yuri Weinstein [Fri, 25 Feb 2022 22:43:52 +0000 (14:43 -0800)]
Merge pull request #44613 from ifed01/wip-ifed-fix-ram-gridy-fsck-pac

pacific: os/bluestore: make shared blob fsck much less RAM-greedy.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
3 years agoMerge pull request #43313 from idryomov/wip-keyring-resolve-error-pacific
Yuri Weinstein [Fri, 25 Feb 2022 22:40:04 +0000 (14:40 -0800)]
Merge pull request #43313 from idryomov/wip-keyring-resolve-error-pacific

pacific: auth,mon: don't log "unable to find a keyring" error when key is given

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #44975 from vshankar/wip-54241
Yuri Weinstein [Fri, 25 Feb 2022 18:05:53 +0000 (10:05 -0800)]
Merge pull request #44975 from vshankar/wip-54241

pacific: mds: ignore unknown client op when tracking op latency

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
3 years agoMerge pull request #45100 from vshankar/wip-54217
Yuri Weinstein [Fri, 25 Feb 2022 16:23:31 +0000 (08:23 -0800)]
Merge pull request #45100 from vshankar/wip-54217

pacific: mds: kill session when mds do ms_handle_remote_reset

Reviewed-by: Xiubo Li <xiubli@redhat.com>
3 years agoMerge pull request #45099 from vshankar/wip-54220
Yuri Weinstein [Fri, 25 Feb 2022 16:22:05 +0000 (08:22 -0800)]
Merge pull request #45099 from vshankar/wip-54220

pacific: mds: fix seg fault in expire_recursive

Reviewed-by: Nikhilkumar Shelke <nshelke@redhat.com>
3 years agoMerge pull request #45016 from Vicente-Cheng/wip-54194-pacific
Yuri Weinstein [Fri, 25 Feb 2022 16:21:09 +0000 (08:21 -0800)]
Merge pull request #45016 from Vicente-Cheng/wip-54194-pacific

pacific: mds: mds_oft_prefetch_dirfrags default to false

Reviewed-by: Venky Shankar vshankar@redhat.com
3 years agoMerge pull request #44910 from batrick/i54161
Yuri Weinstein [Fri, 25 Feb 2022 16:19:52 +0000 (08:19 -0800)]
Merge pull request #44910 from batrick/i54161

pacific: mon/MDSMonitor: sanity assert when inline data turned on in MDSMap from v16.2.4 -> v16.2.[567]

Reviewed-by: Venky Shankar vshankar@redhat.com
3 years agoMerge pull request #44504 from MrFreezeex/wip-53761-pacific
Yuri Weinstein [Fri, 25 Feb 2022 16:19:15 +0000 (08:19 -0800)]
Merge pull request #44504 from MrFreezeex/wip-53761-pacific

pacific: qa/suites/fs: add prefetch_dirfrags false to thrasher suite

Reviewed-by: Venky Shankar vshankar@redhat.com
3 years agoMerge pull request #42932 from kotreshhr/wip-52384-pacific
Yuri Weinstein [Fri, 25 Feb 2022 16:18:28 +0000 (08:18 -0800)]
Merge pull request #42932 from kotreshhr/wip-52384-pacific

pacific: mgr/volumes: Subvolume removal and clone failure fixes

Reviewed-by: Venky Shankar vshankar@redhat.com
3 years agoMerge pull request #44869 from neha-ojha/wip-mgr-fixes-pacific
Yuri Weinstein [Thu, 24 Feb 2022 20:44:15 +0000 (12:44 -0800)]
Merge pull request #44869 from neha-ojha/wip-mgr-fixes-pacific

pacific: mgr: various fixes for mgr scalability

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Adam King adking@redhat.com
Reviewed-by: Nizamudeen A <nia@redhat.com>
3 years agoMerge pull request #45123 from zdover23/wip-hardware-recs-backport-2022-02-23-pacific
zdover23 [Wed, 23 Feb 2022 23:16:40 +0000 (09:16 +1000)]
Merge pull request #45123 from zdover23/wip-hardware-recs-backport-2022-02-23-pacific

doc/start: remove journal info from hardware recs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
3 years agoMerge pull request #45075 from chrisphoffman/wip-54298-pacific
Yuri Weinstein [Wed, 23 Feb 2022 17:55:57 +0000 (09:55 -0800)]
Merge pull request #45075 from chrisphoffman/wip-54298-pacific

pacific: cls/rbd: GroupSnapshotNamespace comparator violates ordering rules

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
3 years agoMerge pull request #45072 from idryomov/wip-qemu-task-rbd-package-pacific
Yuri Weinstein [Wed, 23 Feb 2022 16:54:08 +0000 (08:54 -0800)]
Merge pull request #45072 from idryomov/wip-qemu-task-rbd-package-pacific

pacific: qa/tasks/qemu: make sure block-rbd.so is installed

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #44629 from sebastian-philipp/pacific-backport-44367
Adam King [Wed, 23 Feb 2022 05:18:47 +0000 (00:18 -0500)]
Merge pull request #44629 from sebastian-philipp/pacific-backport-44367

pacific: mgr/cephadm: Fix count for OSDs with OSD specs

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #44632 from sebastian-philipp/pacific-backport-44306
Adam King [Tue, 22 Feb 2022 22:51:27 +0000 (17:51 -0500)]
Merge pull request #44632 from sebastian-philipp/pacific-backport-44306

pacific: cephadm: deal with ambiguity within normalize_image_digest

Reviewed-by: Adam King <adking@redhat.com>
3 years agodoc/start: remove journal info from hardware recs 45123/head
Zac Dover [Sun, 5 Dec 2021 14:44:09 +0000 (00:44 +1000)]
doc/start: remove journal info from hardware recs

This PR removes mentions of journaling from the hardware
recommendations.

Journaling was a FileStore-related practice. BlueStore is
the default backend for Ceph OSDs and has been since
Luminous. The documentation should reflect that.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
(cherry picked from commit 276bbd8f2b516993813fedfdfd1371e87bb0d44a)

3 years agomgr/cephadm: Fix count for OSDs with OSD specs 44629/head
Sebastian Wagner [Mon, 20 Dec 2021 15:08:07 +0000 (16:08 +0100)]
mgr/cephadm: Fix count for OSDs with OSD specs

osd counting is special

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
Conflicts:
src/pybind/mgr/cephadm/module.py
src/pybind/mgr/cephadm/tests/test_cephadm.py

3 years agoMerge pull request #44996 from pcfe/patch-1
Adam King [Mon, 21 Feb 2022 18:46:58 +0000 (13:46 -0500)]
Merge pull request #44996 from pcfe/patch-1

pacific: doc/cephadm: Add CentOS Stream install instructions

Reviewed-by: Adam King <adking@redhat.com>
3 years agocephadm: count number of docker.io occurrences. 44632/head
Sebastian Wagner [Tue, 14 Dec 2021 12:33:27 +0000 (13:33 +0100)]
cephadm: count number of docker.io occurrences.

We have to be super careful adding any new hardcoded
regstistry names to the code base. Let's make this an informed
decision next time.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 0fe2e54db774271e4fc18b45aba36b66cbc71779)

3 years agomgr/cephadm: improve doc strings for ambigous config values
Sebastian Wagner [Tue, 14 Dec 2021 12:09:35 +0000 (13:09 +0100)]
mgr/cephadm: improve doc strings for ambigous config values

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 391ddf15530d4063d68b03dce1da9999490930d6)

3 years agocephadm: sync normalize_image_digest with mgr/cephadm
Sebastian Wagner [Tue, 14 Dec 2021 12:08:38 +0000 (13:08 +0100)]
cephadm: sync normalize_image_digest with mgr/cephadm

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit 84bef14999e0422cf3a2c456c8972211303f91f1)

3 years agomgr/cephadm: normalize_image_digest: reworked
Sebastian Wagner [Tue, 14 Dec 2021 12:05:59 +0000 (13:05 +0100)]
mgr/cephadm: normalize_image_digest: reworked

Reworked to only care about known short names. I coudn't
find any algorithm, as this problem is ambiguous.

Fixes: https://tracker.ceph.com/issues/53594
Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit a48b8344bd9ebb47c23e589f2c7ecdd18a6f3085)

3 years agomgr/cephadm: normalize_image_digest: comment to tests
Sebastian Wagner [Tue, 14 Dec 2021 11:32:05 +0000 (12:32 +0100)]
mgr/cephadm: normalize_image_digest: comment to tests

`pytest --doctest` ftw. Be aware those tests actually fail, as the
implementation contradicts the function documentation.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
(cherry picked from commit dc19e3a867b015d8f4c6f5128603d71be3ce5b0d)

3 years agoMerge pull request #44882 from adk3798/pacific-validate-yumdnf-baseurl
Adam King [Mon, 21 Feb 2022 18:36:24 +0000 (13:36 -0500)]
Merge pull request #44882 from adk3798/pacific-validate-yumdnf-baseurl

pacific: cephadm: validate that the constructed YumDnf baseurl is usable

Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #44880 from adk3798/pacific-shared-ceph-ceph-volume
Adam King [Mon, 21 Feb 2022 18:34:53 +0000 (13:34 -0500)]
Merge pull request #44880 from adk3798/pacific-shared-ceph-ceph-volume

pacific: cephadm: add shared_ceph_folder opt to ceph-volume subcommand

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoMerge pull request #44877 from adk3798/pacific-parse-ipv6-route
Adam King [Mon, 21 Feb 2022 18:33:01 +0000 (13:33 -0500)]
Merge pull request #44877 from adk3798/pacific-parse-ipv6-route

pacific: cephadm: _parse_ipv6_route: Fix parsing ifs w/o route

Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #44661 from sebastian-philipp/pacific-backport-44600
Adam King [Mon, 21 Feb 2022 18:31:46 +0000 (13:31 -0500)]
Merge pull request #44661 from sebastian-philipp/pacific-backport-44600

pacific: mgr/cephadm: auto-enable mirroring module when deploying service

Reviewed-by: Michael Fritch <mfritch@suse.com>
3 years agoMerge pull request #44658 from sebastian-philipp/pacific-backport-43889
Adam King [Mon, 21 Feb 2022 18:30:32 +0000 (13:30 -0500)]
Merge pull request #44658 from sebastian-philipp/pacific-backport-43889

pacific: mgr/cephadm: store contianer registry credentials in config-key

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #45046 from mgfritch/backport-44905-pacific
Adam King [Mon, 21 Feb 2022 18:29:33 +0000 (13:29 -0500)]
Merge pull request #45046 from mgfritch/backport-44905-pacific

pacific: cephadm: chown the prometheus data dir during redeploy

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
3 years agoMerge pull request #44742 from idryomov/wip-rbd-switch-arguments-fix-pacific
Deepika Upadhyay [Mon, 21 Feb 2022 10:14:01 +0000 (15:44 +0530)]
Merge pull request #44742 from idryomov/wip-rbd-switch-arguments-fix-pacific

pacific: rbd: add missing switch arguments for recognition by get_command_spec()

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #44690 from MrFreezeex/wip-53938-pacific
Deepika Upadhyay [Mon, 21 Feb 2022 10:13:16 +0000 (15:43 +0530)]
Merge pull request #44690 from MrFreezeex/wip-53938-pacific

pacific: cls/journal: skip disconnected clients when calculating min_commit_position

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #44094 from MrFreezeex/wip-53386-pacific
Deepika Upadhyay [Mon, 21 Feb 2022 10:12:51 +0000 (15:42 +0530)]
Merge pull request #44094 from MrFreezeex/wip-53386-pacific

pacific: rbd-mirror: make RemoveImmediateUpdate test synchronous

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
3 years agoMerge pull request #44754 from idryomov/wip-rbd-mirror-delprop-races-pacific
Deepika Upadhyay [Mon, 21 Feb 2022 10:12:07 +0000 (15:42 +0530)]
Merge pull request #44754 from idryomov/wip-rbd-mirror-delprop-races-pacific

pacific: rbd-mirror: fix races in snapshot-based mirroring deletion propagation

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
3 years agoMerge pull request #44998 from idryomov/wip-53419-pacific
Deepika Upadhyay [Mon, 21 Feb 2022 10:09:50 +0000 (15:39 +0530)]
Merge pull request #44998 from idryomov/wip-53419-pacific

pacific: librbd: fix various memory leaks

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #44999 from idryomov/wip-52522-pacific
Deepika Upadhyay [Mon, 21 Feb 2022 10:09:28 +0000 (15:39 +0530)]
Merge pull request #44999 from idryomov/wip-52522-pacific

pacific: librbd: fix use-after-free on ictx in list_descendants()

Reviewed-by: Mykola Golub <mgolub@suse.com>
3 years agoMerge pull request #45005 from sunnyku/wip-54168-pacific
Deepika Upadhyay [Mon, 21 Feb 2022 10:08:33 +0000 (15:38 +0530)]
Merge pull request #45005 from sunnyku/wip-54168-pacific

pacific: mgr/rbd_support: fix schedule remove

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
3 years agomds: kill session when mds do ms_handle_remote_reset 45100/head
IvanGuan [Tue, 18 Jan 2022 13:01:59 +0000 (21:01 +0800)]
mds: kill session when mds do ms_handle_remote_reset

if the mds decide to reuse the old connection it will
do reset_session and should  also kill the session
which are open state in MDSDaemon::ms_handle_remote_reset
to prevent the situation client session is stuck in
opening state and never has chance to becaome open.

the root cause is client missed the request_open
reply but the mds session has become open already.
so we should kill the session in mds side and let
mds recreate the session when received the connect
request from client.

Fixes: http://tracker.ceph.com/issues/53911
Signed-off-by: YunfeiGuan <yunfeiguan@xtaotech.com>
(cherry picked from commit 3651deb4e0b0c102adcaddce79ee4e053f033418)

3 years agoMerge pull request #45008 from idryomov/wip-rbd-help-positional-optional-pacific
Deepika Upadhyay [Mon, 21 Feb 2022 10:01:42 +0000 (15:31 +0530)]
Merge pull request #45008 from idryomov/wip-rbd-help-positional-optional-pacific

pacific: rbd: mark optional positional arguments as such in help output

Reviewed-by: Sunny Kumar <sunkumar@redhat.com>
3 years agomds: fix seg fault in expire_recursive 45099/head
胡玮文 [Thu, 6 Jan 2022 07:43:29 +0000 (15:43 +0800)]
mds: fix seg fault in expire_recursive

Range-based for should not be used when we are altering the container.
Use iterator explicitly instead.

Fixes: https://tracker.ceph.com/issues/53805
Signed-off-by: 胡玮文 <huww98@outlook.com>
(cherry picked from commit d48a2cf7e2481cf9758f2934464ec6d9c35d898b)

3 years agoMerge pull request #45044 from votdev/wip-54284-pacific
Volker Theile [Fri, 18 Feb 2022 18:22:08 +0000 (19:22 +0100)]
Merge pull request #45044 from votdev/wip-54284-pacific

3 years agocls/rbd: GroupSnapshotNamespace comparator violates ordering rules 45075/head
Ilya Dryomov [Mon, 14 Feb 2022 12:04:00 +0000 (13:04 +0100)]
cls/rbd: GroupSnapshotNamespace comparator violates ordering rules

For

  GroupSnapshotNamespace a(1, "group-1", "snap-2");
  GroupSnapshotNamespace b(1, "group-2", "snap-1");

both a < b and b < a evaluate to true.  This violates STL strict weak
ordering requirements which is a problem because GroupSnapshotNamespace
is used as a key in std::map (ictx->snap_ids at least), etc.

Fixes: https://tracker.ceph.com/issues/49792
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 830e72ab9d66c8f5703ea27da5249b02dd16ccd0)

3 years agoqa/suites/rbd: make sure block-rbd.so is installed 45072/head
Ilya Dryomov [Wed, 16 Feb 2022 09:32:26 +0000 (10:32 +0100)]
qa/suites/rbd: make sure block-rbd.so is installed

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 8f0fd0af3da8581c47dc916303615264714a0489)

3 years agoqa/tasks/qemu: make sure block-rbd.so is installed
Ilya Dryomov [Tue, 15 Feb 2022 13:57:51 +0000 (14:57 +0100)]
qa/tasks/qemu: make sure block-rbd.so is installed

Fixes: https://tracker.ceph.com/issues/54286
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 525ff61cfc8516b4d7bed6f819b00a0b6cb7be0a)

3 years agomgr/dashboard: "Please expand your cluster first" shouldn't be shown if cluster is... 45044/head
Volker Theile [Thu, 17 Feb 2022 08:11:54 +0000 (09:11 +0100)]
mgr/dashboard: "Please expand your cluster first" shouldn't be shown if cluster is already meaningfully running

This PR will assume that a cluster is already up and fully running. If this should not be the expected behaviour, deployment tools have to set 'INSTALLED' explicitly. Without this assumption it might happen that upgraded and fully running clusters, e.g. Octopus -> Pacific, will show the 'Expand Cluster' on first log in.

cephadm will take care that the bootstrap phase will write the necessary key to show the 'Expand cluster' page.

Fixes: https://tracker.ceph.com/issues/54215
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit 48fff60b63785ec07f71d3e59394b0c08357247c)

Conflicts:
src/cephadm/cephadm

Signed-off-by: Volker Theile <vtheile@suse.com>
3 years agomgr/cephadm: add a test for enabling cephfs mirroring module 44661/head
John Mulligan [Tue, 18 Jan 2022 18:31:03 +0000 (13:31 -0500)]
mgr/cephadm: add a test for enabling cephfs mirroring module

Add a test that checks that when cephfs mirror service is enabled
the mirroring mgr module gets enabled.

Actually-written-by: Sebastian Wagner <sewagner@redhat.com>
Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit bcb4fa70f9f739dce3e1c111db0a322804350f9d)

Conflicts:
src/pybind/mgr/cephadm/tests/test_services.py

3 years agomgr/cephadm: auto-enable mirroring module when deploying service
John Mulligan [Thu, 13 Jan 2022 19:01:51 +0000 (14:01 -0500)]
mgr/cephadm: auto-enable mirroring module when deploying service

Automatically enable the mgr's mirroring module when creating
cephfs-mirror services. This will trigger a mgr respawn.

Fixes: https://tracker.ceph.com/issues/50593
Based roughly on 50dc1d0decb2fbf7b9129bddba940969410be5cd

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit e030130fd1d680ebf9d2c6c9f95f44109fbf3036)

3 years agomgr/cephadm: store contianer registry credentials in config-key 44658/head
Daniel Pivonka [Wed, 10 Nov 2021 16:18:06 +0000 (11:18 -0500)]
mgr/cephadm: store contianer registry credentials in config-key

Signed-off-by: Daniel Pivonka <dpivonka@redhat.com>
(cherry picked from commit 2c29286b370b1079a35e0e0d47c33a7dabab3468)

Conflicts:
src/pybind/mgr/cephadm/module.py
src/pybind/mgr/cephadm/serve.py

3 years agoMerge pull request #44725 from cfsnyder/wip-53703-pacific
Yuri Weinstein [Wed, 16 Feb 2022 23:15:19 +0000 (15:15 -0800)]
Merge pull request #44725 from cfsnyder/wip-53703-pacific

pacific: osdc: add set_error in BufferHead, when split set_error to right

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
3 years agoMerge pull request #44865 from cbodley/wip-54090
Yuri Weinstein [Wed, 16 Feb 2022 21:21:53 +0000 (13:21 -0800)]
Merge pull request #44865 from cbodley/wip-54090

pacific: qa: remove centos8 from supported distros

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44545 from cfsnyder/wip-53718-pacific
Yuri Weinstein [Wed, 16 Feb 2022 21:20:44 +0000 (13:20 -0800)]
Merge pull request #44545 from cfsnyder/wip-53718-pacific

pacific: osd/OSDMapMapping: fix spurious threadpool timeout errors

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44879 from adk3798/pacific-co-locate-doc
Adam King [Wed, 16 Feb 2022 20:12:09 +0000 (15:12 -0500)]
Merge pull request #44879 from adk3798/pacific-co-locate-doc

pacific: doc/cephadm: Co-location of daemons

Reviewed-by: Michael Fritch mfritch@suse.com
3 years agoMerge pull request #44146 from k0ste/wip-53435-pacific
Adam King [Wed, 16 Feb 2022 20:07:18 +0000 (15:07 -0500)]
Merge pull request #44146 from k0ste/wip-53435-pacific

pacific: mgr/prometheus: Fix the per method stats exported

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #44830 from adk3798/pacific-rm-iscsi
Adam King [Wed, 16 Feb 2022 20:04:57 +0000 (15:04 -0500)]
Merge pull request #44830 from adk3798/pacific-rm-iscsi

pacific: mgr/cephadm/iscsi: use `mon_command` in `post_remove` instead of `check_mon_command`

Reviewed-by: Michael Fritch mfritch@suse.com
3 years agoMerge pull request #44829 from adk3798/pacific-cpu-limit
Adam King [Wed, 16 Feb 2022 20:03:26 +0000 (15:03 -0500)]
Merge pull request #44829 from adk3798/pacific-cpu-limit

pacific: mgr/cephadm: allow miscellaneous container args at service level

Reviewed-by: Michael Fritch mfritch@suse.com
3 years agoMerge pull request #44637 from sebastian-philipp/pacific-backport-44566
Adam King [Wed, 16 Feb 2022 20:00:47 +0000 (15:00 -0500)]
Merge pull request #44637 from sebastian-philipp/pacific-backport-44566

pacific: mgr/cephadm: fix minor grammar nit in Dry-Runs message

Reviewed-by: Michael Fritch mfritch@suse.com
3 years agoMerge pull request #44634 from sebastian-philipp/pacific-backport-44498
Adam King [Wed, 16 Feb 2022 19:59:31 +0000 (14:59 -0500)]
Merge pull request #44634 from sebastian-philipp/pacific-backport-44498

pacific: cephadm: check if cephadm is root after cli is parsed

Reviewed-by: John Mulligan <jmulligan@redhat.com>
3 years agoMerge pull request #44633 from sebastian-philipp/pacific-backport-44394
Adam King [Wed, 16 Feb 2022 19:57:27 +0000 (14:57 -0500)]
Merge pull request #44633 from sebastian-philipp/pacific-backport-44394

pacific: Enable autotune for osd_memory_target on bootstrap

Reviewed-by: Melissa Li melissali@redhat.com
3 years agoMerge pull request #44631 from sebastian-philipp/pacific-backport-44482
Adam King [Wed, 16 Feb 2022 19:56:22 +0000 (14:56 -0500)]
Merge pull request #44631 from sebastian-philipp/pacific-backport-44482

pacific: mgr/cephadm: nfs migration: avoid port conflicts

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #44630 from sebastian-philipp/pacific-backport-43833
Adam King [Wed, 16 Feb 2022 19:55:06 +0000 (14:55 -0500)]
Merge pull request #44630 from sebastian-philipp/pacific-backport-43833

pacific: mgr/cephadm: fix tcmu-runner cephadm_stray_daemon

Reviewed-by: Melissa Li melissali@redhat.com
3 years agoMerge pull request #44628 from sebastian-philipp/pacific-backport-44374
Adam King [Wed, 16 Feb 2022 19:53:05 +0000 (14:53 -0500)]
Merge pull request #44628 from sebastian-philipp/pacific-backport-44374

pacific: mgr/cephadm: fix 'mgr/cephadm: spec.virtual_ip  param should be used by the ingress daemon

Reviewed-by: Adam King <adking@redhat.com>
3 years agoMerge pull request #44723 from cfsnyder/wip-53608-pacific
Yuri Weinstein [Wed, 16 Feb 2022 19:07:41 +0000 (11:07 -0800)]
Merge pull request #44723 from cfsnyder/wip-53608-pacific

pacific: os/bluestore: avoid premature onode release.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
3 years agoMerge pull request #44721 from cfsnyder/wip-53535-pacific
Yuri Weinstein [Wed, 16 Feb 2022 19:07:07 +0000 (11:07 -0800)]
Merge pull request #44721 from cfsnyder/wip-53535-pacific

pacific: mon/MgrStatMonitor: do not spam subscribers (mgr) with service_map

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agoMerge pull request #44698 from cfsnyder/wip-53942-pacific
Yuri Weinstein [Wed, 16 Feb 2022 19:05:56 +0000 (11:05 -0800)]
Merge pull request #44698 from cfsnyder/wip-53942-pacific

pacific: mon/OSDMonitor: avoid null dereference if stats are not available

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agopybind/mgr/devicehealth/module.py: use NotifyType enum 44869/head
Neha Ojha [Mon, 14 Feb 2022 22:37:57 +0000 (22:37 +0000)]
pybind/mgr/devicehealth/module.py: use NotifyType enum

master/quincy use a precreated .mgr pool and does not
need this commit

Signed-off-by: Neha Ojha <nojha@redhat.com>
3 years agomgr: only queue notify events that modules ask for
Sage Weil [Thu, 2 Dec 2021 15:23:45 +0000 (10:23 -0500)]
mgr: only queue notify events that modules ask for

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit ee4e3ecd6e0754854c00f3dfa0eaaa17bbf3603d)

3 years agopybind/mgr: annotate which events modules consume
Sage Weil [Thu, 2 Dec 2021 15:23:22 +0000 (10:23 -0500)]
pybind/mgr: annotate which events modules consume

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 95ca3a444505b5bd38f4961ed63760ac45abcf64)

 Conflicts:
src/pybind/mgr/mds_autoscaler/module.py - trivial resolution

3 years agopybind/mgr: introduce NotifyType enum
Sage Weil [Thu, 2 Dec 2021 15:22:48 +0000 (10:22 -0500)]
pybind/mgr: introduce NotifyType enum

Note that we don't annotate the dashboard NotificationQueue because it is
used internally by the dashboard with other events.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 1ac480df45058e103a2f4954950b8a4edb23610a)

 Conflicts:
src/pybind/mgr/cephadm/module.py - trivial resolution
src/pybind/mgr/dashboard/module.py - trivial resolution
src/pybind/mgr/localpool/module.py - trivial resolution
src/pybind/mgr/mds_autoscaler/module.py - trivial resolution

3 years agomgr: stop issuing events that no modules consume
Sage Weil [Thu, 2 Dec 2021 15:19:16 +0000 (10:19 -0500)]
mgr: stop issuing events that no modules consume

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit d3c8f171e9aaa9cbf970b6dadda5837ca9b030c7)

3 years agomgr: limit changes to pg_num
Sage Weil [Tue, 30 Nov 2021 23:18:31 +0000 (18:18 -0500)]
mgr: limit changes to pg_num

We need to avoid making drastic changes to pg_num that outpace pgp_num or
else we will may hit the per-osd pg limits.

Fixes: https://tracker.ceph.com/issues/53442
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 3b2a11249aff6ee608efc95212d6723df180cd07)

 Conflicts:
src/common/options/mgr.yaml.in - old way of specifying config settings

3 years agomgr/progress: avoid inefficient dump of all pg stats
Sage Weil [Fri, 3 Dec 2021 18:48:32 +0000 (13:48 -0500)]
mgr/progress: avoid inefficient dump of all pg stats

We only use a handful of fields, and the pg dump includes a gazillion
fields that we waste CPU copying to python-land.  This tends to lead to
long ClusterState::lock hold times, leading to long ms_dispatch delays
and generally gumming up the works.

Instead, create a new "pg_progress" item that dumps only the fields that
mgr/progress needs.

Fixes: https://tracker.ceph.com/issues/53475
Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit f5973ccef415571c72560a04968592e8a6daf93a)

3 years agopybind/mgr/mgr_module: cache mgr_ip
Sage Weil [Tue, 23 Nov 2021 17:32:40 +0000 (12:32 -0500)]
pybind/mgr/mgr_module: cache mgr_ip

This does not change for the lifetime of an active mgr module.  No need to
keep calling back into Mgr to re-fetch it.

Signed-off-by: Sage Weil <sage@newdream.net>
(cherry picked from commit 994832e8e50c3566e21928f6be61e45a4411e556)

 Conflicts:
src/pybind/mgr/mgr_module.py - self._db_lock = threading.Lock() DNE in pacific

3 years agoMerge pull request #45028 from rhcs-dashboard/wip-54269-pacific
Ernesto Puerta [Wed, 16 Feb 2022 16:16:47 +0000 (17:16 +0100)]
Merge pull request #45028 from rhcs-dashboard/wip-54269-pacific

pacific: mgr/dashboard: Directories Menu Can't Use on Ceph File System Dashboard

Reviewed-by: Sarthak0702 <NOT@FOUND>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #45052 from rhcs-dashboard/wip-54295-pacific
Ernesto Puerta [Wed, 16 Feb 2022 16:06:19 +0000 (17:06 +0100)]
Merge pull request #45052 from rhcs-dashboard/wip-54295-pacific

pacific: mgr/dashboard: change privacy protocol field from required to optional

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44747 from cybozu/wip-53070-pacific
Daniel Gryniewicz [Wed, 16 Feb 2022 15:12:36 +0000 (10:12 -0500)]
Merge pull request #44747 from cybozu/wip-53070-pacific

pacific: rgw: fix lock scope in ObjectCache::get()

3 years agoMerge pull request #44750 from rhcs-dashboard/wip-53987-pacific
Ernesto Puerta [Wed, 16 Feb 2022 13:06:34 +0000 (14:06 +0100)]
Merge pull request #44750 from rhcs-dashboard/wip-53987-pacific

pacific: mgr: TTL Cache in mgr module

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: neha-ojha <NOT@FOUND>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agomgr/dashboard: add validation for snmp v3 engine id 45052/head
Avan Thakkar [Tue, 15 Feb 2022 13:13:36 +0000 (18:43 +0530)]
mgr/dashboard: add validation for snmp v3 engine id

Fixes: https://tracker.ceph.com/issues/54270
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
(cherry picked from commit 2866db1eac7d726201f5bb34abdb32981c783f0e)

3 years agomgr/dashboard: change privacy protocol field from required to optional
Avan Thakkar [Mon, 14 Feb 2022 12:18:39 +0000 (17:48 +0530)]
mgr/dashboard: change privacy protocol field from required to optional

Fixes: https://tracker.ceph.com/issues/54270
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Privacy protocol field shouldn't be a required field.

(cherry picked from commit 2d8f2b8195a0f0c7a21d4ec5061b1b51a3aade2c)

3 years agoMerge pull request #44543 from cfsnyder/wip-53659-pacific
Brad Hubbard [Wed, 16 Feb 2022 00:56:12 +0000 (10:56 +1000)]
Merge pull request #44543 from cfsnyder/wip-53659-pacific

pacific: mon: prevent new sessions during shutdown

Reviewed-by: Neha Ojha <nojha@redhat.com>
3 years agocephadm: chown the prometheus data dir during redeploy 45046/head
Michael Fritch [Fri, 4 Feb 2022 18:11:05 +0000 (11:11 -0700)]
cephadm: chown the prometheus data dir during redeploy

some builds of prometheus run with a uid 65534 (nobody) where other
builds of prometheus run with a uid of 0 (root)

Fixes: https://tracker.ceph.com/issues/54159
Signed-off-by: Michael Fritch <mfritch@suse.com>
(cherry picked from commit 21fb80aaab0b333d997d8241e17cf9749a37e065)

3 years agodoc/cephadm: Add CentOS Stream install instructions 44996/head
Patrick C. F. Ernzer [Sun, 13 Feb 2022 01:05:44 +0000 (02:05 +0100)]
doc/cephadm: Add CentOS Stream install instructions

Tested on CentOS Stream 9

Signed-off-by: Patrick C. F. Ernzer <pcfe@pcfe.net>.
3 years agocephadm/ingress: make frontend stat bind on localhost 44628/head
Guillaume Abrioux [Fri, 11 Feb 2022 16:39:18 +0000 (17:39 +0100)]
cephadm/ingress: make frontend stat bind on localhost

The current configuration of keepalived makes it do
a curl on localhost:9999 in order to check the endpoint is alive.
Given the endpoint only binds on the vip addr, that doesn't work.

Fixes: https://tracker.ceph.com/issues/53807
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
3 years agoMerge pull request #44756 from votdev/wip-53929-pacific
Ernesto Puerta [Tue, 15 Feb 2022 10:56:50 +0000 (11:56 +0100)]
Merge pull request #44756 from votdev/wip-53929-pacific

pacific: mgr/dashboard: Notification banners at the top of the UI have fixed height

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
3 years agoMerge pull request #44938 from rhcs-dashboard/wip-54091-pacific
Ernesto Puerta [Tue, 15 Feb 2022 10:54:55 +0000 (11:54 +0100)]
Merge pull request #44938 from rhcs-dashboard/wip-54091-pacific

pacific: mgr/dashboard: cephadm e2e job improvements

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44935 from rhcs-dashboard/pacific-test-applitools
Ernesto Puerta [Tue, 15 Feb 2022 10:53:55 +0000 (11:53 +0100)]
Merge pull request #44935 from rhcs-dashboard/pacific-test-applitools

pacific: mgr/dashboard: set appropriate baseline branch for applitools

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44876 from aaSharma14/wip-53930-pacific
Ernesto Puerta [Tue, 15 Feb 2022 10:53:19 +0000 (11:53 +0100)]
Merge pull request #44876 from aaSharma14/wip-53930-pacific

pacific: mgr/dashboard: Improve notifications for osd nearfull, full

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
3 years agoMerge pull request #44789 from guits/wip-54021-pacific
Guillaume Abrioux [Mon, 14 Feb 2022 20:17:59 +0000 (21:17 +0100)]
Merge pull request #44789 from guits/wip-54021-pacific

pacific: ceph-volume: improve mpath devices support

3 years agoMerge pull request #44884 from guits/wip-54127-pacific
Guillaume Abrioux [Mon, 14 Feb 2022 19:09:22 +0000 (20:09 +0100)]
Merge pull request #44884 from guits/wip-54127-pacific

pacific: ceph-volume: fix error 'KeyError' with inventory