]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
15 months agoqa/cephadm: only fail on CEPHADM_ error in logs 56897/head
Adam King [Mon, 8 Apr 2024 19:11:02 +0000 (15:11 -0400)]
qa/cephadm: only fail on CEPHADM_ error in logs

Rather than failing for any instance of
[ERR], [WRN], or [SEC]. The orch/cephadm suite
does a lot of stuff that can cause these various
warnings to breifly appear. Trying to catch all
cases has been difficult and the suite has been
red for some time. This patch makes it so it
instead only matches log messages that
include CEPHADM_ on top of having [ERR], [WRN],
or [SEC] as those warnings have been the ones
that have actually lead us to cephadm bugs, while
the others are pretty much always just noise in
these tests. This patch does not apply this
to the mds_upgrade_sequence, nfs, or rbd-iscsi
sections as those are symlinked from other suites
and I didn't want to affect those suites tests
directly with this change.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit b4522dd332d40a54b9e0be58bd96aeaa345f8977)

Conflicts:
qa/suites/orch/cephadm/smb/tasks/deploy_smb_basic.yaml
qa/suites/orch/cephadm/smb/tasks/deploy_smb_domain.yaml
qa/suites/orch/cephadm/smoke/start.yaml
qa/suites/orch/cephadm/workunits/task/test_host_drain.yaml
qa/suites/orch/cephadm/workunits/task/test_monitoring_stack_basic.yaml
qa/suites/orch/cephadm/workunits/task/test_rgw_multisite.yaml
qa/suites/orch/cephadm/workunits/task/test_set_mon_crush_locations.yaml

15 months agoqa/tasks/cephadm: add option to limit what matches in log error scraping
Adam King [Mon, 8 Apr 2024 18:27:26 +0000 (14:27 -0400)]
qa/tasks/cephadm: add option to limit what matches in log error scraping

This is specifically being added with the orch/cephadm suite
in mind, where coming up with a viable ignorelist has proved
difficult. The orch testing does a lot of actions that can
cause thigns like an OSD or MON daemon to be down very
briefly, and I've found the vast majority of the time we
really don't want to fail the test when these pop up as cephadm
testing really only benefits from catching the CEPHADM_ errors/
warnings rather than eveyr possible one. Rather than continuing to
play whack-a-mole with the errors in the logs, this
patch should allow us to limit what we fail on to at
least get the suite in a good spot again. We can always
phase out the uses of this new "log-only_match" option
later in a more controlled way, and adding it shouldn't
affect log scraping for any of the tests that aren't
facing a similar issue.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 443c5913c5247349845a860dba8f6b69def5de75)

15 months agoMerge pull request #56861 from rhcs-dashboard/wip-65458-squid
Ivo Almeida [Mon, 15 Apr 2024 08:13:07 +0000 (09:13 +0100)]
Merge pull request #56861 from rhcs-dashboard/wip-65458-squid

squid: mgr/dashboard: fix snap schedule delete retention

Reviewed-by: afreen23 <NOT@FOUND>
15 months agoMerge pull request #56871 from zdover23/wip-doc-2024-04-14-backport-56870-to-squid
Anthony D'Atri [Sat, 13 Apr 2024 21:50:59 +0000 (17:50 -0400)]
Merge pull request #56871 from zdover23/wip-doc-2024-04-14-backport-56870-to-squid

squid: doc/cephfs: disambiguate "Reporting Free Space"

15 months agodoc/cephfs: disambiguate "Reporting Free Space" 56871/head
Zac Dover [Sat, 13 Apr 2024 10:48:21 +0000 (20:48 +1000)]
doc/cephfs: disambiguate "Reporting Free Space"

Make a few sentences clearer in doc/cephfs/ceph-auth.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 788ccff18149b16cb01947c5bcf3bde380090df0)

15 months agoMerge pull request #56670 from cbodley/wip-65292-squid
Casey Bodley [Fri, 12 Apr 2024 18:01:41 +0000 (19:01 +0100)]
Merge pull request #56670 from cbodley/wip-65292-squid

squid: rgw/pubsub: CreateTopic validates topic name

Reviewed-by: Yuval Lifshitz <ylifshit@ibm.com>
15 months agoMerge pull request #56069 from cbodley/wip-64818-squid
Casey Bodley [Fri, 12 Apr 2024 17:54:18 +0000 (18:54 +0100)]
Merge pull request #56069 from cbodley/wip-64818-squid

squid: rgw: replicate v2 topic/notification metadata

Reviewed-by: Adam Emerson <aemerson@redhat.com>
15 months agomgr/dashboard: fix snap schedule delete retention 56861/head
Ivo Almeida [Mon, 8 Apr 2024 13:39:49 +0000 (14:39 +0100)]
mgr/dashboard: fix snap schedule delete retention

Fixes: https://tracker.ceph.com/issues/65370
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>
(cherry picked from commit 985637825a48a678b0d88887c5e322de65599fc7)

15 months agoMerge pull request #56856 from guits/update-cv-func-tests-squid
Guillaume Abrioux [Fri, 12 Apr 2024 14:19:04 +0000 (16:19 +0200)]
Merge pull request #56856 from guits/update-cv-func-tests-squid

ceph-volume: update functional testing

15 months agoMerge pull request #56850 from zdover23/wip-doc-2024-04-12-backport-56831-to-squid
Anthony D'Atri [Fri, 12 Apr 2024 14:04:10 +0000 (10:04 -0400)]
Merge pull request #56850 from zdover23/wip-doc-2024-04-12-backport-56831-to-squid

squid: doc/cephfs: refine client-auth (3 of 3)

15 months agoMerge pull request #56750 from ronen-fr/wip-rf-unreserved-squid
Ronen Friedman [Fri, 12 Apr 2024 12:17:43 +0000 (15:17 +0300)]
Merge pull request #56750 from ronen-fr/wip-rf-unreserved-squid

squid:osd/scrub: disable scrub reservation queuing

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
15 months agoceph-volume: update functional testing 56856/head
Guillaume Abrioux [Thu, 11 Apr 2024 07:43:47 +0000 (09:43 +0200)]
ceph-volume: update functional testing

various changes for fixing ceph-volume functional testing.
cleaning up deploy.yml (drop py2 references)

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit 90bfaaa2f0ab7e28c0f78189ccb83635e944e758)

15 months agosquid:osd/scrub: disable scrub reservation queuing 56750/head
Ronen Friedman [Sun, 7 Apr 2024 18:18:50 +0000 (13:18 -0500)]
squid:osd/scrub: disable scrub reservation queuing

as a temporary measure for Squid RC0. There are known problems in the
existing code, with patch PRs being tested. This change disables the
Reserver functionality, until such time as the patches are merged.

Note the following trackers:
- https://tracker.ceph.com/issues/65044
- https://tracker.ceph.com/issues/65072

Once these are solved, this commit should be reverted.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
15 months agoMerge pull request #56766 from ronen-fr/wip-65374-squid
Ronen Friedman [Fri, 12 Apr 2024 09:09:33 +0000 (12:09 +0300)]
Merge pull request #56766 from ronen-fr/wip-65374-squid

squid: tests/scripts: use 'tell pg deep-scrub pgid' instead of 'tell pgid deep-scrub'

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
15 months agodoc/cephfs: refine client-auth (3 of 3) 56850/head
Zac Dover [Thu, 11 Apr 2024 11:18:53 +0000 (21:18 +1000)]
doc/cephfs: refine client-auth (3 of 3)

Refine the final third of doc/cephfs/client-auth.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 44d1eeed703e968b374ec4d12fff3a2466a4c276)

15 months agoMerge pull request #56841 from zdover23/wip-doc-2024-04-12-backport-56784-to-squid
Anthony D'Atri [Fri, 12 Apr 2024 03:12:13 +0000 (23:12 -0400)]
Merge pull request #56841 from zdover23/wip-doc-2024-04-12-backport-56784-to-squid

doc/cephfs: refine client-auth (2 of 3)

15 months agodoc/cephfs: refine client-auth (2 of 3) 56841/head
Zac Dover [Tue, 9 Apr 2024 12:04:46 +0000 (22:04 +1000)]
doc/cephfs: refine client-auth (2 of 3)

Refine the second third of doc/cephfs/client-auth.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 083f56352b4fff662dff3c78b6a0c455605d02ff)

15 months agotest/rgw/notification: fix threading issue in test 56069/head
Yuval Lifshitz [Tue, 9 Apr 2024 10:00:08 +0000 (10:00 +0000)]
test/rgw/notification: fix threading issue in test

test_ps_s3_persistent_topic_stats may delete boto client threads
before they are done.
also, no need to run the http server twice

Fixes: https://tracker.ceph.com/issues/63909
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit 51e365927361db19b2ae2b270bab7f30fe3e51ee)

15 months agotest/rgw/notification: use polling instead of sleep
Yuval Lifshitz [Thu, 4 Apr 2024 17:14:31 +0000 (17:14 +0000)]
test/rgw/notification: use polling instead of sleep

when checking persistent stats queue length

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit 5581bc7d14f4c5814a7d43597aa77cacd6a2d125)

15 months agotest/rgw/notification: cleanup leftover topics
Yuval Lifshitz [Thu, 4 Apr 2024 16:25:33 +0000 (16:25 +0000)]
test/rgw/notification: cleanup leftover topics

this may happen when a test fails, and does not cleanup topics
it created. other tests that verify the number of topics may fail
because of that.
all tests that verify number of topics, should delete all topics at the
start of the test.

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit 9ae0de17e1a8c3a0527b04981eeb7e632da985ac)

15 months agotest/rgw/notification: use simpler multithreaded http server
Yuval Lifshitz [Mon, 25 Mar 2024 11:11:31 +0000 (11:11 +0000)]
test/rgw/notification: use simpler multithreaded http server

Fixes: https://tracker.ceph.com/issues/63909
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit 673adcbdbd01e64c8b76c3176e062571fb8710ac)

15 months agotest/rgw/notifications: do not delay http responses to notifications
Yuval Lifshitz [Tue, 20 Feb 2024 10:12:26 +0000 (10:12 +0000)]
test/rgw/notifications: do not delay http responses to notifications

we also increase the wating time after rerunning the http server
to allow fo the persistent notifications retries to finish

Fixes: https://tracker.ceph.com/issues/63909
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit ca6688696b9ac9f838ffe21951331b6a1974dae8)

15 months agorgw/notification: Load bucket attrs before calling publish_reserve.
kchheda3 [Thu, 4 Apr 2024 18:30:40 +0000 (14:30 -0400)]
rgw/notification: Load bucket attrs before calling publish_reserve.

As part of PR# 55657, publish_reserve would reload bucket to ensure bucket_attrs are loaded. However for lc events, where the bucket attrs were already loaded, the reloading was causing crash but there was no obvious root cause, so to avoid the crashes, remove reloading of bucket in publish_reserve and put the onus on callers to load the bucket before calling publish_reserve.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit fa5d370663804db9752c99e79659aae031a7bf85)

15 months agotest/rgw/noitifications: run basic/http/migration tests with 2 RGWs
Yuval Lifshitz [Mon, 18 Mar 2024 16:44:39 +0000 (16:44 +0000)]
test/rgw/noitifications: run basic/http/migration tests with 2 RGWs

this is useful to:
* cover the cls_lock code with persistent notifications
* cover v1 to v2 migration collisions between RGWs

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit c0e540e6f11aa32b347ea6e7e9f03cad5d1913e8)

15 months agorgw/notifications: run v2 migration tests in teuthology
Yuval Lifshitz [Thu, 14 Mar 2024 12:21:19 +0000 (12:21 +0000)]
rgw/notifications: run v2 migration tests in teuthology

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
(cherry picked from commit 67d43ae2538a611373159346855dab8701e176fb)

15 months agoRGW/Rados: Migrate topics to data path v2
Ali Masarwa [Tue, 9 Jan 2024 21:51:54 +0000 (23:51 +0200)]
RGW/Rados: Migrate topics to data path v2

also add migration tests

Signed-off-by: Ali Masarwa <amasarwa@redhat.com>
(cherry picked from commit 1a7d1454801e3d7b888aa734c4e3b609febf417f)

15 months agorgw/notification: Support generating multisite sync delete events.
kchheda3 [Tue, 27 Feb 2024 20:11:48 +0000 (15:11 -0500)]
rgw/notification: Support generating multisite sync delete events.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit 37069ac96b53d2d4927b9d6f743790847861ec8e)

15 months agorgw/notification: Make the Replication events to be aws compatible.
kchheda3 [Tue, 27 Feb 2024 19:24:43 +0000 (14:24 -0500)]
rgw/notification: Make the Replication events to be aws compatible.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit 3ea110bb89a7f666de25f8a128fa8bbb231e304f)

15 months agorgw/notification: Make the Lifecycle events AWS compatible
kchheda3 [Tue, 27 Feb 2024 19:06:47 +0000 (14:06 -0500)]
rgw/notification: Make the Lifecycle events AWS compatible

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit ea6611327c9aa678c524e0d71e954bd735d60789)

15 months agoPendingReleaseNotes: announce the notification_v2 feature and its migration
Casey Bodley [Wed, 13 Mar 2024 22:47:35 +0000 (18:47 -0400)]
PendingReleaseNotes: announce the notification_v2 feature and its migration

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4f0ff0f80ba0ef6273c31345e93244ccc37cfa69)

15 months agodoc/radosgw: document the notification_v2 zone feature
Casey Bodley [Wed, 13 Mar 2024 22:35:40 +0000 (18:35 -0400)]
doc/radosgw: document the notification_v2 zone feature

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 30d97250f945e0afdfe9ca52a97a42922091d4dd)

15 months agodoc/radosgw: generalize description of Zone Features
Casey Bodley [Wed, 13 Mar 2024 22:20:55 +0000 (18:20 -0400)]
doc/radosgw: generalize description of Zone Features

so they make sense outside of multisite configurations

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0a97eb89e1d9c99e96fd215e9dc31febc2e44de5)

15 months agodoc/radosgw: move the Zone Features section to its own page
Casey Bodley [Wed, 13 Mar 2024 22:12:42 +0000 (18:12 -0400)]
doc/radosgw: move the Zone Features section to its own page

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d5ac275694bce35d37d7226b2f63934e3c277cbd)

15 months agoqa/rgw: notification suite creates a realm
Casey Bodley [Thu, 7 Mar 2024 23:07:38 +0000 (18:07 -0500)]
qa/rgw: notification suite creates a realm

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7ef94cdb1c97f537561a7c8bb226d9d22ff709a8)

15 months agoqa/rgw: teach rgw.py to create an optional realm
Casey Bodley [Wed, 6 Mar 2024 19:04:46 +0000 (14:04 -0500)]
qa/rgw: teach rgw.py to create an optional realm

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4ac0b7df3c15140bc9d904a1e12fac482c7130be)

15 months agorgw/notifications: delete persistent queue only if topic is deleted
Yuval Lifshitz [Tue, 13 Feb 2024 16:36:51 +0000 (16:36 +0000)]
rgw/notifications: delete persistent queue only if topic is deleted

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 666e79f1fb78fe8128791e9e23159571f76cfe70)

15 months agorgw/notifications: handle migration state between v1 and v2
Yuval Lifshitz [Sat, 10 Feb 2024 16:38:30 +0000 (16:38 +0000)]
rgw/notifications: handle migration state between v1 and v2

test instructions:
https://gist.github.com/yuvalif/21449e301732b719cd1ed97c3eeeabb2

* during migration all topic and notification operations must fail with HTTP error code 503
* read operations should return the values of the v1 topics and notifications
* sending notifications should continue based on v1 values

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit bcd79d23df63e72839c652cbc20ebb67a024ba85)

15 months agorgw: move rgw_rest_pubsub.h out of driver/rados/
Casey Bodley [Tue, 13 Feb 2024 15:57:08 +0000 (10:57 -0500)]
rgw: move rgw_rest_pubsub.h out of driver/rados/

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5b2ae726a95c9922a5c5ba984d6e170dd115128b)

15 months agorgw/pubsub: topic apis forward post body
Casey Bodley [Mon, 12 Feb 2024 21:44:00 +0000 (16:44 -0500)]
rgw/pubsub: topic apis forward post body

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9d46c21bc6e522b1d85056e1e545125743aace65)

15 months agorgw/topic: metadata handler adds/removes persistent queues
Casey Bodley [Thu, 18 Jan 2024 20:34:16 +0000 (15:34 -0500)]
rgw/topic: metadata handler adds/removes persistent queues

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 633a5bef314b5fbc3087271d684f22f9beafb21a)

15 months agorgw/topic: remove unused RGWSI_Topic_RADOS
Casey Bodley [Wed, 10 Jan 2024 22:40:11 +0000 (17:40 -0500)]
rgw/topic: remove unused RGWSI_Topic_RADOS

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit be9cc0c7bc163f4e578bd5459866a2c1e5c8a2dd)

15 months agorgw/topic: add rgwrados::topic interface for topic metadata
Casey Bodley [Wed, 10 Jan 2024 20:33:25 +0000 (15:33 -0500)]
rgw/topic: add rgwrados::topic interface for topic metadata

add a new interface for topic metadata that doesn't depend on metadata
backends. this low-level interface is used by both RadosStore and the
topic metadata handler

remove Driver::delete_bucket_topic_mapping() from sal because the omap
object is deleted internally by rgwrados::topic::remove()

remove the RGWRados::topics_pool_ctx member

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6d1d036afb2d1624674fef43f2e70ef3b3ae2859)

15 months agorgw/topic: metadata key format in rgw_pubsub.h
Casey Bodley [Wed, 10 Jan 2024 22:07:27 +0000 (17:07 -0500)]
rgw/topic: metadata key format in rgw_pubsub.h

the format of topic metadata keys is agnostic to the backend, so the
parsing/formatting functions should be in rgw_pubsub.h

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1caf954dab136e7d5cff97f76ce362c1bfb8d3f4)

15 months agorgw: paginate ListTopics
Casey Bodley [Tue, 9 Jan 2024 23:55:40 +0000 (18:55 -0500)]
rgw: paginate ListTopics

rename read_topics()/write_topics() to 'v1' and only call them from
internal v1 call paths

public get_topics() now calls read_topics_v1() for the v1 case, and does
the paginated listing with driver->meta_list_keys_next() for v2

RGWPSListTopicsOp now uses the NextToken request/response params with
the paginated get_topics(), limiting responses to 100 entries like AWS

'radosgw-admin topic list' also paginates the listing according to
--max-entries to avoid reading everything into memory at once

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit db6c73a0cdcf60a920c91b6d4506df36d98b7308)

15 months agorgw/mdlog: add complete_entry() for most common usage
Casey Bodley [Fri, 5 Jan 2024 15:26:00 +0000 (10:26 -0500)]
rgw/mdlog: add complete_entry() for most common usage

metadata sync ignores entries other than MDLOG_STATUS_COMPLETE, so we
don't need to write separate prepare/complete entries. metadata
mutations can just call complete_entry() on success

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7827b2527f6b656331ae337ebc196c17b2716e48)

15 months agorgw/metadata: add RGWMetadataLister abstraction
Casey Bodley [Sun, 3 Dec 2023 19:10:30 +0000 (14:10 -0500)]
rgw/metadata: add RGWMetadataLister abstraction

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 6000732706890fa5d276d22972d1d83be32bb5c8)

15 months agorgw: RGWPubSub requires SiteConfig
Casey Bodley [Sun, 4 Feb 2024 20:17:47 +0000 (15:17 -0500)]
rgw: RGWPubSub requires SiteConfig

RGWPubSub constructor takes SiteConfig instead of zonegroup map

replace do_all_zonegroups_support_notification_v2() with a generic
function rgw::all_zonegroups_support() that handles non-realm
configurations too

remove unused sal::ZoneGroup::supports_feature()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 1fccdad21d5bcdebbdd7167b7ed18391a1e8d1e2)

15 months agorgw/rados: RGWTopicMetadataHandler doesn't depend on RGWPubSub
Casey Bodley [Sun, 4 Feb 2024 19:04:11 +0000 (14:04 -0500)]
rgw/rados: RGWTopicMetadataHandler doesn't depend on RGWPubSub

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0b8b62bed0e79dd54d1baac240695b2a398d407f)

15 months agorgw/rados: store SiteConfig with RGWServices
Casey Bodley [Sun, 4 Feb 2024 18:43:10 +0000 (13:43 -0500)]
rgw/rados: store SiteConfig with RGWServices

make the SiteConfig available to all of RGWRados via svc.site instead
of storing it in sal::RadosStore

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0057bb7d0f02974f3a8cb1d34e73ead8d92c3062)

15 months agoradosgw-admin: make SiteConfig available to commands
Casey Bodley [Sun, 4 Feb 2024 16:37:57 +0000 (11:37 -0500)]
radosgw-admin: make SiteConfig available to commands

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 0f333f1cc5a265015d4d436e1275ffadc2d0d8d4)

15 months agorgw/multisite-notification: allow enabling v1/v2 in tests
Yuval Lifshitz [Thu, 28 Dec 2023 12:05:09 +0000 (12:05 +0000)]
rgw/multisite-notification: allow enabling v1/v2 in tests

v1 could be enabled only in local tests. teuthology tests would run with v2

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit d2db9df224659cf711aee80479b5d462ff3ca716)

15 months agotest/rgw/notifications: support running tests in multisite environment
Yuval Lifshitz [Thu, 21 Dec 2023 18:18:45 +0000 (18:18 +0000)]
test/rgw/notifications: support running tests in multisite environment

both locally and in teuthology

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 7502215d70a423e642e5147a0d444f627a7979f2)

15 months agorgw/multisite-notification: Add integration test for bucket notifications in multisit...
kchheda3 [Mon, 11 Dec 2023 20:47:30 +0000 (15:47 -0500)]
rgw/multisite-notification: Add integration test for bucket notifications in multisite config.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit 1ed8df24ae0f279a6e7d294231b5e6c1e45fb663)

15 months agorgw/multisite-notification: Add omap object to store the mapping between bucket and...
kchheda3 [Thu, 21 Dec 2023 22:23:05 +0000 (17:23 -0500)]
rgw/multisite-notification: Add omap object to store the mapping between bucket and topics.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit df668e9f65cec50227f671eb2e624c0ee02f2c37)

15 months agorgw/multisite-notification: Add support to replicate bucket notifications in multisit...
kchheda3 [Tue, 5 Dec 2023 17:26:48 +0000 (12:26 -0500)]
rgw/multisite-notification: Add support to replicate bucket notifications in multisite config.

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit ab74e2cfe4e3738b85d38a132f835d1dca113258)

15 months agorgw/multisite-notification: Add support to replicate topic creation in multisite...
kchheda3 [Mon, 27 Nov 2023 20:51:02 +0000 (15:51 -0500)]
rgw/multisite-notification: Add support to replicate topic creation  in multisite config

Signed-off-by: kchheda3 <kchheda3@bloomberg.net>
(cherry picked from commit c4a147d079cb9f268340e5f79ec5350a94296658)

15 months agotest/rgw/notifications: split tests between basic, kafka and amqp
Yuval Lifshitz [Thu, 22 Feb 2024 09:58:00 +0000 (09:58 +0000)]
test/rgw/notifications: split tests between basic, kafka and amqp

see comment: https://tracker.ceph.com/issues/64184#note-3

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
(cherry picked from commit 6f28930602a2dfdcd402537c1a274df1836109ce)

15 months agoMerge pull request #56778 from epuertat/wip-65368-squid
Ernesto Puerta [Wed, 10 Apr 2024 08:55:38 +0000 (10:55 +0200)]
Merge pull request #56778 from epuertat/wip-65368-squid

squid: install-deps: enable copr ceph/grpc

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56779 from zdover23/wip-doc-2024-04-09-backport-56754-to-squid
Anthony D'Atri [Tue, 9 Apr 2024 21:26:56 +0000 (17:26 -0400)]
Merge pull request #56779 from zdover23/wip-doc-2024-04-09-backport-56754-to-squid

squid: doc/cephfs: refine client-auth (1 of 3)

15 months agoMerge pull request #56797 from zdover23/wip-doc-2024-04-10-backport-56796-to-squid
Anthony D'Atri [Tue, 9 Apr 2024 20:59:01 +0000 (16:59 -0400)]
Merge pull request #56797 from zdover23/wip-doc-2024-04-10-backport-56796-to-squid

squid: doc/mgr: remove Zabbix 1 information

15 months agodoc/mgr: remove Zabbix 1 information 56797/head
Zac Dover [Tue, 9 Apr 2024 20:04:28 +0000 (06:04 +1000)]
doc/mgr: remove Zabbix 1 information

Remove information about the installation of the Zabbix module and link
to a discussion of the reasoning behind Ceph's refusal to support
Zabbix.

John Jasen developed a procedure explaining how to install "Zabbix 2".
This commit removes outdated procedures and explains why those
procedures were removed. Immediately following this explanation, the
text includes an explanation of how to install "Zabbix 2".

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 29ac627956c5e5cdd4ed5f1316c20bc9e1fe0ff7)

15 months agoMerge pull request #56783 from epuertat/wip-65390-squid
Ernesto Puerta [Tue, 9 Apr 2024 14:49:24 +0000 (16:49 +0200)]
Merge pull request #56783 from epuertat/wip-65390-squid

squid: mgr/dashboard: update NVMeoF gRPC: "listener add" changes

15 months agomgr/dashboard: update NVMeoF gRPC 56783/head
Ernesto Puerta [Tue, 2 Apr 2024 17:36:21 +0000 (19:36 +0200)]
mgr/dashboard: update NVMeoF gRPC

To include latest changes to the listener add call.

Fixes: https://tracker.ceph.com/issues/65268
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 7357e1f326c03baabfb870fb424756537dc8830a)

15 months agodoc/cephfs: refine client-auth (1 of 3) 56779/head
Zac Dover [Mon, 8 Apr 2024 12:07:38 +0000 (22:07 +1000)]
doc/cephfs: refine client-auth (1 of 3)

Refine the first third of doc/cephfs/client-auth.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 82c5bac5df553f1e13a2df3a107b2692efa6fc27)

15 months agoinstall-deps: enable copr ceph/grpc 56778/head
Nizamudeen A [Wed, 27 Mar 2024 16:29:55 +0000 (21:59 +0530)]
install-deps: enable copr ceph/grpc

In dashboard, to generate nvmeof apis in el8 this is needed so that it
can download the python3-grpcio packages.
https://copr.fedorainfracloud.org/coprs/ceph/grpc/

Fixes: https://tracker.ceph.com/issues/65184
Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 257dc69d324445f6c71c33d9ac248f0bf0e961ec)

15 months agoMerge pull request #56668 from batrick/wip-65289-squid
Leonid S. Usov [Tue, 9 Apr 2024 10:10:55 +0000 (13:10 +0300)]
Merge pull request #56668 from batrick/wip-65289-squid

squid: mds/quiesce: prevent an overflow of the wait duration

15 months agoMerge pull request #56695 from ceph/wip-lusov-qdb-clear-squid
Leonid S. Usov [Mon, 8 Apr 2024 22:31:48 +0000 (01:31 +0300)]
Merge pull request #56695 from ceph/wip-lusov-qdb-clear-squid

squid: mds/quiesce-db: always clear the db if a membership is lost

15 months agoMerge pull request #56761 from adk3798/squid-cephadm-pin-pyfakefs-version
Adam King [Mon, 8 Apr 2024 18:24:42 +0000 (14:24 -0400)]
Merge pull request #56761 from adk3798/squid-cephadm-pin-pyfakefs-version

squid: cephadm: pin pyfakefs version for tox tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>
15 months agotests/scripts: use 'tell pg deep-scrub pgid' instead of 'tell pgid 56766/head
Ronen Friedman [Sun, 7 Apr 2024 07:48:35 +0000 (02:48 -0500)]
tests/scripts: use 'tell pg deep-scrub pgid' instead of 'tell pgid
deep-scrub'

as older OSD versions do not support the former.

Fixes: https://tracker.ceph.com/issues/64972
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
(cherry picked from commit dda89e77ae6d0a1c7f351c74192417bf60fde3f4)

15 months agocephadm: pin pyfakefs version for tox tests 56761/head
Adam King [Mon, 8 Apr 2024 14:48:18 +0000 (10:48 -0400)]
cephadm: pin pyfakefs version for tox tests

5.4.0 seems to have caused a number of permission issues
in the test. In order to unblock make check, let's pin it
to 5.3.5 for now.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit ce71ce10b7551a649c9a64be992c0eddfa7e7243)

15 months agoMerge pull request #56739 from zdover23/wip-doc-2024-05-07-backport-56738-to-squid
Anthony D'Atri [Sun, 7 Apr 2024 12:28:13 +0000 (08:28 -0400)]
Merge pull request #56739 from zdover23/wip-doc-2024-05-07-backport-56738-to-squid

squid: doc/dev: refine "Concepts" 4 of 3

15 months agodoc/dev: refine "Concepts" 4 of 3 56739/head
Zac Dover [Sat, 6 Apr 2024 04:43:16 +0000 (14:43 +1000)]
doc/dev: refine "Concepts" 4 of 3

s/PG's PG logs/PG's logs/

re https://github.com/ceph/ceph/pull/56727#discussion_r1553541922

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 221d734760b6f680e4d17cd2f0bfcd39f914c8d6)

15 months agoMerge pull request #56731 from zdover23/wip-doc-2024-04-05-backport-56727-to-squid
Zac Dover [Sat, 6 Apr 2024 04:40:04 +0000 (14:40 +1000)]
Merge pull request #56731 from zdover23/wip-doc-2024-04-05-backport-56727-to-squid

squid: doc/dev: refine "Concepts" 3 of 3

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agoMerge pull request #56706 from afreen23/wip-65208-squid
Nizamudeen A [Fri, 5 Apr 2024 17:27:45 +0000 (22:57 +0530)]
Merge pull request #56706 from afreen23/wip-65208-squid

squid: mgr/dashboard:Update encryption and tags in bucket form

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56708 from afreen23/wip-65303-squid
Nizamudeen A [Fri, 5 Apr 2024 17:26:36 +0000 (22:56 +0530)]
Merge pull request #56708 from afreen23/wip-65303-squid

squid: mgr/dashboard:Use advanced fieldset for rbd image

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56564 from batrick/wip-65214-squid
Yuri Weinstein [Fri, 5 Apr 2024 14:59:24 +0000 (07:59 -0700)]
Merge pull request #56564 from batrick/wip-65214-squid

squid: mds: do not dispatch aborted internal requests

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Leonid Usov <leonid.usov@ibm.com>
15 months agoMerge pull request #56423 from batrick/wip-65093-squid
Yuri Weinstein [Fri, 5 Apr 2024 14:58:25 +0000 (07:58 -0700)]
Merge pull request #56423 from batrick/wip-65093-squid

squid: qa/suites/fs/workload: enable snap_schedule early

Reviewed-by: Venky Shankar <vshankar@redhat.com>
15 months agoMerge pull request #56402 from batrick/wip-65063-squid
Yuri Weinstein [Fri, 5 Apr 2024 14:57:41 +0000 (07:57 -0700)]
Merge pull request #56402 from batrick/wip-65063-squid

squid: qa/cephfs: ignorelist clog of MDS_UP_LESS_THAN_MAX

Reviewed-by: Venky Shankar <vshankar@redhat.com>
15 months agodoc/dev: refine "Concepts" 3 of 3 56731/head
Zac Dover [Fri, 5 Apr 2024 06:43:47 +0000 (16:43 +1000)]
doc/dev: refine "Concepts" 3 of 3

Refine the third third of the entries in the section "Concepts" in
doc/dev/peering.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 512e2dc9d52fdd14dfdd87ccd141e535d3098fb7)

15 months agoMerge pull request #56724 from zdover23/wip-doc-2024-04-05-backport-56646-to-squid
Anthony D'Atri [Fri, 5 Apr 2024 12:33:58 +0000 (08:33 -0400)]
Merge pull request #56724 from zdover23/wip-doc-2024-04-05-backport-56646-to-squid

squid: doc/dev: refine "Concepts" 2 of 3

15 months agodoc/dev: refine "Concepts" 2 of 3 56724/head
Zac Dover [Wed, 3 Apr 2024 05:02:29 +0000 (15:02 +1000)]
doc/dev: refine "Concepts" 2 of 3

Refine the second third of the entries in the section "Concepts" in
doc/dev/peering.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 7bb35282da27f0c18ffbe36f2c0852a827c5bc2f)

15 months agomgr/dashboard:Use advanced fieldset for rbd image 56708/head
Afreen [Wed, 27 Mar 2024 09:35:58 +0000 (15:05 +0530)]
mgr/dashboard:Use advanced fieldset for rbd image

Fixes https://tracker.ceph.com/issues/65025

- using cd-advanced-fieldset component for advanced section
- this will align advanced sections of forms in dashboard
- changed heading of sections of rbd-configuration to h5 to better represent it
  as a sub section.
- added unit tests for rbd image config

Signed-off-by: Afreen <afreen23.git@gmail.com>
(cherry picked from commit 25552117b433dbe39bf245aeda99761def5749eb)

15 months agomgr/dashboard:Update encryption and tags in bucket form 56706/head
Afreen [Mon, 25 Mar 2024 08:39:08 +0000 (14:09 +0530)]
mgr/dashboard:Update encryption and tags in bucket form

Fixes https://tracker.ceph.com/issues/65110

- renamed Security to Encryption
- aligned security fieldset with rest of the form using `cd-help-text`
  and aligning it with object locking in UX
- aligned security fieldset top use `cd-help-text`
- changed help text of Tags

Signed-off-by: Afreen <afreen23.git@gmail.com>
(cherry picked from commit 96b21d7cf5bb698102dc0cfce6fef05204b82d2e)

15 months agoMerge pull request #56691 from afreen23/wip-65156-squid
Nizamudeen A [Thu, 4 Apr 2024 16:35:13 +0000 (22:05 +0530)]
Merge pull request #56691 from afreen23/wip-65156-squid

squid: mgr/dashboard: Add advanced fieldset component

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56636 from cbodley/wip-qa-squid-quincy-upgrade-distro
Casey Bodley [Thu, 4 Apr 2024 14:57:16 +0000 (15:57 +0100)]
Merge pull request #56636 from cbodley/wip-qa-squid-quincy-upgrade-distro

squid: qa/upgrade/quincy: bump ubuntu from 20->22

Reviewed-by: Adam King <adking@redhat.com>
15 months agoMerge pull request #56647 from zdover23/wip-doc-2024-04-03-backport-56228-to-squid
Zac Dover [Thu, 4 Apr 2024 12:55:25 +0000 (22:55 +1000)]
Merge pull request #56647 from zdover23/wip-doc-2024-04-03-backport-56228-to-squid

squid: doc: fixing doc/cephfs/fs-volumes

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
15 months agoMerge pull request #56683 from zdover23/wip-doc-2024-04-04-backport-56662-to-squid
Zac Dover [Thu, 4 Apr 2024 11:39:28 +0000 (21:39 +1000)]
Merge pull request #56683 from zdover23/wip-doc-2024-04-04-backport-56662-to-squid

squid: doc/mgr: credit John Jasen for Zabbix 2

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agomds/quiesce-db: always clear the db if a membership is lost 56695/head
Leonid Usov [Sun, 17 Mar 2024 16:22:34 +0000 (12:22 -0400)]
mds/quiesce-db: always clear the db if a membership is lost

Fixes: https://tracker.ceph.com/issues/64912
Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
(cherry picked from commit e1c8e08961e8199f1da74dc9d1eb2a940e2908d5)

15 months agomgr/dashboard: Add advanced fieldset component 56691/head
Afreen [Fri, 1 Mar 2024 07:26:25 +0000 (12:56 +0530)]
mgr/dashboard: Add advanced fieldset component

Fixes https://tracker.ceph.com/issues/65024

- adds a new shared component for displaying advanced fields in /shared
- utilizes that component in bucket form
- placement targets are under advanced fieldset
- minor help text changes in object locking help text
- updated e2e tests

Signed-off-by: Afreen <afreen23.git@gmail.com>
(cherry picked from commit d0979e9c5223231a0fd3ef6408323d2fd5cd2ac5)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/shared/components/components.module.ts

15 months agoMerge pull request #56620 from afreen23/wip-65210-squid
Nizamudeen A [Thu, 4 Apr 2024 05:48:13 +0000 (11:18 +0530)]
Merge pull request #56620 from afreen23/wip-65210-squid

squid: mgr/dashboard: Mark placement targets as non-required

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agoMerge pull request #56659 from zdover23/wip-doc-2024-04-03-backport-56635-to-squid
Zac Dover [Thu, 4 Apr 2024 04:33:28 +0000 (14:33 +1000)]
Merge pull request #56659 from zdover23/wip-doc-2024-04-03-backport-56635-to-squid

squid: doc/dev: refine "Concepts"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
15 months agodoc/mgr: credit John Jasen for Zabbix 2 56683/head
Zac Dover [Wed, 3 Apr 2024 12:21:04 +0000 (22:21 +1000)]
doc/mgr: credit John Jasen for Zabbix 2

Credit John Jasen for the development of the procedures describing the
installation of Zabbix 2.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 28aa556448cf02dc943653e76a7ef89e10ae93e9)

15 months agoMerge pull request #56553 from rzarzynski/wip-ksirivad-osdmap-encode-bug-squid
Laura Flores [Wed, 3 Apr 2024 18:41:55 +0000 (13:41 -0500)]
Merge pull request #56553 from rzarzynski/wip-ksirivad-osdmap-encode-bug-squid

squid: src/osd/OSDMap.cc: Fix encoder to produce same bytestream

15 months agoMerge pull request #56656 from aaSharma14/wip-65284-squid
Aashish Sharma [Wed, 3 Apr 2024 17:10:18 +0000 (22:40 +0530)]
Merge pull request #56656 from aaSharma14/wip-65284-squid

squid: mgr/dashboard: replace deprecated table panel in grafana with a newer table panel

Reviewed-by: Nizamudeen A <nia@redhat.com>
15 months agorgw/pubsub: CreateTopic validates topic name 56670/head
Casey Bodley [Thu, 28 Mar 2024 17:47:30 +0000 (13:47 -0400)]
rgw/pubsub: CreateTopic validates topic name

existing topics may have invalid names, so this is only enforced by
CreateTopic

Fixes: https://tracker.ceph.com/issues/65212
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ea6a7a516234237e1f3c5cfda966c3d53fc174ed)

15 months agomds/quiesce: prevent an overflow of the wait duration 56668/head
Leonid Usov [Thu, 28 Mar 2024 05:32:26 +0000 (01:32 -0400)]
mds/quiesce: prevent an overflow of the wait duration

QuiesceTimeInterval::max() may overflow inside of a call to
std::condition_variable::wait_for and result in a busy-loop,
making the call to timeout immediately

The solution is to cap the wait duration to a value which can
certainly fit in whichever clock std library is using internally.

Fixes: https://tracker.ceph.com/issues/65276
Signed-off-by: Leonid Usov <leonid.usov@ibm.com>
(cherry picked from commit 508e870ee383265b8489e18a4c73854616a4110a)

15 months agoMerge pull request #56343 from cbodley/wip-65007-squid
Casey Bodley [Wed, 3 Apr 2024 13:35:50 +0000 (14:35 +0100)]
Merge pull request #56343 from cbodley/wip-65007-squid

squid: RGW: allow user disabling presigned urls in rgw configuration

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
15 months agodoc/dev: refine "Concepts" 56659/head
Zac Dover [Tue, 2 Apr 2024 12:23:13 +0000 (22:23 +1000)]
doc/dev: refine "Concepts"

Refine the text in the first third of the entries in the section
"Concepts" in doc/dev/peering.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 530b11be24c6e702d4d9b9227649db8ca34591e4)

15 months agomgr/dashboard: replace deprecated table panel in grafana with a newer 56656/head
Aashish Sharma [Wed, 27 Mar 2024 08:16:24 +0000 (13:46 +0530)]
mgr/dashboard: replace deprecated table panel in grafana with a newer
table panel

Fixes: https://tracker.ceph.com/issues/65174
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
(cherry picked from commit c2f4aa7887d8322bc58cd1cb322746246ccebe32)

15 months agodoc: fixing doc/cephfs/fs-volumes 56647/head
neeraj pratap singh [Fri, 15 Mar 2024 17:36:18 +0000 (23:06 +0530)]
doc: fixing doc/cephfs/fs-volumes

Fixes: https://tracker.ceph.com/issues/65267
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>
(cherry picked from commit 906ff4e4c240856d2e27cf9ac8c059479a86c5c9)