]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
5 months agoosd/scrub: use ObjectSection & ArraySection 61506/head
Ronen Friedman [Fri, 24 Jan 2025 16:37:34 +0000 (10:37 -0600)]
osd/scrub: use ObjectSection & ArraySection

Use the RAII wrappers for the dump Formatters,
instead of manually opening and closing sections.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
5 months agoMerge pull request #61258 from guits/cv-hints
Guillaume Abrioux [Thu, 23 Jan 2025 11:57:13 +0000 (12:57 +0100)]
Merge pull request #61258 from guits/cv-hints

ceph-volume: type annotations

5 months agoMerge pull request #61253 from Matan-B/wip-matanb-crimson-tri_mutex_pointer
Matan Breizman [Thu, 23 Jan 2025 10:47:26 +0000 (12:47 +0200)]
Merge pull request #61253 from Matan-B/wip-matanb-crimson-tri_mutex_pointer

crimson/common/tri_mutex: improve logging

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
5 months agoMerge pull request #56446 from guojidan/metrics
Matan Breizman [Thu, 23 Jan 2025 10:42:25 +0000 (12:42 +0200)]
Merge pull request #56446 from guojidan/metrics

crimson/mgr/client.cc: daemon_health_metrics support

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
5 months agoMerge pull request #61489 from dmick/wip-fix-container-tags
Dan Mick [Thu, 23 Jan 2025 02:57:57 +0000 (18:57 -0800)]
Merge pull request #61489 from dmick/wip-fix-container-tags

container/build.sh: fix up org vs. repo naming

5 months agocontainer/build.sh: fix up org vs. repo naming 61489/head
Dan Mick [Thu, 23 Jan 2025 02:28:15 +0000 (18:28 -0800)]
container/build.sh: fix up org vs. repo naming

release builds were using the wrong container repo name because of
confused variable naming and inadequate separation.  Keep the hostname,
org name, and repo name in separate variables, and assemble the full
path with a version when tagging is done.

Signed-off-by: Dan Mick <dan.mick@redhat.com>
5 months agoMerge pull request #57997 from guojidan/migration-child
Ilya Dryomov [Wed, 22 Jan 2025 18:46:41 +0000 (19:46 +0100)]
Merge pull request #57997 from guojidan/migration-child

test/librbd: add unit test for flatten migration

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
5 months agoMerge pull request #61277 from lightmelodies/rgw
J. Eric Ivancich [Wed, 22 Jan 2025 15:50:23 +0000 (10:50 -0500)]
Merge pull request #61277 from lightmelodies/rgw

rgw/lc: make lc worker thread name shorter

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
5 months agoMerge pull request #60800 from rkachach/fix_issue_deps_refactoring
Adam King [Wed, 22 Jan 2025 13:18:37 +0000 (08:18 -0500)]
Merge pull request #60800 from rkachach/fix_issue_deps_refactoring

mgr/cephadm: Refactoring and consolidating services dependencies calculation logic

Reviewed-by: Adam King <adking@redhat.com>
5 months agoMerge pull request #61462 from zdover23/wip-doc-2025-01-21-cephfs-disaster-recovery...
Zac Dover [Wed, 22 Jan 2025 11:57:08 +0000 (21:57 +1000)]
Merge pull request #61462 from zdover23/wip-doc-2025-01-21-cephfs-disaster-recovery-experts

doc/cephfs: edit disaster-recovery-experts (4 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
5 months agoMerge pull request #61436 from soumyakoduri/wip-skoduri-dbstore
Soumya Koduri [Wed, 22 Jan 2025 09:35:34 +0000 (15:05 +0530)]
Merge pull request #61436 from soumyakoduri/wip-skoduri-dbstore

rgw/dbstore: Update bucket attrs as part of put_info()

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>
5 months agoMerge pull request #60647 from soumyakoduri/wip-skoduri-lc-crash
Soumya Koduri [Wed, 22 Jan 2025 09:35:01 +0000 (15:05 +0530)]
Merge pull request #60647 from soumyakoduri/wip-skoduri-lc-crash

rgw: Add object expirer API to the SAL interface

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
5 months agotest/librbd: add unit test for flatten migration 57997/head
junxiang Mu [Wed, 11 Sep 2024 03:06:18 +0000 (11:06 +0800)]
test/librbd: add unit test for flatten migration

Fixes: https://tracker.ceph.com/issues/65743
Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: junxiang Mu <1948535941@qq.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
5 months agoMerge pull request #59050 from mohit84/issue_67270
Yuri Weinstein [Wed, 22 Jan 2025 00:53:51 +0000 (16:53 -0800)]
Merge pull request #59050 from mohit84/issue_67270

mon: Create a local copy of scrub_state to avoid a crash

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
5 months agoMerge pull request #60984 from ljflores/wip-fix-telemetry-tests
Laura Flores [Tue, 21 Jan 2025 20:09:11 +0000 (14:09 -0600)]
Merge pull request #60984 from ljflores/wip-fix-telemetry-tests

qa/suites/upgrade/telemetry-upgrade: update telemetry tests

5 months agoMerge pull request #61007 from kamoltat/wip-ksirivad-fix-68602
Yuri Weinstein [Tue, 21 Jan 2025 18:56:39 +0000 (10:56 -0800)]
Merge pull request #61007 from kamoltat/wip-ksirivad-fix-68602

qa/tasks/thrashosds-health.yaml: Add osds down to ignorelist

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
5 months agoMerge pull request #58052 from jamiepryde/isal-default
Yuri Weinstein [Tue, 21 Jan 2025 18:54:52 +0000 (10:54 -0800)]
Merge pull request #58052 from jamiepryde/isal-default

core: Change the default plugin for Ceph erasure coded pools from Jerasure to ISA-L

Reviewed-by: Afreen Misbah <afreen23.git@gmail.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
5 months agoMerge pull request #56180 from yehudasa/wip-objecter-local-read
Yuri Weinstein [Tue, 21 Jan 2025 18:52:41 +0000 (10:52 -0800)]
Merge pull request #56180 from yehudasa/wip-objecter-local-read

objecter: use configurable to set local and balanced reads

Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
5 months agoMerge pull request #61098 from anoopcs9/add-libcephfs-daemon-containerfile
Dan Mick [Tue, 21 Jan 2025 18:37:30 +0000 (10:37 -0800)]
Merge pull request #61098 from anoopcs9/add-libcephfs-daemon-containerfile

Containerfile: Add `libcephfs-daemon` package

5 months agoMerge pull request #61175 from ronen-fr/wip-rf-nvme-this
Ronen Friedman [Tue, 21 Jan 2025 17:33:07 +0000 (19:33 +0200)]
Merge pull request #61175 from ronen-fr/wip-rf-nvme-this

mon/nvme: fix unused lambda capture warnings
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: baum <alexander.indenbaum@gmail.com>
5 months agoMerge pull request #61282 from oshrey16/wip-lua-oshrey
Yuval Lifshitz [Tue, 21 Jan 2025 16:49:48 +0000 (18:49 +0200)]
Merge pull request #61282 from oshrey16/wip-lua-oshrey

rgw/lua: add configurable runtime limit for the lua state

5 months agoMerge pull request #61388 from AliMasarweh/wip-alimasa-bucket-logging-log-info
Ali Masarwa [Tue, 21 Jan 2025 15:45:08 +0000 (17:45 +0200)]
Merge pull request #61388 from AliMasarweh/wip-alimasa-bucket-logging-log-info

RGW\logging: added missing fields

Reviewed-by: yuvalif<ylifshit@redhat.com>
5 months agodoc/cephfs: edit disaster-recovery-experts (4 of x) 61462/head
Zac Dover [Tue, 21 Jan 2025 05:53:19 +0000 (15:53 +1000)]
doc/cephfs: edit disaster-recovery-experts (4 of x)

Edit the seventh and final section of
doc/cephfs/disaster-recovery-experts.rst in preparation for adding
deeper explanations of the contexts in which one should use the various
commands listed on that page.

The section edited in this commit is

 * Using an alternate metadata pool for recovery

A future commit might beneficially put this section into the format of
an ordered list. If so, such a commit should only reformat the
content and should not make any changes to the English. It's enough to
verify content or format. Let's not overload our editorial faculties by
forcing ourselves to walk and chew gum at the same time.

Follows https://github.com/ceph/ceph/pull/61442

https://tracker.ceph.com/issues/69557

Signed-off-by: Zac Dover <zac.dover@proton.me>
5 months agoRGW\logging: adding transport layer fields 61388/head
Ali Masarwa [Tue, 14 Jan 2025 15:42:48 +0000 (17:42 +0200)]
RGW\logging: adding transport layer fields

Signed-off-by: Ali Masarwa <amasarwa@redhat.com>
5 months agoMerge pull request #61249 from zdover23/wip-doc-2025-01-07-cephfs-snapshots
Zac Dover [Tue, 21 Jan 2025 05:26:08 +0000 (15:26 +1000)]
Merge pull request #61249 from zdover23/wip-doc-2025-01-07-cephfs-snapshots

doc/cephfs: edit grammar in snapshots.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
5 months agoMerge pull request #61267 from athanatos/sjust/wip-rep-pipeline-fix
Matan Breizman [Mon, 20 Jan 2025 14:14:22 +0000 (16:14 +0200)]
Merge pull request #61267 from athanatos/sjust/wip-rep-pipeline-fix

crimson/.../replicated_request: fix pg lifetime in with_pg finally

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
5 months agoMerge pull request #61442 from zdover23/wip-doc-2025-01-18-cephfs-disaster-recovery...
Zac Dover [Mon, 20 Jan 2025 12:38:28 +0000 (22:38 +1000)]
Merge pull request #61442 from zdover23/wip-doc-2025-01-18-cephfs-disaster-recovery-experts

doc/cephfs: edit disaster-recovery-experts (3 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
5 months agoceph-volume: update unit tests 61258/head
Guillaume Abrioux [Wed, 15 Jan 2025 12:37:31 +0000 (12:37 +0000)]
ceph-volume: update unit tests

This commit consolidates all necessary changes to address unit tests
related to the introduction of Python type annotations.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: remove duplicate code
Guillaume Abrioux [Wed, 15 Jan 2025 12:33:45 +0000 (12:33 +0000)]
ceph-volume: remove duplicate code

This check is already performed in `arg_validators.py`.
By the way, this check is only valid when the user passes a VG/LV with
the `<vg>/<lv>` format.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: add type annotations to devices.lvm.batch
Guillaume Abrioux [Thu, 9 Jan 2025 10:22:15 +0000 (10:22 +0000)]
ceph-volume: add type annotations to devices.lvm.batch

This commit adds the Python type annotations to `devices.lvm.batch`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: add type annotations to devices.lvm.listing
Guillaume Abrioux [Wed, 8 Jan 2025 16:02:53 +0000 (16:02 +0000)]
ceph-volume: add type annotations to devices.lvm.listing

This commit adds the Python type annotations to `devices.lvm.listing`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: improve clarity and consistency in List.single_report
Guillaume Abrioux [Wed, 8 Jan 2025 15:54:45 +0000 (15:54 +0000)]
ceph-volume: improve clarity and consistency in List.single_report

- Rename the parameter `arg` to `osd` for better readability.
- Simplify the check for numeric OSD identifiers using `isdigit()`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: add type annotations to devices.lvm.common
Guillaume Abrioux [Wed, 8 Jan 2025 15:52:21 +0000 (15:52 +0000)]
ceph-volume: add type annotations to devices.lvm.common

This commit adds the Python type annotations to `devices.lvm.common`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: add type annotations to devices.lvm.prepare
Guillaume Abrioux [Wed, 8 Jan 2025 15:51:35 +0000 (15:51 +0000)]
ceph-volume: add type annotations to devices.lvm.prepare

This commit adds the Python type annotations to `devices.lvm.prepare`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: add type annotations to devices.lvm.trigger
Guillaume Abrioux [Wed, 8 Jan 2025 15:50:40 +0000 (15:50 +0000)]
ceph-volume: add type annotations to devices.lvm.trigger

This commit adds the Python type annotations to `devices.lvm.trigger`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: add type annotations to devices.lvm.main
Guillaume Abrioux [Wed, 8 Jan 2025 14:27:33 +0000 (14:27 +0000)]
ceph-volume: add type annotations to devices.lvm.main

This commit adds the Python type annotations to `devices.lvm.main`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: add type annotations to devices.lvm.deactivate
Guillaume Abrioux [Wed, 8 Jan 2025 14:17:29 +0000 (14:17 +0000)]
ceph-volume: add type annotations to devices.lvm.deactivate

This commit adds the Python type annotations to `devices.lvm.deactivate`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: add type annotations to devices.lvm.create
Guillaume Abrioux [Wed, 8 Jan 2025 14:13:16 +0000 (14:13 +0000)]
ceph-volume: add type annotations to devices.lvm.create

This commit adds the Python type annotations to `devices.lvm.create`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: add type annotations to devices.lvm.activate
Guillaume Abrioux [Wed, 8 Jan 2025 13:58:26 +0000 (13:58 +0000)]
ceph-volume: add type annotations to devices.lvm.activate

This commit adds the Python type annotations to `devices.lvm.activate`.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoceph-volume: fix type annotation in `objectore`
Guillaume Abrioux [Wed, 8 Jan 2025 13:40:54 +0000 (13:40 +0000)]
ceph-volume: fix type annotation in `objectore`

This commit addresses some python type annotations errors
in `objectstore` code.

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
5 months agoMerge pull request #61040 from rhcs-dashboard/smb-module-check
Pedro Gonzalez Gomez [Mon, 20 Jan 2025 07:19:02 +0000 (08:19 +0100)]
Merge pull request #61040 from rhcs-dashboard/smb-module-check

mgr/module: Smb module check

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
5 months agodoc/cephfs: edit disaster-recovery-experts (3 of x) 61442/head
Zac Dover [Sat, 18 Jan 2025 04:04:14 +0000 (14:04 +1000)]
doc/cephfs: edit disaster-recovery-experts (3 of x)

Edit the fifth and sixth sections of
doc/cephfs/disaster-recovery-experts.rst in preparation for adding
deeper explanations of the contexts in which one should use the various
commands listed on that page.

The sections edited in this commit are

 - MDS Map Reset
 - Recovery From Mission Metadata Objects

Follows https://github.com/ceph/ceph/pull/61427

https://tracker.ceph.com/issues/69557

Signed-off-by: Zac Dover <zac.dover@proton.me>
5 months agorgw/dbstore: Update bucket attrs as part of put_info() 61436/head
Soumya Koduri [Fri, 17 Jan 2025 18:48:25 +0000 (00:18 +0530)]
rgw/dbstore: Update bucket attrs as part of put_info()

With commit#8c025045332a8005c6e82308fc17a33d38058734, changes
were made to call put_info while erasing bucket attrs. But
DBStore wasn't updating attrs as part of put_info operation.
This commit addresses the same.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
5 months agoMerge pull request #61445 from zdover23/wip-doc-2025-01-19-cephfs-disaster-recovery...
Anthony D'Atri [Sun, 19 Jan 2025 15:59:25 +0000 (10:59 -0500)]
Merge pull request #61445 from zdover23/wip-doc-2025-01-19-cephfs-disaster-recovery-experts

doc/cephfs: disaster-recovery-experts cleanup

5 months agodoc/cephfs: disaster-recovery-experts cleanup 61445/head
Zac Dover [Sun, 19 Jan 2025 12:49:52 +0000 (22:49 +1000)]
doc/cephfs: disaster-recovery-experts cleanup

Properly wrap a poorly-formatted paragraph that looks just awful in an
80-column viewport and change MDS to "MDS daemons" where the latter
makes the sentence a lot clearer.

Signed-off-by: Zac Dover <zac.dover@proton.me>
5 months agoMerge pull request #61427 from zdover23/wip-doc-2025-01-17-cephfs-disaster-recovery...
Zac Dover [Sun, 19 Jan 2025 07:31:28 +0000 (17:31 +1000)]
Merge pull request #61427 from zdover23/wip-doc-2025-01-17-cephfs-disaster-recovery-experts-2

doc/cephfs: edit disaster-recovery-experts (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
6 months agorgw/lua: add configurable runtime limit for the lua state 61282/head
Oshrey Avraham [Sun, 19 Jan 2025 00:23:36 +0000 (02:23 +0200)]
rgw/lua: add configurable runtime limit for the lua state

These changes improve the robustness and reliability of Lua script execution in the RADOS Gateway by enforcing configurable limit on execution time.

- Enhanced `lua_state_guard` to support configurable runtime limits for Lua scripts.
- Updated rgw.yaml.in to include new configuration options for Lua runtime limits.
- Added tests in `test_rgw_lua.cc` to verify Lua script execution with different runtime constraints.
- Updated Lua scripting documentation to reflect the new runtime limit configuration.

Signed-off-by: Oshry Avraham <oshryabiz@gmail.com>
6 months agodoc/cephfs: edit disaster-recovery-experts (2 of x) 61427/head
Zac Dover [Fri, 17 Jan 2025 12:33:49 +0000 (22:33 +1000)]
doc/cephfs: edit disaster-recovery-experts (2 of x)

Edit the third and fourth sections of
doc/cephfs/disaster-recovery-experts.rst in preparation for adding
deeper explanations of the contexts in which one should use the various
commands listed on that page.

Follows https://github.com/ceph/ceph/pull/61426

https://tracker.ceph.com/issues/69557

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
6 months agoMerge pull request #61426 from zdover23/wip-doc-2025-01-17-cephfs-disaster-recovery...
Zac Dover [Sat, 18 Jan 2025 03:43:18 +0000 (13:43 +1000)]
Merge pull request #61426 from zdover23/wip-doc-2025-01-17-cephfs-disaster-recovery-experts

doc/cephfs: remove link to undeveloped tool

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
6 months agoqa/suites/upgrade/telemetry-upgrade: ignore TELEMETRY_CHANGED warning 60984/head
Laura Flores [Sat, 18 Jan 2025 03:27:02 +0000 (21:27 -0600)]
qa/suites/upgrade/telemetry-upgrade: ignore TELEMETRY_CHANGED warning

Even though we opt in right away after upgrading, which minimizes the chance
of this warning coming up, the cluster log still detects it sometimes and
fails the test. So, we can ignore it since it's expected for the warning
to come up for a few seconds in some cases.

Signed-off-by: Laura Flores <lflores@ibm.com>
6 months agoqa/workunits: fix some details on telemetry workunits
Laura Flores [Fri, 17 Jan 2025 22:57:10 +0000 (16:57 -0600)]
qa/workunits: fix some details on telemetry workunits

1. For the telemetry upgrade tests, we want to
   make sure we enable the perf channel before we
   upgrade to test that out.

2. We also want to be sure to set up ident information.
   in the workunits used before the upgrade.

3. We can remove the "opt out" commands in the before-upgrade
   workunits since we want to stay opted in during the upgrade.
   We only opt out after all the checks are complete.

4. We mistakenly test "ceph telemetry show" twice when one
   of then should be "ceph telemetry show-all".

Signed-off-by: Laura Flores <lflores@ibm.com>
6 months agoMerge pull request #61440 from zdover23/wip-doc-2025-01-18-backport-61429-to-squid
Zac Dover [Fri, 17 Jan 2025 23:56:48 +0000 (09:56 +1000)]
Merge pull request #61440 from zdover23/wip-doc-2025-01-18-backport-61429-to-squid

doc/releases: edit warning for balancer issues

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
6 months agodoc/releases: edit warning for balancer issues 61440/head
Zac Dover [Fri, 17 Jan 2025 22:22:55 +0000 (08:22 +1000)]
doc/releases: edit warning for balancer issues

Rephrase the balancer-related warning issued to people upgrading to
Ceph 19.2.0.

Follows https://github.com/ceph/ceph/pull/61429

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
6 months agoqa/suites/upgrade/telemetry-upgrade: replace quincy tests with squid tests
Laura Flores [Fri, 6 Dec 2024 17:51:25 +0000 (11:51 -0600)]
qa/suites/upgrade/telemetry-upgrade: replace quincy tests with squid tests

Since main is now pre-Tentacle, we should replace the quincy upgrade
path with squid.

I have kept the quincy workunits since they are still
referenced in the overall upgrade suite. I will remove these
in a separate PR since changing the overall upgrade tests reaches
beyond the scope of just telemetry.

Signed-off-by: Laura Flores <lflores@ibm.com>
6 months agoqa/workunits: remove inaccurate reference to quincy from telemetry reef-->x test
Laura Flores [Thu, 5 Dec 2024 23:01:22 +0000 (23:01 +0000)]
qa/workunits: remove inaccurate reference to quincy from telemetry reef-->x test

This comment was mistakenly carried over in
https://github.com/ljflores/ceph/commit/6d25c053127f6aaaf942e674dca30ffd42658b9d.

Signed-off-by: Laura Flores <lflores@ibm.com>
6 months agoqa/workunits: add new collections to telemetry upgrade workunits (reef)
Laura Flores [Thu, 5 Dec 2024 22:46:19 +0000 (22:46 +0000)]
qa/workunits: add new collections to telemetry upgrade workunits (reef)

This commit changes the workunit that tests the upgrade path from
reef to x.

The logic to verify collections is now much more robust. Previously,
the test relied on developers to make it aware of any new collections
it needed to verify. Now, I have added logic to check that the # of
known collections matches the # of reported collections. If it does not,
the test errors out, which will force developers to add any new
collections to the upgrade test along with adding it to the telemetry
code.

The test also verifies all collections instead of focusing only on new
collections that were added after the base collections.

Signed-off-by: Laura Flores <lflores@ibm.com>
6 months agorgw: Add objects expirer API to the SAL interface 60647/head
Soumya Koduri [Wed, 6 Nov 2024 17:05:38 +0000 (22:35 +0530)]
rgw: Add objects expirer API to the SAL interface

Add the API `process_expired_objects` which is used to process and
delete the expired objects (swift or temporarily restored from cloud)
to the Zipper interface.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
6 months agoMerge pull request #61429 from laimis9133/squid-doc-patch
Anthony D'Atri [Fri, 17 Jan 2025 18:29:18 +0000 (13:29 -0500)]
Merge pull request #61429 from laimis9133/squid-doc-patch

doc/releases /squid: add warning for balancer issues

6 months agoMerge pull request #61407 from clwluvw/replication-status
J. Eric Ivancich [Fri, 17 Jan 2025 18:15:21 +0000 (13:15 -0500)]
Merge pull request #61407 from clwluvw/replication-status

rgw: update attrs with replication status to reflect in response

Reviewed-by: Alex Wojno <awojno@bloomberg.net>
6 months agoMerge pull request #61406 from clwluvw/checkpoint-dstbucket
J. Eric Ivancich [Fri, 17 Jan 2025 18:14:42 +0000 (13:14 -0500)]
Merge pull request #61406 from clwluvw/checkpoint-dstbucket

rgw: set dest bucket id in bucket sync checkpoint

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
6 months agoMerge pull request #61271 from clwluvw/datasync-usermode-err
J. Eric Ivancich [Fri, 17 Jan 2025 18:14:10 +0000 (13:14 -0500)]
Merge pull request #61271 from clwluvw/datasync-usermode-err

rgw: set retcode on verify_bucket_permission in RGWObjFetchCR

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 months agoMerge pull request #61157 from aiivashchenko/fix_68195
J. Eric Ivancich [Fri, 17 Jan 2025 18:13:54 +0000 (13:13 -0500)]
Merge pull request #61157 from aiivashchenko/fix_68195

rgw: add missing last_modified field to swift api

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 months agodoc/releases/squid: add warning for balancer issues 61429/head
Laimis Juzeliūnas [Fri, 17 Jan 2025 15:28:28 +0000 (17:28 +0200)]
doc/releases/squid: add warning for balancer issues

This change is to update 19.2.0 Squid release documentation with a warning for users regarding potential placement group balancer issues.

Signed-off-by: Laimis Juzeliunas <laimis.juzeliunas@oxylabs.io>
6 months agoMerge pull request #61387 from yuvalif/wip-yuval-69473
J. Eric Ivancich [Fri, 17 Jan 2025 16:20:28 +0000 (11:20 -0500)]
Merge pull request #61387 from yuvalif/wip-yuval-69473

test/rgw/lua: install luarocks

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>
6 months agoMerge pull request #59143 from Sean10/fix_rados_pybind_zero_param
Yuri Weinstein [Fri, 17 Jan 2025 16:08:51 +0000 (08:08 -0800)]
Merge pull request #59143 from Sean10/fix_rados_pybind_zero_param

pybind/rados: fix the incorrect order of offset,length in WriteOp.zero

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
6 months agoMerge pull request #61384 from sj14/sj/delete-policy-204
J. Eric Ivancich [Fri, 17 Jan 2025 14:31:31 +0000 (09:31 -0500)]
Merge pull request #61384 from sj14/sj/delete-policy-204

rgw: S3 Delete Bucket Policy should return 204 on success

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 months agoMerge pull request #61370 from yehudasa/wip-69536
J. Eric Ivancich [Fri, 17 Jan 2025 14:30:07 +0000 (09:30 -0500)]
Merge pull request #61370 from yehudasa/wip-69536

rgw: RGWRados::get_olh() needs to use the correct attr

Reviewed-by: Casey Bodley <cbodley@redhat.com>
6 months agoMerge pull request #61285 from cbodley/wip-69462
J. Eric Ivancich [Fri, 17 Jan 2025 14:26:00 +0000 (09:26 -0500)]
Merge pull request #61285 from cbodley/wip-69462

rgw: /admin/user api dumps account-related user info

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
6 months agoMerge pull request #61244 from awojno-bloomberg/fix-bucket-sync-status
J. Eric Ivancich [Fri, 17 Jan 2025 14:25:19 +0000 (09:25 -0500)]
Merge pull request #61244 from awojno-bloomberg/fix-bucket-sync-status

rgw: fix radosgw-admin use after free in bucket sync status

Reviewed-By: Adam Emerson <aemerson@ibm.com>
Reviewed-By: Oguzhan Ozmen <oozmen@bloomberg.net>
6 months agodoc/cephfs: remove link to undeveloped tool 61426/head
Zac Dover [Fri, 17 Jan 2025 12:23:17 +0000 (22:23 +1000)]
doc/cephfs: remove link to undeveloped tool

Remove a link to a tracker for a planned-but-never-developed tool that
would have facilitated the importation or exportation of a RADOS pool to
a file. This is a part of a project to improve the CephFS Disaster
Recovery documentation, which was requested by Eugen Block. See the
tracker issue below for more on this project.

Follows https://github.com/ceph/ceph/pull/61411

See https://github.com/ceph/ceph/pull/61411#discussion_r1918525109

https://tracker.ceph.com/issues/69557

Signed-off-by: Zac Dover <zac.dover@proton.me>
6 months agoMerge pull request #61411 from zdover23/wip-doc-2025-01-16-cephfs-disaster-recovery...
Zac Dover [Fri, 17 Jan 2025 12:18:07 +0000 (22:18 +1000)]
Merge pull request #61411 from zdover23/wip-doc-2025-01-16-cephfs-disaster-recovery-experts

doc/cephfs: edit disaster-recovery-experts

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
6 months agoMerge pull request #59155 from badone/wip-tracker-67489-fixed-4-for-verify
SrinivasaBharathKanta [Fri, 17 Jan 2025 10:20:27 +0000 (15:50 +0530)]
Merge pull request #59155 from badone/wip-tracker-67489-fixed-4-for-verify

qa/suites: Make fixed-4 an option for verify tests

6 months agoMerge pull request #60556 from aclamk/wip-aclamk-bluefs-truncate-allocations-main
Adam Kupczyk [Fri, 17 Jan 2025 08:31:32 +0000 (09:31 +0100)]
Merge pull request #60556 from aclamk/wip-aclamk-bluefs-truncate-allocations-main

os/bluestore: Make truncate() drop unused allocations - addendum

6 months agoMerge pull request #60363 from aclamk/wip-aclamk-fix-bluefs-bdev-expand
Adam Kupczyk [Fri, 17 Jan 2025 08:31:15 +0000 (09:31 +0100)]
Merge pull request #60363 from aclamk/wip-aclamk-fix-bluefs-bdev-expand

os/bluestore: Create additional bdev labels when expanding block device.

6 months agoMerge pull request #61369 from laimis9133/patch-2
Anthony D'Atri [Fri, 17 Jan 2025 01:46:57 +0000 (20:46 -0500)]
Merge pull request #61369 from laimis9133/patch-2

doc/radosgw/config-ref: fix lifecycle workload tuning description

6 months agoMerge pull request #61412 from gbregman/main
Gil Bregman [Fri, 17 Jan 2025 00:11:55 +0000 (02:11 +0200)]
Merge pull request #61412 from gbregman/main

mgr/cephadm/nvmeof: Add SPDK iobuf options field to NVMeOF configuration

6 months agoMerge pull request #60265 from MaxKellermann/timeout_round_up
Yuri Weinstein [Thu, 16 Jan 2025 23:06:36 +0000 (15:06 -0800)]
Merge pull request #60265 from MaxKellermann/timeout_round_up

msg/async: round timeouts up (fix busy loops)

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
6 months agoMerge pull request #48539 from chrisphoffman/wip-57864
Ilya Dryomov [Thu, 16 Jan 2025 18:13:44 +0000 (19:13 +0100)]
Merge pull request #48539 from chrisphoffman/wip-57864

qa/tasks: Include stderr on tasks badness check

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
6 months agomgr/cephadm/nvmeof: Add SPDK iobuf options field to NVMeOF configuration 61412/head
Gil Bregman [Thu, 16 Jan 2025 12:21:33 +0000 (14:21 +0200)]
mgr/cephadm/nvmeof: Add SPDK iobuf options field to NVMeOF configuration
Fixes https://tracker.ceph.com/issues/69554

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>
6 months agotest/rgw/lua: install luarocks 61387/head
Yuval Lifshitz [Tue, 14 Jan 2025 17:30:32 +0000 (17:30 +0000)]
test/rgw/lua: install luarocks

since it is only a suggestion for debian.
this is a regression from: https://github.com/ceph/ceph/pull/60666

Fixes: https://tracker.ceph.com/issues/69473
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
6 months agoMerge pull request #61393 from abhishek-kane/mapping_email
Adam King [Thu, 16 Jan 2025 13:01:22 +0000 (08:01 -0500)]
Merge pull request #61393 from abhishek-kane/mapping_email

Add Abhishek Kane into peoplemap, githubmap, organizationmap, mailmap

Reviewed-by: Adam King <adking@redhat.com>
6 months agodoc/cephfs: edit disaster-recovery-experts 61411/head
Zac Dover [Thu, 16 Jan 2025 11:51:46 +0000 (21:51 +1000)]
doc/cephfs: edit disaster-recovery-experts

Edit the first two sections of doc/cephfs/disaster-recovery-experts.rst
in preparation for adding deeper explanations of the contexts in which
one should use the various commands listed on that page.

https://tracker.ceph.com/issues/69557

Signed-off-by: Zac Dover <zac.dover@proton.me>
6 months agoMerge pull request #61349 from VallariAg/wip-vallari-nvmeof-ceph-version
Vallari Agrawal [Thu, 16 Jan 2025 09:57:46 +0000 (15:27 +0530)]
Merge pull request #61349 from VallariAg/wip-vallari-nvmeof-ceph-version

mon: do not show nvmeof in 'ceph versions' output

6 months agoMerge pull request #60889 from anoopcs9/fix-invalid-access-mds
Milind Changire [Thu, 16 Jan 2025 09:08:39 +0000 (14:38 +0530)]
Merge pull request #60889 from anoopcs9/fix-invalid-access-mds

mds: Fix invalid access of mdr->dn[0].back()

6 months agoAdd Abhishek Kane into peoplemap, githubmap, organizationmap, mailmap 61393/head
abhishek-kane [Wed, 15 Jan 2025 11:20:25 +0000 (16:50 +0530)]
Add Abhishek Kane into peoplemap, githubmap, organizationmap, mailmap

Signed-off-by: abhishek-kane <abhishek.kane@ibm.com>
6 months agomon: do not show nvmeof in 'ceph versions' output 61349/head
Vallari Agrawal [Mon, 13 Jan 2025 11:05:27 +0000 (16:35 +0530)]
mon: do not show nvmeof in 'ceph versions' output

NVMeoF gateway version is independent of ceph version
so 'ceph version' shows wrong nvmeof version in output
(i.e. instead of gateway version, it shows Ceph version).
Hence, remove nvmeof in 'ceph versions' output.

To check for gateway version, use 'gw info' command.

Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
6 months agoMerge pull request #61218 from dmick/wip-containerfile-ceph-version
Dan Mick [Wed, 15 Jan 2025 20:28:39 +0000 (12:28 -0800)]
Merge pull request #61218 from dmick/wip-containerfile-ceph-version

container/Containerfile: replace CEPH_VERSION label for backward compat

6 months agorgw: update attrs with replication status to reflect in response 61407/head
Seena Fallah [Sun, 3 Nov 2024 22:18:57 +0000 (23:18 +0100)]
rgw: update attrs with replication status to reflect in response

In handle_replication_status_header() when the COMPLETED status is
being set to object's attr, it needs to be reflected to the current
live attrs for the corresponding request so it can be reflected in
the first try than the second.

Fixes: https://tracker.ceph.com/issues/69549
Signed-off-by: Seena Fallah <seenafallah@gmail.com>
6 months agorgw: set dest bucket id in bucket sync checkpoint 61406/head
Seena Fallah [Mon, 28 Oct 2024 01:06:53 +0000 (02:06 +0100)]
rgw: set dest bucket id in bucket sync checkpoint

bucket_source_sync_checkpoint() requires bucket id to lookup for
full-sync object and the dest pipe doesn't have it. by replcaning
the bucket key in the pipe with the bucket info we will ensure the
bucket id is set.

Fixes: https://tracker.ceph.com/issues/69548
Signed-off-by: Seena Fallah <seenafallah@gmail.com>
6 months agoMerge pull request #60847 from cbodley/wip-rgw-boto-headbucket-stats
Casey Bodley [Wed, 15 Jan 2025 19:25:40 +0000 (14:25 -0500)]
Merge pull request #60847 from cbodley/wip-rgw-boto-headbucket-stats

examples/rgw: register boto extension for HeadBucket stats

Reviewed-by: Seena Fallah <seenafallah@gmail.com>
6 months agoMerge pull request #61211 from yuvalif/wip-yuval-logging-tenant
Yuval Lifshitz [Wed, 15 Jan 2025 16:23:44 +0000 (18:23 +0200)]
Merge pull request #61211 from yuvalif/wip-yuval-logging-tenant

rgw/logging: add more validations to conf, cross tenant support and cleanup

Reviewed-By: Adam Emerson <aemerson@ibm.com>, Ali Masarwe <ali.masarwa@ibm.com>
6 months agoMerge pull request #61373 from anthonyeleven/fix-file-descriptions
Zac Dover [Wed, 15 Jan 2025 15:25:50 +0000 (01:25 +1000)]
Merge pull request #61373 from anthonyeleven/fix-file-descriptions

AsyncMessenger.cc : improve error messages

Reviewed-by: Zac Dover <zac.dover@proton.me>
6 months agoRGW\log: adding some missing members
Ali Masarwa [Mon, 9 Dec 2024 10:53:05 +0000 (12:53 +0200)]
RGW\log: adding some missing members

Signed-off-by: Ali Masarwa <amasarwa@redhat.com>
6 months agoMerge pull request #61289 from ronen-fr/wip-rf-catcher-fix
Ronen Friedman [Wed, 15 Jan 2025 09:54:06 +0000 (11:54 +0200)]
Merge pull request #61289 from ronen-fr/wip-rf-catcher-fix

common: fix md_config_cacher_t
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
6 months agorgw: S3 Delete Bucket Policy should return 204 on success 61384/head
Simon Jürgensmeyer [Wed, 15 Jan 2025 08:08:38 +0000 (09:08 +0100)]
rgw: S3 Delete Bucket Policy should return 204 on success

Currently, RGW returns a 200 on a successful DELETE on a bucket policy but
the S3 API expects a 204:

    https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketPolicy.html

The sample response in the example above is a 204.

This patch checks the op_ret in `RGWDeleteBucketPolicy::send_response()`
and on a success we turn it to a 204 (or STATUS_NO_CONTENT).

Fixes: https://tracker.ceph.com/issues/69539
Signed-off-by: Simon Jürgensmeyer <simon.juergensmeyer@hetzner-cloud.de>
6 months agoMerge pull request #61072 from pecastro/doc_and_test_fixes
Anthony D'Atri [Wed, 15 Jan 2025 03:03:36 +0000 (22:03 -0500)]
Merge pull request #61072 from pecastro/doc_and_test_fixes

Doc and test fixes

6 months agoAsyncMessenger.cc : improve error messages 61373/head
Anthony D'Atri [Wed, 15 Jan 2025 00:17:14 +0000 (19:17 -0500)]
AsyncMessenger.cc : improve error messages

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
6 months agoqa/tasks: Include stderr on tasks badness check. 48539/head
Christopher Hoffman [Tue, 18 Oct 2022 20:14:19 +0000 (20:14 +0000)]
qa/tasks: Include stderr on tasks badness check.

Make sure that first_in_ceph_log() doesn't return None (which is
treated as success/"no badness" by the caller) if the cluster log file
is missing.

Fixes: https://tracker.ceph.com/issues/57864
Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Christopher Hoffman <choffman@redhat.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
6 months agorgw: RGWRados::get_olh() needs to use the correct attr 61370/head
Yehuda Sadeh [Tue, 14 Jan 2025 18:57:06 +0000 (13:57 -0500)]
rgw: RGWRados::get_olh() needs to use the correct attr

Fixes: https://tracker.ceph.com/issues/69536
Signed-off-by: Yehuda Sadeh <ysadehwe@ibm.com>
6 months agodoc/radosgw/config-ref: fix lifecycle workload tuning description 61369/head
Laimis Juzeliūnas [Tue, 14 Jan 2025 17:38:08 +0000 (19:38 +0200)]
doc/radosgw/config-ref: fix lifecycle workload tuning description

This commit updates RGW Config Reference - Lifecycle Settings section. In particular it changes the section where two different tunings for two different workloads are suggested. It aims for a more understandable description as asked by the contributors.

Signed-off-by: Laimis Juzeliunas <laimis.juzeliunas@oxylabs.io>