]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agoexporter: promethize counter names 51813/head
Pere Diaz Bou [Thu, 13 Apr 2023 16:55:04 +0000 (18:55 +0200)]
exporter: promethize counter names

Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>
Fixes: https://tracker.ceph.com/issues/59475
(cherry picked from commit 7087e76a036e35a733b185e987f0c9adc8f52007)

2 years agoMerge pull request #50748 from rhcs-dashboard/wip-59215-reef
Avan [Mon, 29 May 2023 12:24:21 +0000 (17:54 +0530)]
Merge pull request #50748 from rhcs-dashboard/wip-59215-reef

reef: exporter: user only counter dump/schema commands for extacting counters

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #51792 from zdover23/wip-doc-2023-05-28-backport-51587-to-reef
Anthony D'Atri [Sat, 27 May 2023 23:54:38 +0000 (19:54 -0400)]
Merge pull request #51792 from zdover23/wip-doc-2023-05-28-backport-51587-to-reef

reef: doc/rados: edit bluestore-config-ref.rst (2 of x)

2 years agodoc/rados: edit bluestore-config-ref.rst (2 of x) 51792/head
Zac Dover [Fri, 26 May 2023 18:44:18 +0000 (04:44 +1000)]
doc/rados: edit bluestore-config-ref.rst (2 of x)

Edit the second part of doc/rados/configuration/bluestore-config-ref.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthoy D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 1ceeab30ebde767aaf675fe46e79b9b7d28b881a)

2 years agoMerge pull request #51760 from ljflores/wip-reef-backport-pr-51270
yaarith [Fri, 26 May 2023 22:46:21 +0000 (18:46 -0400)]
Merge pull request #51760 from ljflores/wip-reef-backport-pr-51270

reef: mgr/telemetry: compile all channels and collections in selftest

Reviewed-by: Yaarit Hatuka <yaarit@redhat.com>
2 years agoMerge pull request #51789 from zdover23/wip-doc-2023-05-27-backport-51771-to-reef
Anthony D'Atri [Fri, 26 May 2023 22:39:42 +0000 (18:39 -0400)]
Merge pull request #51789 from zdover23/wip-doc-2023-05-27-backport-51771-to-reef

reef: doc/rados: edit bluestore-config-ref.rst (1 of x)

2 years agodoc/rados: edit bluestore-config-ref.rst (1 of x) 51789/head
Zac Dover [Fri, 26 May 2023 08:59:36 +0000 (18:59 +1000)]
doc/rados: edit bluestore-config-ref.rst (1 of x)

Edit the first part of doc/rados/configuration/bluestore-config-ref.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 3879e4ad417dddef1247521c199925a679d490ac)

2 years agoMerge pull request #51764 from cbodley/wip-61454
Casey Bodley [Fri, 26 May 2023 17:34:12 +0000 (13:34 -0400)]
Merge pull request #51764 from cbodley/wip-61454

reef: rgw/rados: remove sal handles from RGWRados::Object ops

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agorgw/rados: remove sal handles from RGWRados::Object ops 51764/head
Casey Bodley [Sat, 18 Feb 2023 18:11:45 +0000 (13:11 -0500)]
rgw/rados: remove sal handles from RGWRados::Object ops

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f4df0acb01171cff86c8ec1ab455798f2b4bf6dd)

Conflicts:
src/rgw/driver/rados/rgw_rados.cc need_to_log_data()
src/rgw/driver/rados/rgw_rados.h reindex_obj()

2 years agorgw/sal: StoreObject doesn't store separate attrs
Casey Bodley [Sat, 18 Feb 2023 20:19:34 +0000 (15:19 -0500)]
rgw/sal: StoreObject doesn't store separate attrs

use RGWObjState::attrset instead

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ad878f6c26424efa1017c8f861ece21b6b7d7704)

2 years agoMerge pull request #51717 from soumyakoduri/wip-skoduri-reef
Yuri Weinstein [Thu, 25 May 2023 20:12:13 +0000 (16:12 -0400)]
Merge pull request #51717 from soumyakoduri/wip-skoduri-reef

[reef] RGW backports

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #51762 from neha-ojha/wip-reef-rc
Laura Flores [Thu, 25 May 2023 19:19:14 +0000 (14:19 -0500)]
Merge pull request #51762 from neha-ojha/wip-reef-rc

reef: src/ceph_release: dev -> rc

2 years agoMerge pull request #51731 from cbodley/wip-59351-again
Casey Bodley [Thu, 25 May 2023 19:15:36 +0000 (15:15 -0400)]
Merge pull request #51731 from cbodley/wip-59351-again

reef: cmake: cls_rgw and cls_fifo depend on fmt

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
2 years agoMerge pull request #51735 from smanjara/wip-61406-reef
Yuri Weinstein [Thu, 25 May 2023 17:20:42 +0000 (13:20 -0400)]
Merge pull request #51735 from smanjara/wip-61406-reef

reef: rgw/multisite: handle shard_id correctly in data sync when bucket num_shards is 0

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years agoMerge pull request #51724 from smanjara/wip-61387-reef
Yuri Weinstein [Thu, 25 May 2023 17:20:00 +0000 (13:20 -0400)]
Merge pull request #51724 from smanjara/wip-61387-reef

reef: rgw/multisite: fix ret handling in bucket read_sync_status()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2 years ago doc/dev/release-checklists.rst: reef is in rc phase 51762/head
Neha Ojha [Thu, 25 May 2023 16:24:16 +0000 (16:24 +0000)]
 doc/dev/release-checklists.rst: reef is in rc phase

Signed-off-by: Neha Ojha <nojha@redhat.com>
2 years agodoc/dev/release-checklists.rst: add after dev freeze section
Neha Ojha [Tue, 14 Mar 2023 18:45:30 +0000 (18:45 +0000)]
doc/dev/release-checklists.rst: add after dev freeze section

- add release to redmine
- add milestone

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 1e048aade95880fe3c227f551bc91ac420701a8d)

2 years agosrc/ceph_release: dev -> rc
Neha Ojha [Thu, 25 May 2023 16:08:50 +0000 (16:08 +0000)]
src/ceph_release: dev -> rc

reef is now in rc phase

Signed-off-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #51001 from ifed01/wip-ifed-bluefs-duplicate-dir-link-ree
Yuri Weinstein [Thu, 25 May 2023 16:04:21 +0000 (12:04 -0400)]
Merge pull request #51001 from ifed01/wip-ifed-bluefs-duplicate-dir-link-ree

reef: os/bluestore/bluefs: fix dir_link might add link that already exists in compact log

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2 years agoMerge pull request #50930 from ifed01/wip-ifed-fix-spillover-alert-ree
Yuri Weinstein [Thu, 25 May 2023 16:02:57 +0000 (12:02 -0400)]
Merge pull request #50930 from ifed01/wip-ifed-fix-spillover-alert-ree

reef: os/bluestore: fix spillover alert

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
2 years agomgr/telemetry: compile all channels and collections in selftest 51760/head
Laura Flores [Thu, 27 Apr 2023 18:22:47 +0000 (18:22 +0000)]
mgr/telemetry: compile all channels and collections in selftest

The selftest currently compiles only channels and collections that
are on by default. This change tests the compilation of all channels
and collections.

Signed-off-by: Laura Flores <lflores@redhat.com>
(cherry picked from commit 70c5a4d54833f20a07f48488f29d9410d19f6bd4)

2 years agoMerge pull request #51755 from zdover23/wip-doc-2023-05-25-backport-51754-to-reef
Anthony D'Atri [Thu, 25 May 2023 11:14:13 +0000 (07:14 -0400)]
Merge pull request #51755 from zdover23/wip-doc-2023-05-25-backport-51754-to-reef

reef: doc/rados: fix link in common.rst

2 years agodoc/rados: fix link in common.rst 51755/head
Zac Dover [Thu, 25 May 2023 09:01:49 +0000 (19:01 +1000)]
doc/rados: fix link in common.rst

Fix a link in doc/rados/configuration/common.rst that was missing its
final letter, causing a 404 error when readers attempted to follow it.

This bug was reported by stalwart friend of the Ceph documentation
project Eugen Block, who is here credited as a co-author. This bug was
reported at https://pad.ceph.com/p/Report_Documentation_Bugs.

Co-authored-by: Eugen Block <eblock@nde.ag>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 8128ce20fef3d75c6251691d10af21861c01e155)

2 years agoMerge pull request #51751 from zdover23/wip-doc-2023-05-25-backport-51745-to-reef
Anthony D'Atri [Thu, 25 May 2023 10:02:00 +0000 (06:02 -0400)]
Merge pull request #51751 from zdover23/wip-doc-2023-05-25-backport-51745-to-reef

reef: doc/rados: edit filestore-config-ref.rst

2 years agoMerge pull request #51162 from rhcs-dashboard/dependabot-pr-50580
Nizamudeen A [Thu, 25 May 2023 09:29:50 +0000 (14:59 +0530)]
Merge pull request #51162 from rhcs-dashboard/dependabot-pr-50580

reef: mgr/dashboard: bump webpack and @angular-devkit/build-angular

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2 years agoMerge pull request #51077 from aaSharma14/wip-59447-reef
Nizamudeen A [Thu, 25 May 2023 09:28:50 +0000 (14:58 +0530)]
Merge pull request #51077 from aaSharma14/wip-59447-reef

reef: mgr/dashboard: Fix rbd snapshot creation

Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agodoc/rados: edit filestore-config-ref.rst 51751/head
Zac Dover [Thu, 25 May 2023 00:16:51 +0000 (10:16 +1000)]
doc/rados: edit filestore-config-ref.rst

Edit doc/rados/configuration/filestore-config-ref.rst.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 1202e4207bc182d83f193adb1557359d52687103)

2 years agomgr/dashboard: bump webpack and @angular-devkit/build-angular 51162/head
dependabot[bot] [Sat, 18 Mar 2023 13:11:15 +0000 (13:11 +0000)]
mgr/dashboard: bump webpack and @angular-devkit/build-angular

Bumps [webpack](https://github.com/webpack/webpack) to 5.76.1 and updates ancestor dependency [@angular-devkit/build-angular](https://github.com/angular/angular-cli). These dependencies need to be updated together.

Updates `webpack` from 5.70.0 to 5.76.1
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.70.0...v5.76.1)

Updates `@angular-devkit/build-angular` from 13.3.9 to 13.3.11
- [Release notes](https://github.com/angular/angular-cli/releases)
- [Changelog](https://github.com/angular/angular-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular-cli/compare/13.3.9...13.3.11)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: indirect
- dependency-name: "@angular-devkit/build-angular"
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit a6514bd04a78ab71b6b47706942de3dee9055db5)

2 years agoMerge pull request #51739 from zdover23/wip-doc-2023-05-25-backport-50858-to-reef
Anthony D'Atri [Thu, 25 May 2023 02:14:22 +0000 (22:14 -0400)]
Merge pull request #51739 from zdover23/wip-doc-2023-05-25-backport-50858-to-reef

reef: doc/rados/operations: Acting Set question

2 years agoMerge pull request #51742 from zdover23/wip-doc-2023-05-25-backport-51730-to-reef
Anthony D'Atri [Thu, 25 May 2023 02:12:10 +0000 (22:12 -0400)]
Merge pull request #51742 from zdover23/wip-doc-2023-05-25-backport-51730-to-reef

reef: doc/rados/operations: Fix erasure-code-jerasure.rst fix

2 years agodoc/rados/operations: Fix erasure-code-jerasure.rst fix 51742/head
Anthony D'Atri [Wed, 24 May 2023 13:57:42 +0000 (09:57 -0400)]
doc/rados/operations: Fix erasure-code-jerasure.rst fix

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 2a3177e60d93742d3125f3ef0913dfeb8743e8bf)

2 years agodoc/rados/operations: Acting Set question 51739/head
Zac Dover [Tue, 4 Apr 2023 03:59:46 +0000 (13:59 +1000)]
doc/rados/operations: Acting Set question

Clear up a paragraph that describes the behavior of the Acting Set.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 3bc9cfd83b3b836775433c7f1b075fe397b56463)

2 years agorgw/multisite: parse shard_id value correctly to handle num_shards 0 case 51735/head
Shilpa Jagannath [Tue, 18 Apr 2023 20:40:09 +0000 (16:40 -0400)]
rgw/multisite: parse shard_id value correctly to handle num_shards 0 case

For buckets that have num_shards set to 0, bucket instance
will not have a shard_id delimiter. When this bucket instance is parsed,
we end up assigning a value of -1 to shard_id, which is invalid
in data sync. This change ensures that we represent the shard_id correctly
by giving it a valid number

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit f11b3dce832221a344440787333c41f61a9f9c67)

2 years agoMerge pull request #51712 from amathuria/wip-reef-change-default-scrub-cost
Laura Flores [Wed, 24 May 2023 17:06:58 +0000 (12:06 -0500)]
Merge pull request #51712 from amathuria/wip-reef-change-default-scrub-cost

reef: osd: Change scrub cost in case of mClock scheduler

2 years agoMerge pull request #51683 from athanatos/sjust/wip-scrub-59049-reef
Laura Flores [Wed, 24 May 2023 17:06:43 +0000 (12:06 -0500)]
Merge pull request #51683 from athanatos/sjust/wip-scrub-59049-reef

reef: src/osd/scrubber: process MapsCompared event syncronously

2 years agoMerge pull request #51323 from rhcs-dashboard/wip-59621-reef
Nizamudeen A [Wed, 24 May 2023 14:58:27 +0000 (20:28 +0530)]
Merge pull request #51323 from rhcs-dashboard/wip-59621-reef

reef: mgr/dashboard: fix the rbd mirroring configure check

Reviewed-by: Pegonzal <NOT@FOUND>
2 years agocmake: cls_rgw and cls_fifo depend on fmt 51731/head
Casey Bodley [Mon, 22 May 2023 16:33:06 +0000 (12:33 -0400)]
cmake: cls_rgw and cls_fifo depend on fmt

Fixes: https://tracker.ceph.com/issues/58953
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 72164b129ea56b82e1f3e8b58764ba14bb4bf7df)

2 years agoMerge pull request #51711 from Matan-B/wip-51388-reef
Matan [Wed, 24 May 2023 09:11:19 +0000 (11:11 +0200)]
Merge pull request #51711 from Matan-B/wip-51388-reef

reef: qa/suites/crimson: Enhance rbd api testing

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #51716 from Matan-B/wip-51312-reef
Matan [Wed, 24 May 2023 09:10:39 +0000 (11:10 +0200)]
Merge pull request #51716 from Matan-B/wip-51312-reef

reef: crimson/osd/ops_executer: Fix usage of Message's connection

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agorgw/sync-policy: Support disabling per-bucket replication 51717/head
Soumya Koduri [Fri, 20 Jan 2023 10:33:15 +0000 (16:03 +0530)]
rgw/sync-policy: Support disabling per-bucket replication

When the zones replicate, allow disabling replication
for specific buckets using sync policy.

These are the semantics to be followed while resolving the policy
conflicts -

==================================================
zonegroup bucket Result
==================================================
enabled enabled enabled
allowed enabled
forbidden disabled
allowed enabled enabled
allowed disabled
forbidden disabled
forbidden enabled disabled
allowed disabled
forbidden disabled

In case multiple group policies are set to reflect for any sync pair
(<source-zone,source-bucket>, <dest-zone,dest-bucket>), the following
rules are applied in the order -
1) Even if one policy status is FORBIDDEN, the sync will be disabled
2) Atleast one policy should be ENABLED for the sync to be allowed.

Various cases tested are outlined here -
https://docs.google.com/document/d/19oBQA-bYxLBR4BnekA2DTwJJaTFvjAfrqAk9G3RGU0I/edit#heading=h.4qac9dpc76m

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit aeb3a746180675bc5fcfaea4af839daadb0abdd7)

2 years agoMerge pull request #51725 from zdover23/wip-doc-2023-05-24-backport-51273-to-reef
zdover23 [Wed, 24 May 2023 03:15:18 +0000 (13:15 +1000)]
Merge pull request #51725 from zdover23/wip-doc-2023-05-24-backport-51273-to-reef

reef: doc: Update jerasure.org references

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc: Update jerasure.org references 51725/head
Anthony D'Atri [Tue, 23 May 2023 23:13:33 +0000 (19:13 -0400)]
doc: Update jerasure.org references

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 5e60e0de275f7260aeae9e664ca22ebfdf8fc5f9)

2 years agorgw/multisite: fix ret handling in bucket read_sync_status() 51724/head
Shilpa Jagannath [Mon, 3 Apr 2023 17:44:35 +0000 (13:44 -0400)]
rgw/multisite: fix ret handling in bucket read_sync_status()

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 91642f4757f91eee45df0b18f79cc3dc38c46d63)

2 years agoMerge pull request #51615 from adk3798/wip-59636-reef
Adam King [Tue, 23 May 2023 23:18:48 +0000 (19:18 -0400)]
Merge pull request #51615 from adk3798/wip-59636-reef

reef: mgr/cephadm: allow configuring anonymous access for grafana

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #51663 from sseshasa/wip-61336-reef
Yuri Weinstein [Tue, 23 May 2023 22:59:10 +0000 (18:59 -0400)]
Merge pull request #51663 from sseshasa/wip-61336-reef

reef: osd/scheduler: Reset ephemeral changes to mClock built-in profile

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51666 from amathuria/wip-61303-reef
Yuri Weinstein [Tue, 23 May 2023 22:57:24 +0000 (18:57 -0400)]
Merge pull request #51666 from amathuria/wip-61303-reef

reef: mClockScheduler: Set priority cutoff in the mClock Scheduler

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agoMerge pull request #51720 from zdover23/wip-doc-2023-05-24-backport-51679-to-reef
Anthony D'Atri [Tue, 23 May 2023 22:29:08 +0000 (18:29 -0400)]
Merge pull request #51720 from zdover23/wip-doc-2023-05-24-backport-51679-to-reef

reef: doc/mgr: edit "leaderboard" in telemetry.rst

2 years agoMerge pull request #51714 from ljflores/wip-backport-pr-50482
Laura Flores [Tue, 23 May 2023 22:14:43 +0000 (17:14 -0500)]
Merge pull request #51714 from ljflores/wip-backport-pr-50482

reef: .github/milestone.yml: add reef

2 years agodoc/mgr: edit "leaderboard" in telemetry.rst 51720/head
Zac Dover [Mon, 22 May 2023 20:06:52 +0000 (06:06 +1000)]
doc/mgr: edit "leaderboard" in telemetry.rst

Standardize the presentation of commands in the "Leaderboard" section of
doc/mgr/telemetry.rst.

Follow-up to https://github.com/ceph/ceph/pull/50977

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 4935ad8aed2aa796015473de5b38cc973ba85ba1)

2 years agoMerge pull request #51128 from petrutlucian94/wip-59480-reef
Ilya Dryomov [Tue, 23 May 2023 18:59:28 +0000 (20:59 +0200)]
Merge pull request #51128 from petrutlucian94/wip-59480-reef

reef: win32_deps_build: avoid pip

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge PR #50755 into reef
Patrick Donnelly [Tue, 23 May 2023 18:21:42 +0000 (14:21 -0400)]
Merge PR #50755 into reef

* refs/pull/50755/head:
mds: add config to decide whether to mark dentry bad
qa: add missing scan_links step for data scan recovery
qa/tasks/cephfs: test damage to dentry's first is caught
qa/tasks/cephfs: use rank_asok and allow specifying rank
qa/tasks: allow specifying timeout command prefix to ceph
mds: provide test configs for creating first corruption
mds: catch damage to dentry's first field
mds: add debugging for pre_cow_old_inode
mds: cleanup code

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2 years agorgw/multisite: Fix issue with parsing replication rules
Soumya Koduri [Tue, 16 May 2023 10:10:36 +0000 (15:40 +0530)]
rgw/multisite: Fix issue with parsing replication rules

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 570de795721d13ddd6d39e28416b6d6f48477501)

2 years agorgw/archive: Correct the bucket sync status cmd output
Soumya Koduri [Mon, 20 Mar 2023 12:15:33 +0000 (17:45 +0530)]
rgw/archive: Correct the bucket sync status cmd output

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit d7af6c228253d05a16dd3be993f150518f1b6f66)

2 years agorgw: Disable logging for datalog and bilog if zone doesn't export data
Soumya Koduri [Sat, 8 Apr 2023 12:23:24 +0000 (17:53 +0530)]
rgw: Disable logging for datalog and bilog if zone doesn't export data

Zones (such as archive zone) which do not export data should have
sync logging disabled.

Fixes# https://tracker.ceph.com/issues/59106

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 85f0cb592aa0d755c101d2e2845ad736a7697e3e)

2 years agorgw/cloud-transition: New attrs to detect cloudtiered objects
Soumya Koduri [Tue, 25 Apr 2023 18:04:28 +0000 (23:34 +0530)]
rgw/cloud-transition: New attrs to detect cloudtiered objects

Add new attrs "RGW_ATTR_CLOUD_TIER_TYPE" and "RGW_ATTR_CLOUD_TIER_CONFIG"
to store details about cloud-tiered objects so that they get synced accordingly
in a multisite environment.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 0f690117439384827cdfffaf662b71a99f73b34a)

2 years agorgw: Dump tier_* config in RGWObjManifest
Soumya Koduri [Mon, 24 Apr 2023 17:44:15 +0000 (23:14 +0530)]
rgw: Dump tier_* config in RGWObjManifest

This was missed as part of refactoring done to encoder
and decoder methods (https://github.com/ceph/ceph/commit/2414c7584eb90bfff42d512879a0aa6b220d359e)

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 4db632567d29ebc5df753faac9c80444884cc785)

2 years agorgw/cloudtransition: Allow multisite zones to sync cloudtiered objects
Soumya Koduri [Wed, 19 Apr 2023 18:03:28 +0000 (23:33 +0530)]
rgw/cloudtransition: Allow multisite zones to sync cloudtiered objects

In a multisite configuration, zones should be able to fetch & sync
cloud-transitioned objects as well. To allow this, a new header
'x-rgwx-sync-cloudtiered' is added to be used by sync client to GET
such objects.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit cec19b09efc5a56a5f5600000675b951428b98ce)

2 years agorgw/cloudtier: Fix bug with decoding tier_targets
Soumya Koduri [Wed, 19 Apr 2023 11:14:24 +0000 (16:44 +0530)]
rgw/cloudtier: Fix bug with decoding tier_targets

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit e33946b89ee9a3c61f0c4e08597f67b42ca4850b)

2 years agorgw/rados: cosmetic cleanup of attr handling
Casey Bodley [Sun, 19 Feb 2023 22:48:11 +0000 (17:48 -0500)]
rgw/rados: cosmetic cleanup of attr handling

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8ee9b3fc5e7e81a79d637593eca9add365c0b784)

2 years agorgw/rados: don't use attrset[] to create entries that don't exist
Casey Bodley [Sun, 19 Feb 2023 22:47:43 +0000 (17:47 -0500)]
rgw/rados: don't use attrset[] to create entries that don't exist

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 33273476730c70f4a5103151c36947030f11348e)

2 years agorgw: add s3 error message to cloud-tiered objects
Casey Bodley [Mon, 20 Feb 2023 13:52:39 +0000 (08:52 -0500)]
rgw: add s3 error message to cloud-tiered objects

better to tell the s3 client why their request failed than to log it as
an ERROR; this is normal/expected when cloud tiering is enabled

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 5713a040c38c5c17fa2ae4913067a602a412b639)

2 years agorgw: catch manifest decode errors in GetObj/PutObj/CopyObj
Casey Bodley [Sun, 19 Feb 2023 22:24:48 +0000 (17:24 -0500)]
rgw: catch manifest decode errors in GetObj/PutObj/CopyObj

if we read an object that doesn't have a manifest attribute,
RGWRados::get_obj_state_impl() adds an empty one and causes decode to
throw an end_of_buffer exception

Fixes: https://tracker.ceph.com/issues/58794
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 77e0b9848cb458427b5dcf86d2c19769ea65e0bc)

2 years agocrimson/osd/ops_executer: Fix usage of Message's connection 51716/head
Matan Breizman [Tue, 2 May 2023 09:07:00 +0000 (09:07 +0000)]
crimson/osd/ops_executer: Fix usage of Message's connection

See #50835.
In crimson, conn is independently maintained outside Message.
Therefore, when trying to use the message's connection for
`get_orig_source_inst()` we won't be able to get the peer address.

Fixes: https://tracker.ceph.com/issues/59589
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit d569703a5f35a48ef042d61702eaa86a6e2d48bd)

2 years agoMerge pull request #51710 from Matan-B/wip-47749-reef
Matan [Tue, 23 May 2023 15:56:11 +0000 (17:56 +0200)]
Merge pull request #51710 from Matan-B/wip-47749-reef

reef: crimson/os/seastore/btree: link fixedkvbtree's nodes and logical extents with forward and backward pointers, and drop the pin_set

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51709 from Matan-B/wip-51381-reef
Matan [Tue, 23 May 2023 15:55:36 +0000 (17:55 +0200)]
Merge pull request #51709 from Matan-B/wip-51381-reef

reef: crimson/osd/osd_operations/client_request: Fix client blocklisting

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51708 from Matan-B/wip-48717-reef
Matan [Tue, 23 May 2023 15:55:09 +0000 (17:55 +0200)]
Merge pull request #51708 from Matan-B/wip-48717-reef

reef: crimson/os: support multicore seastore

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51707 from Matan-B/wip-50835-reef
Matan [Tue, 23 May 2023 15:54:36 +0000 (17:54 +0200)]
Merge pull request #51707 from Matan-B/wip-50835-reef

reef: crimson/net: change ConnectionRef to be a local_shared_foreign_ptr

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years ago.github/milestone.yml: add reef 51714/head
Neha Ojha [Fri, 10 Mar 2023 22:13:17 +0000 (22:13 +0000)]
.github/milestone.yml: add reef

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 573d0ff5d58a07a8ae27daea7cc69cbe323b5865)

2 years agoMerge pull request #51248 from mchangir/wip-59411-reef
Yuri Weinstein [Tue, 23 May 2023 14:52:53 +0000 (10:52 -0400)]
Merge pull request #51248 from mchangir/wip-59411-reef

reef: mgr/snap_schedule: add debug log for paths failing snapshot creation

Reviewed-by: Kotresh HR <khiremat@redhat.com>
2 years agoMerge pull request #51142 from joscollin/wip-59481-reef
Yuri Weinstein [Tue, 23 May 2023 14:51:57 +0000 (10:51 -0400)]
Merge pull request #51142 from joscollin/wip-59481-reef

reef: tools/cephfs/top/CMakeList.txt: check the minimum compatible python version for cephfs-top

Reviewed-by: Kotresh HR <khiremat@redhat.com>
2 years agoMerge pull request #51704 from petrutlucian94/wip-61365-reef
Ilya Dryomov [Tue, 23 May 2023 14:48:59 +0000 (16:48 +0200)]
Merge pull request #51704 from petrutlucian94/wip-61365-reef

reef: rbd-wnbd: improve image map error message

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoosd: mClock default_min reservation should be 0 51712/head
Aishwarya Mathuria [Tue, 23 May 2023 10:43:08 +0000 (16:13 +0530)]
osd: mClock default_min reservation should be 0

All the mClock profiles enforce no minimum reservation for the background best-effort client.
For this purpose, the default_min value should 0 and not 1.

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
(cherry picked from commit 05848b4957713c5c98c249e15eb8de01d1c6a1e6)

2 years agoosd: Change scrub cost in case of mClock scheduler
Aishwarya Mathuria [Fri, 19 May 2023 11:46:11 +0000 (17:16 +0530)]
osd: Change scrub cost in case of mClock scheduler

With osd_op_queue as WPQ, high costs were assigned to scrub in order to throttle it effectively.
In the case of mClock scheduler, mClock parameters are used to do the throttling and the cost should represent a realistic value.

Fixes: https://tracker.ceph.com/issues/61313
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
(cherry picked from commit 43544d296c5fc14aaf638ddf21de9ea9d99f6059)

2 years agoqa/suites/crimson: Introduce rbd_python_api_tests.yaml 51711/head
Matan Breizman [Mon, 8 May 2023 10:53:00 +0000 (10:53 +0000)]
qa/suites/crimson: Introduce rbd_python_api_tests.yaml

Test python api with new image format.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 01958e648ef6ea0689e30e946be71b28a641303c)

2 years agoqa/suites/crimson: Skip unsupported tests (Crimson)
Matan Breizman [Mon, 8 May 2023 10:50:19 +0000 (10:50 +0000)]
qa/suites/crimson: Skip unsupported tests (Crimson)

Align with `rbd_api_tests` and skip deep_copy and breaklock tests
in Crimson.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 5823c04542c2a6870cb038da752e752ea2a3cf9a)

2 years agotest/*rbd: Enable supported Crimson test
Matan Breizman [Wed, 3 May 2023 10:38:55 +0000 (10:38 +0000)]
test/*rbd: Enable supported Crimson test

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 6895fb33e8308d8bfabe14d92e156b02044b9c8e)

2 years agocrimson/tools/store_nbd: read logical extents via 51710/head
Xuehan Xu [Mon, 8 May 2023 08:15:55 +0000 (08:15 +0000)]
crimson/tools/store_nbd: read logical extents via
TransactionManager::read_pin()

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 33b56a04d51443f5f035bfc267da16d4a5dfe28c)

2 years agocrimson/os/seastore/cache: add comment about backref_extent_entry_t
Xuehan Xu [Thu, 23 Mar 2023 09:59:12 +0000 (09:59 +0000)]
crimson/os/seastore/cache: add comment about backref_extent_entry_t

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 62974a65897edce2afdb16ba99baea0193a0d4a6)

2 years agotest/crimson/seastore: complement lba test with logical extents
Xuehan Xu [Sat, 11 Mar 2023 03:46:14 +0000 (03:46 +0000)]
test/crimson/seastore: complement lba test with logical extents

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 3c4f8c761333dcc1a4e24e73f37808720c8f684f)

2 years agotest/crimson/seastore: check intra-fixedkv-btree parent->child trackers during unittests
Xuehan Xu [Mon, 29 Aug 2022 08:12:00 +0000 (16:12 +0800)]
test/crimson/seastore: check intra-fixedkv-btree parent->child trackers during unittests

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 302bc3c2d95cd36012008e334444f689ddb6694f)

2 years agocrimson/os/seastore/btree: drop btree_pin_set_t
Xuehan Xu [Mon, 27 Mar 2023 02:20:59 +0000 (02:20 +0000)]
crimson/os/seastore/btree: drop btree_pin_set_t

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 4a3dfc0f630d6e635bd82801e0107be78d3d2c6d)

2 years agocrimson/os/seastore/transaction_manager: follow leaf<->logical extent pointers to...
Xuehan Xu [Sat, 6 May 2023 09:26:18 +0000 (17:26 +0800)]
crimson/os/seastore/transaction_manager: follow leaf<->logical extent pointers to read extent

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 89c2d0b3af7feb4124b7a5ac7f54026c205663b5)

2 years agocrimson/os/seastore/lba_manager: link lba leaf nodes with logical extents by pointers
Xuehan Xu [Tue, 25 Oct 2022 06:03:43 +0000 (14:03 +0800)]
crimson/os/seastore/lba_manager: link lba leaf nodes with logical extents by pointers

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit cce850d75609c7c34bce0920e4e12ba9b9513229)

2 years agocrimson/os/seastore/btree: "templatize" btree leaf node to distinguish leaf nodes...
Xuehan Xu [Thu, 27 Oct 2022 07:21:32 +0000 (15:21 +0800)]
crimson/os/seastore/btree: "templatize" btree leaf node to distinguish leaf nodes with(out) children

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 55e1924e3818e7fb574893372ba7dfad4fa54014)

2 years agocrimson/os/seastore/btree: link fixed-kv-btree and root_block with pointers
Xuehan Xu [Thu, 20 Oct 2022 09:41:25 +0000 (17:41 +0800)]
crimson/os/seastore/btree: link fixed-kv-btree and root_block with pointers

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 4d9b60e75047fa0d7e5acce601bc15f49d253349)

2 years agocrimson/os/seastore: more debug logs
Xuehan Xu [Thu, 20 Oct 2022 05:35:08 +0000 (13:35 +0800)]
crimson/os/seastore: more debug logs

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 25b001db29bbd58c627f3ab64428f5aaac14a399)

2 years agocrimson/os/seastore/backref_manager: retrieve live backref extents throught the backr...
Xuehan Xu [Wed, 17 Aug 2022 10:07:42 +0000 (18:07 +0800)]
crimson/os/seastore/backref_manager: retrieve live backref extents throught the backref tree

After involving intra-fixed-kv-btree parent-child pointers, we need to keep the
invariant that it's only when extents are not in transactions' read_set that
we can directly query cache with inspecting the transaction

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 45440fadd20fa21deaddbe6db4e0c4e84015c9bf)

2 years agocrimson/os/seastore/btree: avoid searching transactions' read_set when retrieving...
Xuehan Xu [Thu, 13 Oct 2022 06:27:34 +0000 (14:27 +0800)]
crimson/os/seastore/btree: avoid searching transactions' read_set when retrieving btree nodes

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit c29051c4c747c5b8415b8600affc24c0025507fb)

2 years agocrimson/os/seastore/btree: search fixed-kv-btree by parent<->child pointers
Xuehan Xu [Thu, 13 Oct 2022 03:50:17 +0000 (11:50 +0800)]
crimson/os/seastore/btree: search fixed-kv-btree by parent<->child pointers

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 7c3305f0149808f800dc6c852d4f1f755a490ec4)

2 years agocrimson/os/seastore/cache: invalidate out-dated extent when initiating Cache
Xuehan Xu [Thu, 13 Oct 2022 02:57:09 +0000 (10:57 +0800)]
crimson/os/seastore/cache: invalidate out-dated extent when initiating Cache

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 686d1206532200733d69b9430227bdb6bf2d9eb3)

2 years agocrimson/os/seastore/cached_extent: improve the representation of "has_been_invalidated"
Xuehan Xu [Wed, 12 Oct 2022 06:37:39 +0000 (14:37 +0800)]
crimson/os/seastore/cached_extent: improve the representation of "has_been_invalidated"

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 1b4c591ef51920211de978f490b4125177571e08)

2 years agocrimson/os/seastore/btree: don't go to leaf nodes when updating internal mappings
Xuehan Xu [Tue, 31 Jan 2023 06:36:42 +0000 (14:36 +0800)]
crimson/os/seastore/btree: don't go to leaf nodes when updating internal mappings

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit a86c7bd651bc7a89a49e57f9a87e8baeb4291c41)

2 years agocrimson/os/seastore/btree: introduce parent<->child pointers for fixed-kv-btree nodes
Xuehan Xu [Tue, 11 Oct 2022 02:34:16 +0000 (10:34 +0800)]
crimson/os/seastore/btree: introduce parent<->child pointers for fixed-kv-btree nodes

maintain correct parent<->child pointers when modifying the btree

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 71051f997fca8ff0fdcd55586e3150bffa2a916a)

2 years agocrimson/os/seastore/cached_extent: add on_invalidated interface
Xuehan Xu [Tue, 16 Aug 2022 09:05:52 +0000 (17:05 +0800)]
crimson/os/seastore/cached_extent: add on_invalidated interface

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit e5aabe6c306061b2ce3aff5ca30a9049c1a7f9bf)

2 years agocrimson/os/seastore/cached_extent: add on_replace_prior interface
Xuehan Xu [Tue, 16 Aug 2022 08:48:28 +0000 (16:48 +0800)]
crimson/os/seastore/cached_extent: add on_replace_prior interface

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit c56af30a9c199b4a5d2cba9d2208b8c1d8a6abfb)

2 years agocrimson/os/seastore/cached_extent: duplicate_for_write accepts the transaction ref...
Xuehan Xu [Tue, 13 Sep 2022 07:53:53 +0000 (15:53 +0800)]
crimson/os/seastore/cached_extent: duplicate_for_write accepts the transaction ref as its parameter

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 06ae16141470a9052224fbe76c842fa13bbf4520)

2 years agocrimson/os/seastore/cached_extent: record transaction id in pending CachedExtents
Xuehan Xu [Tue, 13 Sep 2022 07:50:35 +0000 (15:50 +0800)]
crimson/os/seastore/cached_extent: record transaction id in pending CachedExtents

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 8240be8f324b244061b17bb2c1676b192a12dca2)

2 years agocrimson/osd/osd_operations/client_request: Fix client blocklisting 51709/head
Matan Breizman [Sun, 7 May 2023 13:29:22 +0000 (13:29 +0000)]
crimson/osd/osd_operations/client_request: Fix client blocklisting

See #50835.
In crimson, conn is independently maintained outside Message.
Therefore, when trying to use the message's connection for `get_peer_addr()`
we won't be able to get the peer address.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 9c1531ca404582725bfcb758dd489626fda31454)

2 years agocrimson/os: make ZNS support shard metadata 51708/head
chunmei-liu [Tue, 22 Nov 2022 16:59:56 +0000 (08:59 -0800)]
crimson/os: make ZNS support shard metadata

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
(cherry picked from commit ad5b0c12a788bca998cb95bfbaa745d9856c56ba)

2 years agocrimson/os: fix formatter build error
chunmei [Thu, 6 Apr 2023 07:19:40 +0000 (07:19 +0000)]
crimson/os: fix formatter build error

Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit c401a4af889fb68468447453716059cb73eb405a)