]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agomgr/dashboard: bump webpack and @angular-devkit/build-angular 51162/head
dependabot[bot] [Sat, 18 Mar 2023 13:11:15 +0000 (13:11 +0000)]
mgr/dashboard: bump webpack and @angular-devkit/build-angular

Bumps [webpack](https://github.com/webpack/webpack) to 5.76.1 and updates ancestor dependency [@angular-devkit/build-angular](https://github.com/angular/angular-cli). These dependencies need to be updated together.

Updates `webpack` from 5.70.0 to 5.76.1
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.70.0...v5.76.1)

Updates `@angular-devkit/build-angular` from 13.3.9 to 13.3.11
- [Release notes](https://github.com/angular/angular-cli/releases)
- [Changelog](https://github.com/angular/angular-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular-cli/compare/13.3.9...13.3.11)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: indirect
- dependency-name: "@angular-devkit/build-angular"
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit a6514bd04a78ab71b6b47706942de3dee9055db5)

2 years agoMerge pull request #51742 from zdover23/wip-doc-2023-05-25-backport-51730-to-reef
Anthony D'Atri [Thu, 25 May 2023 02:12:10 +0000 (22:12 -0400)]
Merge pull request #51742 from zdover23/wip-doc-2023-05-25-backport-51730-to-reef

reef: doc/rados/operations: Fix erasure-code-jerasure.rst fix

2 years agodoc/rados/operations: Fix erasure-code-jerasure.rst fix 51742/head
Anthony D'Atri [Wed, 24 May 2023 13:57:42 +0000 (09:57 -0400)]
doc/rados/operations: Fix erasure-code-jerasure.rst fix

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 2a3177e60d93742d3125f3ef0913dfeb8743e8bf)

2 years agoMerge pull request #51712 from amathuria/wip-reef-change-default-scrub-cost
Laura Flores [Wed, 24 May 2023 17:06:58 +0000 (12:06 -0500)]
Merge pull request #51712 from amathuria/wip-reef-change-default-scrub-cost

reef: osd: Change scrub cost in case of mClock scheduler

2 years agoMerge pull request #51683 from athanatos/sjust/wip-scrub-59049-reef
Laura Flores [Wed, 24 May 2023 17:06:43 +0000 (12:06 -0500)]
Merge pull request #51683 from athanatos/sjust/wip-scrub-59049-reef

reef: src/osd/scrubber: process MapsCompared event syncronously

2 years agoMerge pull request #51323 from rhcs-dashboard/wip-59621-reef
Nizamudeen A [Wed, 24 May 2023 14:58:27 +0000 (20:28 +0530)]
Merge pull request #51323 from rhcs-dashboard/wip-59621-reef

reef: mgr/dashboard: fix the rbd mirroring configure check

Reviewed-by: Pegonzal <NOT@FOUND>
2 years agoMerge pull request #51711 from Matan-B/wip-51388-reef
Matan [Wed, 24 May 2023 09:11:19 +0000 (11:11 +0200)]
Merge pull request #51711 from Matan-B/wip-51388-reef

reef: qa/suites/crimson: Enhance rbd api testing

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #51716 from Matan-B/wip-51312-reef
Matan [Wed, 24 May 2023 09:10:39 +0000 (11:10 +0200)]
Merge pull request #51716 from Matan-B/wip-51312-reef

reef: crimson/osd/ops_executer: Fix usage of Message's connection

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51725 from zdover23/wip-doc-2023-05-24-backport-51273-to-reef
zdover23 [Wed, 24 May 2023 03:15:18 +0000 (13:15 +1000)]
Merge pull request #51725 from zdover23/wip-doc-2023-05-24-backport-51273-to-reef

reef: doc: Update jerasure.org references

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc: Update jerasure.org references 51725/head
Anthony D'Atri [Tue, 23 May 2023 23:13:33 +0000 (19:13 -0400)]
doc: Update jerasure.org references

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 5e60e0de275f7260aeae9e664ca22ebfdf8fc5f9)

2 years agoMerge pull request #51615 from adk3798/wip-59636-reef
Adam King [Tue, 23 May 2023 23:18:48 +0000 (19:18 -0400)]
Merge pull request #51615 from adk3798/wip-59636-reef

reef: mgr/cephadm: allow configuring anonymous access for grafana

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #51663 from sseshasa/wip-61336-reef
Yuri Weinstein [Tue, 23 May 2023 22:59:10 +0000 (18:59 -0400)]
Merge pull request #51663 from sseshasa/wip-61336-reef

reef: osd/scheduler: Reset ephemeral changes to mClock built-in profile

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51666 from amathuria/wip-61303-reef
Yuri Weinstein [Tue, 23 May 2023 22:57:24 +0000 (18:57 -0400)]
Merge pull request #51666 from amathuria/wip-61303-reef

reef: mClockScheduler: Set priority cutoff in the mClock Scheduler

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agoMerge pull request #51720 from zdover23/wip-doc-2023-05-24-backport-51679-to-reef
Anthony D'Atri [Tue, 23 May 2023 22:29:08 +0000 (18:29 -0400)]
Merge pull request #51720 from zdover23/wip-doc-2023-05-24-backport-51679-to-reef

reef: doc/mgr: edit "leaderboard" in telemetry.rst

2 years agoMerge pull request #51714 from ljflores/wip-backport-pr-50482
Laura Flores [Tue, 23 May 2023 22:14:43 +0000 (17:14 -0500)]
Merge pull request #51714 from ljflores/wip-backport-pr-50482

reef: .github/milestone.yml: add reef

2 years agodoc/mgr: edit "leaderboard" in telemetry.rst 51720/head
Zac Dover [Mon, 22 May 2023 20:06:52 +0000 (06:06 +1000)]
doc/mgr: edit "leaderboard" in telemetry.rst

Standardize the presentation of commands in the "Leaderboard" section of
doc/mgr/telemetry.rst.

Follow-up to https://github.com/ceph/ceph/pull/50977

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 4935ad8aed2aa796015473de5b38cc973ba85ba1)

2 years agoMerge pull request #51128 from petrutlucian94/wip-59480-reef
Ilya Dryomov [Tue, 23 May 2023 18:59:28 +0000 (20:59 +0200)]
Merge pull request #51128 from petrutlucian94/wip-59480-reef

reef: win32_deps_build: avoid pip

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge PR #50755 into reef
Patrick Donnelly [Tue, 23 May 2023 18:21:42 +0000 (14:21 -0400)]
Merge PR #50755 into reef

* refs/pull/50755/head:
mds: add config to decide whether to mark dentry bad
qa: add missing scan_links step for data scan recovery
qa/tasks/cephfs: test damage to dentry's first is caught
qa/tasks/cephfs: use rank_asok and allow specifying rank
qa/tasks: allow specifying timeout command prefix to ceph
mds: provide test configs for creating first corruption
mds: catch damage to dentry's first field
mds: add debugging for pre_cow_old_inode
mds: cleanup code

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2 years agocrimson/osd/ops_executer: Fix usage of Message's connection 51716/head
Matan Breizman [Tue, 2 May 2023 09:07:00 +0000 (09:07 +0000)]
crimson/osd/ops_executer: Fix usage of Message's connection

See #50835.
In crimson, conn is independently maintained outside Message.
Therefore, when trying to use the message's connection for
`get_orig_source_inst()` we won't be able to get the peer address.

Fixes: https://tracker.ceph.com/issues/59589
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit d569703a5f35a48ef042d61702eaa86a6e2d48bd)

2 years agoMerge pull request #51710 from Matan-B/wip-47749-reef
Matan [Tue, 23 May 2023 15:56:11 +0000 (17:56 +0200)]
Merge pull request #51710 from Matan-B/wip-47749-reef

reef: crimson/os/seastore/btree: link fixedkvbtree's nodes and logical extents with forward and backward pointers, and drop the pin_set

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51709 from Matan-B/wip-51381-reef
Matan [Tue, 23 May 2023 15:55:36 +0000 (17:55 +0200)]
Merge pull request #51709 from Matan-B/wip-51381-reef

reef: crimson/osd/osd_operations/client_request: Fix client blocklisting

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51708 from Matan-B/wip-48717-reef
Matan [Tue, 23 May 2023 15:55:09 +0000 (17:55 +0200)]
Merge pull request #51708 from Matan-B/wip-48717-reef

reef: crimson/os: support multicore seastore

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51707 from Matan-B/wip-50835-reef
Matan [Tue, 23 May 2023 15:54:36 +0000 (17:54 +0200)]
Merge pull request #51707 from Matan-B/wip-50835-reef

reef: crimson/net: change ConnectionRef to be a local_shared_foreign_ptr

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years ago.github/milestone.yml: add reef 51714/head
Neha Ojha [Fri, 10 Mar 2023 22:13:17 +0000 (22:13 +0000)]
.github/milestone.yml: add reef

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 573d0ff5d58a07a8ae27daea7cc69cbe323b5865)

2 years agoMerge pull request #51248 from mchangir/wip-59411-reef
Yuri Weinstein [Tue, 23 May 2023 14:52:53 +0000 (10:52 -0400)]
Merge pull request #51248 from mchangir/wip-59411-reef

reef: mgr/snap_schedule: add debug log for paths failing snapshot creation

Reviewed-by: Kotresh HR <khiremat@redhat.com>
2 years agoMerge pull request #51142 from joscollin/wip-59481-reef
Yuri Weinstein [Tue, 23 May 2023 14:51:57 +0000 (10:51 -0400)]
Merge pull request #51142 from joscollin/wip-59481-reef

reef: tools/cephfs/top/CMakeList.txt: check the minimum compatible python version for cephfs-top

Reviewed-by: Kotresh HR <khiremat@redhat.com>
2 years agoMerge pull request #51704 from petrutlucian94/wip-61365-reef
Ilya Dryomov [Tue, 23 May 2023 14:48:59 +0000 (16:48 +0200)]
Merge pull request #51704 from petrutlucian94/wip-61365-reef

reef: rbd-wnbd: improve image map error message

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoosd: mClock default_min reservation should be 0 51712/head
Aishwarya Mathuria [Tue, 23 May 2023 10:43:08 +0000 (16:13 +0530)]
osd: mClock default_min reservation should be 0

All the mClock profiles enforce no minimum reservation for the background best-effort client.
For this purpose, the default_min value should 0 and not 1.

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
(cherry picked from commit 05848b4957713c5c98c249e15eb8de01d1c6a1e6)

2 years agoosd: Change scrub cost in case of mClock scheduler
Aishwarya Mathuria [Fri, 19 May 2023 11:46:11 +0000 (17:16 +0530)]
osd: Change scrub cost in case of mClock scheduler

With osd_op_queue as WPQ, high costs were assigned to scrub in order to throttle it effectively.
In the case of mClock scheduler, mClock parameters are used to do the throttling and the cost should represent a realistic value.

Fixes: https://tracker.ceph.com/issues/61313
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
(cherry picked from commit 43544d296c5fc14aaf638ddf21de9ea9d99f6059)

2 years agoqa/suites/crimson: Introduce rbd_python_api_tests.yaml 51711/head
Matan Breizman [Mon, 8 May 2023 10:53:00 +0000 (10:53 +0000)]
qa/suites/crimson: Introduce rbd_python_api_tests.yaml

Test python api with new image format.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 01958e648ef6ea0689e30e946be71b28a641303c)

2 years agoqa/suites/crimson: Skip unsupported tests (Crimson)
Matan Breizman [Mon, 8 May 2023 10:50:19 +0000 (10:50 +0000)]
qa/suites/crimson: Skip unsupported tests (Crimson)

Align with `rbd_api_tests` and skip deep_copy and breaklock tests
in Crimson.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 5823c04542c2a6870cb038da752e752ea2a3cf9a)

2 years agotest/*rbd: Enable supported Crimson test
Matan Breizman [Wed, 3 May 2023 10:38:55 +0000 (10:38 +0000)]
test/*rbd: Enable supported Crimson test

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 6895fb33e8308d8bfabe14d92e156b02044b9c8e)

2 years agocrimson/tools/store_nbd: read logical extents via 51710/head
Xuehan Xu [Mon, 8 May 2023 08:15:55 +0000 (08:15 +0000)]
crimson/tools/store_nbd: read logical extents via
TransactionManager::read_pin()

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 33b56a04d51443f5f035bfc267da16d4a5dfe28c)

2 years agocrimson/os/seastore/cache: add comment about backref_extent_entry_t
Xuehan Xu [Thu, 23 Mar 2023 09:59:12 +0000 (09:59 +0000)]
crimson/os/seastore/cache: add comment about backref_extent_entry_t

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 62974a65897edce2afdb16ba99baea0193a0d4a6)

2 years agotest/crimson/seastore: complement lba test with logical extents
Xuehan Xu [Sat, 11 Mar 2023 03:46:14 +0000 (03:46 +0000)]
test/crimson/seastore: complement lba test with logical extents

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 3c4f8c761333dcc1a4e24e73f37808720c8f684f)

2 years agotest/crimson/seastore: check intra-fixedkv-btree parent->child trackers during unittests
Xuehan Xu [Mon, 29 Aug 2022 08:12:00 +0000 (16:12 +0800)]
test/crimson/seastore: check intra-fixedkv-btree parent->child trackers during unittests

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 302bc3c2d95cd36012008e334444f689ddb6694f)

2 years agocrimson/os/seastore/btree: drop btree_pin_set_t
Xuehan Xu [Mon, 27 Mar 2023 02:20:59 +0000 (02:20 +0000)]
crimson/os/seastore/btree: drop btree_pin_set_t

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 4a3dfc0f630d6e635bd82801e0107be78d3d2c6d)

2 years agocrimson/os/seastore/transaction_manager: follow leaf<->logical extent pointers to...
Xuehan Xu [Sat, 6 May 2023 09:26:18 +0000 (17:26 +0800)]
crimson/os/seastore/transaction_manager: follow leaf<->logical extent pointers to read extent

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 89c2d0b3af7feb4124b7a5ac7f54026c205663b5)

2 years agocrimson/os/seastore/lba_manager: link lba leaf nodes with logical extents by pointers
Xuehan Xu [Tue, 25 Oct 2022 06:03:43 +0000 (14:03 +0800)]
crimson/os/seastore/lba_manager: link lba leaf nodes with logical extents by pointers

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit cce850d75609c7c34bce0920e4e12ba9b9513229)

2 years agocrimson/os/seastore/btree: "templatize" btree leaf node to distinguish leaf nodes...
Xuehan Xu [Thu, 27 Oct 2022 07:21:32 +0000 (15:21 +0800)]
crimson/os/seastore/btree: "templatize" btree leaf node to distinguish leaf nodes with(out) children

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 55e1924e3818e7fb574893372ba7dfad4fa54014)

2 years agocrimson/os/seastore/btree: link fixed-kv-btree and root_block with pointers
Xuehan Xu [Thu, 20 Oct 2022 09:41:25 +0000 (17:41 +0800)]
crimson/os/seastore/btree: link fixed-kv-btree and root_block with pointers

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 4d9b60e75047fa0d7e5acce601bc15f49d253349)

2 years agocrimson/os/seastore: more debug logs
Xuehan Xu [Thu, 20 Oct 2022 05:35:08 +0000 (13:35 +0800)]
crimson/os/seastore: more debug logs

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 25b001db29bbd58c627f3ab64428f5aaac14a399)

2 years agocrimson/os/seastore/backref_manager: retrieve live backref extents throught the backr...
Xuehan Xu [Wed, 17 Aug 2022 10:07:42 +0000 (18:07 +0800)]
crimson/os/seastore/backref_manager: retrieve live backref extents throught the backref tree

After involving intra-fixed-kv-btree parent-child pointers, we need to keep the
invariant that it's only when extents are not in transactions' read_set that
we can directly query cache with inspecting the transaction

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 45440fadd20fa21deaddbe6db4e0c4e84015c9bf)

2 years agocrimson/os/seastore/btree: avoid searching transactions' read_set when retrieving...
Xuehan Xu [Thu, 13 Oct 2022 06:27:34 +0000 (14:27 +0800)]
crimson/os/seastore/btree: avoid searching transactions' read_set when retrieving btree nodes

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit c29051c4c747c5b8415b8600affc24c0025507fb)

2 years agocrimson/os/seastore/btree: search fixed-kv-btree by parent<->child pointers
Xuehan Xu [Thu, 13 Oct 2022 03:50:17 +0000 (11:50 +0800)]
crimson/os/seastore/btree: search fixed-kv-btree by parent<->child pointers

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 7c3305f0149808f800dc6c852d4f1f755a490ec4)

2 years agocrimson/os/seastore/cache: invalidate out-dated extent when initiating Cache
Xuehan Xu [Thu, 13 Oct 2022 02:57:09 +0000 (10:57 +0800)]
crimson/os/seastore/cache: invalidate out-dated extent when initiating Cache

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 686d1206532200733d69b9430227bdb6bf2d9eb3)

2 years agocrimson/os/seastore/cached_extent: improve the representation of "has_been_invalidated"
Xuehan Xu [Wed, 12 Oct 2022 06:37:39 +0000 (14:37 +0800)]
crimson/os/seastore/cached_extent: improve the representation of "has_been_invalidated"

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 1b4c591ef51920211de978f490b4125177571e08)

2 years agocrimson/os/seastore/btree: don't go to leaf nodes when updating internal mappings
Xuehan Xu [Tue, 31 Jan 2023 06:36:42 +0000 (14:36 +0800)]
crimson/os/seastore/btree: don't go to leaf nodes when updating internal mappings

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit a86c7bd651bc7a89a49e57f9a87e8baeb4291c41)

2 years agocrimson/os/seastore/btree: introduce parent<->child pointers for fixed-kv-btree nodes
Xuehan Xu [Tue, 11 Oct 2022 02:34:16 +0000 (10:34 +0800)]
crimson/os/seastore/btree: introduce parent<->child pointers for fixed-kv-btree nodes

maintain correct parent<->child pointers when modifying the btree

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 71051f997fca8ff0fdcd55586e3150bffa2a916a)

2 years agocrimson/os/seastore/cached_extent: add on_invalidated interface
Xuehan Xu [Tue, 16 Aug 2022 09:05:52 +0000 (17:05 +0800)]
crimson/os/seastore/cached_extent: add on_invalidated interface

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit e5aabe6c306061b2ce3aff5ca30a9049c1a7f9bf)

2 years agocrimson/os/seastore/cached_extent: add on_replace_prior interface
Xuehan Xu [Tue, 16 Aug 2022 08:48:28 +0000 (16:48 +0800)]
crimson/os/seastore/cached_extent: add on_replace_prior interface

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit c56af30a9c199b4a5d2cba9d2208b8c1d8a6abfb)

2 years agocrimson/os/seastore/cached_extent: duplicate_for_write accepts the transaction ref...
Xuehan Xu [Tue, 13 Sep 2022 07:53:53 +0000 (15:53 +0800)]
crimson/os/seastore/cached_extent: duplicate_for_write accepts the transaction ref as its parameter

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 06ae16141470a9052224fbe76c842fa13bbf4520)

2 years agocrimson/os/seastore/cached_extent: record transaction id in pending CachedExtents
Xuehan Xu [Tue, 13 Sep 2022 07:50:35 +0000 (15:50 +0800)]
crimson/os/seastore/cached_extent: record transaction id in pending CachedExtents

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 8240be8f324b244061b17bb2c1676b192a12dca2)

2 years agocrimson/osd/osd_operations/client_request: Fix client blocklisting 51709/head
Matan Breizman [Sun, 7 May 2023 13:29:22 +0000 (13:29 +0000)]
crimson/osd/osd_operations/client_request: Fix client blocklisting

See #50835.
In crimson, conn is independently maintained outside Message.
Therefore, when trying to use the message's connection for `get_peer_addr()`
we won't be able to get the peer address.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 9c1531ca404582725bfcb758dd489626fda31454)

2 years agocrimson/os: make ZNS support shard metadata 51708/head
chunmei-liu [Tue, 22 Nov 2022 16:59:56 +0000 (08:59 -0800)]
crimson/os: make ZNS support shard metadata

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
(cherry picked from commit ad5b0c12a788bca998cb95bfbaa745d9856c56ba)

2 years agocrimson/os: fix formatter build error
chunmei [Thu, 6 Apr 2023 07:19:40 +0000 (07:19 +0000)]
crimson/os: fix formatter build error

Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit c401a4af889fb68468447453716059cb73eb405a)

2 years agocrimson/os: fix unittest error for multicore seastore
chunmei [Fri, 17 Mar 2023 05:56:44 +0000 (05:56 +0000)]
crimson/os: fix unittest error for multicore seastore

Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit 91572c1799f312418bf5d98a7cf910be5b8431ff)

2 years agocrimson/os: static assign segments to each shard
chunmei [Tue, 4 Apr 2023 07:37:16 +0000 (07:37 +0000)]
crimson/os: static assign segments to each shard

and make device sharded

Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit ad4f7bbf15785b851db8e7d6f8a8608a362b2f74)

2 years agocirmson/os: seperate futurizedStore interface
chunmei [Fri, 3 Mar 2023 04:19:21 +0000 (04:19 +0000)]
cirmson/os: seperate futurizedStore interface

into futurizedStore and futurizedShardStore
and modification the other parts in osd and objectstore

Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit 7a673d006da9fec985b2653a2c16b9f4581398f9)

2 years agocrimson/seastore: fix heap-used-after-free bug
chunmei [Tue, 18 Apr 2023 09:33:52 +0000 (09:33 +0000)]
crimson/seastore: fix heap-used-after-free bug

Signed-off-by: chunmei <chunmei.liu@intel.com>
(cherry picked from commit 6a121c0852332ec122b5e659744eb622f2e15c58)

2 years agocrimson/osd: specialize messages to use the external connection 51707/head
Yingxin Cheng [Mon, 3 Apr 2023 08:39:52 +0000 (16:39 +0800)]
crimson/osd: specialize messages to use the external connection

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit ebda4900736dfe4ee38163ae7adcc7d06d07e3a9)

2 years agocrimson/net: change ConnectionRef to be a local_shared_foreign_ptr
Yingxin Cheng [Mon, 3 Apr 2023 02:00:11 +0000 (10:00 +0800)]
crimson/net: change ConnectionRef to be a local_shared_foreign_ptr

Make it possible for connections and messages to be dispatched in
arbitrary core without asynchronous foreign copy.

The local_shared_foreign_ptr conn cannot be moved to another core
implicitly, maintain it outside Message independently.

Do asynchronous foreign copy to the new ConnectionRef only in
with_remote_shard_state_and_op().

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit b54153ecaddda890401d12d6c5ceadb0030df4f1)

2 years agocrimson/osd: drop unused methods in ClientRequest
Yingxin Cheng [Mon, 3 Apr 2023 01:54:49 +0000 (09:54 +0800)]
crimson/osd: drop unused methods in ClientRequest

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 1f419b779e8610a19bfa4141d12403c7271686cf)

2 years agotest/crimson/test_messenger_thrash: free msgr after its connections
Yingxin Cheng [Mon, 3 Apr 2023 01:51:02 +0000 (09:51 +0800)]
test/crimson/test_messenger_thrash: free msgr after its connections

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 3fde061a49ba31c3d64b4a0c2798fc4972100a19)

2 years agocrimson/common: improve local_shared_foreign_ptr
Yingxin Cheng [Mon, 3 Apr 2023 01:48:38 +0000 (09:48 +0800)]
crimson/common: improve local_shared_foreign_ptr

Make ptr nullable and add necessary operators.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit f96ae0b367825c0e4433a57a086e03f75b1eab2e)

2 years agoMerge pull request #51673 from Matan-B/wip-50546-reef-2
Matan [Tue, 23 May 2023 13:13:58 +0000 (15:13 +0200)]
Merge pull request #51673 from Matan-B/wip-50546-reef-2

reef: crimson/osd: Support librados::OPERATION_ORDERSNAP

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
2 years agoMerge pull request #51649 from adk3798/reef-require-image-flag-inspect
Adam King [Tue, 23 May 2023 12:39:49 +0000 (08:39 -0400)]
Merge pull request #51649 from adk3798/reef-require-image-flag-inspect

reef: cephadm: require --image is passed to inspect-image

Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
2 years agoMerge pull request #51650 from adk3798/reef-cephadm-byte-compile
Adam King [Tue, 23 May 2023 12:36:41 +0000 (08:36 -0400)]
Merge pull request #51650 from adk3798/reef-cephadm-byte-compile

reef: cephadm: call compile_dir to byte compile zipapp py files

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #51614 from adk3798/wip-59631-reef
Adam King [Tue, 23 May 2023 12:35:44 +0000 (08:35 -0400)]
Merge pull request #51614 from adk3798/wip-59631-reef

reef: mgr/cephadm: support for nfs backed by VIP

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #51613 from adk3798/wip-59550-reef
Adam King [Tue, 23 May 2023 12:34:46 +0000 (08:34 -0400)]
Merge pull request #51613 from adk3798/wip-59550-reef

reef: mgr/cephadm: asyncio based universal timeout for ssh/cephadm commands

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #51342 from k0ste/wip-59634-reef
Adam King [Tue, 23 May 2023 12:32:34 +0000 (08:32 -0400)]
Merge pull request #51342 from k0ste/wip-59634-reef

reef: pybind/mgr: improve error when inbuf is given to commands that don't use it

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2 years agorbd-wnbd: improve image map error message 51704/head
Lucian Petrut [Mon, 22 May 2023 10:01:19 +0000 (10:01 +0000)]
rbd-wnbd: improve image map error message

"rbd-wnbd map" commands are delegated to the centralized
ceph-rbd Windows service, which in turn will create the
actual image mapping. This allows the daemons to continue
running even after the current session ends.

rbd-wnbd communicates with the service through a named pipe.
If the operation fails, we're only getting an error code.

This change will print the according error message based on the
error code, suggesting the user to check the logs or use
foreground mode ("-f") for additional information.

We could also capture the entire subprocess console output and
print it in case of errors. This can be quite verbose, so we'll
keep it simple for now.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
(cherry picked from commit 5bbcfc82970bd3a76536a7dd554096bec70ddc2d)

2 years agoMerge pull request #51693 from zdover23/wip-doc-2023-05-23-backport-51682-to-reef
Anthony D'Atri [Tue, 23 May 2023 12:10:34 +0000 (08:10 -0400)]
Merge pull request #51693 from zdover23/wip-doc-2023-05-23-backport-51682-to-reef

reef: doc/glossary: update bluestore entry

2 years agoMerge pull request #51672 from k0ste/wip-61342-reef
Pere Diaz Bou [Tue, 23 May 2023 11:16:44 +0000 (13:16 +0200)]
Merge pull request #51672 from k0ste/wip-61342-reef

reef: mgr/prometheus: fix pool_objects_repaired and daemon_health_metrics format

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #51636 from Matan-B/wip-50584-reef
Matan [Tue, 23 May 2023 11:11:07 +0000 (13:11 +0200)]
Merge pull request #51636 from Matan-B/wip-50584-reef

reef: qa/suites/crimson-rados/thrash: Enable supported tests and ops

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51625 from Matan-B/wip-47974-reef
Matan [Tue, 23 May 2023 11:08:56 +0000 (13:08 +0200)]
Merge pull request #51625 from Matan-B/wip-47974-reef

reef: crimson/os/seastore: evict cold data to slower devices

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2 years agoMerge pull request #51588 from Svelar/reef-vstart-filestore-cleanup
Ilya Dryomov [Tue, 23 May 2023 10:52:50 +0000 (12:52 +0200)]
Merge pull request #51588 from Svelar/reef-vstart-filestore-cleanup

reef: vstart: remove filestore options and config snippets

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #51644 from Matan-B/wip-51167-reef
Matan [Tue, 23 May 2023 08:32:49 +0000 (10:32 +0200)]
Merge pull request #51644 from Matan-B/wip-51167-reef

reef: crimson/qa: make crimson run multicore in teuthology test

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51641 from Matan-B/wip-50457-reef
Matan [Tue, 23 May 2023 08:31:41 +0000 (10:31 +0200)]
Merge pull request #51641 from Matan-B/wip-50457-reef

reef: qa/suites/crimson-rados/rbd: Add new rbd image format api tests

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51675 from Matan-B/wip-47637-reef
Matan [Tue, 23 May 2023 08:30:50 +0000 (10:30 +0200)]
Merge pull request #51675 from Matan-B/wip-47637-reef

reef: crimson/osd: bring support for client blocklisting

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51674 from Matan-B/wip-51198-reef
Matan [Tue, 23 May 2023 08:29:50 +0000 (10:29 +0200)]
Merge pull request #51674 from Matan-B/wip-51198-reef

reef: crimson/osd/snaptrim_event: Handle missing clone obc case

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoqa/suites/crimson-rados/thrash: Enable supported thrashers 51636/head
Matan Breizman [Sun, 19 Mar 2023 11:56:03 +0000 (11:56 +0000)]
qa/suites/crimson-rados/thrash: Enable supported thrashers

Balanced/Localized reads are now supported.
snap_remove and rollback are supported as well.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit b888cfa3da956aad527d05c3dbe507efb368363f)

2 years agoqa/suites/crimson-rados/thrash: Add snap_remove/create weights
Matan Breizman [Sun, 19 Mar 2023 11:53:38 +0000 (11:53 +0000)]
qa/suites/crimson-rados/thrash: Add snap_remove/create weights

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 8839f829d60e0f0a3e6feca37b8eaa167bccdfd0)

2 years agoqa/suites/crimson-rados: Remove rados_api_test.yaml.disabled
Matan Breizman [Sun, 19 Mar 2023 11:49:03 +0000 (11:49 +0000)]
qa/suites/crimson-rados: Remove rados_api_test.yaml.disabled

This test is already included in:
crimson-rados/basic/tasks/rados_api_tests.yaml

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit bb23ddc5dc31ced752db0310d9857ba268005182)

2 years agoMerge pull request #51635 from Matan-B/wip-49980-reef
Matan [Tue, 23 May 2023 08:24:52 +0000 (10:24 +0200)]
Merge pull request #51635 from Matan-B/wip-49980-reef

reef: crimson/os/seastore/journal: generalize recordsubmitter

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51629 from Matan-B/wip-50491-reef
Matan [Tue, 23 May 2023 08:23:05 +0000 (10:23 +0200)]
Merge pull request #51629 from Matan-B/wip-50491-reef

reef: test/librados/snapshots*: Enable Crimson supported tests

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agodoc/glossary: update bluestore entry 51693/head
Zac Dover [Mon, 22 May 2023 21:41:09 +0000 (07:41 +1000)]
doc/glossary: update bluestore entry

Update the BlueStore entry in the glossary, explaining that as of Reef
BlueStore and only BlueStore (and not FileStore) is the storage backend
for Ceph.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit bcee264276128f622c35e3aab81fdecb2b8afc10)

2 years agotest/crimson/seasatore: add evction test 51625/head
Zhang Song [Fri, 17 Feb 2023 06:25:20 +0000 (14:25 +0800)]
test/crimson/seasatore: add evction test

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit c53446df687dd82860ba012f89ae783be8a4f51c)

2 years agocrimson/os/seastore/EPM: cleanup alloc_new_extent
Zhang Song [Fri, 17 Feb 2023 06:23:36 +0000 (14:23 +0800)]
crimson/os/seastore/EPM: cleanup alloc_new_extent

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 5bc650a0605ef9559ac32fe70930d446652c2d1d)

2 years agocrimson/os/seasotre/segment_manager_group: ensure the types of devices are same
Zhang Song [Wed, 1 Mar 2023 10:19:54 +0000 (18:19 +0800)]
crimson/os/seasotre/segment_manager_group: ensure the types of devices are same

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 61156355385aaa06cf9d15870a8830256874714d)

2 years agocrimson/os/seastore: add EPHEMERAL_COLD device type
Zhang Song [Tue, 14 Feb 2023 09:47:35 +0000 (17:47 +0800)]
crimson/os/seastore: add EPHEMERAL_COLD device type

rename SEGMENTED_EPHEMERAL to EPHEMERAL_MAIN

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit b40304092b6ce8b29e97437642928cc84afa1be5)

2 years agocrimson/os/seastore/EPM/BackgroundProcess: introduce more eviction policy
Zhang Song [Wed, 11 Jan 2023 10:17:41 +0000 (18:17 +0800)]
crimson/os/seastore/EPM/BackgroundProcess: introduce more eviction policy

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 45c53a26c662b2c23efdecaaa186b6ac8811c2ea)

2 years agocrimson/os/seastore/EPM/BackgroundProcess: reserve space on cold cleaner
Zhang Song [Tue, 10 Jan 2023 10:52:04 +0000 (18:52 +0800)]
crimson/os/seastore/EPM/BackgroundProcess: reserve space on cold cleaner

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 4aea0e266de43281a5d8c24fe91034bdaae45d98)

2 years agocrimson/os/seastore: get segment info from cold segment provider
Zhang Song [Tue, 10 Jan 2023 10:02:50 +0000 (18:02 +0800)]
crimson/os/seastore: get segment info from cold segment provider

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 7b0ccecbced037bbceca707d62d81fe2accc5a39)

2 years agocrimson/os/seastore: support mark_space_{free, used} on cold_cleaner
Zhang Song [Tue, 10 Jan 2023 10:01:25 +0000 (18:01 +0800)]
crimson/os/seastore: support mark_space_{free, used} on cold_cleaner

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 783c5e7662006c6efca7ade795e6e731b0b18332)

2 years agosrc/vstart.sh: support specify the type of secondary device
Zhang Song [Tue, 10 Jan 2023 09:59:21 +0000 (17:59 +0800)]
src/vstart.sh: support specify the type of secondary device

doc/crimson: add options and example about SeaStore

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit b40f6421f0d6f7966e13941ba13e70b5639bce93)

2 years agocrimson/os/seastore: change the category of COLL_BLOCK
Zhang Song [Tue, 10 Jan 2023 09:01:57 +0000 (17:01 +0800)]
crimson/os/seastore: change the category of COLL_BLOCK

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 9283b72e476f306a8926aa2f7ae191ca8705fcd4)

2 years agocrimson/os/seastore: specify the device type explicitly when constructing BlockSegmen...
Zhang Song [Mon, 6 Mar 2023 07:06:19 +0000 (15:06 +0800)]
crimson/os/seastore: specify the device type explicitly when constructing BlockSegmentManager

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 1e515e8561519ded43b1046942057d705441ee22)

2 years agocrimson/os/seastore: make segment_manager work with HDD
Zhang Song [Tue, 10 Jan 2023 09:01:30 +0000 (17:01 +0800)]
crimson/os/seastore: make segment_manager work with HDD

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 89939e7ad7c1dffaa1c842be601fa372d074a86a)

2 years agocrimson/os/seasotre: add CLEANER_COLD transaction type
Zhang Song [Tue, 10 Jan 2023 09:00:12 +0000 (17:00 +0800)]
crimson/os/seasotre: add CLEANER_COLD transaction type

rename CLEANER to CLEANER_MAIN

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 0d20f1595a0c839e780540b61ff68f52be29a99b)