]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agoqa/suites/crimson-rados/thrash: Enable supported thrashers 51636/head
Matan Breizman [Sun, 19 Mar 2023 11:56:03 +0000 (11:56 +0000)]
qa/suites/crimson-rados/thrash: Enable supported thrashers

Balanced/Localized reads are now supported.
snap_remove and rollback are supported as well.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit b888cfa3da956aad527d05c3dbe507efb368363f)

2 years agoqa/suites/crimson-rados/thrash: Add snap_remove/create weights
Matan Breizman [Sun, 19 Mar 2023 11:53:38 +0000 (11:53 +0000)]
qa/suites/crimson-rados/thrash: Add snap_remove/create weights

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 8839f829d60e0f0a3e6feca37b8eaa167bccdfd0)

2 years agoqa/suites/crimson-rados: Remove rados_api_test.yaml.disabled
Matan Breizman [Sun, 19 Mar 2023 11:49:03 +0000 (11:49 +0000)]
qa/suites/crimson-rados: Remove rados_api_test.yaml.disabled

This test is already included in:
crimson-rados/basic/tasks/rados_api_tests.yaml

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit bb23ddc5dc31ced752db0310d9857ba268005182)

2 years agoMerge pull request #51635 from Matan-B/wip-49980-reef
Matan [Tue, 23 May 2023 08:24:52 +0000 (10:24 +0200)]
Merge pull request #51635 from Matan-B/wip-49980-reef

reef: crimson/os/seastore/journal: generalize recordsubmitter

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51629 from Matan-B/wip-50491-reef
Matan [Tue, 23 May 2023 08:23:05 +0000 (10:23 +0200)]
Merge pull request #51629 from Matan-B/wip-50491-reef

reef: test/librados/snapshots*: Enable Crimson supported tests

Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51623 from Matan-B/wip-50317-reef
Matan [Tue, 23 May 2023 08:00:32 +0000 (10:00 +0200)]
Merge pull request #51623 from Matan-B/wip-50317-reef

reef: crimson/os/seastore/epm: also do prepare_write for rbm

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51677 from zdover23/wip-doc-2023-05-23-backport-51646-to-reef
Anthony D'Atri [Mon, 22 May 2023 21:21:38 +0000 (17:21 -0400)]
Merge pull request #51677 from zdover23/wip-doc-2023-05-23-backport-51646-to-reef

reef: doc/dev/crimson: Improve crimson.rst

2 years agoMerge pull request #51678 from zdover23/wip-doc-2023-05-23-backport-51651-to-reef
Anthony D'Atri [Mon, 22 May 2023 21:21:30 +0000 (17:21 -0400)]
Merge pull request #51678 from zdover23/wip-doc-2023-05-23-backport-51651-to-reef

reef: doc: note deprecation of Cache Tiering in Reef

2 years agoMerge pull request #51612 from adk3798/wip-59548-reef
Adam King [Mon, 22 May 2023 20:38:12 +0000 (16:38 -0400)]
Merge pull request #51612 from adk3798/wip-59548-reef

reef: mgr/cephadm: allow setting mon crush locations through mon service spec

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoMerge pull request #51611 from adk3798/wip-59546-reef
Adam King [Mon, 22 May 2023 20:36:58 +0000 (16:36 -0400)]
Merge pull request #51611 from adk3798/wip-59546-reef

reef: cephadm: reschedule haproxy from an offline host

Reviewed-by: Michael Fritch <mfritch@suse.com>
2 years agoMerge pull request #51208 from guits/wip-59255-reef
Adam King [Mon, 22 May 2023 20:36:04 +0000 (16:36 -0400)]
Merge pull request #51208 from guits/wip-59255-reef

reef: ceph-volume: fix drive-group issue that expects the batch_args to be a string

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #51205 from guits/wip-59516-reef
Adam King [Mon, 22 May 2023 20:35:10 +0000 (16:35 -0400)]
Merge pull request #51205 from guits/wip-59516-reef

reef: ceph-volume: fix batch refactor issue

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #51194 from guits/wip-59523-reef
Adam King [Mon, 22 May 2023 20:34:22 +0000 (16:34 -0400)]
Merge pull request #51194 from guits/wip-59523-reef

reef: ceph-volume: quick fix in zap.py

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #51191 from guits/wip-59519-reef
Adam King [Mon, 22 May 2023 20:33:30 +0000 (16:33 -0400)]
Merge pull request #51191 from guits/wip-59519-reef

reef: orchestrator: add `--no-destroy` arg to `ceph orch osd rm`

Reviewed-by: Adam King <adking@redhat.com>
2 years agodoc: note deprecation of Cache Tiering in Reef 51678/head
Zac Dover [Mon, 22 May 2023 01:35:26 +0000 (11:35 +1000)]
doc: note deprecation of Cache Tiering in Reef

Add a note to the documentation that cache tiering is deprecated in
Reef.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 7f9eaf7a8ef95d611df81463424cfd06472db356)

2 years agodoc/dev/crimson: Improve crimson.rst 51677/head
Anthony D'Atri [Sun, 21 May 2023 12:39:56 +0000 (08:39 -0400)]
doc/dev/crimson: Improve crimson.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit 524901d3cfbf73cc86284a2623ada493836c13ec)

2 years agoMerge pull request #51593 from cbodley/wip-61219-reef
Yuri Weinstein [Mon, 22 May 2023 19:57:29 +0000 (15:57 -0400)]
Merge pull request #51593 from cbodley/wip-61219-reef

reef: rgw: use string_view to parse Accept header

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #51592 from cbodley/wip-61298-reef
Yuri Weinstein [Mon, 22 May 2023 19:56:52 +0000 (15:56 -0400)]
Merge pull request #51592 from cbodley/wip-61298-reef

reef: qa/rgw: add POOL_APP_NOT_ENABLED to log-ignorelist

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agoMerge pull request #51569 from amathuria/wip-61232-reef
Yuri Weinstein [Mon, 22 May 2023 19:40:41 +0000 (15:40 -0400)]
Merge pull request #51569 from amathuria/wip-61232-reef

reef: qa/tasks: Changing default mClock profile to high_recovery_ops

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2 years agoMerge pull request #51409 from kamoltat/wip-ksirivad-reef-backport-50857
Yuri Weinstein [Mon, 22 May 2023 19:39:38 +0000 (15:39 -0400)]
Merge pull request #51409 from kamoltat/wip-ksirivad-reef-backport-50857

reef:mon/Monitor.cc: exit function if !osdmon()->is_writeable()

Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #50977 from yaarith/wip-59390-reef
Laura Flores [Mon, 22 May 2023 19:38:54 +0000 (14:38 -0500)]
Merge pull request #50977 from yaarith/wip-59390-reef

reef: mgr/telemetry: add leaderboard description and documentation

2 years agoMerge pull request #51357 from rhcs-dashboard/wip-59633-reef
Adam King [Mon, 22 May 2023 18:33:25 +0000 (14:33 -0400)]
Merge pull request #51357 from rhcs-dashboard/wip-59633-reef

reef: mgr/cephadm: update monitoring stack versions

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #51645 from Matan-B/wip-51448-reef
Matan [Mon, 22 May 2023 15:36:06 +0000 (17:36 +0200)]
Merge pull request #51645 from Matan-B/wip-51448-reef

reef: crimson/osd/scheduler/mclock_scheduler: Fix OSD unable to start

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51643 from Matan-B/wip-50411-reef
Matan [Mon, 22 May 2023 15:35:21 +0000 (17:35 +0200)]
Merge pull request #51643 from Matan-B/wip-50411-reef

reef: crimson/osd: start operations asynchrously

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51642 from Matan-B/wip-51333-reef
Matan [Mon, 22 May 2023 15:34:23 +0000 (17:34 +0200)]
Merge pull request #51642 from Matan-B/wip-51333-reef

reef: crimson/osd/objclass: Fix compilation warning

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51640 from Matan-B/wip-50227-reef
Matan [Mon, 22 May 2023 15:33:08 +0000 (17:33 +0200)]
Merge pull request #51640 from Matan-B/wip-50227-reef

reef: crimson/osd/heartbeat: share osdmap to peers when necessary

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51639 from Matan-B/wip-51202-reef
Matan [Mon, 22 May 2023 15:31:38 +0000 (17:31 +0200)]
Merge pull request #51639 from Matan-B/wip-51202-reef

reef: crimson/osd/pg: make clone object's version consistent with pglog

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51638 from Matan-B/wip-51180-reef
Matan [Mon, 22 May 2023 15:30:39 +0000 (17:30 +0200)]
Merge pull request #51638 from Matan-B/wip-51180-reef

reef: crimson/osd/snaptrim_event: get SnapSetContext from head obc

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51637 from Matan-B/wip-50530-reef
Matan [Mon, 22 May 2023 15:29:23 +0000 (17:29 +0200)]
Merge pull request #51637 from Matan-B/wip-50530-reef

reef: crimson/osd: Handle rollback to head obejct

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51634 from Matan-B/wip-50840-reef
Matan [Mon, 22 May 2023 15:25:34 +0000 (17:25 +0200)]
Merge pull request #51634 from Matan-B/wip-50840-reef

reef: crimson/os/seastore: avoid segment nonce collision

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51633 from Matan-B/wip-50839-reef
Matan [Mon, 22 May 2023 15:24:56 +0000 (17:24 +0200)]
Merge pull request #51633 from Matan-B/wip-50839-reef

reef: crimson/os/seastore: ensure transaction commit order

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51632 from Matan-B/wip-50653-reef
Matan [Mon, 22 May 2023 15:23:53 +0000 (17:23 +0200)]
Merge pull request #51632 from Matan-B/wip-50653-reef

reef: crimson/os/seastore/cache: consider EXIST_CLEAN extents as pending ones

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51631 from Matan-B/wip-50555-reef
Matan [Mon, 22 May 2023 15:18:22 +0000 (17:18 +0200)]
Merge pull request #51631 from Matan-B/wip-50555-reef

reef: crimson/os/seastore: implement get_stat() for RBM

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51630 from Matan-B/wip-50492-reef
Matan [Mon, 22 May 2023 15:17:19 +0000 (17:17 +0200)]
Merge pull request #51630 from Matan-B/wip-50492-reef

reef: crimson/common/log: Fix debug logging macros

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51627 from Matan-B/wip-50602-reef
Matan [Mon, 22 May 2023 15:15:37 +0000 (17:15 +0200)]
Merge pull request #51627 from Matan-B/wip-50602-reef

reef: crimson/os/seastore: don't set INVALID extents to CLEAN when reading extents

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51626 from Matan-B/wip-50556-reef
Matan [Mon, 22 May 2023 15:14:04 +0000 (17:14 +0200)]
Merge pull request #51626 from Matan-B/wip-50556-reef

reef: crimson: fix build error

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51624 from Matan-B/wip-49116-reef
Matan [Mon, 22 May 2023 15:11:02 +0000 (17:11 +0200)]
Merge pull request #51624 from Matan-B/wip-49116-reef

reef: crimson/osd: Support balance reads

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #51652 from zdover23/wip-doc-2023-05-22-backport-51319-to-reef
Anthony D'Atri [Mon, 22 May 2023 08:55:31 +0000 (04:55 -0400)]
Merge pull request #51652 from zdover23/wip-doc-2023-05-22-backport-51319-to-reef

reef: doc: deprecate the cache tiering

2 years agodoc: deprecate the cache tiering 51652/head
Radosław Zarzyński [Tue, 2 May 2023 15:52:23 +0000 (17:52 +0200)]
doc: deprecate the cache tiering

This topic has been discussed many times; recently at the Dev
Summit of Cephalocon 2023.

This commit is the minial version of the work, contained entirely
within the `doc`. However, likely it will be expanded as there
were ideas like e.g. adding cache tiering back experimental feature
list (Sam) to warn users when deploying a new cluster.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>
(cherry picked from commit 535b8db33ea03fbab7ef0c4df5251658f956b0c5)

2 years agomgr/cephadm: verify mon spec exists before trying to grab from spec store 51612/head
Adam King [Tue, 2 May 2023 22:59:18 +0000 (18:59 -0400)]
mgr/cephadm: verify mon spec exists before trying to grab from spec store

In a normal deployment, we generally shouldn't have
to worry about this. This is more for teuthology
which does deployments in a weird way that can cause
there to be no mon spec in the cluster. Fixes an issue
seen when backporting the mon crush location work
to quincy where an upgrade test would fail with

```
[WRN] UPGRADE_REDEPLOY_DAEMON: Upgrading daemon mon.b on host smithi047 failed.
    Upgrade daemon: mon.b: Service mon not found.
```

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 8aab7beefbb52f47573ad7fce932552ad5c0b2fa)

2 years agocrimson/osd/scheduler/mclock_scheduler: Fix OSD unable to start 51645/head
Matan Breizman [Thu, 11 May 2023 14:18:46 +0000 (14:18 +0000)]
crimson/osd/scheduler/mclock_scheduler: Fix OSD unable to start

https://github.com/ceph/ceph/pull/49975 Introduced changes to
mclock conf value types which caused the osd to stall while booting.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 5cb12bb4af1f1c9cd3051429d8f6c0c01535ab04)

2 years agocrimson/osd/snaptrim_event: preserve interrupt condition when creating 51643/head
Xuehan Xu [Fri, 5 May 2023 05:22:11 +0000 (05:22 +0000)]
crimson/osd/snaptrim_event: preserve interrupt condition when creating
subops

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 58f527893e0a5c051c91d6df0c8d3217c1811875)

2 years agocrimson/osd: start operations asynchrously
Xuehan Xu [Wed, 8 Mar 2023 05:52:00 +0000 (05:52 +0000)]
crimson/osd: start operations asynchrously

Fixes: https://tracker.ceph.com/issues/58928
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit a1963937e421bfdaf2e9735dc2faf750c9a9e08e)

2 years agocrimson/osd/objclass: Compilation warning 51642/head
Matan Breizman [Wed, 3 May 2023 09:39:14 +0000 (09:39 +0000)]
crimson/osd/objclass: Compilation warning

```
In copy constructor ‘ceph::buffer::v15_2_0::list::list(const ceph::buffer::v15_2_0::list&)’,
    inlined from ‘OSDOp::OSDOp(const OSDOp&)’ at ../src/osd/osd_types.h:4081:8,
    inlined from ‘int cls_cxx_snap_revert(cls_method_context_t, snapid_t)’ at ../src/crimson/osd/objclass.cc:279:37:
../src/include/buffer.h:945:20: warning: ‘op.OSDOp::indata.ceph::buffer::v15_2_0::list::_len’ is used uninitialized [-Wuninitialized]
  945 |         _len(other._len),
      |              ~~~~~~^~~~
../src/crimson/osd/objclass.cc: In function ‘int cls_cxx_snap_revert(cls_method_context_t, snapid_t)’:
../src/crimson/osd/objclass.cc:279:9: note: ‘op’ declared here
  279 |   OSDOp op{op = CEPH_OSD_OP_ROLLBACK};
      |
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 99a8d19b05c175777ec7877eb58bb87270602ba5)

2 years agocrimson/osd/heartbeat: add latest epoch sent concept 51640/head
Matan Breizman [Wed, 29 Mar 2023 16:42:09 +0000 (16:42 +0000)]
crimson/osd/heartbeat: add latest epoch sent concept

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 12af68c24d2ea0cfd540f4da0a4f765b8212396d)

2 years agocrimson/osd/shard_services: Add logs to load_map_bls
Matan Breizman [Wed, 29 Mar 2023 09:21:12 +0000 (09:21 +0000)]
crimson/osd/shard_services: Add logs to load_map_bls

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 7550d423b1baf2a7bd701b0391a192dc30e9482f)

2 years agocrimson/osd/heartbeat: add a rename todo
Matan Breizman [Thu, 27 Apr 2023 16:07:01 +0000 (16:07 +0000)]
crimson/osd/heartbeat: add a rename todo

In continuation to: bfbbc311f1f

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 0075349661a838abc04f110630bc4f370116c391)

2 years agocrimson/osd/heartbeat: Rename Peer's created at epoch
Matan Breizman [Wed, 29 Mar 2023 09:19:29 +0000 (09:19 +0000)]
crimson/osd/heartbeat: Rename Peer's created at epoch

As these getter and setter represent the epoch at which the peer was added

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit bfbbc311f1fc029ef5df3415cada325fbfed4bde)

2 years agocrimson/osd/heartbeat: share osdmap to peers when necessary
Xuehan Xu [Thu, 23 Feb 2023 06:50:56 +0000 (06:50 +0000)]
crimson/osd/heartbeat: share osdmap to peers when necessary

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 670bbaa4d858de3dd50d857370f76bbd1a13c702)

2 years agocrimson/osd/pg: make clone object's version consistent with pglog 51639/head
Xuehan Xu [Tue, 25 Apr 2023 06:21:07 +0000 (06:21 +0000)]
crimson/osd/pg: make clone object's version consistent with pglog

Right now, the head object's version is 1 less than clone object's
version in the pg log; while it's exactly the other way around when
it comes to the versions recorded in the object info of head/clone
objects

Fixes: https://tracker.ceph.com/issues/59536
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 1fd6324bb8baaee4d413f188c78b821f84e5d4ff)

2 years agocrimson/osd/snaptrim_event: avoid leaking txn ops for updating head obcs 51638/head
Xuehan Xu [Thu, 27 Apr 2023 03:32:04 +0000 (03:32 +0000)]
crimson/osd/snaptrim_event: avoid leaking txn ops for updating head obcs
when removing clones

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 08625c27f9e158ffb46d0d5d008f500ee9677b42)

2 years agocrimson/osd/snaptrim_event: encode head oi with no oid
Xuehan Xu [Thu, 27 Apr 2023 03:31:20 +0000 (03:31 +0000)]
crimson/osd/snaptrim_event: encode head oi with no oid

This is required by PGBackend::load_metadata

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit ba8732d2a942b4faede4bd3d88764f1da0aa5ad8)

2 years agocrimson/osd/snaptrim_event: don't clear head obc's snaps
Xuehan Xu [Thu, 27 Apr 2023 03:29:58 +0000 (03:29 +0000)]
crimson/osd/snaptrim_event: don't clear head obc's snaps

There may be valid snaps for a head object, although some snap is
being removed

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit f5194e60a25d89f0b0673a96629ea7db42487ac0)

2 years agocrimson/osd: get SnapSetContext from head obc
Xuehan Xu [Sun, 23 Apr 2023 03:59:17 +0000 (03:59 +0000)]
crimson/osd: get SnapSetContext from head obc

clone obc doesn't have snapset context

Fixes: https://tracker.ceph.com/issues/59511
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit a41588532a0339bd647daaf52a6207b67c4459b5)

2 years agotest/librados/snapshots_cxx: Add WriteRollback test 51637/head
Matan Breizman [Tue, 21 Mar 2023 10:50:58 +0000 (10:50 +0000)]
test/librados/snapshots_cxx: Add WriteRollback test

This test is an example for https://tracker.ceph.com/issues/59114

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit b4f453d3a2d8bbdd3b48c7101ef91303c8b2b00b)

2 years agocrimson/osd/pg_backend: Add a note about unhandled rollback case
Matan Breizman [Sun, 19 Mar 2023 11:26:41 +0000 (11:26 +0000)]
crimson/osd/pg_backend: Add a note about unhandled rollback case

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit f616fafbd6e8beeb1cb0852407014227b3c34421)

2 years agotest/librados/snapshots: Add FutureSnapRollback test
Matan Breizman [Wed, 15 Mar 2023 11:11:47 +0000 (11:11 +0000)]
test/librados/snapshots: Add FutureSnapRollback test

Add a test we rollback to a future snap id. Meaning, no mutation is
followed after taking the third snapshot.
The read op is expected to be applied to the head object.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 047c1832c41afc3ce17c724375a6817fa1672b67)

2 years agocrimson/osd/pg_backend: Rename clone_obc to resolved_obc
Matan Breizman [Wed, 15 Mar 2023 11:19:10 +0000 (11:19 +0000)]
crimson/osd/pg_backend: Rename clone_obc to resolved_obc

The resolved_obc being used is not necessarily a clone object.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit a9100256d03dc94805ebf41c1e7e67eb70f8a47e)

2 years agocrimson/osd/pg_backend:: rollback to support resolved head
Matan Breizman [Wed, 15 Mar 2023 10:00:29 +0000 (10:00 +0000)]
crimson/osd/pg_backend:: rollback to support resolved head

Consider the following `resolve_oid()` case:
```
    // Because oid.snap > ss.seq, we are trying to read from a snapshot
    // taken after the most recent write to this object. Read from head.
```
In this case, a no-op is expected as the head object can read from.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 18f3a67cb53c473fd74a92ab7d5211dff63b88fa)

2 years agocrimson/osd/pg_backend: PGBackend::rollback take RWWRITE lock
Matan Breizman [Tue, 14 Mar 2023 13:33:40 +0000 (13:33 +0000)]
crimson/osd/pg_backend: PGBackend::rollback take RWWRITE lock

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 1e1c4318e7fb4cd3d7f585b3431236904ccaea31)

2 years agocrimson/os/seastore/cbjournal: reset start address for replay if committed_to exceeds... 51635/head
myoungwon oh [Thu, 20 Apr 2023 07:26:38 +0000 (07:26 +0000)]
crimson/os/seastore/cbjournal: reset start address for replay if committed_to exceeds cbjournal_end()

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 6b453c9b2f88cc334a43cf31619ba3e7afd3f3f1)

2 years agotest/crimson/seastore/cbjournal: call close after a unittest is done and relocate...
myoungwon oh [Fri, 7 Apr 2023 01:43:43 +0000 (01:43 +0000)]
test/crimson/seastore/cbjournal: call close after a unittest is done and relocate open and close related functions

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 7bac32cd710116ac2410e8881bbc0ea935921c08)

2 years agocrimson/os/seastore/journal: add CircularJournalSpace
myoungwon oh [Thu, 6 Apr 2023 01:56:10 +0000 (01:56 +0000)]
crimson/os/seastore/journal: add CircularJournalSpace

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 59cb8c1050b08bd3fa002c0c401605b977631be9)

2 years agocrimson/os/seastore/journal: introduce JournalAllocator to generalize SegmentAllocator
myoungwon oh [Wed, 5 Apr 2023 04:37:29 +0000 (04:37 +0000)]
crimson/os/seastore/journal: introduce JournalAllocator to generalize SegmentAllocator

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit bba25dbce6491dcc80887ac70e012616c9934c47)

2 years agocrimson/os/seastore/journal: relocate RecordSubmitter related classes
myoungwon oh [Mon, 27 Mar 2023 00:49:18 +0000 (00:49 +0000)]
crimson/os/seastore/journal: relocate RecordSubmitter related classes

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 5abf9b5c98e07159bd8c90555b2579f1ff7b89a5)

2 years agocrimson/os/seastore: check the type of segment head and segment tail during mount 51634/head
Zhang Song [Tue, 4 Apr 2023 01:34:06 +0000 (09:34 +0800)]
crimson/os/seastore: check the type of segment head and segment tail during mount

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit b890de1c2fe489e05804a2724ad3493c0114732e)

2 years agocrimson/os/seastore: avoid segment nonce collision
Zhang Song [Mon, 3 Apr 2023 11:29:27 +0000 (19:29 +0800)]
crimson/os/seastore: avoid segment nonce collision

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 367577efa7440d6d7414b9772a3b130109852535)

2 years agocrimson/os/seastore: ensure transaction commit order 51633/head
Zhang Song [Mon, 3 Apr 2023 11:15:21 +0000 (19:15 +0800)]
crimson/os/seastore: ensure transaction commit order

The previous implementation acquired the throttler first,
which could result in an unordered wakeup sequence.

Signed-off-by: Zhang Song <zhangsong325@gmail.com>
(cherry picked from commit 0740e21e3248d227301d099d6778e94baf62a9e8)

2 years agocrimson/os/seastore/cache: use CachedExtent::is_mutable() where appropriate 51632/head
Yingxin Cheng [Mon, 27 Mar 2023 09:38:17 +0000 (17:38 +0800)]
crimson/os/seastore/cache: use CachedExtent::is_mutable() where appropriate

CachedExtent::is_mutable() should only be used to check whether need to
call duplicate_for_write(extent).

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
(cherry picked from commit 865285a53cf3a4a2880c2b7b6a35b5b20a55553d)

2 years agocrimson/os/seastore/cache: consider EXIST_CLEAN extents as pending ones
Xuehan Xu [Fri, 24 Mar 2023 10:43:57 +0000 (18:43 +0800)]
crimson/os/seastore/cache: consider EXIST_CLEAN extents as pending ones

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit f34faf363e69053d18d50263ee02ca5948b6462c)

2 years agocrimson/os/seastore/rbm: implement get_stat() 51631/head
myoungwon oh [Thu, 16 Mar 2023 06:35:59 +0000 (15:35 +0900)]
crimson/os/seastore/rbm: implement get_stat()

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
(cherry picked from commit 3c21cff441739a93d9936ed0d7b4fdc4a32ca51c)

2 years agocrimson/common/log: Fix debug logging macros 51630/head
Matan Breizman [Sun, 12 Mar 2023 19:52:28 +0000 (19:52 +0000)]
crimson/common/log: Fix debug logging macros

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 5b33995073d70921a1412c07d710f4bdc6e1b0fe)

2 years agotest/librados/snapshots*: Enable supported tests 51629/head
Matan Breizman [Mon, 13 Mar 2023 08:55:48 +0000 (08:55 +0000)]
test/librados/snapshots*: Enable supported tests

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 04083372485b62233c3b5a1e6a12d28aa2b34524)

2 years agocrimson/os/seastore: don't set INVALID extents to CLEAN when reading 51627/head
Xuehan Xu [Tue, 21 Mar 2023 05:52:43 +0000 (05:52 +0000)]
crimson/os/seastore: don't set INVALID extents to CLEAN when reading
extents

CLEAN_PENDING extents may be invalidated before read completes:

1. transaction A retired an laddr, which lead to a RetirePlaceHolder in
   Cache
2. transaction B try to read that extent, and replace A's
   RetirePlaceHolder with it;
3. transaction A commits and invalidate that extent;
4. transaction B complete reading that extent;

In this case, we shouldn't set the extent's state to CLEAN

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 729dabd8ff20a747e02c15113e005a6d61acc8d7)

2 years agocrimson: fix build error 51626/head
Aravind Ramesh [Thu, 16 Mar 2023 07:02:39 +0000 (12:32 +0530)]
crimson: fix build error

With -DWITH_SEASTAR=ON, build is failing.

src/crimson/osd/osd_operations/snaptrim_event.cc:174:29: error: ‘sleep’ is not a member of ‘seastar’

Signed-off-by: Aravind Ramesh <Aravind.Ramesh@wdc.com>
(cherry picked from commit 917cba60ca40b187ac5da915956e8523c5e161d0)

2 years agocrimson/osd/object_context_loader: Hide private methods 51624/head
Matan Breizman [Sun, 5 Mar 2023 10:22:10 +0000 (10:22 +0000)]
crimson/osd/object_context_loader: Hide private methods

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit b1632dbf17d0118b6a6c7aff1da15c62cbec6637)

2 years agocrimson/osd: Introduce with_head_and_clone_obc()
Matan Breizman [Sun, 5 Mar 2023 09:31:03 +0000 (09:31 +0000)]
crimson/osd: Introduce with_head_and_clone_obc()

In continuation to 7ca2690be956a36f61c7729946b94ccd970dd9c7:
Now that the head ref is no longer a member of obc, we need a new
substitute way to get the head when needed.

When loading a clone object, the head object is loaded
first (See with_clone_obc). Therefore we can make use of this design
to move the loaded head forward to the relevant func (See with_head_and_clone_obc).
Usually, we wouldn't need to make use of both the head and the clone obc in the
same function. However, SnapTrimObjSubEvent::remove_or_update is an abnormal usage.

Note: We want to avoid holding any unneeded references to obcs
to allow the obc_registery to evict no longer valid obc.
Therefore, with_obc() which references only a single obc is the
preferred entry point for loading obcs.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 3ea1d8e09d43a742d52931bbc508f80cdd8ceaa0)

2 years agocrimson/osd/object_context_loader: with_clone_obc to move head
Matan Breizman [Sun, 19 Feb 2023 10:50:04 +0000 (10:50 +0000)]
crimson/osd/object_context_loader: with_clone_obc to move head

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 4bf99c69f2506babc566395ea1f090c30bd70d30)

2 years agocrimson/osd/object_conetxt*: Rename get_ro_ss to get_head_ss
Matan Breizman [Sun, 19 Feb 2023 13:12:15 +0000 (13:12 +0000)]
crimson/osd/object_conetxt*: Rename get_ro_ss to get_head_ss

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit efcf292905135419f9f6f929f9bd3194e3661e61)

2 years agocrimson/osd: Remove head memeber from ObjectContext
Matan Breizman [Sun, 19 Feb 2023 10:49:47 +0000 (10:49 +0000)]
crimson/osd: Remove head memeber from ObjectContext

Before this patch, ObjectContext had a head member which was used
to get the head obc of a clone object.
This member caused the head object to being referenced while
attempting to 'clear_replica_obc' (Since we only evict un-referenced
obc from the obc_registery).
This mechanism, of obtaining the head, is no longer needed since
'with_clone_obc' loads the head object context first (using
oid.get_head).

In this commit, head is removed from ObjectContext class and users
are removed as well.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit e702dc39606bfcc3899307453a6b54af11b8b803)

2 years agocrimson/osd/pg: Add can_discard_op case
Matan Breizman [Sun, 19 Feb 2023 11:35:03 +0000 (11:35 +0000)]
crimson/osd/pg: Add can_discard_op case

discard op in the case where same_primary_since is later than
the MOSDOp's map epoch

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit c6ab21791545598473bd044ce602eea2bba892d3)

2 years agocrimson/osd/pg: Implement context_registry_on_change()
Matan Breizman [Tue, 7 Feb 2023 19:48:42 +0000 (19:48 +0000)]
crimson/osd/pg: Implement context_registry_on_change()

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit f468c673253e3fdd30677e424674f58a8affd963)

2 years agocommon/instrusive_lru: Implement for_each
Matan Breizman [Tue, 7 Feb 2023 19:47:55 +0000 (19:47 +0000)]
common/instrusive_lru: Implement for_each

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 9f8e2329882a2d5101489b7c2445285a43d4ed13)

2 years agoqa/*/crimson: Seperate Crimson's rbd api testing
Matan Breizman [Sun, 5 Feb 2023 14:53:11 +0000 (14:53 +0000)]
qa/*/crimson: Seperate Crimson's rbd api testing

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit b73d8fd860b13f970292e7e78a19df76c9dc15fb)

2 years agocrimson/osd/osd_operations: Add RepRequest::with_pg() to the pipeline
Matan Breizman [Sun, 5 Feb 2023 12:01:41 +0000 (12:01 +0000)]
crimson/osd/osd_operations: Add RepRequest::with_pg() to the pipeline

Wait on a PG to advance to the request's map epoch.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit cc53ae651249a24ab3e6844823bec6fe65fd7cd3)

2 years agocrimson/osd/pg: add replica_clear_repop_obc() and log_operation()
Matan Breizman [Wed, 7 Dec 2022 13:19:26 +0000 (13:19 +0000)]
crimson/osd/pg: add replica_clear_repop_obc() and log_operation()

Clear invalid obc from cache.

Fixes: https://tracker.ceph.com/issues/58089
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 91cbe881397dbf548559a94582b244f0d4ebf8cb)

2 years agocommon/intrusive_lru: Add clear_range()
Matan Breizman [Wed, 7 Dec 2022 13:17:23 +0000 (13:17 +0000)]
common/intrusive_lru: Add clear_range()

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit c4d996e0b733a8a198274949a9d77c1f2fed7fbd)

2 years agoosd/osd_types_fmt: Add fmt::formatter<pg_log_entry_t>
Matan Breizman [Sun, 22 Jan 2023 11:32:01 +0000 (11:32 +0000)]
osd/osd_types_fmt: Add fmt::formatter<pg_log_entry_t>

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit f4300d4aa4390f78fe1fc7b09558cfa765ea9260)

2 years agoqa/suites/crimson-rados/thrash/workloads: Enable small-objects-balanced/localized
Matan Breizman [Tue, 6 Dec 2022 14:23:06 +0000 (14:23 +0000)]
qa/suites/crimson-rados/thrash/workloads: Enable small-objects-balanced/localized

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 5b457365fa9283d2638e454ff481b45288f73a8b)

2 years agocrimson/osd/osd_operation/client_request: do_process() reorder
Matan Breizman [Tue, 29 Nov 2022 11:04:22 +0000 (11:04 +0000)]
crimson/osd/osd_operation/client_request: do_process() reorder

* Move error checking to the beginning of do_process()

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit b1d75849312bea7b91b16336738116928c687528)

2 years agocrimson/osd/osd_operations/client_request: fix is_misdirected()
Matan Breizman [Tue, 29 Nov 2022 11:00:52 +0000 (11:00 +0000)]
crimson/osd/osd_operations/client_request: fix is_misdirected()

In the case of balanced read the op is not misdirected.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit b63ab1f4aefc7a557814769cf2d9bd48ba6a2b8c)

2 years agocrimson/osd/osd_operations/client_request: Add logs around do_process()
Matan Breizman [Tue, 29 Nov 2022 10:59:55 +0000 (10:59 +0000)]
crimson/osd/osd_operations/client_request: Add logs around do_process()

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 2f77bbb79c179597dedae36855f531963aecfa54)

2 years agocrimson/osd/osd_operations/client_request: Skip do_recover_missing() on replica
Matan Breizman [Sun, 11 Dec 2022 13:44:17 +0000 (13:44 +0000)]
crimson/osd/osd_operations/client_request: Skip do_recover_missing() on replica

* assert internal_client_request is on primary since
  do_recover_missing is also called by internal requests.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 7fd0b34521d14423e3bdaf260895cd005dbf918a)

2 years agocrimson/osd/pg: Expand can_discard_op on BALANCE/LOCALIZE reads
Matan Breizman [Tue, 29 Nov 2022 10:51:58 +0000 (10:51 +0000)]
crimson/osd/pg: Expand can_discard_op on BALANCE/LOCALIZE reads

See: 69def1433d49bd8543e26e9f1ebf00dfbc5059bd

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
(cherry picked from commit 53e26d16ff386fc384fa1a4b6d631145e587b225)

2 years agocrimson/os/seastore/epm: also do prepare_write for rbm 51623/head
Xuehan Xu [Wed, 1 Mar 2023 07:09:08 +0000 (07:09 +0000)]
crimson/os/seastore/epm: also do prepare_write for rbm

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
(cherry picked from commit 1923f8db92d0d2063b8af54ab39b043d6737c8d3)

2 years agoMerge pull request #51619 from zdover23/wip-doc-2023-05-21-backport-51618-to-reef
Anthony D'Atri [Sat, 20 May 2023 22:17:12 +0000 (18:17 -0400)]
Merge pull request #51619 from zdover23/wip-doc-2023-05-21-backport-51618-to-reef

reef: doc: Add missing `ceph` command in documentation section `REPLACING A…

2 years agodoc: Add missing `ceph` command in documentation section `REPLACING AN OSD` 51619/head
Alexander Proschek [Sat, 20 May 2023 21:06:09 +0000 (14:06 -0700)]
doc: Add missing `ceph` command in documentation section `REPLACING AN OSD`

Signed-off-by: Alexander Proschek <alexander.proschek@protonmail.com>
Signed-off-by: Alexander Proschek <alexander.proschek@protonmail.com>
(cherry picked from commit 0557d5e465556adba6d25db62a40ba55a5dd2400)

2 years agodoc/cephadm: document setting mon crush locations through spec
Adam King [Thu, 12 Jan 2023 20:13:15 +0000 (15:13 -0500)]
doc/cephadm: document setting mon crush locations through spec

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit cd96b70b676369298bfb06acb394d415918d0683)

2 years agoqa/suites/orch/cephadm: teuth test for mon crush locations
Adam King [Thu, 12 Jan 2023 20:00:12 +0000 (15:00 -0500)]
qa/suites/orch/cephadm: teuth test for mon crush locations

Trying to add a feature where mon crush locations
can be set through the orchestrator using the mon
service spec. This is meant to be a test for that.

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 8c52a0a77b2ddad077d1de47ea67b89904e47e99)

2 years agomgr/cephadm: Set mon crush locations based on service spec
Adam King [Mon, 28 Nov 2022 16:59:59 +0000 (11:59 -0500)]
mgr/cephadm: Set mon crush locations based on service spec

The part of this that added the --set-crush-location flag
when deploying the mon was handled in another commit. This
piece is to finish the functionality by having cephadm set
the location through commands to handle when multiple
bucket=loc pairs are specified for a single monitor

Fixes: https://tracker.ceph.com/issues/58101
Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 2672a719067bdc71486240e99d98e42678681d44)