]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agolibrbd: always refresh after creating snapshot in CreatePrimaryRequest scrub_incomplete
Ilya Dryomov [Mon, 17 Apr 2023 21:31:37 +0000 (23:31 +0200)]
librbd: always refresh after creating snapshot in CreatePrimaryRequest

Up until now this was conditioned on whether the caller expressed
interest in the ID of the created snapshot and happened to work only
because CreatePrimaryRequest wasn't actually consulting any mirror
snapshot metadata.  This has just changed with unlink_peer() needing to
see an up-to-date complete flag which is set in SetImageStateRequest
following the write out of image state object(s).

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2 years agorbd: remove the incomplete primary snapshot
Prasanna Kumar Kalever [Mon, 6 Mar 2023 09:58:03 +0000 (15:28 +0530)]
rbd: remove the incomplete primary snapshot

Problem:
-------
At a high level, creating a primary snapshot consists of three steps:

1. actually creating a snapshot in the mirror namespace
2. generating a set of image state objects with additional metadata for
   the snapshot
3. marking the snapshot as complete after the image state objects are
   written out

Depending on the circumstances, a request to create a primary snapshot
can be forwarded to rbd-mirror daemon.  If that happens and rbd-mirror
daemon gets axed for some practical reason after completing steps (1)
and/or (2) but before completing step (3), we are left with a
permanently incomplete primary snapshot because upon retrying that
primary snapshot creation request, librbd notices that such snapshot
already exists.  It does not check whether this "pre-existing" snapshot
is complete.

Solution:
--------
As part of the next mirror snapshot create (say triggered by the
scheduler) the unlink_peer() is called, it checks if there exists any
incomplete snapshot and delete them accordingly.

Fixes: https://tracker.ceph.com/issues/58887
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2120624
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2 years agoMerge pull request #49286 from rzarzynski/wip-crimson-snaptrimmer
Radoslaw Zarzynski [Wed, 1 Mar 2023 07:57:27 +0000 (08:57 +0100)]
Merge pull request #49286 from rzarzynski/wip-crimson-snaptrimmer

crimson/osd, osd: bring snap trimming to crimson

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge PR #48720 into main
Venky Shankar [Wed, 1 Mar 2023 02:24:18 +0000 (07:54 +0530)]
Merge PR #48720 into main

* refs/pull/48720/head:
qa: fix api failure issue caused during a dashboard test
PendingReleaseNotes: noted new MDSMap field refuse_client_session
qa: added two testcases
client: do not initiate session if flag refuse_client_session is set
mds: do not reconnect when refuse_client_session is set
mds: add new feature to block clients from establishing sessions

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #49006 from yanghonggang/yhg-upgrade-rocksdb-7.7.3
Neha Ojha [Tue, 28 Feb 2023 23:50:21 +0000 (15:50 -0800)]
Merge pull request #49006 from yanghonggang/yhg-upgrade-rocksdb-7.7.3

rocksdb: Upgrade rocksdb to latest facebook's v7.9.2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2 years agoqa/valgrind.supp: suppress the ColumnFamily ctor called on init 49006/head
Radoslaw Zarzynski [Tue, 28 Feb 2023 20:58:24 +0000 (20:58 +0000)]
qa/valgrind.supp: suppress the ColumnFamily ctor called on init

```
    <frame>
      <ip>0x1712F1E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>rocksdb::BytewiseComparator()</fn>
      <dir>/home/rzarzynski/ceph3/src/rocksdb/util</dir>
      <file>comparator.cc</file>
      <line>304</line>
    </frame>
    <frame>
      <ip>0x162361C</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>rocksdb::ColumnFamilyOptions::ColumnFamilyOptions()</fn>
      <dir>/home/rzarzynski/ceph3/src/rocksdb/options</dir>
      <file>options.cc</file>
      <line>123</line>
    </frame>
    <frame>
      <ip>0x12CA21E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>Options</fn>
      <dir>/home/rzarzynski/ceph3/build/../src/rocksdb/include/rocksdb</dir>
      <file>options.h</file>
      <line>1409</line>
    </frame>
    <frame>
      <ip>0x12CA21E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>RocksDBStore::init(std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;)</fn>
      <dir>/home/rzarzynski/ceph3/build/../src/kv</dir>
      <file>RocksDBStore.cc</file>
      <line>400</line>
    </frame>
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoqa/valgrind.supp: suppress the get-or-create CacheItemHelper
Radoslaw Zarzynski [Tue, 28 Feb 2023 20:42:31 +0000 (20:42 +0000)]
qa/valgrind.supp: suppress the get-or-create CacheItemHelper

```
// Get an CacheItemHelper pointer for value type T and role R.
template <typename T, CacheEntryRole R>
Cache::CacheItemHelper* GetCacheItemHelperForRole() {
  static Cache::CacheItemHelper cache_helper(
      BlocklikeTraits<T>::SizeCallback, BlocklikeTraits<T>::SaveToCallback,
      GetCacheEntryDeleterForRole<T, R>());
  return &cache_helper;
}
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoqa/valgrind.supp: even broader suppression for rocksdb's init
Radoslaw Zarzynski [Tue, 28 Feb 2023 20:35:33 +0000 (20:35 +0000)]
qa/valgrind.supp: even broader suppression for rocksdb's init

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #50308 from adk3798/mock-fqdn-secure-alertmanager
Adam King [Tue, 28 Feb 2023 19:57:44 +0000 (14:57 -0500)]
Merge pull request #50308 from adk3798/mock-fqdn-secure-alertmanager

mgr/cephadm: mock get_fqdn in secure alertmanager config test

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoqa/valgrind.supp: ingore rocksdb leak in option parsing
Radoslaw Zarzynski [Tue, 28 Feb 2023 17:52:12 +0000 (17:52 +0000)]
qa/valgrind.supp: ingore rocksdb leak in option parsing

This suppression is needed since upgrading to v7.9.2.
See PR https://github.com/ceph/ceph/pull/49006.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: fix compilation guard around specialization for libfmt => 9 49286/head
Radoslaw Zarzynski [Tue, 17 Jan 2023 19:11:44 +0000 (19:11 +0000)]
crimson/osd: fix compilation guard around specialization for libfmt => 9

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: handle snap trim's interruption as any other error
Radoslaw Zarzynski [Thu, 22 Dec 2022 15:32:49 +0000 (15:32 +0000)]
crimson/osd: handle snap trim's interruption as any other error

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: handle PG_STATE_SNAPTRIM_ERROR when trimming snaps
Radoslaw Zarzynski [Tue, 20 Dec 2022 15:27:14 +0000 (15:27 +0000)]
crimson/osd: handle PG_STATE_SNAPTRIM_ERROR when trimming snaps

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: errorate SnapTrimEvent
Radoslaw Zarzynski [Mon, 19 Dec 2022 21:44:12 +0000 (21:44 +0000)]
crimson/osd: errorate SnapTrimEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: manage PG_STATE_SNAPTRIM when trimming snaps
Radoslaw Zarzynski [Mon, 19 Dec 2022 20:19:52 +0000 (20:19 +0000)]
crimson/osd: manage PG_STATE_SNAPTRIM when trimming snaps

After a few tries:

```
$ bin/ceph -s
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2022-12-19T20:18:21.856+0000 7ff87b46f700 -1 WARNING: all dangerous and experimental features are enabled.
2022-12-19T20:18:21.857+0000 7ff87b46f700  1 build_initial for_mkfs: 0
2022-12-19T20:18:21.860+0000 7ff87b46f700 -1 WARNING: all dangerous and experimental features are enabled.
2022-12-19T20:18:21.861+0000 7ff87b46f700  1 build_initial for_mkfs: 0
  cluster:
    id:     347aedc9-d72f-4a34-98fc-26cb04b0c2fc
    health: HEALTH_WARN
            11 mgr modules have failed dependencies
            1 pool(s) do not have an application enabled
            1 pool(s) have no replicas configured

  services:
    mon: 1 daemons, quorum a (age 3m)
    mgr: x(active, since 3m)
    osd: 1 osds: 1 up (since 3m), 1 in (since 3m)

  data:
    pools:   1 pools, 1 pgs
    objects: 95 objects, 6 B
    usage:   1.0 GiB used, 100 GiB / 101 GiB avail
    pgs:     1 active+clean+snaptrim

  io:
    client:   1.2 KiB/s wr, 0 op/s rd, 1 op/s wr
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop C++'s ranges to workaround unbuildability on Clang
Radoslaw Zarzynski [Wed, 14 Dec 2022 20:18:19 +0000 (20:18 +0000)]
crimson/osd: drop C++'s ranges to workaround unbuildability on Clang

```
/home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/osd/osd_operations/snaptrim_event.cc:38:33: note: while checking constraint satisfaction for template 'operator|<const std::ranges::views::_Elements<0> &, const std::vector<std::pair<unsigned long, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>>>> &>' required here
    for (const auto &i : subops | std::views::keys) {
                                ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/osd/osd_operations/snaptrim_event.cc:38:33: note: in instantiation of function template specialization 'std::ranges::views::__adaptor::operator|<const std::ranges::views::_Elements<0> &, const std::vector<std::pair<unsigned long, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>>>> &>' requested here
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/iterator_concepts.h:963:7: note: candidate template ignored: constraints not satisfied [with _Tp = std::ranges::ref_view<const std
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: simplify SnapTrimObjSubEvent::start()
Radoslaw Zarzynski [Tue, 13 Dec 2022 13:28:43 +0000 (13:28 +0000)]
crimson/osd: simplify SnapTrimObjSubEvent::start()

Throwing away the NOP-looking-like `maybe_delay` concept became
possible after eradication of the `with_interruption` call nested
within `SnapTrimSubEvent`. Before, on Ubuntu and GCC 11.2.1, lack
of it was the reson behind assertion failures on resetting the
`thread_local` intteruption condition.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop the nested interruptors in snap trimming
Radoslaw Zarzynski [Tue, 13 Dec 2022 18:30:28 +0000 (18:30 +0000)]
crimson/osd: drop the nested interruptors in snap trimming

`crimson::interruptible` does not support that. See the
`DISABLED_nested_interruptors` unit test.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/test: verify nesting of interruptors
Radoslaw Zarzynski [Mon, 12 Dec 2022 19:12:11 +0000 (19:12 +0000)]
crimson/test: verify nesting of interruptors

It doesn't work.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: errorate SnapTrimObjSubEvent::remove_clone()
Radoslaw Zarzynski [Mon, 12 Dec 2022 20:56:42 +0000 (20:56 +0000)]
crimson/osd: errorate SnapTrimObjSubEvent::remove_clone()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: convert SnapTrimObjSubEvent::remove_or_update to ertr
Radoslaw Zarzynski [Sun, 11 Dec 2022 16:38:38 +0000 (16:38 +0000)]
crimson/osd: convert SnapTrimObjSubEvent::remove_or_update to ertr

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: dissect snapset update into SnapTrimObjSubEvent::update_head()
Radoslaw Zarzynski [Sat, 10 Dec 2022 22:42:06 +0000 (22:42 +0000)]
crimson/osd: dissect snapset update into SnapTrimObjSubEvent::update_head()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: dissect head whiteout removal into SnapTrimObjSubEvent::remove_head_whit...
Radoslaw Zarzynski [Sat, 10 Dec 2022 22:28:55 +0000 (22:28 +0000)]
crimson/osd: dissect head whiteout removal into SnapTrimObjSubEvent::remove_head_whiteout()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop data aliases from SnapTrimObjSubEvent::adjust_snaps()
Radoslaw Zarzynski [Sat, 10 Dec 2022 22:03:11 +0000 (22:03 +0000)]
crimson/osd: drop data aliases from SnapTrimObjSubEvent::adjust_snaps()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: dissect clone removal into SnapTrimObjSubEvent::remove_clone()
Radoslaw Zarzynski [Sat, 10 Dec 2022 21:54:26 +0000 (21:54 +0000)]
crimson/osd: dissect clone removal into SnapTrimObjSubEvent::remove_clone()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop data aliases from SnapTrimObjSubEvent::remove_or_update()
Radoslaw Zarzynski [Sat, 10 Dec 2022 21:29:26 +0000 (21:29 +0000)]
crimson/osd: drop data aliases from SnapTrimObjSubEvent::remove_or_update()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: simplify informing SnapMapper about new mappings
Radoslaw Zarzynski [Fri, 9 Dec 2022 01:13:30 +0000 (01:13 +0000)]
crimson/osd: simplify informing SnapMapper about new mappings

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: implement basic osd_snap_trim_sleep support
Radoslaw Zarzynski [Wed, 7 Dec 2022 20:41:07 +0000 (20:41 +0000)]
crimson/osd: implement basic osd_snap_trim_sleep support

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: introduce WaitSubop stage to SnapTrimEvent
Radoslaw Zarzynski [Tue, 6 Dec 2022 14:04:17 +0000 (14:04 +0000)]
crimson/osd: introduce WaitSubop stage to SnapTrimEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: snap trimming informs OpsExecuter about altered objects
Radoslaw Zarzynski [Tue, 6 Dec 2022 14:01:44 +0000 (14:01 +0000)]
crimson/osd: snap trimming informs OpsExecuter about altered objects

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocommon: add debugs to crimson's green_condition_variable
Radoslaw Zarzynski [Tue, 6 Dec 2022 10:55:39 +0000 (10:55 +0000)]
common: add debugs to crimson's green_condition_variable

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop unused declaration of PG::fill_op_params_bump_pg_version()
Radoslaw Zarzynski [Mon, 5 Dec 2022 20:17:19 +0000 (20:17 +0000)]
crimson/osd: drop unused declaration of PG::fill_op_params_bump_pg_version()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoosd: SnapMapper::OSDriver uses crimson's interruptor::green_get().
Radoslaw Zarzynski [Mon, 5 Dec 2022 20:00:55 +0000 (20:00 +0000)]
osd: SnapMapper::OSDriver uses crimson's interruptor::green_get().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/common: bring green_get() to interruptor
Radoslaw Zarzynski [Tue, 6 Dec 2022 12:06:39 +0000 (12:06 +0000)]
crimson/common: bring green_get() to interruptor

The rationale
=============
Calling `get()` on an unavailable future can result in yielding
the reactor, so -- from the interruptor's POV -- the interrupt
condition must be handled exactly like it is done for `yield()`.

Interruptor takes care about that but, at the moment, only for the
interruptible futures; there is mechanism for plain `seastar::future`.
This patch paves the way for handling them as well.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/common: trace management of interrupt_cond in interruptor::async()
Radoslaw Zarzynski [Mon, 5 Dec 2022 19:51:14 +0000 (19:51 +0000)]
crimson/common: trace management of interrupt_cond in interruptor::async()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/common: fix method name in interrupt_cond_t::reset()'s debugs
Radoslaw Zarzynski [Mon, 5 Dec 2022 19:46:36 +0000 (19:46 +0000)]
crimson/common: fix method name in interrupt_cond_t::reset()'s debugs

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoosd: fix intersecion of green threads and interuptible
Radoslaw Zarzynski [Sat, 3 Dec 2022 21:24:56 +0000 (21:24 +0000)]
osd: fix intersecion of green threads and interuptible

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: wire SnapMapper::add_oid() with OpsExecuter
Radoslaw Zarzynski [Fri, 25 Nov 2022 22:48:04 +0000 (22:48 +0000)]
crimson/osd: wire SnapMapper::add_oid() with OpsExecuter

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: settle snap_trimq in PG and wire it up
Radoslaw Zarzynski [Thu, 24 Nov 2022 22:50:17 +0000 (22:50 +0000)]
crimson/osd: settle snap_trimq in PG and wire it up

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: settle SnapMapper within crimson'o PG
Radoslaw Zarzynski [Thu, 24 Nov 2022 16:26:38 +0000 (16:26 +0000)]
crimson/osd: settle SnapMapper within crimson'o PG

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: PerShardState::start_operation() doesn't assume Ret is seastar::future<>
Radoslaw Zarzynski [Wed, 23 Nov 2022 21:30:02 +0000 (21:30 +0000)]
crimson/osd: PerShardState::start_operation() doesn't assume Ret is seastar::future<>

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: implement SnapMapper::OSDriver for crimson
Radoslaw Zarzynski [Tue, 22 Nov 2022 18:21:01 +0000 (18:21 +0000)]
crimson/osd: implement SnapMapper::OSDriver for crimson

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/ertr: let ErrVisitorT return plain value if ValueFuncT returns seastar::future
Radoslaw Zarzynski [Tue, 22 Nov 2022 16:03:42 +0000 (16:03 +0000)]
crimson/ertr: let ErrVisitorT return plain value if ValueFuncT returns seastar::future

The following corner case was unsupported till now:

```cpp
  return os->omap_get_values(
    ch, hoid, keys
  ).safe_then([] (FuturizedStore::omap_values_t&& vals) {
    return 0;
  }, FuturizedStore::read_errorator::all_same_way([] (auto&) {
    return -1;
  })).get();
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocommon/ceph_mutex: add green thread-requiring condition variable for crimson
Radoslaw Zarzynski [Mon, 21 Nov 2022 19:06:02 +0000 (19:06 +0000)]
common/ceph_mutex: add green thread-requiring condition variable for crimson

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd, osd: share SnapMapper with crimson
Radoslaw Zarzynski [Fri, 18 Nov 2022 19:15:27 +0000 (19:15 +0000)]
crimson/osd, osd: share SnapMapper with crimson

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoosd: drop the scrub_purged_snaps() declaration from SnapMapper
Radoslaw Zarzynski [Fri, 18 Nov 2022 18:43:24 +0000 (18:43 +0000)]
osd: drop the scrub_purged_snaps() declaration from SnapMapper

Nowadays this is a dead code; the actual definition is in `OSD`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: SnapMapper::record_purged_snaps() takes abstratced store and txn
Radoslaw Zarzynski [Fri, 18 Nov 2022 17:47:16 +0000 (17:47 +0000)]
crimson/osd: SnapMapper::record_purged_snaps() takes abstratced store and txn

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: convert SnapMapper::record_purged_snaps() to OSTransaction
Radoslaw Zarzynski [Fri, 18 Nov 2022 16:12:10 +0000 (16:12 +0000)]
crimson/osd: convert SnapMapper::record_purged_snaps() to OSTransaction

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: initialize OSDriver in SnapMapper::record_purged_snaps()
Radoslaw Zarzynski [Fri, 18 Nov 2022 15:53:38 +0000 (15:53 +0000)]
crimson/osd: initialize OSDriver in SnapMapper::record_purged_snaps()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocommon, osd: switch SnapMapper::_lookup_purged_snap to use OSDriver
Radoslaw Zarzynski [Thu, 17 Nov 2022 21:29:20 +0000 (21:29 +0000)]
common, osd: switch SnapMapper::_lookup_purged_snap to use OSDriver

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: fix debug in Orderer::clear_and_cancel()
Radoslaw Zarzynski [Thu, 17 Nov 2022 15:22:02 +0000 (15:22 +0000)]
crimson/osd: fix debug in Orderer::clear_and_cancel()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoosd: don't build unnecessary parts of SnapMapper for crimson
Radoslaw Zarzynski [Wed, 16 Nov 2022 16:47:49 +0000 (16:47 +0000)]
osd: don't build unnecessary parts of SnapMapper for crimson

This commits skips building two call-sites of `omap_get_iterator()`
of `ObjectStore` which are bypassing the `OSDriver` abstraction
layer.

1. `SnapMapper::convert_legacy()` likely never be needed by
crimson and we can leave it unchanged.

2. `SnapMapper::Subber` isn't needed right now but will be
needed in the future.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: bring SnapTrimObjSubEvent
Radosław Zarzyński [Tue, 6 Dec 2022 21:20:15 +0000 (22:20 +0100)]
crimson/osd: bring SnapTrimObjSubEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: wire SnapTrimEvent with SnapMapper up
Radosław Zarzyński [Tue, 6 Dec 2022 20:26:42 +0000 (21:26 +0100)]
crimson/osd: wire SnapTrimEvent with SnapMapper up

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: bring SubOpBlocker to SnapTrimEvent
Radoslaw Zarzynski [Wed, 16 Nov 2022 16:40:51 +0000 (16:40 +0000)]
crimson/osd: bring SubOpBlocker to SnapTrimEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: implement the basics of SnapTrimEvent
Radoslaw Zarzynski [Wed, 16 Nov 2022 15:19:41 +0000 (15:19 +0000)]
crimson/osd: implement the basics of SnapTrimEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd/osd_operations: flatten indentation of InternalClientRequest
Radoslaw Zarzynski [Tue, 8 Nov 2022 13:55:37 +0000 (13:55 +0000)]
crimson/osd/osd_operations: flatten indentation of InternalClientRequest

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop the residue of CompoundPeeringRequest
Radoslaw Zarzynski [Fri, 4 Nov 2022 13:07:13 +0000 (13:07 +0000)]
crimson/osd: drop the residue of CompoundPeeringRequest

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: simplify ShardServices::snap_request_reservation()
Radoslaw Zarzynski [Wed, 26 Oct 2022 18:55:22 +0000 (18:55 +0000)]
crimson/osd: simplify ShardServices::snap_request_reservation()

`on_preempt` is not used by snap trimming; we can left its default.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: bring the skeleton of SnapTrimEvent
Radoslaw Zarzynski [Wed, 26 Oct 2022 18:46:37 +0000 (18:46 +0000)]
crimson/osd: bring the skeleton of SnapTrimEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: settle snap_reserver within ShardServices
Radoslaw Zarzynski [Wed, 26 Oct 2022 17:45:50 +0000 (17:45 +0000)]
crimson/osd: settle snap_reserver within ShardServices

This bit will be used by the upcoming `SnapTrimRequest`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: move the SnapTrimmer-related handlers of PeeringState from .h to .cc
Radoslaw Zarzynski [Wed, 26 Oct 2022 11:50:22 +0000 (11:50 +0000)]
crimson/osd: move the SnapTrimmer-related handlers of PeeringState from .h to .cc

This commit is solely about code shuffling.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agomgr/cephadm: mock get_fqdn in secure alertmanager config test 50308/head
Adam King [Tue, 28 Feb 2023 15:45:03 +0000 (10:45 -0500)]
mgr/cephadm: mock get_fqdn in secure alertmanager config test

Before it was expecting "::1" when passed to get_fqdn to
resolve to "localhost" but in some environments it would instead
resolve to "localhost.localdomain" and the test would fail. This
test isn't testing the functionality of get_fqdn, so let's just
mock it so the test passes regardless of how get_fqdn resolves "::1"

Signed-off-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #49836 from rkachach/fix_issue_58548
Adam King [Tue, 28 Feb 2023 14:31:21 +0000 (09:31 -0500)]
Merge pull request #49836 from rkachach/fix_issue_58548

cephadm: using ip instead of short hostname for prometheus urls

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2 years agoMerge pull request #50083 from adk3798/pids-limit
Adam King [Tue, 28 Feb 2023 14:28:34 +0000 (09:28 -0500)]
Merge pull request #50083 from adk3798/pids-limit

cephadm: set pids-limit unlimited for all ceph daemons

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>
2 years agoMerge PR #50268 into main
Venky Shankar [Tue, 28 Feb 2023 13:45:32 +0000 (19:15 +0530)]
Merge PR #50268 into main

* refs/pull/50268/head:
mds: memset the head by default to fix the readable.sh test failure

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #46601 from rkachach/fix_issue_55814
Adam King [Tue, 28 Feb 2023 13:41:50 +0000 (08:41 -0500)]
Merge pull request #46601 from rkachach/fix_issue_55814

mgr/cephadm: adding support for a secure ceph monitoring stack

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #50006 from idryomov/wip-58647
Ilya Dryomov [Tue, 28 Feb 2023 12:31:02 +0000 (13:31 +0100)]
Merge pull request #50006 from idryomov/wip-58647

mon/MgrMap: dump last_failure_osd_epoch and active_clients at top level

Reviewed-by: Ramana Raja <rraja@redhat.com>
2 years agoMerge pull request #50285 from ceph/epuertat-patch-1
Ernesto Puerta [Tue, 28 Feb 2023 11:41:56 +0000 (12:41 +0100)]
Merge pull request #50285 from ceph/epuertat-patch-1

.github: update add-to-project action

Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #50299 from rhcs-dashboard/fix-broken-styles-crud
Nizamudeen A [Tue, 28 Feb 2023 10:57:32 +0000 (16:27 +0530)]
Merge pull request #50299 from rhcs-dashboard/fix-broken-styles-crud

mgr/dashboard: minor fixes to crud form UI

Reviewed-by: Pegonzal <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #50085 from myoungwon/wip-fix-initialization-vstart-rbm
Yingxin [Tue, 28 Feb 2023 08:33:32 +0000 (16:33 +0800)]
Merge pull request #50085 from myoungwon/wip-fix-initialization-vstart-rbm

crimson/os/seastore: RBM fixes during vstart startup time

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge PR #50197 into main
Venky Shankar [Tue, 28 Feb 2023 07:01:59 +0000 (12:31 +0530)]
Merge PR #50197 into main

* refs/pull/50197/head:
cephfs-top: navigate to home screen while selecting sort field, if no filesystems.
cephfs-top: display_menu() isn't triggered when all fs' are removed

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agomgr/dashboard: minor fixes to crud form UI 50299/head
Nizamudeen A [Tue, 28 Feb 2023 06:18:16 +0000 (11:48 +0530)]
mgr/dashboard: minor fixes to crud form UI

Fixes: https://tracker.ceph.com/issues/58864
Signed-off-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #50295 from zdover23/wip-doc-2023-02-28-radosgw-multisite-zonegrou...
Anthony D'Atri [Tue, 28 Feb 2023 06:02:03 +0000 (01:02 -0500)]
Merge pull request #50295 from zdover23/wip-doc-2023-02-28-radosgw-multisite-zonegroup-correction-and-formatting

doc/radosgw: s/zone group/zonegroup/g et alia

2 years agoMerge pull request #50273 from cyx1231st/wip-seastore-fix-submitter-roll-segment
Yingxin [Tue, 28 Feb 2023 05:29:36 +0000 (13:29 +0800)]
Merge pull request #50273 from cyx1231st/wip-seastore-fix-submitter-roll-segment

crimson/os/seastore/journal: allow pending i/o in a full record_submitter when rolling the segment 2

Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
2 years agodoc/radosgw: s/zone group/zonegroup/g et alia 50295/head
Zac Dover [Tue, 28 Feb 2023 02:55:08 +0000 (12:55 +1000)]
doc/radosgw: s/zone group/zonegroup/g et alia

s/zone group/zonegroup/ where simple greps failed to find instances of
"zone group" that were spread across two lines; break a paragraph into
two paragraphs so that each paragraph has a thematic idea of its own.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #50276 from zdover23/wip-doc-2023-02-27-radosgw-multisite-caption...
zdover23 [Tue, 28 Feb 2023 02:10:02 +0000 (12:10 +1000)]
Merge pull request #50276 from zdover23/wip-doc-2023-02-27-radosgw-multisite-caption-zone-data-replication

doc/rgw: caption a diagram

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #48010 from ljflores/wip-lflores-primary-balancer
Laura Flores [Mon, 27 Feb 2023 23:55:52 +0000 (17:55 -0600)]
Merge pull request #48010 from ljflores/wip-lflores-primary-balancer

osd, tools, test/osd: implement read balancer

2 years agoMerge pull request #49482 from amathuria/wip-amathuria-mclock-high-prio-queue
Laura Flores [Mon, 27 Feb 2023 23:55:43 +0000 (17:55 -0600)]
Merge pull request #49482 from amathuria/wip-amathuria-mclock-high-prio-queue

mClock: Add ability to handle high priority operations

2 years agomon/MgrMap: dump last_failure_osd_epoch and active_clients at top level 50006/head
Ilya Dryomov [Mon, 6 Feb 2023 16:56:00 +0000 (17:56 +0100)]
mon/MgrMap: dump last_failure_osd_epoch and active_clients at top level

Currently last_failure_osd_epoch and active_clients are dumped in the
always_on_modules dictionary in "ceph mgr dump" output.  This goes back
to when these fields were added in commits f2986a4400bb ("mon/MgrMonitor:
blacklist previous instance") and df507cde8d71 ("mgr: forward RADOS
client instances for potential blacklist") but is wrong as these fields
have nothing to do with always-on modules.

Fixes: https://tracker.ceph.com/issues/58647
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #48706 from Matan-B/wip-matanb-max-oldest-map
Yuri Weinstein [Mon, 27 Feb 2023 20:12:59 +0000 (12:12 -0800)]
Merge pull request #48706 from Matan-B/wip-matanb-max-oldest-map

OSD: Fix check_past_interval_bounds()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #49990 from rzarzynski/wip-osd-mon-public_bind_in_bindv
Laura Flores [Mon, 27 Feb 2023 20:10:49 +0000 (14:10 -0600)]
Merge pull request #49990 from rzarzynski/wip-osd-mon-public_bind_in_bindv

mon, osd: rework the public_bind_addr support. Bring it to OSD

2 years agoMerge pull request #50047 from rhcs-dashboard/post-css-error-teuth
Yuri Weinstein [Mon, 27 Feb 2023 19:56:27 +0000 (11:56 -0800)]
Merge pull request #50047 from rhcs-dashboard/post-css-error-teuth

mgr/dashboard: fix conflicting peer dependency: postcss@8.4.21

Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agoMerge pull request #49507 from Matan-B/wip-matanb-check_pg_num-edit
Yuri Weinstein [Mon, 27 Feb 2023 19:56:00 +0000 (11:56 -0800)]
Merge pull request #49507 from Matan-B/wip-matanb-check_pg_num-edit

mon/OSDMontior: Simplify check_pg_num()

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>
2 years agoMerge pull request #48192 from amathuria/amathuria-edit-pg-deletion-cost
Yuri Weinstein [Mon, 27 Feb 2023 19:54:33 +0000 (11:54 -0800)]
Merge pull request #48192 from amathuria/amathuria-edit-pg-deletion-cost

osd: Change default value of osd_pg_delete_cost

Reviewed-by: Sridhar Seshasayee <sseshasa@redhat.com>
2 years agoMerge pull request #48657 from alimaredia/wip-rgw-labeled-perf-counters-cache
Ilya Dryomov [Mon, 27 Feb 2023 19:05:54 +0000 (20:05 +0100)]
Merge pull request #48657 from alimaredia/wip-rgw-labeled-perf-counters-cache

Labeled Perf Counters

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agodoc/rgw: caption a diagram 50276/head
Zac Dover [Mon, 27 Feb 2023 09:00:04 +0000 (19:00 +1000)]
doc/rgw: caption a diagram

Add an explanation of a diagram that shows how object data is replicated
between zones within a zonegroup.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years ago.github: update add-to-project action 50285/head
Ernesto Puerta [Mon, 27 Feb 2023 15:01:33 +0000 (16:01 +0100)]
.github: update add-to-project action

Signed-off-by: Ernesto Puerta <37327689+epuertat@users.noreply.github.com>
2 years agoMerge pull request #50274 from zdover23/wip-doc-2023-02-27-radosgw-multisite-remove...
Anthony D'Atri [Mon, 27 Feb 2023 13:43:58 +0000 (08:43 -0500)]
Merge pull request #50274 from zdover23/wip-doc-2023-02-27-radosgw-multisite-remove-tertiary

doc/rgw: remove "tertiary", link to procedure

2 years agoMerge pull request #50230 from idryomov/wip-58833
Ilya Dryomov [Mon, 27 Feb 2023 13:15:31 +0000 (14:15 +0100)]
Merge pull request #50230 from idryomov/wip-58833

librbd: call apply_changes() after setting librados_thread_count

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>
2 years agoMerge pull request #50232 from idryomov/wip-qa-workunits-windows-ownership
Ilya Dryomov [Mon, 27 Feb 2023 10:24:40 +0000 (11:24 +0100)]
Merge pull request #50232 from idryomov/wip-qa-workunits-windows-ownership

CODEOWNERS: assign qa/workunits/windows to RBD

Reviewed-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2 years agomds: memset the head by default to fix the readable.sh test failure 50268/head
Xiubo Li [Mon, 27 Feb 2023 04:55:59 +0000 (12:55 +0800)]
mds: memset the head by default to fix the readable.sh test failure

Introduced-by: cbd7e304020(ceph_fs.h: add 32 bits extended num_retry and num_fwd support)
Fixes: https://tracker.ceph.com/issues/58853
Signed-off-by: Xiubo Li <xiubli@redhat.com>
2 years agoMerge pull request #50264 from zdover23/wip-doc-2023-02-27-radosgw-multisite-infernal...
zdover23 [Mon, 27 Feb 2023 09:01:36 +0000 (19:01 +1000)]
Merge pull request #50264 from zdover23/wip-doc-2023-02-27-radosgw-multisite-infernalis-changes

doc/radosgw: multisite - edit "functional changes"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agocrimson/os/seastore/journal: move segment rolling asserts to ceph_asserts 50273/head
Yingxin Cheng [Mon, 27 Feb 2023 08:18:26 +0000 (16:18 +0800)]
crimson/os/seastore/journal: move segment rolling asserts to ceph_asserts

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocrimson/os/seastore/journal: allow pending i/o in a full record_submitter when rollin...
Yingxin Cheng [Mon, 27 Feb 2023 08:14:16 +0000 (16:14 +0800)]
crimson/os/seastore/journal: allow pending i/o in a full record_submitter when rolling the segment

Fixes: https://tracker.ceph.com/issues/58824
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agodoc/rgw: remove "tertiary", link to procedure 50274/head
Zac Dover [Mon, 27 Feb 2023 08:40:14 +0000 (18:40 +1000)]
doc/rgw: remove "tertiary", link to procedure

Remove the term "tertiary zone" and replace it with "second secondary
zone" (because there is no such thing as a tertiary zone). Link to the
procedure for creating a secondary zone in a place where such a link is
helpful to the reader.

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agodoc/radosgw: multisite - edit "functional changes" 50264/head
Zac Dover [Sun, 26 Feb 2023 16:52:02 +0000 (02:52 +1000)]
doc/radosgw: multisite - edit "functional changes"

Improve the wording of the section "Functional Changes from Infernalis"
in doc/radosgw/multisite.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge PR #50178 into main
Venky Shankar [Mon, 27 Feb 2023 05:52:42 +0000 (11:22 +0530)]
Merge PR #50178 into main

* refs/pull/50178/head:
client: add comment about how quota affect space usage report

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2 years agoMerge PR #44240 into main
Venky Shankar [Mon, 27 Feb 2023 05:36:01 +0000 (11:06 +0530)]
Merge PR #44240 into main

* refs/pull/44240/head:
qa/cephfs: upgrade xfstests_dev.py for FUSE mounts

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
2 years agoMerge pull request #50238 from rhcs-dashboard/fix-fix-58827_2-main
Nizamudeen A [Mon, 27 Feb 2023 04:43:45 +0000 (10:13 +0530)]
Merge pull request #50238 from rhcs-dashboard/fix-fix-58827_2-main

mgr/dashboard: fix constraints.txt again

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>