]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agomgr/telemetry: make sure histograms are formatted in `all` commands 50481/head
Laura Flores [Fri, 18 Nov 2022 20:42:18 +0000 (20:42 +0000)]
mgr/telemetry: make sure histograms are formatted in `all` commands

I noticed that histograms weren't getting formatted in `preview-all`
and `show-all`. I made a few tweaks so they are properly formatted now.

I verified this fix on:
`ceph telemetry preview`
`ceph telemetry preview-all`
`ceph telemetry show`
`ceph telemetry show-all`

Note that only the `all` commands were affected.

Fixes: https://tracker.ceph.com/issues/58050
Signed-off-by: Laura Flores <lflores@redhat.com>
(cherry picked from commit 277350ed15ca7ca899c8f58062ac463f682e2f37)

2 years agoMerge pull request #50387 from zdover23/wip-doc-2023-03-06-backport-50386-to-reef
zdover23 [Thu, 9 Mar 2023 10:09:47 +0000 (20:09 +1000)]
Merge pull request #50387 from zdover23/wip-doc-2023-03-06-backport-50386-to-reef

reef: doc/rgw - fix grammar in table in s3.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #50431 from zdover23/wip-doc-2023-03-08-backport-50430-to-reef
zdover23 [Wed, 8 Mar 2023 04:01:03 +0000 (14:01 +1000)]
Merge pull request #50431 from zdover23/wip-doc-2023-03-08-backport-50430-to-reef

reef: doc/install: update index.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/install: update index.rst 50431/head
Zac Dover [Wed, 8 Mar 2023 01:52:12 +0000 (11:52 +1000)]
doc/install: update index.rst

Update index.rst by making minor grammar improvements. This file was
long overdue for a backport to Reef, Quincy, and Pacific, so this commit
was a good way to pass a human eyeball over the text before making those
backports.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 82d748ff590cfd31fefddc094b10622d090b054f)

2 years agoMerge pull request #50417 from soumyakoduri/wip-skoduri-reef
Casey Bodley [Tue, 7 Mar 2023 13:45:47 +0000 (08:45 -0500)]
Merge pull request #50417 from soumyakoduri/wip-skoduri-reef

[Reef] rgw/tests: Reset endpoint before creating additional zones

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agorgw/tests: Reset endpoint before creating additional zones 50417/head
Soumya Koduri [Mon, 6 Mar 2023 14:23:08 +0000 (19:53 +0530)]
rgw/tests: Reset endpoint before creating additional zones

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Fixes: https://tracker.ceph.com/issues/58822
2 years agoMerge pull request #50399 from zdover23/wip-doc-2023-03-07-backport-50398-to-reef
zdover23 [Mon, 6 Mar 2023 23:13:47 +0000 (09:13 +1000)]
Merge pull request #50399 from zdover23/wip-doc-2023-03-07-backport-50398-to-reef

reef: doc/dev: add full stop to sentence in basic-wo

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #50374 from cbodley/wip-rgw-reef-s3select-json
Casey Bodley [Mon, 6 Mar 2023 22:11:02 +0000 (17:11 -0500)]
Merge pull request #50374 from cbodley/wip-rgw-reef-s3select-json

reef: json integration: identifying new data-source and execute json flow.

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>
2 years agoMerge pull request #50358 from cbodley/wip-qa-rgw-reef-tests
Casey Bodley [Mon, 6 Mar 2023 22:05:09 +0000 (17:05 -0500)]
Merge pull request #50358 from cbodley/wip-qa-rgw-reef-tests

reef: qa/rgw: point all test repos at new ceph-reef branch

Reviewed-by: Ali Maredia <amaredia@redhat.com>
2 years agoMerge pull request #50376 from cbodley/wip-58891-reef
Casey Bodley [Mon, 6 Mar 2023 19:44:13 +0000 (14:44 -0500)]
Merge pull request #50376 from cbodley/wip-58891-reef

reef: rgw/reshard: treat old RGWBucketInfo::num_shards=0 as 1 shard

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agodoc/dev: add full stop to sentence in basic-wo 50399/head
Zac Dover [Mon, 6 Mar 2023 16:15:27 +0000 (02:15 +1000)]
doc/dev: add full stop to sentence in basic-wo

Add a full stop to a sentence in basic-workflow.rst. I looked at this
document and noticed that it wasn't there, and it was just bugging me.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 262b115b7554b7931cc174500eb12ac7a5d67513)

2 years agoMerge pull request #50394 from zdover23/wip-doc-2023-03-06-backport-50382-to-reef
Anthony D'Atri [Mon, 6 Mar 2023 15:04:55 +0000 (10:04 -0500)]
Merge pull request #50394 from zdover23/wip-doc-2023-03-06-backport-50382-to-reef

reef: doc/rgw: fix caption

2 years agodoc/rgw: fix caption 50394/head
Zac Dover [Sun, 5 Mar 2023 06:43:38 +0000 (16:43 +1000)]
doc/rgw: fix caption

Rewrite the third paragraph of the caption to the figure depicting how a
zonegroup works.

Follows https://github.com/ceph/ceph/pull/50316.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 8fa521466023635436305c1af31501bbbe20e857)

2 years agodoc/rgw - fix grammar in table in s3.rst 50387/head
Zac Dover [Mon, 6 Mar 2023 00:09:24 +0000 (10:09 +1000)]
doc/rgw - fix grammar in table in s3.rst

Put an "only" in the correct place. Also, this is a very small change
that will allow me to see if Jenkins is working (it probably isn't).

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 1beab1e8abf59088be628cca972c59202526637d)

2 years agoMerge pull request #50377 from zdover23/wip-doc-2023-03-04-backport-50375-to-reef
Anthony D'Atri [Fri, 3 Mar 2023 23:04:36 +0000 (18:04 -0500)]
Merge pull request #50377 from zdover23/wip-doc-2023-03-04-backport-50375-to-reef

reef: doc/rgw: refine keycloak.rst

2 years agodoc/rgw: refine keycloak.rst 50377/head
Zac Dover [Fri, 3 Mar 2023 19:36:35 +0000 (05:36 +1000)]
doc/rgw: refine keycloak.rst

Full line-edit of doc/radosgw/keycloak.rst. Corrected syntax, grammar,
RST, and broken links.

Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 5706ff1e451e6c7b3c3d40d492985377161b0133)

2 years agorgw/reshard: avoid infinite loop if bi_list() returns -ENOENT 50376/head
Casey Bodley [Wed, 1 Mar 2023 22:20:06 +0000 (17:20 -0500)]
rgw/reshard: avoid infinite loop if bi_list() returns -ENOENT

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3e10944ff551e7bbb3d9083cb351ceb53c6b07b9)

2 years agorgw/rados: RGWSI_BucketIndex_RADOS requires layouts, not num_shards
Casey Bodley [Wed, 1 Mar 2023 21:06:14 +0000 (16:06 -0500)]
rgw/rados: RGWSI_BucketIndex_RADOS requires layouts, not num_shards

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 46f89cbae76d68af83a712f42443466e466681a3)

2 years agorgw/rados: use rgw::num_shards() helper
Casey Bodley [Wed, 1 Mar 2023 15:48:30 +0000 (10:48 -0500)]
rgw/rados: use rgw::num_shards() helper

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c8908fdd4cf4bb6dd14787dd7a49f75b32cc53ce)

2 years agorgw/reshard: refuse to reshard to new_num_shards=0
Casey Bodley [Wed, 1 Mar 2023 15:44:31 +0000 (10:44 -0500)]
rgw/reshard: refuse to reshard to new_num_shards=0

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4db197e858a4cedef4ad86af66d9b2cd2f4e4e31)

2 years agorgw/reshard: use the rgw::num_shards() helper
Casey Bodley [Wed, 1 Mar 2023 15:44:09 +0000 (10:44 -0500)]
rgw/reshard: use the rgw::num_shards() helper

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e376080940605e21ba4189597706371bd7a29d9b)

2 years agorgw: rgw::num_shards() returns 1 instead of 0
Casey Bodley [Wed, 1 Mar 2023 15:02:11 +0000 (10:02 -0500)]
rgw: rgw::num_shards() returns 1 instead of 0

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4497ce51c40fe04688d7a80e9b3241fe89e1db5e)

2 years agojson integration: identifying new data-source and execute json flow. 50374/head
galsalomon66 [Tue, 13 Dec 2022 14:31:43 +0000 (16:31 +0200)]
json integration: identifying new data-source and execute json flow.
s3select submodule

integrate the limit-operator into RGW. current changes is for CSV format

integrate the limit-operator with CSV-flow, and JSON-flow. an update of s3select submodule

a fix for the input-serialization-type selection

debug functionality.

fix for error handling

adding the scan-range feature. i.e. enables the user to define the range of processing(text only)

remove the temp variable(the s3select-layer handles the sql-result setting upon JSON flow)

adding documentation. for JSON, SQL limit-operator, scan-range option.
a fix for JSON test framework in s3select module

investigate crash. replace len with it.length()

editorial changes; non-initialized variables

adding validation for offset&length. (it seems that the offset&length causes Invalid reads, in some cases)

changes related to trino integration.(1) syntax issues related to trino-engine statement conbersions(s3select parser handles that) (2).
(2). trino rejects(HIVE_CURSUR_ERROR) some of the s3select responses do to its size, the change is about control the size of the respond.

update JSON documentation

trino changes: response in paging (limitation on size), trino syntax issues.

Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
(cherry picked from commit 9f20e75a0b611a9de96e67070ce6078e23f6ba41)

2 years agoMerge pull request #50366 from zdover23/wip-doc-2023-03-03-backport-50364-to-reef
Anthony D'Atri [Fri, 3 Mar 2023 15:55:32 +0000 (10:55 -0500)]
Merge pull request #50366 from zdover23/wip-doc-2023-03-03-backport-50364-to-reef

reef: doc/cephadm: Redd up compatibility.rst

2 years agodoc/cephadm: Redd up compatibility.rst 50366/head
Anthony D'Atri [Fri, 3 Mar 2023 07:44:26 +0000 (02:44 -0500)]
doc/cephadm: Redd up compatibility.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
(cherry picked from commit d608dec43b84c9a06990458219dda769829e05f8)

2 years agoMerge pull request #50361 from zdover23/wip-doc-2023-03-03-backport-50354-to-reef
Anthony D'Atri [Fri, 3 Mar 2023 06:48:42 +0000 (01:48 -0500)]
Merge pull request #50361 from zdover23/wip-doc-2023-03-03-backport-50354-to-reef

reef: doc/rgw: clean zone-sync.svg

2 years agoMerge pull request #50355 from zdover23/wip-doc-2023-03-03-backport-50351-to-reef
zdover23 [Fri, 3 Mar 2023 05:05:38 +0000 (15:05 +1000)]
Merge pull request #50355 from zdover23/wip-doc-2023-03-03-backport-50351-to-reef

reef: doc/radosgw: format admonitions

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rgw: clean zone-sync.svg 50361/head
Zac Dover [Thu, 2 Mar 2023 19:24:00 +0000 (05:24 +1000)]
doc/rgw: clean zone-sync.svg

Refine zone-sync.svg (fix kerning and use fonts that render properly).

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 064fdc9d9b4ad3393bfb5069d2f97dc622aaf12d)

2 years agoqa/rgw: point all test repos at new ceph-reef branch 50358/head
Casey Bodley [Thu, 2 Mar 2023 23:30:26 +0000 (18:30 -0500)]
qa/rgw: point all test repos at new ceph-reef branch

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agoqa/smoke: use rgw/s3tests-branch.yaml
Casey Bodley [Thu, 2 Mar 2023 23:29:44 +0000 (18:29 -0500)]
qa/smoke: use rgw/s3tests-branch.yaml

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agodoc/radosgw: format admonitions 50355/head
Zac Dover [Thu, 2 Mar 2023 18:04:30 +0000 (04:04 +1000)]
doc/radosgw: format admonitions

Break up the text of two similar admonitions into three paragraphs (in
each of the two instances). This makes the content of the admonition
much easier to read at a glance.

Signed-off-by: Zac Dover <zac.dover@proton.me>
(cherry picked from commit 4db68f5711f48b190f0135a3a7a6df8dfcf5abfa)

2 years agoMerge pull request #50338 from zdover23/wip-doc-2023-03-02-backport-50335-to-reef
zdover23 [Thu, 2 Mar 2023 03:59:45 +0000 (13:59 +1000)]
Merge pull request #50338 from zdover23/wip-doc-2023-03-02-backport-50335-to-reef

reef: doc/cephadm: update cephadm compatability and stability page

Reviewed-by: Adam King <adking@redhat.com>
2 years agodoc/cephadm: update cephadm compatability and stability page 50338/head
Adam King [Wed, 1 Mar 2023 21:10:41 +0000 (16:10 -0500)]
doc/cephadm: update cephadm compatability and stability page

This page is very out of date. This commit probably doesn't
cover everything there is to say about stability and compatability
in cephadm, but it at least gets it noticeably closer to reality

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 383f46aa79e7a6f5571405a352ad023983560625)

2 years agoMerge pull request #50316 from zdover23/wip-doc-2023-03-01-radosgw-multisite-diagram...
zdover23 [Wed, 1 Mar 2023 18:04:25 +0000 (04:04 +1000)]
Merge pull request #50316 from zdover23/wip-doc-2023-03-01-radosgw-multisite-diagram-object-replication-text-revision

doc/rgw: improve diagram caption

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rgw: improve diagram caption 50316/head
Zac Dover [Wed, 1 Mar 2023 07:36:18 +0000 (17:36 +1000)]
doc/rgw: improve diagram caption

Improve the syntax and semantics of the caption of the diagram
"Replication of Object Data Between Zones" in doc/radosgw/multisite.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #50256 from yuvalif/wip-yuval-zipper-notifications
Yuval Lifshitz [Wed, 1 Mar 2023 14:13:55 +0000 (16:13 +0200)]
Merge pull request #50256 from yuvalif/wip-yuval-zipper-notifications

rgw/notifications: add bucket notification configuration to zipper

Reviewed-By: dang, cbodley
2 years agoMerge pull request #48181 from huynp1999/wip-rgw-kafka-add-sasl-mechanism
Yuval Lifshitz [Wed, 1 Mar 2023 14:12:47 +0000 (16:12 +0200)]
Merge pull request #48181 from huynp1999/wip-rgw-kafka-add-sasl-mechanism

src/rgw: SASL mechanism implem

Reviewed-By: ivancich, anthonyeleven, yuvalif
2 years agoMerge pull request #49286 from rzarzynski/wip-crimson-snaptrimmer
Radoslaw Zarzynski [Wed, 1 Mar 2023 07:57:27 +0000 (08:57 +0100)]
Merge pull request #49286 from rzarzynski/wip-crimson-snaptrimmer

crimson/osd, osd: bring snap trimming to crimson

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge PR #48720 into main
Venky Shankar [Wed, 1 Mar 2023 02:24:18 +0000 (07:54 +0530)]
Merge PR #48720 into main

* refs/pull/48720/head:
qa: fix api failure issue caused during a dashboard test
PendingReleaseNotes: noted new MDSMap field refuse_client_session
qa: added two testcases
client: do not initiate session if flag refuse_client_session is set
mds: do not reconnect when refuse_client_session is set
mds: add new feature to block clients from establishing sessions

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2 years agoMerge pull request #49006 from yanghonggang/yhg-upgrade-rocksdb-7.7.3
Neha Ojha [Tue, 28 Feb 2023 23:50:21 +0000 (15:50 -0800)]
Merge pull request #49006 from yanghonggang/yhg-upgrade-rocksdb-7.7.3

rocksdb: Upgrade rocksdb to latest facebook's v7.9.2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Mark Nelson <mnelson@redhat.com>
2 years agoqa/valgrind.supp: suppress the ColumnFamily ctor called on init 49006/head
Radoslaw Zarzynski [Tue, 28 Feb 2023 20:58:24 +0000 (20:58 +0000)]
qa/valgrind.supp: suppress the ColumnFamily ctor called on init

```
    <frame>
      <ip>0x1712F1E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>rocksdb::BytewiseComparator()</fn>
      <dir>/home/rzarzynski/ceph3/src/rocksdb/util</dir>
      <file>comparator.cc</file>
      <line>304</line>
    </frame>
    <frame>
      <ip>0x162361C</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>rocksdb::ColumnFamilyOptions::ColumnFamilyOptions()</fn>
      <dir>/home/rzarzynski/ceph3/src/rocksdb/options</dir>
      <file>options.cc</file>
      <line>123</line>
    </frame>
    <frame>
      <ip>0x12CA21E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>Options</fn>
      <dir>/home/rzarzynski/ceph3/build/../src/rocksdb/include/rocksdb</dir>
      <file>options.h</file>
      <line>1409</line>
    </frame>
    <frame>
      <ip>0x12CA21E</ip>
      <obj>/home/rzarzynski/ceph3/build/bin/ceph-mon</obj>
      <fn>RocksDBStore::init(std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;)</fn>
      <dir>/home/rzarzynski/ceph3/build/../src/kv</dir>
      <file>RocksDBStore.cc</file>
      <line>400</line>
    </frame>
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoqa/valgrind.supp: suppress the get-or-create CacheItemHelper
Radoslaw Zarzynski [Tue, 28 Feb 2023 20:42:31 +0000 (20:42 +0000)]
qa/valgrind.supp: suppress the get-or-create CacheItemHelper

```
// Get an CacheItemHelper pointer for value type T and role R.
template <typename T, CacheEntryRole R>
Cache::CacheItemHelper* GetCacheItemHelperForRole() {
  static Cache::CacheItemHelper cache_helper(
      BlocklikeTraits<T>::SizeCallback, BlocklikeTraits<T>::SaveToCallback,
      GetCacheEntryDeleterForRole<T, R>());
  return &cache_helper;
}
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoqa/valgrind.supp: even broader suppression for rocksdb's init
Radoslaw Zarzynski [Tue, 28 Feb 2023 20:35:33 +0000 (20:35 +0000)]
qa/valgrind.supp: even broader suppression for rocksdb's init

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #50308 from adk3798/mock-fqdn-secure-alertmanager
Adam King [Tue, 28 Feb 2023 19:57:44 +0000 (14:57 -0500)]
Merge pull request #50308 from adk3798/mock-fqdn-secure-alertmanager

mgr/cephadm: mock get_fqdn in secure alertmanager config test

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
2 years agoqa/valgrind.supp: ingore rocksdb leak in option parsing
Radoslaw Zarzynski [Tue, 28 Feb 2023 17:52:12 +0000 (17:52 +0000)]
qa/valgrind.supp: ingore rocksdb leak in option parsing

This suppression is needed since upgrading to v7.9.2.
See PR https://github.com/ceph/ceph/pull/49006.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: fix compilation guard around specialization for libfmt => 9 49286/head
Radoslaw Zarzynski [Tue, 17 Jan 2023 19:11:44 +0000 (19:11 +0000)]
crimson/osd: fix compilation guard around specialization for libfmt => 9

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: handle snap trim's interruption as any other error
Radoslaw Zarzynski [Thu, 22 Dec 2022 15:32:49 +0000 (15:32 +0000)]
crimson/osd: handle snap trim's interruption as any other error

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: handle PG_STATE_SNAPTRIM_ERROR when trimming snaps
Radoslaw Zarzynski [Tue, 20 Dec 2022 15:27:14 +0000 (15:27 +0000)]
crimson/osd: handle PG_STATE_SNAPTRIM_ERROR when trimming snaps

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: errorate SnapTrimEvent
Radoslaw Zarzynski [Mon, 19 Dec 2022 21:44:12 +0000 (21:44 +0000)]
crimson/osd: errorate SnapTrimEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: manage PG_STATE_SNAPTRIM when trimming snaps
Radoslaw Zarzynski [Mon, 19 Dec 2022 20:19:52 +0000 (20:19 +0000)]
crimson/osd: manage PG_STATE_SNAPTRIM when trimming snaps

After a few tries:

```
$ bin/ceph -s
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2022-12-19T20:18:21.856+0000 7ff87b46f700 -1 WARNING: all dangerous and experimental features are enabled.
2022-12-19T20:18:21.857+0000 7ff87b46f700  1 build_initial for_mkfs: 0
2022-12-19T20:18:21.860+0000 7ff87b46f700 -1 WARNING: all dangerous and experimental features are enabled.
2022-12-19T20:18:21.861+0000 7ff87b46f700  1 build_initial for_mkfs: 0
  cluster:
    id:     347aedc9-d72f-4a34-98fc-26cb04b0c2fc
    health: HEALTH_WARN
            11 mgr modules have failed dependencies
            1 pool(s) do not have an application enabled
            1 pool(s) have no replicas configured

  services:
    mon: 1 daemons, quorum a (age 3m)
    mgr: x(active, since 3m)
    osd: 1 osds: 1 up (since 3m), 1 in (since 3m)

  data:
    pools:   1 pools, 1 pgs
    objects: 95 objects, 6 B
    usage:   1.0 GiB used, 100 GiB / 101 GiB avail
    pgs:     1 active+clean+snaptrim

  io:
    client:   1.2 KiB/s wr, 0 op/s rd, 1 op/s wr
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop C++'s ranges to workaround unbuildability on Clang
Radoslaw Zarzynski [Wed, 14 Dec 2022 20:18:19 +0000 (20:18 +0000)]
crimson/osd: drop C++'s ranges to workaround unbuildability on Clang

```
/home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/osd/osd_operations/snaptrim_event.cc:38:33: note: while checking constraint satisfaction for template 'operator|<const std::ranges::views::_Elements<0> &, const std::vector<std::pair<unsigned long, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>>>> &>' required here
    for (const auto &i : subops | std::views::keys) {
                                ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/crimson/osd/osd_operations/snaptrim_event.cc:38:33: note: in instantiation of function template specialization 'std::ranges::views::__adaptor::operator|<const std::ranges::views::_Elements<0> &, const std::vector<std::pair<unsigned long, crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, seastar::future<>>>> &>' requested here
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/iterator_concepts.h:963:7: note: candidate template ignored: constraints not satisfied [with _Tp = std::ranges::ref_view<const std
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: simplify SnapTrimObjSubEvent::start()
Radoslaw Zarzynski [Tue, 13 Dec 2022 13:28:43 +0000 (13:28 +0000)]
crimson/osd: simplify SnapTrimObjSubEvent::start()

Throwing away the NOP-looking-like `maybe_delay` concept became
possible after eradication of the `with_interruption` call nested
within `SnapTrimSubEvent`. Before, on Ubuntu and GCC 11.2.1, lack
of it was the reson behind assertion failures on resetting the
`thread_local` intteruption condition.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop the nested interruptors in snap trimming
Radoslaw Zarzynski [Tue, 13 Dec 2022 18:30:28 +0000 (18:30 +0000)]
crimson/osd: drop the nested interruptors in snap trimming

`crimson::interruptible` does not support that. See the
`DISABLED_nested_interruptors` unit test.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/test: verify nesting of interruptors
Radoslaw Zarzynski [Mon, 12 Dec 2022 19:12:11 +0000 (19:12 +0000)]
crimson/test: verify nesting of interruptors

It doesn't work.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: errorate SnapTrimObjSubEvent::remove_clone()
Radoslaw Zarzynski [Mon, 12 Dec 2022 20:56:42 +0000 (20:56 +0000)]
crimson/osd: errorate SnapTrimObjSubEvent::remove_clone()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: convert SnapTrimObjSubEvent::remove_or_update to ertr
Radoslaw Zarzynski [Sun, 11 Dec 2022 16:38:38 +0000 (16:38 +0000)]
crimson/osd: convert SnapTrimObjSubEvent::remove_or_update to ertr

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: dissect snapset update into SnapTrimObjSubEvent::update_head()
Radoslaw Zarzynski [Sat, 10 Dec 2022 22:42:06 +0000 (22:42 +0000)]
crimson/osd: dissect snapset update into SnapTrimObjSubEvent::update_head()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: dissect head whiteout removal into SnapTrimObjSubEvent::remove_head_whit...
Radoslaw Zarzynski [Sat, 10 Dec 2022 22:28:55 +0000 (22:28 +0000)]
crimson/osd: dissect head whiteout removal into SnapTrimObjSubEvent::remove_head_whiteout()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop data aliases from SnapTrimObjSubEvent::adjust_snaps()
Radoslaw Zarzynski [Sat, 10 Dec 2022 22:03:11 +0000 (22:03 +0000)]
crimson/osd: drop data aliases from SnapTrimObjSubEvent::adjust_snaps()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: dissect clone removal into SnapTrimObjSubEvent::remove_clone()
Radoslaw Zarzynski [Sat, 10 Dec 2022 21:54:26 +0000 (21:54 +0000)]
crimson/osd: dissect clone removal into SnapTrimObjSubEvent::remove_clone()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop data aliases from SnapTrimObjSubEvent::remove_or_update()
Radoslaw Zarzynski [Sat, 10 Dec 2022 21:29:26 +0000 (21:29 +0000)]
crimson/osd: drop data aliases from SnapTrimObjSubEvent::remove_or_update()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: simplify informing SnapMapper about new mappings
Radoslaw Zarzynski [Fri, 9 Dec 2022 01:13:30 +0000 (01:13 +0000)]
crimson/osd: simplify informing SnapMapper about new mappings

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: implement basic osd_snap_trim_sleep support
Radoslaw Zarzynski [Wed, 7 Dec 2022 20:41:07 +0000 (20:41 +0000)]
crimson/osd: implement basic osd_snap_trim_sleep support

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: introduce WaitSubop stage to SnapTrimEvent
Radoslaw Zarzynski [Tue, 6 Dec 2022 14:04:17 +0000 (14:04 +0000)]
crimson/osd: introduce WaitSubop stage to SnapTrimEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: snap trimming informs OpsExecuter about altered objects
Radoslaw Zarzynski [Tue, 6 Dec 2022 14:01:44 +0000 (14:01 +0000)]
crimson/osd: snap trimming informs OpsExecuter about altered objects

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocommon: add debugs to crimson's green_condition_variable
Radoslaw Zarzynski [Tue, 6 Dec 2022 10:55:39 +0000 (10:55 +0000)]
common: add debugs to crimson's green_condition_variable

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop unused declaration of PG::fill_op_params_bump_pg_version()
Radoslaw Zarzynski [Mon, 5 Dec 2022 20:17:19 +0000 (20:17 +0000)]
crimson/osd: drop unused declaration of PG::fill_op_params_bump_pg_version()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoosd: SnapMapper::OSDriver uses crimson's interruptor::green_get().
Radoslaw Zarzynski [Mon, 5 Dec 2022 20:00:55 +0000 (20:00 +0000)]
osd: SnapMapper::OSDriver uses crimson's interruptor::green_get().

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/common: bring green_get() to interruptor
Radoslaw Zarzynski [Tue, 6 Dec 2022 12:06:39 +0000 (12:06 +0000)]
crimson/common: bring green_get() to interruptor

The rationale
=============
Calling `get()` on an unavailable future can result in yielding
the reactor, so -- from the interruptor's POV -- the interrupt
condition must be handled exactly like it is done for `yield()`.

Interruptor takes care about that but, at the moment, only for the
interruptible futures; there is mechanism for plain `seastar::future`.
This patch paves the way for handling them as well.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/common: trace management of interrupt_cond in interruptor::async()
Radoslaw Zarzynski [Mon, 5 Dec 2022 19:51:14 +0000 (19:51 +0000)]
crimson/common: trace management of interrupt_cond in interruptor::async()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/common: fix method name in interrupt_cond_t::reset()'s debugs
Radoslaw Zarzynski [Mon, 5 Dec 2022 19:46:36 +0000 (19:46 +0000)]
crimson/common: fix method name in interrupt_cond_t::reset()'s debugs

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoosd: fix intersecion of green threads and interuptible
Radoslaw Zarzynski [Sat, 3 Dec 2022 21:24:56 +0000 (21:24 +0000)]
osd: fix intersecion of green threads and interuptible

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: wire SnapMapper::add_oid() with OpsExecuter
Radoslaw Zarzynski [Fri, 25 Nov 2022 22:48:04 +0000 (22:48 +0000)]
crimson/osd: wire SnapMapper::add_oid() with OpsExecuter

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: settle snap_trimq in PG and wire it up
Radoslaw Zarzynski [Thu, 24 Nov 2022 22:50:17 +0000 (22:50 +0000)]
crimson/osd: settle snap_trimq in PG and wire it up

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: settle SnapMapper within crimson'o PG
Radoslaw Zarzynski [Thu, 24 Nov 2022 16:26:38 +0000 (16:26 +0000)]
crimson/osd: settle SnapMapper within crimson'o PG

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: PerShardState::start_operation() doesn't assume Ret is seastar::future<>
Radoslaw Zarzynski [Wed, 23 Nov 2022 21:30:02 +0000 (21:30 +0000)]
crimson/osd: PerShardState::start_operation() doesn't assume Ret is seastar::future<>

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: implement SnapMapper::OSDriver for crimson
Radoslaw Zarzynski [Tue, 22 Nov 2022 18:21:01 +0000 (18:21 +0000)]
crimson/osd: implement SnapMapper::OSDriver for crimson

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/ertr: let ErrVisitorT return plain value if ValueFuncT returns seastar::future
Radoslaw Zarzynski [Tue, 22 Nov 2022 16:03:42 +0000 (16:03 +0000)]
crimson/ertr: let ErrVisitorT return plain value if ValueFuncT returns seastar::future

The following corner case was unsupported till now:

```cpp
  return os->omap_get_values(
    ch, hoid, keys
  ).safe_then([] (FuturizedStore::omap_values_t&& vals) {
    return 0;
  }, FuturizedStore::read_errorator::all_same_way([] (auto&) {
    return -1;
  })).get();
```
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocommon/ceph_mutex: add green thread-requiring condition variable for crimson
Radoslaw Zarzynski [Mon, 21 Nov 2022 19:06:02 +0000 (19:06 +0000)]
common/ceph_mutex: add green thread-requiring condition variable for crimson

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd, osd: share SnapMapper with crimson
Radoslaw Zarzynski [Fri, 18 Nov 2022 19:15:27 +0000 (19:15 +0000)]
crimson/osd, osd: share SnapMapper with crimson

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoosd: drop the scrub_purged_snaps() declaration from SnapMapper
Radoslaw Zarzynski [Fri, 18 Nov 2022 18:43:24 +0000 (18:43 +0000)]
osd: drop the scrub_purged_snaps() declaration from SnapMapper

Nowadays this is a dead code; the actual definition is in `OSD`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: SnapMapper::record_purged_snaps() takes abstratced store and txn
Radoslaw Zarzynski [Fri, 18 Nov 2022 17:47:16 +0000 (17:47 +0000)]
crimson/osd: SnapMapper::record_purged_snaps() takes abstratced store and txn

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: convert SnapMapper::record_purged_snaps() to OSTransaction
Radoslaw Zarzynski [Fri, 18 Nov 2022 16:12:10 +0000 (16:12 +0000)]
crimson/osd: convert SnapMapper::record_purged_snaps() to OSTransaction

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: initialize OSDriver in SnapMapper::record_purged_snaps()
Radoslaw Zarzynski [Fri, 18 Nov 2022 15:53:38 +0000 (15:53 +0000)]
crimson/osd: initialize OSDriver in SnapMapper::record_purged_snaps()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocommon, osd: switch SnapMapper::_lookup_purged_snap to use OSDriver
Radoslaw Zarzynski [Thu, 17 Nov 2022 21:29:20 +0000 (21:29 +0000)]
common, osd: switch SnapMapper::_lookup_purged_snap to use OSDriver

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: fix debug in Orderer::clear_and_cancel()
Radoslaw Zarzynski [Thu, 17 Nov 2022 15:22:02 +0000 (15:22 +0000)]
crimson/osd: fix debug in Orderer::clear_and_cancel()

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoosd: don't build unnecessary parts of SnapMapper for crimson
Radoslaw Zarzynski [Wed, 16 Nov 2022 16:47:49 +0000 (16:47 +0000)]
osd: don't build unnecessary parts of SnapMapper for crimson

This commits skips building two call-sites of `omap_get_iterator()`
of `ObjectStore` which are bypassing the `OSDriver` abstraction
layer.

1. `SnapMapper::convert_legacy()` likely never be needed by
crimson and we can leave it unchanged.

2. `SnapMapper::Subber` isn't needed right now but will be
needed in the future.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: bring SnapTrimObjSubEvent
Radosław Zarzyński [Tue, 6 Dec 2022 21:20:15 +0000 (22:20 +0100)]
crimson/osd: bring SnapTrimObjSubEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: wire SnapTrimEvent with SnapMapper up
Radosław Zarzyński [Tue, 6 Dec 2022 20:26:42 +0000 (21:26 +0100)]
crimson/osd: wire SnapTrimEvent with SnapMapper up

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: bring SubOpBlocker to SnapTrimEvent
Radoslaw Zarzynski [Wed, 16 Nov 2022 16:40:51 +0000 (16:40 +0000)]
crimson/osd: bring SubOpBlocker to SnapTrimEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: implement the basics of SnapTrimEvent
Radoslaw Zarzynski [Wed, 16 Nov 2022 15:19:41 +0000 (15:19 +0000)]
crimson/osd: implement the basics of SnapTrimEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd/osd_operations: flatten indentation of InternalClientRequest
Radoslaw Zarzynski [Tue, 8 Nov 2022 13:55:37 +0000 (13:55 +0000)]
crimson/osd/osd_operations: flatten indentation of InternalClientRequest

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: drop the residue of CompoundPeeringRequest
Radoslaw Zarzynski [Fri, 4 Nov 2022 13:07:13 +0000 (13:07 +0000)]
crimson/osd: drop the residue of CompoundPeeringRequest

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: simplify ShardServices::snap_request_reservation()
Radoslaw Zarzynski [Wed, 26 Oct 2022 18:55:22 +0000 (18:55 +0000)]
crimson/osd: simplify ShardServices::snap_request_reservation()

`on_preempt` is not used by snap trimming; we can left its default.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: bring the skeleton of SnapTrimEvent
Radoslaw Zarzynski [Wed, 26 Oct 2022 18:46:37 +0000 (18:46 +0000)]
crimson/osd: bring the skeleton of SnapTrimEvent

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: settle snap_reserver within ShardServices
Radoslaw Zarzynski [Wed, 26 Oct 2022 17:45:50 +0000 (17:45 +0000)]
crimson/osd: settle snap_reserver within ShardServices

This bit will be used by the upcoming `SnapTrimRequest`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agocrimson/osd: move the SnapTrimmer-related handlers of PeeringState from .h to .cc
Radoslaw Zarzynski [Wed, 26 Oct 2022 11:50:22 +0000 (11:50 +0000)]
crimson/osd: move the SnapTrimmer-related handlers of PeeringState from .h to .cc

This commit is solely about code shuffling.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agomgr/cephadm: mock get_fqdn in secure alertmanager config test 50308/head
Adam King [Tue, 28 Feb 2023 15:45:03 +0000 (10:45 -0500)]
mgr/cephadm: mock get_fqdn in secure alertmanager config test

Before it was expecting "::1" when passed to get_fqdn to
resolve to "localhost" but in some environments it would instead
resolve to "localhost.localdomain" and the test would fail. This
test isn't testing the functionality of get_fqdn, so let's just
mock it so the test passes regardless of how get_fqdn resolves "::1"

Signed-off-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #49836 from rkachach/fix_issue_58548
Adam King [Tue, 28 Feb 2023 14:31:21 +0000 (09:31 -0500)]
Merge pull request #49836 from rkachach/fix_issue_58548

cephadm: using ip instead of short hostname for prometheus urls

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2 years agoMerge pull request #50083 from adk3798/pids-limit
Adam King [Tue, 28 Feb 2023 14:28:34 +0000 (09:28 -0500)]
Merge pull request #50083 from adk3798/pids-limit

cephadm: set pids-limit unlimited for all ceph daemons

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>