]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
2 years agoqa/lsan.supp: update heap_profiler suppression and ASAN_OPTIONS 51973/head
Matan Breizman [Thu, 8 Jun 2023 14:14:15 +0000 (14:14 +0000)]
qa/lsan.supp: update heap_profiler suppression and ASAN_OPTIONS

In continuation to: 8c099a534044bf7182e04f250e342aab76bc3e54

Regsiter singleton leak seems to be replaced by:
```
-----------------------------------------------------
Suppressions used:
  count      bytes template
      1          8 ^InitModule
-----------------------------------------------------
```

In similarity to Crimson's suppression: 6ed8d839b421442a64410444ca8f88f157ae28b3

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2 years agoMerge pull request #50054 from rhcs-dashboard/cache-invalidation
Nizamudeen A [Wed, 7 Jun 2023 08:29:50 +0000 (13:59 +0530)]
Merge pull request #50054 from rhcs-dashboard/cache-invalidation

mgr/dashboard: RBD cache invalidation

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #51927 from ljflores/wip-rook-tests
Laura Flores [Tue, 6 Jun 2023 18:35:35 +0000 (13:35 -0500)]
Merge pull request #51927 from ljflores/wip-rook-tests

qa/suites/rados: remove rook coverage from the rados suite

2 years agoMerge pull request #51943 from dvanders/update-contact-dan
Anthony D'Atri [Tue, 6 Jun 2023 17:18:04 +0000 (13:18 -0400)]
Merge pull request #51943 from dvanders/update-contact-dan

Update affiliation for Dan van der Ster

2 years agoMerge pull request #51925 from ljflores/wip-tracker-59192
Laura Flores [Tue, 6 Jun 2023 16:14:32 +0000 (11:14 -0500)]
Merge pull request #51925 from ljflores/wip-tracker-59192

qa/suites/rados: whitelist POOL_APP_NOT_ENABLED for cls tests

2 years agoUpdate affiliation for Dan van der Ster 51943/head
Dan van der Ster [Tue, 6 Jun 2023 15:08:44 +0000 (08:08 -0700)]
Update affiliation for Dan van der Ster

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Signed-off-by: Dan van der Ster <dan.vanderster@clyso.com>
2 years agoMerge pull request #51875 from Matan-B/wip-matanb-crimson-supress-tcmalloc
Samuel Just [Tue, 6 Jun 2023 14:48:30 +0000 (07:48 -0700)]
Merge pull request #51875 from Matan-B/wip-matanb-crimson-supress-tcmalloc

crimson/osd/lsan_suppressions.cc: Add MallocExtension::Initialize()

Reviewed-by: Samuel Just <sjust@redhat.com>
2 years agoMerge pull request #51914 from zdover23/wip-doc-2023-06-05-rados-operations-pools... 52484/head 52756/head
zdover23 [Tue, 6 Jun 2023 12:23:12 +0000 (22:23 +1000)]
Merge pull request #51914 from zdover23/wip-doc-2023-06-05-rados-operations-pools-2-of-x

doc/rados: edit pools.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #51932 from stackhpc/typo-fix
zdover23 [Tue, 6 Jun 2023 12:15:45 +0000 (22:15 +1000)]
Merge pull request #51932 from stackhpc/typo-fix

doc/rados/operations: fix typo in balancer.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rados: edit pools.rst (2 of x) 51914/head
Zac Dover [Mon, 5 Jun 2023 02:13:28 +0000 (12:13 +1000)]
doc/rados: edit pools.rst (2 of x)

Edit doc/operations/rados/pools.rst.

There remains confusion in this part of the document regarding pg_num
and pgp_num. pg_num and pgp_num are not explained with sufficient
clarity. A future commit will clear up this confusion. There is also
some potential confusion between on the one hand the strings "pg-num"
and "pgp-num" and on the other hand "pg_num" and "pgp_num". The strings
with the hyphens are used in dummy commands, and the strings with the
underscores are used as key names. I think it possible that this could
confuse a reader, but I am open to discussion on the matter.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agodoc/rados/operations: fix typo in balancer.rst 51932/head
Pierre Riteau [Tue, 6 Jun 2023 06:31:07 +0000 (08:31 +0200)]
doc/rados/operations: fix typo in balancer.rst

Signed-off-by: Pierre Riteau <pierre@stackhpc.com>
2 years agoMerge pull request #50341 from yangdongsheng/ubbd
Ilya Dryomov [Mon, 5 Jun 2023 21:12:28 +0000 (23:12 +0200)]
Merge pull request #50341 from yangdongsheng/ubbd

rbd: add support for new device type of ubbd

Reviewed-by: Mykola Golub <mykola.golub@clyso.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoqa/suites/rados: whitelist POOL_APP_NOT_ENABLED for rados cls tests 51925/head
Laura Flores [Mon, 5 Jun 2023 20:35:54 +0000 (15:35 -0500)]
qa/suites/rados: whitelist POOL_APP_NOT_ENABLED for rados cls tests

Fixes: https://tracker.ceph.com/issues/59192
Signed-off-by: Laura Flores <lflores@redhat.com>
2 years agoqa/suites/rados: remove rook coverage from the rados suite 51927/head
Laura Flores [Mon, 5 Jun 2023 20:23:42 +0000 (15:23 -0500)]
qa/suites/rados: remove rook coverage from the rados suite

The rook team relies on a daily CI system to validate
rook changes. It doesn't seem that the teuthology tests
are maintained, so it makes sense to remove them from the
rados suite.

By removing this symlink, rook test coverage will remain
in the orch suite, and coverage will only be removed from the
rados suite.

Workaround for: https://tracker.ceph.com/issues/58585
Signed-off-by: Laura Flores <lflores@redhat.com>
2 years agoMerge pull request #51294 from cfsnyder/wip-59588
Pere Diaz Bou [Mon, 5 Jun 2023 20:15:35 +0000 (22:15 +0200)]
Merge pull request #51294 from cfsnyder/wip-59588

mgr/dashboard: add 'omit_usage' query param to dashboard api 'get rbd' endpoint

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agoMerge pull request #51729 from NitzanMordhai/wip-nitzan-striper-api-aio-futex-wait
Yuri Weinstein [Mon, 5 Jun 2023 15:35:35 +0000 (11:35 -0400)]
Merge pull request #51729 from NitzanMordhai/wip-nitzan-striper-api-aio-futex-wait

test: futex fail if more notification sent after destroy

Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agoMerge pull request #51706 from Wangwenjuan1105/wwj_clist_lat
Yuri Weinstein [Mon, 5 Jun 2023 15:34:55 +0000 (11:34 -0400)]
Merge pull request #51706 from Wangwenjuan1105/wwj_clist_lat

Bluestore: fix bluestore collection_list latency perf counter

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
2 years agoMerge pull request #51668 from YiteGu/fix-TrackedOp-event-order
Yuri Weinstein [Mon, 5 Jun 2023 15:34:31 +0000 (11:34 -0400)]
Merge pull request #51668 from YiteGu/fix-TrackedOp-event-order

osd/TrackedOp: Fix TrackedOp event order

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years agoMerge pull request #51424 from NitzanMordhai/wip-nitzan-monclient-reconnect-finish...
Yuri Weinstein [Mon, 5 Jun 2023 15:33:45 +0000 (11:33 -0400)]
Merge pull request #51424 from NitzanMordhai/wip-nitzan-monclient-reconnect-finish-auth-with-error

mon/MonClient: before complete auth with error, reopen session

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2 years agoMerge pull request #50651 from rosinL/cleanup
Yuri Weinstein [Mon, 5 Jun 2023 15:32:51 +0000 (11:32 -0400)]
Merge pull request #50651 from rosinL/cleanup

Cleanup the LevelDB residue

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2 years ago[CHERRY-PICKED FOR TESTING ONLY] crimson: Enable tcmalloc when using seastar 51875/head
Mark Nelson [Wed, 27 Apr 2022 15:06:22 +0000 (15:06 +0000)]
[CHERRY-PICKED FOR TESTING ONLY] crimson: Enable tcmalloc when using seastar

classic-osds have always caused significant memory fragmentation
when using the libc memory allocator due to the way that Ceph
tends to utilize memory.  In recent testing, crimson-osd was found
to use 25-27GB of RAM with the stock 3GB bluestore cache settings
(osd_memory_target is only used when tcmalloc is available).  Upon
further testing, it was found that the classic OSD is even worse,
using between 32-33GB of RAM after a 5 minute 4K sequential
write test when using libc malloc.

The good news is that it appears that crimson-osd is able to use
tcmalloc for alienstore without significant modification. Better
still, it drastically reduces memory usage.  In the same test that
resulted in 25GB RSS memory usage for crimson-osd with libc malloc,
a tcmalloc linked version took around 9GB (with an 8GB
osd_memory_target).  Since we do not yet (afaik) expose classic OSD
debugging in crimson it is tough to tell why we are still a little
over, but it's clear that for alienstore we are going to need to
use tcmalloc as we do in classic.

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 9f31ad7b7388996307808e26722260aed29d4ea1)

2 years agotest/crimson: embed lsan_suppressions into tests executables
Matan Breizman [Mon, 5 Jun 2023 11:58:28 +0000 (11:58 +0000)]
test/crimson: embed lsan_suppressions into tests executables

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2 years agoMerge pull request #51915 from zdover23/wip-doc-2023-06-05-rados-operations-pools...
Anthony D'Atri [Mon, 5 Jun 2023 10:53:03 +0000 (06:53 -0400)]
Merge pull request #51915 from zdover23/wip-doc-2023-06-05-rados-operations-pools-typo

doc/rados: pools.rst: "decreaesed"

2 years agomgr/dashboard: invalidate rbd image cache on CRUD ops 50054/head
Pere Diaz Bou [Thu, 9 Feb 2023 13:10:19 +0000 (14:10 +0100)]
mgr/dashboard: invalidate rbd image cache on CRUD ops

Signed-off-by: Pere Diaz Bou <pdiazbou@redhat.com>
Fixes: https://tracker.ceph.com/issues/58710
2 years agoMerge pull request #51746 from rosinL/fix-61227
Matan [Mon, 5 Jun 2023 09:10:50 +0000 (11:10 +0200)]
Merge pull request #51746 from rosinL/fix-61227

crimson/osd/ops_executer: fix double apply_stats() when writing

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
2 years agodoc/rados: pools.rst: "decreaesed" 51915/head
Zac Dover [Mon, 5 Jun 2023 03:26:20 +0000 (13:26 +1000)]
doc/rados: pools.rst: "decreaesed"

s/decreased/decreased/

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #51908 from zdover23/wip-doc-2023-06-04-rados-operations-pools...
zdover23 [Mon, 5 Jun 2023 00:16:12 +0000 (10:16 +1000)]
Merge pull request #51908 from zdover23/wip-doc-2023-06-04-rados-operations-pools-1-of-x

doc/rados: edit pools.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rados: edit pools.rst (1 of x) 51908/head
Zac Dover [Sun, 4 Jun 2023 05:23:39 +0000 (15:23 +1000)]
doc/rados: edit pools.rst (1 of x)

Edit doc/rados/operations/pools.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #51901 from zdover23/wip-doc-2023-06-03-rados-troubleshooting...
zdover23 [Sat, 3 Jun 2023 23:18:27 +0000 (09:18 +1000)]
Merge pull request #51901 from zdover23/wip-doc-2023-06-03-rados-troubleshooting-log-and-debug-2-of-x

doc/rados: edit log-and-debug.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rados: edit log-and-debug.rst (2 of x) 51901/head
Zac Dover [Sat, 3 Jun 2023 01:49:05 +0000 (11:49 +1000)]
doc/rados: edit log-and-debug.rst (2 of x)

Edit roughly the second half of
doc/rados/troubleshooting/log-and-debug.rst.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #51895 from zdover23/wip-doc-2023-06-03-rados-troubleshooting...
Anthony D'Atri [Sat, 3 Jun 2023 22:31:06 +0000 (18:31 -0400)]
Merge pull request #51895 from zdover23/wip-doc-2023-06-03-rados-troubleshooting-troubleshooting-mon-1-of-x

doc/rados: edit troubleshooting-mon.rst (1 of x)

2 years agoMerge pull request #51885 from zdover23/wip-doc-2023-06-02-rados-troubleshooting...
zdover23 [Sat, 3 Jun 2023 22:04:32 +0000 (08:04 +1000)]
Merge pull request #51885 from zdover23/wip-doc-2023-06-02-rados-troubleshooting-log-and-debug-1-of-x

doc/rados: edit log-and-debug.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rados: edit log-and-debug.rst (1 of x) 51885/head
Zac Dover [Fri, 2 Jun 2023 05:43:36 +0000 (15:43 +1000)]
doc/rados: edit log-and-debug.rst (1 of x)

Edit roughly the first half of
doc/rados/troubleshooting/log-and-debug.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agodoc/rados: edit troubleshooting-mon.rst (1 of x) 51895/head
Zac Dover [Sat, 3 Jun 2023 04:03:37 +0000 (14:03 +1000)]
doc/rados: edit troubleshooting-mon.rst (1 of x)

Edit the first 150 lines of
doc/rados/troubleshooting/troubleshooting-mon.rst.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #50576 from rzarzynski/wip-script-unmerged-backport
Radoslaw Zarzynski [Sat, 3 Jun 2023 19:54:59 +0000 (21:54 +0200)]
Merge pull request #50576 from rzarzynski/wip-script-unmerged-backport

script: ceph-backport.sh respects --force when  PR is not merged in main

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
2 years agoMerge pull request #51896 from zdover23/wip-doc-2023-06-03-rados-operations-pg-repair
Anthony D'Atri [Sat, 3 Jun 2023 08:36:41 +0000 (04:36 -0400)]
Merge pull request #51896 from zdover23/wip-doc-2023-06-03-rados-operations-pg-repair

doc/rados: fix typos in pg-repair.rst

2 years agodoc/rados: fix typos in pg-repair.rst 51896/head
Zac Dover [Sat, 3 Jun 2023 05:17:57 +0000 (15:17 +1000)]
doc/rados: fix typos in pg-repair.rst

Fix one RST formatting error and one mistakenly-left-in plural noun in
doc/rados/operations/pg-repair.rst.

https://tracker.ceph.com/issues/58485

Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #51821 from markhpc/wip-bs-rocksdb-cf-tuning
Yuri Weinstein [Fri, 2 Jun 2023 16:37:04 +0000 (12:37 -0400)]
Merge pull request #51821 from markhpc/wip-bs-rocksdb-cf-tuning

common/options: Update RocksDB CF Tuning

Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agoMerge pull request #51872 from zdover23/wip-doc-2023-06-01-cephadm-48180-cleanup...
zdover23 [Fri, 2 Jun 2023 05:51:12 +0000 (13:51 +0800)]
Merge pull request #51872 from zdover23/wip-doc-2023-06-01-cephadm-48180-cleanup-3-of-x

doc: update the cephadm download instructions

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #51851 from Matan-B/wip-matanb-cmake-allocator-status
Samuel Just [Thu, 1 Jun 2023 19:56:18 +0000 (12:56 -0700)]
Merge pull request #51851 from Matan-B/wip-matanb-cmake-allocator-status

CMakeLists.txt: increase verbosity for selected allocator

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>
2 years agocrimson/osd/lsan_suppressions.cc: Add MallocExtension::Initialize()
Matan Breizman [Thu, 1 Jun 2023 15:12:02 +0000 (15:12 +0000)]
crimson/osd/lsan_suppressions.cc: Add MallocExtension::Initialize()

In similarity to lsan.supp:
```
leak:^MallocExtension::Register
```

Shouls suppress this:
```
==2692420==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 45 byte(s) in 1 object(s) allocated from:
    #0 0x5641d4ffaedd in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/unittest-object-data-handler+0x7c01edd) (BuildId: 7aeb00a388469f4de54fedaaceb47fc006ce2906)
..
    #7 0x7fb22a6426f0 in MallocExtension::Initialize() (/lib/x86_64-linux-gnu/libtcmalloc.so.4+0x2a6f0) (BuildId: eeef3d1257388a806e122398dbce3157ee568ef4)
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2 years agoMerge pull request #51176 from linuxbox2/wip-fix-rgwfile-ver
Casey Bodley [Thu, 1 Jun 2023 14:25:32 +0000 (10:25 -0400)]
Merge pull request #51176 from linuxbox2/wip-fix-rgwfile-ver

rgw/file: advance rgw_file interface version

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2 years agodoc: update the cephadm download instructions 51872/head
John Mulligan [Mon, 19 Sep 2022 17:46:48 +0000 (13:46 -0400)]
doc: update the cephadm download instructions

Starting with reef, cephadm is a compiled (zipapp) python application.
The cephadm script has been renamed and thus the old curl-based
download instructions will no loner work. While cephadm still has
no dependencies outside the Python stdlib, this will be changed in
future versions so it is no longer appropriate to just download the
source file of cephadm and run it either.

This change updates the `Install cephadm` section of the doc to explain
how to acquire a "compiled" version of cephadm as well as:
* moving and tweaking the note that the two installation methods are
  distinct
* adding a new note linking to instructions on building cephadm
* moving the distribution-specific installations before the curl-based
  installation to subtly hint that we prefer you to get it using
  packages if you can
* Noting cephadm's minimal required python verision and how to run it
  with a particular python version.

Note from Zac Dover, June 1, 2023: Note: This commit is a cherry-pick of
d11cf0e, which was introduced by John Mulligan in #48180. This is one of
three commits introduced in that PR, and this cherry-pick cleans up
omissions I (Zac Dover) inadvertently introduced while attempting to
rectify the merge conflicts in #51843. This should be the final
main-branch-targeting commit that cleans up PR#51483.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d11cf0e82aab8d4cef9d423e5d463a373eaf383a)

2 years agoMerge pull request #51857 from zdover23/wip-doc-2023-05-31-cephadm-48180-cleanup...
zdover23 [Thu, 1 Jun 2023 11:24:10 +0000 (19:24 +0800)]
Merge pull request #51857 from zdover23/wip-doc-2023-05-31-cephadm-48180-cleanup-1-of-x

doc: make instructions to get an updated cephadm common

Reviewed-by : Adam King <adking@redhat.com>

2 years agoMerge pull request #51844 from rhcs-dashboard/fix-qa-failure-orch
Nizamudeen A [Thu, 1 Jun 2023 06:29:23 +0000 (11:59 +0530)]
Merge pull request #51844 from rhcs-dashboard/fix-qa-failure-orch

mgr/dashboard: fix test_dashboard_e2e.sh failure

Reviewed-by: Pegonzal <NOT@FOUND>
2 years agoMerge pull request #51347 from adk3798/maint-host-no-write
Adam King [Wed, 31 May 2023 23:52:20 +0000 (19:52 -0400)]
Merge pull request #51347 from adk3798/maint-host-no-write

mgr/cephadm: also don't write client files/tuned profiles to maintenance hosts

Reviewed-by: Michael Fritch <mfritch@suse.com>
2 years agoMerge pull request #51701 from rkachach/fix_issue_61361
Adam King [Wed, 31 May 2023 18:25:49 +0000 (14:25 -0400)]
Merge pull request #51701 from rkachach/fix_issue_61361

cephadm: delete /tmp/cephadm-<fsid> when removing the cluster

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #51660 from rkachach/fix_issue_61330
Adam King [Wed, 31 May 2023 18:18:31 +0000 (14:18 -0400)]
Merge pull request #51660 from rkachach/fix_issue_61330

cephadm: Adding support to configure public_network cfg section

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #50805 from rkachach/fix_issue_59254
Adam King [Wed, 31 May 2023 18:06:06 +0000 (14:06 -0400)]
Merge pull request #50805 from rkachach/fix_issue_59254

mgr/cephadm: show meaningful messages when failing to execute cmds

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2 years agoMerge pull request #51816 from AliMasarweh/wip-alimasa-admin-delete-persistent-topic
Yuval Lifshitz [Wed, 31 May 2023 18:01:02 +0000 (21:01 +0300)]
Merge pull request #51816 from AliMasarweh/wip-alimasa-admin-delete-persistent-topic

RGW:notifications: persistent topics are not deleted via radosgw-admin

2 years agoMerge pull request #50614 from phlogistonjohn/jjm-nfs-haproxy-cfg
Adam King [Wed, 31 May 2023 18:00:13 +0000 (14:00 -0400)]
Merge pull request #50614 from phlogistonjohn/jjm-nfs-haproxy-cfg

mgr/cephadm: add support for nfs with haproxy protocol

Reviewed-by: Adam King <adking@redhat.com>
2 years agoMerge pull request #51859 from zdover23/wip-doc-2023-05-31-cephadm-48180-cleanup...
Anthony D'Atri [Wed, 31 May 2023 16:48:27 +0000 (12:48 -0400)]
Merge pull request #51859 from zdover23/wip-doc-2023-05-31-cephadm-48180-cleanup-2-of-x

doc: add instructions for compiling cephadm

2 years agodoc: add instructions for compiling cephadm 51859/head
John Mulligan [Mon, 22 May 2023 18:20:19 +0000 (14:20 -0400)]
doc: add instructions for compiling cephadm

Now that cephadm is based on zipapp, add a short section to the
developer docs explaining how to build cephadm yourself.

Note: This commit is a cherry-pick of
9ad38033cc5c7f177cb8fe3bae696682687e0346, which was introduced by John
Mulligan in #48180. This is one of three commits introduced in that PR,
and this cherry-pick cleans up omissions I (Zac Dover) inadvertently
introduced while attempting to rectify the merge conflicts in #51843. I
expect that one more cherry-picked commit (specifically,
d11cf0e82aab8d4cef9d423e5d463a373eaf383a, which cannot be merged easily
until d7921e88d69b4bc355da9c0327cc33e59e7d7abb has been merged into
main, for reasons that are too
Rick-and-Morty-there-should-never-be-more-than-one-dot to go into here)
will follow this one.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit 9ad38033cc5c7f177cb8fe3bae696682687e0346)

2 years agodoc: make instructions to get an updated cephadm common 51857/head
John Mulligan [Wed, 24 May 2023 17:42:26 +0000 (13:42 -0400)]
doc: make instructions to get an updated cephadm common

As discussed in person and over the ceph orch weekly, we want all users
to use a recent supported version of cephadm. Previously, the
instructions only had those downloading cephadm with curl using the
"add-repo" and "install" commands to get a up-to-date cephadm build.
According to ADK we've seen cases of users get "old" distro packages
in the past. Change the instructions so that the "update cephadm" steps
are common after acquiring a "bootstrap copy" of cephadm.

Note: This commit is a cherry-pick of
d7921e88d69b4bc355da9c0327cc33e59e7d7abb, which was introduced by John
Mulligan in https://github.com/ceph/ceph/pull/48180. This is one of
three commits introduced in that PR, and this cherry-pick cleans up
omissions I (Zac Dover) inadvertently introduced while attempting to
rectify the merge conflicts in https://github.com/ceph/ceph/pull/51843.
I expect that two more cherry-picked commits will follow this one.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d7921e88d69b4bc355da9c0327cc33e59e7d7abb)

2 years agoMerge pull request #51828 from petrutlucian94/it_decrement_issue
Ilya Dryomov [Wed, 31 May 2023 13:07:12 +0000 (15:07 +0200)]
Merge pull request #51828 from petrutlucian94/it_decrement_issue

librbd: avoid decrementing iterator before first element

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #51814 from idryomov/wip-61382
Ilya Dryomov [Wed, 31 May 2023 13:05:16 +0000 (15:05 +0200)]
Merge pull request #51814 from idryomov/wip-61382

osd/OSDCap: allow rbd.metadata_list method under rbd-read-only profile

Reviewed-by: Mykola Golub <mgolub@suse.com>
2 years agoRGW:notifications: persistent topics are not deleted via radosgw-admin 51816/head
Ali Masarwa [Mon, 29 May 2023 14:40:15 +0000 (17:40 +0300)]
RGW:notifications: persistent topics are not deleted via radosgw-admin

Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2 years agoCMakeLists.txt: increase verbosity for selected allocator 51851/head
Matan Breizman [Wed, 31 May 2023 11:06:16 +0000 (11:06 +0000)]
CMakeLists.txt: increase verbosity for selected allocator

Unless the allocator was set on command line, we will select one based on the following order:
```
"specify memory allocator to use. currently tcmalloc, tcmalloc_minimal, \
jemalloc, and libc is supported. if not specified, will try to find tcmalloc, \
and then jemalloc. If neither of then is found. use the one in libc.")
```
with this change, cmake will explicitly message the compiler selected,
otherwise we have no option to identify the one which is being used.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2 years agoMerge pull request #51810 from yuvalif/wip-yuval-fix-61477
Yuval Lifshitz [Wed, 31 May 2023 08:48:42 +0000 (11:48 +0300)]
Merge pull request #51810 from yuvalif/wip-yuval-fix-61477

test/rgw/notifications: fix kafka consumer shutdown issue

2 years agomgr/dashboard: fix test_dashboard_e2e.sh failure 51844/head
Nizamudeen A [Wed, 31 May 2023 05:39:25 +0000 (11:09 +0530)]
mgr/dashboard: fix test_dashboard_e2e.sh failure

The qa e2e is failing because the script is not adapted with cypress 10.

Fixes: https://tracker.ceph.com/issues/61519
Signed-off-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #51843 from zdover23/wip-doc-2023-05-31-compiled-cephadm
colemitchell [Tue, 30 May 2023 23:43:26 +0000 (19:43 -0400)]
Merge pull request #51843 from zdover23/wip-doc-2023-05-31-compiled-cephadm

doc: update the cephadm download instructions

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
2 years agoMerge pull request #51380 from Matan-B/wip-matanb-c-last-sent-epoch
Samuel Just [Tue, 30 May 2023 23:04:43 +0000 (16:04 -0700)]
Merge pull request #51380 from Matan-B/wip-matanb-c-last-sent-epoch

crimson/osd/heartbeat: Improve maybe_share_osdmap behavior

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2 years agodoc: update the cephadm download instructions 51843/head
John Mulligan [Mon, 19 Sep 2022 17:46:48 +0000 (13:46 -0400)]
doc: update the cephadm download instructions

Starting with reef, cephadm is a compiled (zipapp) python application.
The cephadm script has been renamed and thus the old curl-based
download instructions will no loner work. While cephadm still has
no dependencies outside the Python stdlib, this will be changed in
future versions so it is no longer appropriate to just download the
source file of cephadm and run it either.

This change updates the `Install cephadm` section of the doc to explain
how to acquire a "compiled" version of cephadm as well as:
* moving and tweaking the note that the two installation methods are
  distinct
* adding a new note linking to instructions on building cephadm
* moving the distribution-specific installations before the curl-based
  installation to subtly hint that we prefer you to get it using
  packages if you can
* Noting cephadm's minimal required python verision and how to run it
  with a particular python version.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d11cf0e82aab8d4cef9d423e5d463a373eaf383a)

doc: make instructions to get an updated cephadm common

As discussed in person and over the ceph orch weekly, we want all users
to use a recent supported version of cephadm. Previously, the
instructions only had those downloading cephadm with curl using the
"add-repo" and "install" commands to get a up-to-date cephadm build.
According to ADK we've seen cases of users get "old" distro packages
in the past. Change the instructions so that the "update cephadm" steps
are common after acquiring a "bootstrap copy" of cephadm.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
(cherry picked from commit d7921e88d69b4bc355da9c0327cc33e59e7d7abb)

2 years agoMerge pull request #48180 from phlogistonjohn/jjm-cephadm-compile-doc
zdover23 [Tue, 30 May 2023 22:44:53 +0000 (06:44 +0800)]
Merge pull request #48180 from phlogistonjohn/jjm-cephadm-compile-doc

doc: Updates to doc for compiled cephadm

Reviewed-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #51475 from aravind-wdc/wip-crimson-zbd
Samuel Just [Tue, 30 May 2023 20:43:23 +0000 (13:43 -0700)]
Merge pull request #51475 from aravind-wdc/wip-crimson-zbd

crimson/os/seastore: fix ceph_assert in segment_manager.h

Reviewed-by: chunmei-liu <chunmei.liu@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agoMerge pull request #51765 from pdvian/wip-fix-versions
Yuri Weinstein [Tue, 30 May 2023 19:23:40 +0000 (15:23 -0400)]
Merge pull request #51765 from pdvian/wip-fix-versions

mon: Fix ceph versions command

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2 years agoMerge pull request #51838 from cbodley/wip-doc-rgw-reshard-feature-reef
Anthony D'Atri [Tue, 30 May 2023 16:43:38 +0000 (12:43 -0400)]
Merge pull request #51838 from cbodley/wip-doc-rgw-reshard-feature-reef

doc/rgw: fix multisite resharding feature to say Reef instead of Quincy

2 years agoMerge pull request #51818 from idryomov/wip-61495
Ilya Dryomov [Tue, 30 May 2023 16:36:01 +0000 (18:36 +0200)]
Merge pull request #51818 from idryomov/wip-61495

Revert "test: adjust rbd test case guards to handle new defaults"

Reviewed-by: Mykola Golub <mgolub@suse.com>
2 years agodoc: make instructions to get an updated cephadm common 48180/head
John Mulligan [Wed, 24 May 2023 17:42:26 +0000 (13:42 -0400)]
doc: make instructions to get an updated cephadm common

As discussed in person and over the ceph orch weekly, we want all users
to use a recent supported version of cephadm. Previously, the
instructions only had those downloading cephadm with curl using the
"add-repo" and "install" commands to get a up-to-date cephadm build.
According to ADK we've seen cases of users get "old" distro packages
in the past. Change the instructions so that the "update cephadm" steps
are common after acquiring a "bootstrap copy" of cephadm.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agodoc: update the cephadm download instructions
John Mulligan [Mon, 19 Sep 2022 17:46:48 +0000 (13:46 -0400)]
doc: update the cephadm download instructions

Starting with reef, cephadm is a compiled (zipapp) python application.
The cephadm script has been renamed and thus the old curl-based
download instructions will no loner work. While cephadm still has
no dependencies outside the Python stdlib, this will be changed in
future versions so it is no longer appropriate to just download the
source file of cephadm and run it either.

This change updates the `Install cephadm` section of the doc to explain
how to acquire a "compiled" version of cephadm as well as:
* moving and tweaking the note that the two installation methods are
  distinct
* adding a new note linking to instructions on building cephadm
* moving the distribution-specific installations before the curl-based
  installation to subtly hint that we prefer you to get it using
  packages if you can
* Noting cephadm's minimal required python verision and how to run it
  with a particular python version.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agodoc: add instructions for compiling cephadm
John Mulligan [Mon, 22 May 2023 18:20:19 +0000 (14:20 -0400)]
doc: add instructions for compiling cephadm

Now that cephadm is based on zipapp, add a short section to the
developer docs explaining how to build cephadm yourself.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agodoc/rgw: fix multisite resharding feature to say Reef instead of Quincy 51838/head
Casey Bodley [Tue, 30 May 2023 15:30:05 +0000 (11:30 -0400)]
doc/rgw: fix multisite resharding feature to say Reef instead of Quincy

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2 years agomgr/dashboard: add 'omit_usage' query param to dashboard api 'get rbd' endpoint 51294/head
Cory Snyder [Sun, 30 Apr 2023 10:45:41 +0000 (10:45 +0000)]
mgr/dashboard: add 'omit_usage' query param to dashboard api 'get rbd' endpoint

Allows RBD info to be retrieved without getting associated usage info. This
can be useful for large RBDs where the process of gathering such usage info
is sometimes very slow.

Fixes: https://tracker.ceph.com/issues/59588
Signed-off-by: Cory Snyder <csnyder@1111systems.com>
2 years agoMerge pull request #51763 from dang/wip-dang-61251
Daniel Gryniewicz [Tue, 30 May 2023 12:14:48 +0000 (08:14 -0400)]
Merge pull request #51763 from dang/wip-dang-61251

RGW - Use correct multipart upload mtime

2 years agolibrbd: avoid decrementing iterator before first element 51828/head
Lucian Petrut [Thu, 27 Apr 2023 12:44:51 +0000 (12:44 +0000)]
librbd: avoid decrementing iterator before first element

While trying to merge delayed requests, SimpleSchedulerObjectDispatch
can end up iterating before the first element.

With llvm, this leads to a crash:
https://paste.opendev.org/raw/bxnQqqDtIrkOfVvRfkZ6/

This change adds a check, ensuring that we won't decrement the iterator
before the first map element.

Fixes: https://tracker.ceph.com/issues/61503
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
2 years agoMerge pull request #51798 from zdover23/wip-doc-2023-05-29-rados-operations-balancer
zdover23 [Tue, 30 May 2023 07:34:58 +0000 (15:34 +0800)]
Merge pull request #51798 from zdover23/wip-doc-2023-05-29-rados-operations-balancer

doc/rados: edit balancer.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agoMerge pull request #51750 from zhscn/fix-scan-order
Yingxin [Tue, 30 May 2023 01:45:02 +0000 (09:45 +0800)]
Merge pull request #51750 from zhscn/fix-scan-order

crimson/os/seastore/backref_manager: scan backref entries by journal seq

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
2 years agocommon/options: Update RocksDB CF Tuning 51821/head
Mark Nelson [Tue, 30 May 2023 00:20:14 +0000 (00:20 +0000)]
common/options: Update RocksDB CF Tuning

Signed-off-by: Mark Nelson <mark.nelson@clyso.com>
2 years agoRevert "test: adjust rbd test case guards to handle new defaults" 51818/head
Ilya Dryomov [Mon, 29 May 2023 15:40:05 +0000 (17:40 +0200)]
Revert "test: adjust rbd test case guards to handle new defaults"

This reverts commit feb2fc02404775bc262677a2d0434faec0348c53 which
appears to have caused us to lose old format coverage in the Python
bindings tests (rbd_python_api_tests_old_format.yaml).

Unset RBD_FEATURES enviroment variable means "old format".  This
shouldn't be mucked with in any way, see require_new_format() and
create_image() methods in particular.

Fixes: https://tracker.ceph.com/issues/61495
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #51748 from rhcs-dashboard/rgw-roles-fix
Nizamudeen A [Mon, 29 May 2023 14:13:45 +0000 (19:43 +0530)]
Merge pull request #51748 from rhcs-dashboard/rgw-roles-fix

mgr/dashboard: fix the rgw roles page

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
2 years agodoc/rados: edit balancer.rst 51798/head
Zac Dover [Mon, 29 May 2023 01:18:00 +0000 (09:18 +0800)]
doc/rados: edit balancer.rst

Edit doc/rados/operations/balancer.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoosd/OSDCap: allow rbd.metadata_list method under rbd-read-only profile 51814/head
Ilya Dryomov [Sat, 27 May 2023 10:28:40 +0000 (12:28 +0200)]
osd/OSDCap: allow rbd.metadata_list method under rbd-read-only profile

This was missed in commit acc447d5de7b ("osd/OSDCap: rbd profile
permits use of rbd.metadata_list cls method") which adjusted only
"profile rbd" OSD cap.  Listing image metadata is an essential part
of opening the image and "profile rbd-read-only" OSD cap must allow
it too.

While at it, constrain the existing grant for rbd profile from "any
object in the pool" to just "rbd_info object in the global namespace of
the pool" as this is where pool-level image metadata actually lives.

Fixes: https://tracker.ceph.com/issues/61382
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2 years agoMerge pull request #51408 from AliMasarweh/wip-alimasa-topics-op-by-bucket
Ali Masarwa [Mon, 29 May 2023 12:41:04 +0000 (15:41 +0300)]
Merge pull request #51408 from AliMasarweh/wip-alimasa-topics-op-by-bucket

RGW: add admin interfaces to get and delete notifications by bucket

2 years agotest/rgw/notifications: fix kafka consumer shutdown issue 51810/head
Yuval Lifshitz [Mon, 29 May 2023 11:54:00 +0000 (11:54 +0000)]
test/rgw/notifications: fix kafka consumer shutdown issue

Fixes: https://tracker.ceph.com/issues/61477
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
2 years agoMerge pull request #51396 from rhcs-dashboard/access-landing-page-readonly-users
Nizamudeen A [Mon, 29 May 2023 09:56:07 +0000 (15:26 +0530)]
Merge pull request #51396 from rhcs-dashboard/access-landing-page-readonly-users

mgr/dashboard: fix issues with read-only user on landing page

Reviewed-by: Nizamudeen A <nia@redhat.com>
2 years agomgr/dashboard: fix issues with read-only user on landing page 51396/head
Pedro Gonzalez Gomez [Mon, 8 May 2023 20:51:40 +0000 (22:51 +0200)]
mgr/dashboard: fix issues with read-only user on landing page

Fixes: https://tracker.ceph.com/issues/61418
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Signed-off-by: Nizamudeen A <nia@redhat.com>
2 years agoMerge pull request #51662 from yuvalif/wip-yuval-61328
Yuval Lifshitz [Sun, 28 May 2023 15:06:38 +0000 (18:06 +0300)]
Merge pull request #51662 from yuvalif/wip-yuval-61328

rgw/notification/kafka: simplify kafka connection memory management

2 years agoMerge pull request #51575 from yuvalif/wip-yuval-fix-61254
Yuval Lifshitz [Sun, 28 May 2023 15:06:08 +0000 (18:06 +0300)]
Merge pull request #51575 from yuvalif/wip-yuval-fix-61254

rgw/kafka: remove potential race condition between creation and deletion of endpoint

2 years agoRGW: add admin interfaces to get and delete notifications from bucket 51408/head
Ali Masarwa [Mon, 8 May 2023 13:55:01 +0000 (16:55 +0300)]
RGW: add admin interfaces to get and delete notifications from bucket

Signed-off-by: Ali Masarwa <ali.saed.masarwa@gmail.com>
2 years agotest: futex fail if more notification sent after destroy 51729/head
Nitzan Mordechai [Wed, 24 May 2023 12:40:35 +0000 (12:40 +0000)]
test: futex fail if more notification sent after destroy
When testing with more then 1 completion, we may hit an issue
with semaphors been notify after destroy.
we should add wait for each completion and not destroy the sem.
before all notified.

Fixes: https://tracker.ceph.com/issues/57310
Signed-off-by: Nitzan Mordechai <nmordech@redhat.com>
2 years agoMerge pull request #51787 from zdover23/wip-doc-2023-05-27-rados-config-bluestore...
zdover23 [Sat, 27 May 2023 22:53:16 +0000 (08:53 +1000)]
Merge pull request #51787 from zdover23/wip-doc-2023-05-27-rados-config-bluestore-config-ref-2-of-x

doc/rados: edit bluestore-config-ref.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rados: edit bluestore-config-ref.rst (2 of x) 51787/head
Zac Dover [Fri, 26 May 2023 18:44:18 +0000 (04:44 +1000)]
doc/rados: edit bluestore-config-ref.rst (2 of x)

Edit the second part of doc/rados/configuration/bluestore-config-ref.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthoy D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoMerge pull request #51771 from zdover23/wip-doc-2023-05-26-rados-config-bluestore...
zdover23 [Fri, 26 May 2023 21:41:02 +0000 (07:41 +1000)]
Merge pull request #51771 from zdover23/wip-doc-2023-05-26-rados-config-bluestore-config-ref-1-of-x

doc/rados: edit bluestore-config-ref.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2 years agodoc/rados: edit bluestore-config-ref.rst (1 of x) 51771/head
Zac Dover [Fri, 26 May 2023 08:59:36 +0000 (18:59 +1000)]
doc/rados: edit bluestore-config-ref.rst (1 of x)

Edit the first part of doc/rados/configuration/bluestore-config-ref.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>
2 years agoRGW - Use correct multipart upload mtime 51763/head
Daniel Gryniewicz [Thu, 25 May 2023 18:09:54 +0000 (14:09 -0400)]
RGW - Use correct multipart upload mtime

During multipart listing, the mtime of the uploads were not being
loaded, resulting in the current time being returned.  Fix this by
setting the correct mtime.

Fixes: https://tracker.ceph.com/issues/61251
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
2 years agodoc/cephadm: document haproxy protocol options 50614/head
John Mulligan [Tue, 21 Mar 2023 20:47:35 +0000 (16:47 -0400)]
doc/cephadm: document haproxy protocol options

There are now two ways to run NFS with ingress using haxproxy. This
makes three distinct modes for ingress. Document these.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agoqa/cephadm: teuthology test for nfs ingress-mode=haproxy-protocol
John Mulligan [Tue, 21 Mar 2023 20:20:42 +0000 (16:20 -0400)]
qa/cephadm: teuthology test for nfs ingress-mode=haproxy-protocol

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agomgr/cephadm: add test explicitly validating proxy protocol
John Mulligan [Tue, 21 Mar 2023 19:40:17 +0000 (15:40 -0400)]
mgr/cephadm: add test explicitly validating proxy protocol

Adds a test case that specifically ensures that the proper values
appear in the configuration files for both nfs and haproxy when
the proxy protocol is enabled.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agomgr/cephadm: parametrize test for haproxy protocol support
John Mulligan [Tue, 21 Mar 2023 17:51:48 +0000 (13:51 -0400)]
mgr/cephadm: parametrize test for haproxy protocol support

Make a few small tweaks to an existing test to validate that it
functions similarly when the haxproxy protocol is disabled or enabled.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2 years agomgr/cephadm: minor reorganizing of nfs service test
John Mulligan [Tue, 21 Mar 2023 15:42:25 +0000 (11:42 -0400)]
mgr/cephadm: minor reorganizing of nfs service test

This change makes some formatting and organizational changes to one of
the nfs + ingress test cases. This is in preparation for a future change
that will parametrize this function allowing it to test haproxy config
when proxy protocol is enabled and when not.

Signed-off-by: John Mulligan <jmulligan@redhat.com>