]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agodoc/cephadm: remove step to restart the mgr 39932/head
Michael Fritch [Wed, 10 Mar 2021 04:06:27 +0000 (21:06 -0700)]
doc/cephadm: remove step to restart the mgr

a restart of the mgr does not appear to be necessary
after a `ceph config-key set ...`

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agodoc/cephadm: use `reconfig` instead of `redeploy`
Michael Fritch [Wed, 10 Mar 2021 04:06:20 +0000 (21:06 -0700)]
doc/cephadm: use `reconfig` instead of `redeploy`

`reconfig` can be used to apply a change to either
the tls/ssl cert or a custom configuration file (j2)

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agodoc/cephadm: update custom j2 config-key name
Michael Fritch [Wed, 10 Mar 2021 04:06:12 +0000 (21:06 -0700)]
doc/cephadm: update custom j2 config-key name

introduced by:
cd79c9912ab35ee6296d613edc7830410a141e05

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #39981 from xxhdx1985126/wip-obc-tri_mutex-bug
Kefu Chai [Wed, 10 Mar 2021 03:43:36 +0000 (11:43 +0800)]
Merge pull request #39981 from xxhdx1985126/wip-obc-tri_mutex-bug

crimson/osd: fix two obc lock related bugs

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/common: use excl_lock_from_excl when promoting tri_mutex from excl to excl 39981/head
Xuehan Xu [Tue, 9 Mar 2021 03:06:26 +0000 (11:06 +0800)]
crimson/common: use excl_lock_from_excl when promoting tri_mutex from excl to excl

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/osd: acquire locks strictly by OSDOp type
Xuehan Xu [Mon, 8 Mar 2021 07:51:49 +0000 (15:51 +0800)]
crimson/osd: acquire locks strictly by OSDOp type

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agoMerge pull request #39826 from tchaikov/wip-ninja
Kefu Chai [Wed, 10 Mar 2021 00:30:13 +0000 (08:30 +0800)]
Merge pull request #39826 from tchaikov/wip-ninja

run-make: switch over to ninja-build

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #39946 from TheJJ/fix-ceph-crash-caps
zdover23 [Tue, 9 Mar 2021 23:38:37 +0000 (09:38 +1000)]
Merge pull request #39946 from TheJJ/fix-ceph-crash-caps

doc/mgr/crash: explain needed crash upload permissions

Reviewed-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge PR #39955 into master
Sage Weil [Tue, 9 Mar 2021 22:32:59 +0000 (17:32 -0500)]
Merge PR #39955 into master

* refs/pull/39955/head:
mgr/cephadm: do not prime service cache on reconfig

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoMerge PR #37711 into master
Patrick Donnelly [Tue, 9 Mar 2021 21:41:53 +0000 (13:41 -0800)]
Merge PR #37711 into master

* refs/pull/37711/head:
qa/cephfs: create class LocalCephFSMount
vstart_runner: reuse FuseMount methods in LocalFuseMount
vstart_runner: reuse KernelMount methods in LocalKernelMount
qa/cephfs: move common and generic methods mount.py
qa/cephfs: refactor FuseMount.mount method
qa/cephfs: refactor KernelMount.mount method

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #37945 into master
Patrick Donnelly [Tue, 9 Mar 2021 21:41:00 +0000 (13:41 -0800)]
Merge PR #37945 into master

* refs/pull/37945/head:
mds: add opened files/inodes and pinned i_caps metric payload support

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #39408 into master
Patrick Donnelly [Tue, 9 Mar 2021 21:40:09 +0000 (13:40 -0800)]
Merge PR #39408 into master

* refs/pull/39408/head:
test: add test for validating cephfs-mirror daemon service status
pybind/mirroring: interface to fetch mirror daemon status
cephfs-mirror: register mirror daemon with service manager
cephfs-mirror: remove unneeded json_spirit header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #39502 into master
Patrick Donnelly [Tue, 9 Mar 2021 21:38:57 +0000 (13:38 -0800)]
Merge PR #39502 into master

* refs/pull/39502/head:
qa: add sleep for blocklisting to take effect

Reviewed-by: Rishabh Dave <ridave@redhat.com>
4 years agoMerge PR #39687 into master
Patrick Donnelly [Tue, 9 Mar 2021 21:37:35 +0000 (13:37 -0800)]
Merge PR #39687 into master

* refs/pull/39687/head:
tools/cephfs-shell: continue file listing even on error
tools/cephfs-shell: refactoring of code related listing
tools/cephfs-shell: fix listing of symbolic links

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #39814 into master
Patrick Donnelly [Tue, 9 Mar 2021 21:36:40 +0000 (13:36 -0800)]
Merge PR #39814 into master

* refs/pull/39814/head:
qa/cephfs: move is_blocklisted() to filesystem.CephCluster

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agomgr/cephadm: do not prime service cache on reconfig 39955/head
Sage Weil [Tue, 9 Mar 2021 18:15:20 +0000 (12:15 -0600)]
mgr/cephadm: do not prime service cache on reconfig

Ceph daemon reconfig does not need any daemon state refresh since we don't
do a restart--we just rewrite the ceph.conf.  This also avoids priming
our cache with a 'starting' state when the daemon wasn't touched.

Fixes: https://tracker.ceph.com/issues/49675
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #39948 from tchaikov/wip-clang-tidy
Kefu Chai [Tue, 9 Mar 2021 17:25:14 +0000 (01:25 +0800)]
Merge pull request #39948 from tchaikov/wip-clang-tidy

crimson,common: cleanups inspired by clang-tidy

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson: pass by `const` or `const&` if possible 39948/head
Kefu Chai [Tue, 9 Mar 2021 14:30:56 +0000 (22:30 +0800)]
crimson: pass by `const` or `const&` if possible

more constness, and better performance.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39930 from zdover23/wip-doc-cephadm-convert-adoption-process...
zdover23 [Tue, 9 Mar 2021 14:43:07 +0000 (00:43 +1000)]
Merge pull request #39930 from zdover23/wip-doc-cephadm-convert-adoption-process-2021-Mar-09

doc/cephadm: rewrite "adoption process"

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agocrimson/mon: comment unused parameter
Kefu Chai [Tue, 9 Mar 2021 14:30:06 +0000 (22:30 +0800)]
crimson/mon: comment unused parameter

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson: do not rely on implicit conversion to bool
Kefu Chai [Tue, 9 Mar 2021 14:28:09 +0000 (22:28 +0800)]
crimson: do not rely on implicit conversion to bool

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson: use `auto*` instead of `auto` for pointer
Kefu Chai [Tue, 9 Mar 2021 14:26:23 +0000 (22:26 +0800)]
crimson: use `auto*` instead of `auto` for pointer

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: use moved string for constructing vector<>
Kefu Chai [Tue, 9 Mar 2021 14:25:23 +0000 (22:25 +0800)]
crimson/admin: use moved string for constructing vector<>

it is the last consumer of cmdline, so move it away.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: avoid using C-style cast
Kefu Chai [Tue, 9 Mar 2021 14:23:59 +0000 (22:23 +0800)]
crimson/admin: avoid using C-style cast

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/str_list: remove unused "using"
Kefu Chai [Tue, 9 Mar 2021 14:22:57 +0000 (22:22 +0800)]
common/str_list: remove unused "using"

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/cephadm: rewrite "adoption process" 39930/head
Zac Dover [Mon, 8 Mar 2021 22:06:05 +0000 (08:06 +1000)]
doc/cephadm: rewrite "adoption process"

This rewrites the "Adoption Process" section
of the "Converting and Existing Cluster to
cephadm" chapter of the Cephadm guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agodoc/mgr/crash: explain needed crash upload permissions 39946/head
Jonas Jelten [Tue, 9 Mar 2021 13:10:52 +0000 (14:10 +0100)]
doc/mgr/crash: explain needed crash upload permissions

Signed-off-by: Jonas Jelten <jj@sft.lol>
4 years agocmake: silence "You are in 'detached HEAD' state" warning 39826/head
Kefu Chai [Thu, 4 Mar 2021 10:25:33 +0000 (18:25 +0800)]
cmake: silence "You are in 'detached HEAD' state" warning

git warns us when the repo is in 'detached HEAD' state, like:

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in
this state without impacting any branches by switching back to a branch.
...
Turn off this advice by setting config variable advice.detachedHead to
false

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agonasm-wrapper: use += for appending string to a variable
Kefu Chai [Thu, 4 Mar 2021 09:17:15 +0000 (17:17 +0800)]
nasm-wrapper: use += for appending string to a variable

less repeating, better readablity

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: do not pass "-fdiagnostics-color" to as
Kefu Chai [Thu, 4 Mar 2021 08:46:05 +0000 (16:46 +0800)]
cmake: do not pass "-fdiagnostics-color" to as

this option can be understood by GCC. see
https://gcc.gnu.org/onlinedocs/gcc/Diagnostic-Message-Formatting-Options.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agonasm-wrapper: swallow -MD option
Kefu Chai [Thu, 4 Mar 2021 08:06:51 +0000 (16:06 +0800)]
nasm-wrapper: swallow -MD option

see also

- https://discourse.cmake.org/t/cmake-generating-invalid-ninja-build-rules-for-asm/1389
- https://gitlab.kitware.com/cmake/cmake/-/merge_requests/4460
- https://gitlab.kitware.com/cmake/cmake/-/merge_requests/4838

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: define BUILD_BYPRODUCTS for Boost
Kefu Chai [Thu, 4 Mar 2021 07:46:42 +0000 (15:46 +0800)]
cmake: define BUILD_BYPRODUCTS for Boost

so if a cmake generator needs any one of them, it knowns how to build
it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: do not specify CMAKE_GENERATOR for external projects
Kefu Chai [Thu, 4 Mar 2021 08:20:16 +0000 (16:20 +0800)]
cmake: do not specify CMAKE_GENERATOR for external projects

as per CMake document,

> Without this option, the same generator as the main build will be
> used.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: use "" for no-op INSTALL_COMMAND
Kefu Chai [Thu, 4 Mar 2021 07:16:37 +0000 (15:16 +0800)]
cmake: use "" for no-op INSTALL_COMMAND

as suggested by CMake document, see
https://cmake.org/cmake/help/latest/module/ExternalProject.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: define BUILD_BYPRODUCTS for spdk-ext
Kefu Chai [Thu, 4 Mar 2021 07:04:13 +0000 (15:04 +0800)]
cmake: define BUILD_BYPRODUCTS for spdk-ext

so if a cmake generator needs any one of them, it knowns how to build
it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodo_cmake: always pass -DWITH_PYTHON3 to cmake
Kefu Chai [Thu, 4 Mar 2021 06:30:17 +0000 (14:30 +0800)]
do_cmake: always pass -DWITH_PYTHON3 to cmake

do not pretend that we support python2 anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodo_cmake,run-make: switch to ninja
Kefu Chai [Thu, 4 Mar 2021 06:28:00 +0000 (14:28 +0800)]
do_cmake,run-make: switch to ninja

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoceph.spec: prepare for out-of-source build
Kefu Chai [Thu, 4 Mar 2021 04:56:11 +0000 (12:56 +0800)]
ceph.spec: prepare for out-of-source build

see also
https://fedoraproject.org/wiki/Changes/CMake_to_do_out-of-source_builds

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39226 from rhcs-dashboard/host-maintenance-feature
Ernesto Puerta [Tue, 9 Mar 2021 09:09:14 +0000 (10:09 +0100)]
Merge pull request #39226 from rhcs-dashboard/host-maintenance-feature

mgr/dashboard: Host Maintenance Feature

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #39909 from smithfarm/wip-revert-make-build-on-suse
Kefu Chai [Tue, 9 Mar 2021 08:58:58 +0000 (16:58 +0800)]
Merge pull request #39909 from smithfarm/wip-revert-make-build-on-suse

rpm: refrain from using %make_build macro on SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39936 from tchaikov/wip-workflow-use-known-sha1
Kefu Chai [Tue, 9 Mar 2021 06:09:45 +0000 (14:09 +0800)]
Merge pull request #39936 from tchaikov/wip-workflow-use-known-sha1

.github/workflows: use a released sha1

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years ago.github/workflows: use a released sha1 39936/head
Kefu Chai [Tue, 9 Mar 2021 04:16:26 +0000 (12:16 +0800)]
.github/workflows: use a released sha1

otherwise the action cannot be found

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39892 from tchaikov/wip-github-workflow-milestone
Kefu Chai [Mon, 8 Mar 2021 19:42:32 +0000 (03:42 +0800)]
Merge pull request #39892 from tchaikov/wip-github-workflow-milestone

.github: correct the regex in milestone workflow

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years ago.github/workflows: rename labeler.yml to pr-triage.yml 39892/head
Kefu Chai [Mon, 8 Mar 2021 18:46:46 +0000 (02:46 +0800)]
.github/workflows: rename labeler.yml to pr-triage.yml

to reflect the change in the actions in it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years ago.github/workflows: use @{sha1} for actions
Kefu Chai [Mon, 8 Mar 2021 18:26:34 +0000 (02:26 +0800)]
.github/workflows: use @{sha1} for actions

more secure this way. see also https://julienrenaux.fr/2019/12/20/github-actions-security-risk/

point the sha1 to

* labeler@v3
* milestone@main HEAD

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years ago.github/workflows: move milestone.yml into labeler.yml
Kefu Chai [Sun, 7 Mar 2021 11:21:25 +0000 (19:21 +0800)]
.github/workflows: move milestone.yml into labeler.yml

no need to have two actions triggered by the same event.

we could backport this labeler.yml to LTS branches.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39246 from rhcs-dashboard/fix-45415-master
Ernesto Puerta [Mon, 8 Mar 2021 16:17:49 +0000 (17:17 +0100)]
Merge pull request #39246 from rhcs-dashboard/fix-45415-master

mgr/dashboard: test prometheus rules through promtool

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: p-se <NOT@FOUND>
4 years agomgr/dashboard: Host Maintenance Feature 39226/head
Nizamudeen A [Tue, 2 Feb 2021 12:26:13 +0000 (17:56 +0530)]
mgr/dashboard: Host Maintenance Feature

In Cluster -> Hosts, I've added additional button to put the selected host on maintenance or exit out of the maintenance mode. Also for some hosts the ok-to-stop tests may trigger some warnings which requires a --force command to pass along with the maintenance enter command to enter a host into maintenance. In UI this is achieved using a confirmation Modal. In addition to this if the check error is It is NOT safe to stop the host then the host wont be able to put into maintenance mode.

Fixes: https://tracker.ceph.com/issues/49101
Signed-off-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge pull request #38605 from mdw-at-linuxbox/wip-master-transit
Casey Bodley [Mon, 8 Mar 2021 16:00:01 +0000 (11:00 -0500)]
Merge pull request #38605 from mdw-at-linuxbox/wip-master-transit

rgw: new vault transit logic for SSE-C

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37536 from xxhdx1985126/wip-crimson-interruptible-future-final
Kefu Chai [Mon, 8 Mar 2021 15:07:55 +0000 (23:07 +0800)]
Merge pull request #37536 from xxhdx1985126/wip-crimson-interruptible-future-final

crimson: add continuation chain interruption capabilities

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39876 from dang/wip-dang-tempest
Daniel Gryniewicz [Mon, 8 Mar 2021 13:36:09 +0000 (08:36 -0500)]
Merge pull request #39876 from dang/wip-dang-tempest

RGW - Fix TempURL user handling

4 years agoMerge pull request #39476 from rhcs-dashboard/fix-host-label
Ernesto Puerta [Mon, 8 Mar 2021 13:29:12 +0000 (14:29 +0100)]
Merge pull request #39476 from rhcs-dashboard/fix-host-label

mgr/dashboard: fix PUT - /api/host/{hostname} while adding labels

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #39825 from mgfritch/cephadm-rgw-nfs-health
Sebastian Wagner [Mon, 8 Mar 2021 12:18:26 +0000 (13:18 +0100)]
Merge pull request #39825 from mgfritch/cephadm-rgw-nfs-health

mgr/cephadm: alias rgw-nfs -> nfs

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #38908 from smithfarm/wip-script-agpl3
Kefu Chai [Mon, 8 Mar 2021 12:02:03 +0000 (20:02 +0800)]
Merge pull request #38908 from smithfarm/wip-script-agpl3

COPYING: add entries for backport scripts

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #39593 from mychoxin/fix_promote_dead_loop
Kefu Chai [Mon, 8 Mar 2021 11:44:04 +0000 (19:44 +0800)]
Merge pull request #39593 from mychoxin/fix_promote_dead_loop

osd: avoid two copy with same src cancel each other

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agorpm: refrain from using %make_build macro on SUSE 39909/head
Nathan Cutler [Mon, 8 Mar 2021 10:14:56 +0000 (11:14 +0100)]
rpm: refrain from using %make_build macro on SUSE

The recently merged commit 0e511973f7b6b0e5dbb3c4caf98f1290f81829e9 replaced

    make %{_smp_mflags}

with

    %make_build

for the stated purpose of hiding the %_smp_mflags macro in a higher-level macro.

But, on SUSE, the higher-level macro (%make_build) expands to:

    make -O %{_smp_mflags}

The addition of the -O flag makes the build considerably slower and increases
the memory requirement. The exact reason for this is unknown - possibly it's due
to a bug in make, although the same slowness was observed with ninja as well.
In any event, this is a deal-breaker when building in the OBS, because the build
infrastructure there is optimized for builds that do not require huge amounts of
memory and we would rather have a fast build with mixed up compiler messages
than a very slow one with synced compiler messages.

Fixes: 0e511973f7b6b0e5dbb3c4caf98f1290f81829e9
Signed-off-by: Nathan Cutler <ncutler@suse.com>
4 years agoMerge pull request #39821 from dillaman/wip-qa-objectstore-logs
Kefu Chai [Mon, 8 Mar 2021 10:43:59 +0000 (18:43 +0800)]
Merge pull request #39821 from dillaman/wip-qa-objectstore-logs

qa: reduce debug log levels for bluestore

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39900 from tchaikov/wip-crimson-watch
Kefu Chai [Mon, 8 Mar 2021 08:23:20 +0000 (16:23 +0800)]
Merge pull request #39900 from tchaikov/wip-crimson-watch

crimson/osd: cleanup Notify::send_completion()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agomgr/dashboard:test prometheus rules through promtool 39246/head
Aashish Sharma [Wed, 3 Feb 2021 07:23:56 +0000 (12:53 +0530)]
mgr/dashboard:test prometheus rules through promtool

This PR intends to add unit testing for prometheus rules using promtool. To run the tests run 'run-promtool-unittests.sh' file.

Fixes: https://tracker.ceph.com/issues/45415
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
4 years agoMerge pull request #39729 from aclamk/wip-rocksdb-aware-bluefs-buffered-io
Kefu Chai [Sun, 7 Mar 2021 16:07:42 +0000 (00:07 +0800)]
Merge pull request #39729 from aclamk/wip-rocksdb-aware-bluefs-buffered-io

Allows RocksDB to distinguish between buffered/direct_io files

Reviewed-by: Mark Nelson <mnelson@redhat.com>
4 years agoMerge pull request #38997 from gerald-yang/patch-48946
Kefu Chai [Sun, 7 Mar 2021 15:54:06 +0000 (23:54 +0800)]
Merge pull request #38997 from gerald-yang/patch-48946

common: Fix assertion when disabling and re-enabling clog_to_monitors

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39775 from rishabh-d-dave/fs-qa-accept-cmds-as-str
Kefu Chai [Sun, 7 Mar 2021 15:52:54 +0000 (23:52 +0800)]
Merge pull request #39775 from rishabh-d-dave/fs-qa-accept-cmds-as-str

qa/ceph_manager: accepts commands as str too

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #39690 from rishabh-d-dave/qa-raw_cluster_cmd
Kefu Chai [Sun, 7 Mar 2021 15:51:05 +0000 (23:51 +0800)]
Merge pull request #39690 from rishabh-d-dave/qa-raw_cluster_cmd

qa/ceph_manger: fixes bugs in CephManager

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #39712 from tchaikov/wip-prometheus-ann
Kefu Chai [Sun, 7 Mar 2021 15:50:02 +0000 (23:50 +0800)]
Merge pull request #39712 from tchaikov/wip-prometheus-ann

mgr/prometheus: add type annotations

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
4 years agoMerge pull request #39740 from tchaikov/wip-pybind-rtype
Kefu Chai [Sun, 7 Mar 2021 15:48:06 +0000 (23:48 +0800)]
Merge pull request #39740 from tchaikov/wip-pybind-rtype

pybind/{rados,cephfs}: use type annotation for rtype docstring

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agocrimson/osd: add constness to Notify 39900/head
Kefu Chai [Sun, 7 Mar 2021 13:51:53 +0000 (21:51 +0800)]
crimson/osd: add constness to Notify

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: cleanup Notify::send_completion()
Kefu Chai [Sun, 7 Mar 2021 13:32:28 +0000 (21:32 +0800)]
crimson/osd: cleanup Notify::send_completion()

Notify::send_completion() should just send the completion message. so,
move irrelevant code out of Notify::send_completion().

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: differentiate Notify with and without watchers
Kefu Chai [Sun, 7 Mar 2021 13:23:54 +0000 (21:23 +0800)]
crimson/osd: differentiate Notify with and without watchers

we don't need to arm the timer, if there is no watchers.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: factor out "maybe" from maybe_send_completion()
Kefu Chai [Sun, 7 Mar 2021 13:04:52 +0000 (21:04 +0800)]
crimson/osd: factor out "maybe" from maybe_send_completion()

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39896 from bolapara/patch-1
Kefu Chai [Sun, 7 Mar 2021 12:23:21 +0000 (20:23 +0800)]
Merge pull request #39896 from bolapara/patch-1

docs: Ubuntu uses 'apt' not 'dnf'

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39895 from tchaikov/wip-crimson-watch
Kefu Chai [Sun, 7 Mar 2021 11:18:05 +0000 (19:18 +0800)]
Merge pull request #39895 from tchaikov/wip-crimson-watch

crimson/osd: watch-notify related cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agodoc/cephadm: use 'apt' to install cephadm on Ubuntu 39896/head
Josh [Sun, 7 Mar 2021 03:59:46 +0000 (21:59 -0600)]
doc/cephadm: use 'apt' to install cephadm on Ubuntu

Adjusted so Ubuntu command uses 'apt' and added Fedora since that uses 'dnf'.

4 years agocrimson/common: deal with seastar::thread in interruptible_future 37536/head
Xuehan Xu [Tue, 10 Nov 2020 09:14:44 +0000 (17:14 +0800)]
crimson/common: deal with seastar::thread in interruptible_future

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/test: add interruptible future unit test
Xuehan Xu [Tue, 20 Oct 2020 10:19:11 +0000 (18:19 +0800)]
crimson/test: add interruptible future unit test

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson/osd: use interruptible future in the PG layer
Xuehan Xu [Sun, 4 Oct 2020 09:51:59 +0000 (17:51 +0800)]
crimson/osd: use interruptible future in the PG layer

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agocrimson: add interruptible future facilities
Xuehan Xu [Sun, 4 Oct 2020 09:50:48 +0000 (17:50 +0800)]
crimson: add interruptible future facilities

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: no need to clear watchers beforehand 39895/head
Kefu Chai [Sun, 7 Mar 2021 02:55:35 +0000 (10:55 +0800)]
crimson/osd: no need to clear watchers beforehand

as long as we don't yield, we can assume the atomicity.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: use transparent comparator
Kefu Chai [Sun, 7 Mar 2021 02:47:20 +0000 (10:47 +0800)]
crimson/osd: use transparent comparator

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39879 from rzarzynski/wip-crimson-notify-timeout
Kefu Chai [Sun, 7 Mar 2021 01:44:12 +0000 (09:44 +0800)]
Merge pull request #39879 from rzarzynski/wip-crimson-notify-timeout

crimson/osd: implement timeout for notify propagation.

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years ago.github: correct the regex in mileston workflow
Kefu Chai [Sat, 6 Mar 2021 16:32:42 +0000 (00:32 +0800)]
.github: correct the regex in mileston workflow

also use pull_request_target event so the action is run in the
context of the base of the pull request. this helps us to overcome
the "Resource not accessible by integration" issue where the action
is run in the context of the pull request.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon: provide description for osd_default_notify_timeout. 39879/head
Radoslaw Zarzynski [Fri, 5 Mar 2021 21:43:07 +0000 (21:43 +0000)]
common: provide description for osd_default_notify_timeout.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: unify initialization in Notify.
Radoslaw Zarzynski [Fri, 5 Mar 2021 21:37:05 +0000 (21:37 +0000)]
crimson/osd: unify initialization in Notify.

Let's be consistent with the `Notify::timeout_timer`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: implement timeout for notify propagation.
Radoslaw Zarzynski [Fri, 5 Mar 2021 18:49:05 +0000 (18:49 +0000)]
crimson/osd: implement timeout for notify propagation.

This missed feature was the root cause of the following
failure at Sepia:

```
2021-03-04T15:40:03.013 INFO:tasks.workunit.client.0.smithi058.stdout:         api_watch_notify: [ RUN      ] LibRadosWatchNotify.WatchNotify2Timeout
2021-03-04T15:40:03.013 INFO:tasks.workunit.client.0.smithi058.stdout:         api_watch_notify: watch_notify2_test_cb from 4394 notify_id 120259084288 cookie 94023196911472
2021-03-04T15:40:03.013 INFO:tasks.workunit.client.0.smithi058.stdout:         api_watch_notify: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-1471-g8bd81c29/rpm/el8/BUILD/ceph-17.0.0-1471-g8bd81c29/src/test/librados/watch_notify.cc:425: Failure
2021-03-04T15:40:03.014 INFO:tasks.workunit.client.0.smithi058.stdout:         api_watch_notify: Expected equality of these values:
2021-03-04T15:40:03.015 INFO:tasks.workunit.client.0.smithi058.stdout:         api_watch_notify:   -110
2021-03-04T15:40:03.015 INFO:tasks.workunit.client.0.smithi058.stdout:         api_watch_notify:   rados_notify2(ioctx, notify_oid, "notify", 6, 1000, &reply_buf, &reply_buf_len)
2021-03-04T15:40:03.015 INFO:tasks.workunit.client.0.smithi058.stdout:         api_watch_notify:     Which is: 0
2021-03-04T15:40:03.016 INFO:tasks.workunit.client.0.smithi058.stdout:         api_watch_notify: [  FAILED  ] LibRadosWatchNotify.WatchNotify2Timeout (3020 ms)
```

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #39886 from tchaikov/wip-github-workflow-milestone
Kefu Chai [Sat, 6 Mar 2021 14:38:46 +0000 (22:38 +0800)]
Merge pull request #39886 from tchaikov/wip-github-workflow-milestone

.github: add workflow for adding milestone

Reviewed-by: Sage Weil <sage@redhat.com>
4 years ago.github: add workflow for adding milestone 39886/head
Kefu Chai [Sat, 6 Mar 2021 13:18:13 +0000 (21:18 +0800)]
.github: add workflow for adding milestone

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39846 from tchaikov/wip-cmake-find-pmem
Kefu Chai [Sat, 6 Mar 2021 09:03:39 +0000 (17:03 +0800)]
Merge pull request #39846 from tchaikov/wip-cmake-find-pmem

cmake: support COMPONENTS param in Findpmem.cmake

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agocmake: make the linkage to pmem::pmemobj public 39846/head
Kefu Chai [Sat, 6 Mar 2021 07:43:33 +0000 (15:43 +0800)]
cmake: make the linkage to pmem::pmemobj public

tools/ceph-dencoder/rbd_types.cc includes Types.h which in turn includes
libpmemobj.h via librbd/cache/pwl/Types.h. and ceph-dencoder pulls in the
rbd_type.cc's linked libraries by linking against rbd_types. but before
this change, rbd_types links against pmem::pmemobj as a PRIVATE library.

so, if we want to pull in rbd_types linkage we should always link
rbd_types as a PUBLIC library. as rbd_types include libpmemobj.h in its
header file.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: link libpmemobj against libpmem
Kefu Chai [Sat, 6 Mar 2021 04:22:39 +0000 (12:22 +0800)]
cmake: link libpmemobj against libpmem

libpmemobj should link against libpmem, but, in CMake, imported library
does not allow PRIVATE linkage. so pmem::pmem is added to the list of
INTERFACE_LINK_LIBRARIES.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: support COMPONENTS param in Findpmem.cmake
Kefu Chai [Fri, 5 Mar 2021 06:04:23 +0000 (14:04 +0800)]
cmake: support COMPONENTS param in Findpmem.cmake

add two components: pmem and pmemobj to this package. so we can find
them and link against them in a more intuitive way.

before this change the COMPONENTS parameter passed to

find_package(pmem ...)

is dropped on the floor and ignored.

after this change, it is checked and taken into consideration.

also, in this change, the exposed variables are renamed from

PMEM_* to pmem_*

to be consistent with the package name. it's encouraged to be consistent
with the package name when it comes to the INCLUDE_DIR and LIBRARIES
variable names.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agorgw/kms/vault - PendingReleaseNotes pointer 38605/head
Marcus Watts [Tue, 2 Mar 2021 04:10:35 +0000 (23:10 -0500)]
rgw/kms/vault - PendingReleaseNotes pointer

The "new" transit logic requires configure changes to be
effective.  Here is a pointer to the revised
documentation for those who already have data encrypted
using the previous version.

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
4 years agorgw/kms/vault - s3tests for both old and new test logic.
Marcus Watts [Tue, 26 Jan 2021 01:49:16 +0000 (20:49 -0500)]
rgw/kms/vault - s3tests for both old and new test logic.

Test both "old" and "new" transit logic with s3tests.  Does not test
migration - that will need to be done separately.  I've added
a "flavor" parameter so the test logic can tell the difference
between the "old" engine and the "new" engine.  The vault
keys creation logic now has options to determine whether
the keys created are exportable (needed for the old transit
engine), or not (should be the case going forward with the
new transit engine.)

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
4 years agorgw/kms/vault - rework unit test logic for new transit logic.
Marcus Watts [Fri, 8 Jan 2021 22:49:20 +0000 (17:49 -0500)]
rgw/kms/vault - rework unit test logic for new transit logic.

The "new" transit logic is organized quite differently
than the old logic, so the existing unit test logic was
very broken.  Additionally, it's possible to test the
input arguments and send_request() has more of them now,
so add logic to verify most of those arguments are correct.

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
4 years agorgw/kms/vault - 0 terminate before rapidjson
Marcus Watts [Fri, 8 Jan 2021 22:43:58 +0000 (17:43 -0500)]
rgw/kms/vault - 0 terminate before rapidjson

bufferlist c_str() doesn't guarantee a trailing nul, which is req'd
by rapidjson.  So, append a nul to make that guarantee explicit.

Also, add an optional "virtual" so unit test logic can override
send_request().

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
4 years agorgw/kms/vault - document configuration for new transit logic
Marcus Watts [Tue, 5 Jan 2021 02:22:07 +0000 (21:22 -0500)]
rgw/kms/vault - document configuration for new transit logic

Using the new transit logic requires slightly different configuration.
additionally there is some backwards compatibility support, which
also needed documentation.

The existing description of how to configure hashicorp vault
to work with ceph was also incomplete.  I've fleshed that out a bit,
including considerably more information on how to use configure
and use the vault secret agent with ceph.

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
4 years agorgw/kms/vault - new transit logic - fix compat logic
Marcus Watts [Tue, 5 Jan 2021 02:21:33 +0000 (21:21 -0500)]
rgw/kms/vault - new transit logic - fix compat logic

Teuthology passes in a vault uri that ends in /export/encryption-key/
So: we need to handle (and ignore) trailing slashes when deciding
to enable compatibility support.

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
4 years agorgw/kms/vault - define attribute for new transit logic
Marcus Watts [Sat, 12 Dec 2020 19:09:44 +0000 (14:09 -0500)]
rgw/kms/vault - define attribute for new transit logic

For the new transit logic; I need to store a wrapped key
per object.  This names that attribute on the rados object.

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
4 years agorgw/kms/vault - "compat" option
Marcus Watts [Tue, 8 Dec 2020 23:09:04 +0000 (18:09 -0500)]
rgw/kms/vault - "compat" option

For transit engine:
"compat" option: 0=new only, 1=old & new, 2=old only.
This is just the option parsing itself: not the actual logic
for make_key | reconstitute_key.

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
4 years agorgw/kms/vault - encryption context - first part
Marcus Watts [Mon, 7 Dec 2020 22:55:22 +0000 (17:55 -0500)]
rgw/kms/vault - encryption context - first part

This includes the logic to process the user provided
encryption context, turn it into "canonical json", and
to add in a default arn if it's not present.

Also present here is the start of logic to distinguish
between "prepare_encrypt" and "prepare_decrypt" at a lower
level; as "make_key" and "reconstitute_key" these will be
the functions that separately create a new datakey using
the vault transit operation, and to retrieve a previously
stored datakey.

Fixes: http://tracker.ceph.com/issues/48746
Signed-off-by: Marcus Watts <mwatts@redhat.com>
4 years agoqa/cephfs: move is_blocklisted() to filesystem.CephCluster 39814/head
Rishabh Dave [Wed, 3 Mar 2021 11:44:22 +0000 (17:14 +0530)]
qa/cephfs: move is_blocklisted() to filesystem.CephCluster

Using self.fs.mon_manager in mount.py can lead to a crash since self.fs
can be None. Move is_blocklisted() to tasks.filesystem.CephCluster where
it can get access to mon_manager without depending on objects
representing Ceph FSs.

Fixes: https://tracker.ceph.com/issues/49511
Signed-off-by: Rishabh Dave <ridave@redhat.com>
4 years agocmake: do not link blk against pmem::pmemobj
Kefu Chai [Fri, 5 Mar 2021 13:15:59 +0000 (21:15 +0800)]
cmake: do not link blk against pmem::pmemobj

as blk does not use pmem::pmemobj, librbd does.

Signed-off-by: Kefu Chai <kchai@redhat.com>