]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agomgr/cephadm/upgrade: do not repeat crash message 40242/head
Sage Weil [Fri, 19 Mar 2021 14:46:09 +0000 (10:46 -0400)]
mgr/cephadm/upgrade: do not repeat crash message

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/upgrade: a little less verbose
Sage Weil [Fri, 19 Mar 2021 14:44:19 +0000 (10:44 -0400)]
mgr/cephadm/upgrade: a little less verbose

The _do_upgrade() method runs a zillion times; try to report fewer
repetitive messages on every iteration.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: don't log not-ok-to-stop at ERR level
Sage Weil [Fri, 19 Mar 2021 14:38:06 +0000 (10:38 -0400)]
mgr/cephadm: don't log not-ok-to-stop at ERR level

This is normal during the upgrade; INF is fine.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: is presumed -> appears
Sage Weil [Fri, 19 Mar 2021 14:37:37 +0000 (10:37 -0400)]
mgr/cephadm: is presumed -> appears

The old wording was weird.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: don't double-log ok-to-stop results
Sage Weil [Fri, 19 Mar 2021 14:37:16 +0000 (10:37 -0400)]
mgr/cephadm: don't double-log ok-to-stop results

The calling upgrade code also reports this.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/upgrade: include upgrade progress in ceph -s
Sage Weil [Fri, 19 Mar 2021 14:31:24 +0000 (10:31 -0400)]
mgr/cephadm/upgrade: include upgrade progress in ceph -s

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #39915 from CongMinYin/fix-vm-io-hang
Jason Dillaman [Wed, 17 Mar 2021 19:35:58 +0000 (15:35 -0400)]
Merge pull request #39915 from CongMinYin/fix-vm-io-hang

librbd/cache/pwl: set max size of continuous data

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
4 years agoMerge pull request #40175 from idryomov/wip-krbd-blkroset-5.12
Ilya Dryomov [Wed, 17 Mar 2021 18:44:14 +0000 (19:44 +0100)]
Merge pull request #40175 from idryomov/wip-krbd-blkroset-5.12

qa: krbd_blkroset.t: update for separate hw and user read-only flags

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #40051 from strenuous-life/wip-cephadm-osd-exact-match-size
Sebastian Wagner [Wed, 17 Mar 2021 17:22:05 +0000 (18:22 +0100)]
Merge pull request #40051 from strenuous-life/wip-cephadm-osd-exact-match-size

mgr/cephadm: When device size contains the decimal, it can not match size exactly

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #39880 from adk3798/upgrade-status
Sebastian Wagner [Wed, 17 Mar 2021 17:08:23 +0000 (18:08 +0100)]
Merge pull request #39880 from adk3798/upgrade-status

mgr/cephadm: add info to 'ceph orch upgrade status' in cephadm

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge PR #40177 into master
Patrick Donnelly [Wed, 17 Mar 2021 16:00:56 +0000 (09:00 -0700)]
Merge PR #40177 into master

* refs/pull/40177/head:
doc: update Windows MSI link

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #40178 from idryomov/wip-rbd-map-sanity-check-fixup
Ilya Dryomov [Wed, 17 Mar 2021 15:43:26 +0000 (16:43 +0100)]
Merge pull request #40178 from idryomov/wip-rbd-map-sanity-check-fixup

krbd: check device node accessibility only if we actually mapped

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #39678 from yehudasa/wip-rgw-aws4-client
Casey Bodley [Wed, 17 Mar 2021 15:42:08 +0000 (11:42 -0400)]
Merge pull request #39678 from yehudasa/wip-rgw-aws4-client

rgw: awsv4 client support

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #36549 from markhpc/wip-bl-dynamic_alen
Kefu Chai [Wed, 17 Mar 2021 14:43:07 +0000 (22:43 +0800)]
Merge pull request #36549 from markhpc/wip-bl-dynamic_alen

common/buffer.cc: Implement dynamic alen in refill_append_space

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #39757 from aclamk/wip-qa-test-bluestore-reshard
Kefu Chai [Wed, 17 Mar 2021 14:41:34 +0000 (22:41 +0800)]
Merge pull request #39757 from aclamk/wip-qa-test-bluestore-reshard

qa: Add bluestore resharding test

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #40032 from aclamk/wip-bdev-remove-reap
Kefu Chai [Wed, 17 Mar 2021 14:40:21 +0000 (22:40 +0800)]
Merge pull request #40032 from aclamk/wip-bdev-remove-reap

blk/BlockDevice: Remove reap_ioc logic

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
4 years agoMerge pull request #40118 from jtlayton/wip-49806
Sebastian Wagner [Wed, 17 Mar 2021 14:19:33 +0000 (15:19 +0100)]
Merge pull request #40118 from jtlayton/wip-49806

doc: fixes for cephadm documentation

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #40179 from tchaikov/wip-cmake-permissive-for-cxx
Kefu Chai [Wed, 17 Mar 2021 13:49:12 +0000 (21:49 +0800)]
Merge pull request #40179 from tchaikov/wip-cmake-permissive-for-cxx

cmake: do not pass -fpermissive when compiling C code

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #40136 from kshtsk/wip-fix-ninja-leap
Kefu Chai [Wed, 17 Mar 2021 13:23:45 +0000 (21:23 +0800)]
Merge pull request #40136 from kshtsk/wip-fix-ninja-leap

ceph.spec.in: use ninja instead of ninja-build for openSUSE

Reviewed-by: Nathan Cutler <ncutler@suse.com>
4 years agodoc: fixes for cephadm documentation 40118/head
Jeff Layton [Fri, 29 Jan 2021 19:15:26 +0000 (14:15 -0500)]
doc: fixes for cephadm documentation

Be sure to note that python 3 is a prerequisite. Minimal centos 8
installs don't have it, for instance.

Also, we probably don't want to hardcode an octopus URL into the
suggested curl command. Change it to fill that in with
"|stable-release|", which should always point to the latest released
version name.

Fixes: https://tracker.ceph.com/issues/49806
Signed-off-by: Jeff Layton <jlayton@redhat.com>
4 years agoMerge pull request #39290 from lxbsz/service
Jason Dillaman [Wed, 17 Mar 2021 12:16:49 +0000 (08:16 -0400)]
Merge pull request #39290 from lxbsz/service

mgr: enhance the rados service map

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: do not pass -fpermissive when compiling C code 40179/head
Kefu Chai [Wed, 17 Mar 2021 12:03:45 +0000 (20:03 +0800)]
cmake: do not pass -fpermissive when compiling C code

silences warnings like

[2/768] /usr/bin/x86_64-w64-mingw32-gcc-posix -DBOOST_ASIO_DISABLE_THREAD_KEYWORD_EXTENSION -DHAVE_CONFIG_H -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -D_POSIX=1 -D_POSIX_=1 -D_POSIX_C_SOURCE=1
-D_POSIX_THREADS=1 -D_REENTRANT -D_THREAD_SAFE -D_WIN32_WINNT=0x0A00 -D__CEPH__ -D__STDC_FORMAT_MACROS -Isrc/include -I../src -I../src/include/win32 -isystem ../build.deps/mingw/boost/include -isystem
include -isystem ../src/xxHash -isystem ../src/rapidjson/include -isystem ../src/fmt/include -O3 -DNDEBUG   -include winsock_wrapper.h -include win32_errno.h -U_FORTIFY_SOURCE -Wall
-fno-strict-aliasing -fsigned-char -Wtype-limits -Wignored-qualifiers -Wpointer-arith -Werror=format-security -Winit-self -Wno-unknown-pragmas -fpermissive -fdiagnostics-color=auto -std=gnu99 -MD -MT
src/CMakeFiles/common-objs.dir/xxHash/xxhash.c.obj -MF src/CMakeFiles/common-objs.dir/xxHash/xxhash.c.obj.d -o src/CMakeFiles/common-objs.dir/xxHash/xxhash.c.obj   -c ../src/xxHash/xxhash.c
cc1: warning: command line option '-fpermissive' is valid for C++/ObjC++ but not for C

see also
https://gcc.gnu.org/onlinedocs/gcc-4.0.4/gcc/C_002b_002b-Dialect-Options.html

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agokrbd: check device node accessibility only if we actually mapped 40178/head
Ilya Dryomov [Mon, 15 Mar 2021 19:30:07 +0000 (20:30 +0100)]
krbd: check device node accessibility only if we actually mapped

Fix a braino that came with commit f6854ac65d2a ("krbd: make sure the
device node is accessible after the mapping").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agodoc: update Windows MSI link 40177/head
Lucian Petrut [Wed, 17 Mar 2021 10:38:55 +0000 (10:38 +0000)]
doc: update Windows MSI link

Instead of having a direct download link, we'll point to the
download page, which will eventually contain other MSI versions as
well (e.g. Quincy).

While at it, we're simplifying the document a bit, dropping
information that's also included in the manual install guide.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
4 years agoMerge pull request #40169 from petrutlucian94/test_fixes
Ilya Dryomov [Wed, 17 Mar 2021 10:16:24 +0000 (11:16 +0100)]
Merge pull request #40169 from petrutlucian94/test_fixes

win32*.sh: use the right target when building the tests

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
4 years agoqa: krbd_blkroset.t: update for separate hw and user read-only flags 40175/head
Ilya Dryomov [Wed, 17 Mar 2021 10:00:33 +0000 (11:00 +0100)]
qa: krbd_blkroset.t: update for separate hw and user read-only flags

Since kernel 5.12, hardware read-only state and user read-only
policy (BLKROGET/SET ioctls) are tracked separately in the block
layer.  As the purpose of our ->set_read_only() method was exactly
that, it was removed.

As a side effect, BLKROSET no longer returns EROFS on an attempt
to make a read-only mapping read-write with "blockdev --setrw".
The policy gets updated, but the device remains read-only as before
because the hardware (== mapping) state is controlled by the driver.

Fixes: https://tracker.ceph.com/issues/49858
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agowin32*.sh: use the right target when building the tests 40169/head
Lucian Petrut [Wed, 17 Mar 2021 07:01:15 +0000 (07:01 +0000)]
win32*.sh: use the right target when building the tests

We switched from make to ninja but we're using the wrong target
when building the tests.

"ninja test" tries to actually run the tests. We'll have to use
"ninja tests" when targeting Windows.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
4 years agoMerge pull request #40022 from athanatos/sjust/wip-warnings
Kefu Chai [Wed, 17 Mar 2021 04:03:24 +0000 (12:03 +0800)]
Merge pull request #40022 from athanatos/sjust/wip-warnings

test/librgw_file_xattr.cc: fix unused warning

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #40070 into master
Patrick Donnelly [Wed, 17 Mar 2021 03:27:15 +0000 (20:27 -0700)]
Merge PR #40070 into master

* refs/pull/40070/head:
lockdep: fix follows/follows_bt resize() size
lockdep: switch follows_bt vector member to std::map

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
4 years agoMerge PR #40038 into master
Patrick Donnelly [Wed, 17 Mar 2021 03:25:41 +0000 (20:25 -0700)]
Merge PR #40038 into master

* refs/pull/40038/head:
test: log the RLIMIT_NOFILE that we set when running this test

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #40028 into master
Patrick Donnelly [Wed, 17 Mar 2021 03:24:35 +0000 (20:24 -0700)]
Merge PR #40028 into master

* refs/pull/40028/head:
client: hold the cct by increasing the cct nref
client: notify and stop the tick thread when destructing the Client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #39848 into master
Patrick Donnelly [Wed, 17 Mar 2021 03:23:13 +0000 (20:23 -0700)]
Merge PR #39848 into master

* refs/pull/39848/head:
mds: fix race of fetching large dirfrag

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
4 years agoMerge PR #39847 into master
Patrick Donnelly [Wed, 17 Mar 2021 03:21:52 +0000 (20:21 -0700)]
Merge PR #39847 into master

* refs/pull/39847/head:
qa/cephfs: don't expect "file exists" error for "mkdir -p"
qa/cephfs: don't take parameter cwd in _create_mntpt()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
4 years agoMerge pull request #40162 from liewegas/bug-49846
Kefu Chai [Wed, 17 Mar 2021 02:52:41 +0000 (10:52 +0800)]
Merge pull request #40162 from liewegas/bug-49846

mgr/prometheus: fix typo in get_collect_time_metrics

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #40152 from TheJJ/fix-osd-class-path-docs
Kefu Chai [Wed, 17 Mar 2021 02:46:19 +0000 (10:46 +0800)]
Merge pull request #40152 from TheJJ/fix-osd-class-path-docs

doc: update osd-class-path error messages

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #40164 from racpatel/DocRBDfix
Jason Dillaman [Wed, 17 Mar 2021 00:21:37 +0000 (20:21 -0400)]
Merge pull request #40164 from racpatel/DocRBDfix

doc/RBD:fixes for ceph-immutable-object-cache daemon enable command

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agodoc/RBD:fixes for ceph-immutable-object-cache daemon enable command 40164/head
Rachanaben Patel [Tue, 16 Mar 2021 22:37:46 +0000 (15:37 -0700)]
doc/RBD:fixes for ceph-immutable-object-cache daemon enable command

Document for rbd-persistent-read-only-cache show how to manage
ceph-immutable-object-cache daemon using systemd.
command example needs fixing.It should be

systemctl enable ceph-immutable-object-cache@ceph-immutable-object-cache.{unique id}

Fixes: https://tracker.ceph.com/issues/49849
Signed-off-by: Rachanaben Patel <racpatel@redhat.com>
4 years agomgr/prometheus: fix typo in get_collect_time_metrics 40162/head
Sage Weil [Tue, 16 Mar 2021 20:10:42 +0000 (15:10 -0500)]
mgr/prometheus: fix typo in get_collect_time_metrics

This causes a failure the first time through this function, but
subsequent calls succeed, making it a bit hard to notice.

Fixes: 58fd057e2c8799fa000b9937aa992e13cbbd485f
Fixes: https://tracker.ceph.com/issues/49846
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge PR #40114 into master
Patrick Donnelly [Tue, 16 Mar 2021 19:27:43 +0000 (12:27 -0700)]
Merge PR #40114 into master

* refs/pull/40114/head:
doc: reorganize Windows docs

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge PR #40075 into master
Sage Weil [Tue, 16 Mar 2021 18:07:51 +0000 (14:07 -0400)]
Merge PR #40075 into master

* refs/pull/40075/head:
qa/suites/rados/cephadm/upgrade: add centos upgrade on latest octopus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge PR #39931 into master
Sage Weil [Tue, 16 Mar 2021 17:04:55 +0000 (13:04 -0400)]
Merge PR #39931 into master

* refs/pull/39931/head:
mgr/cephadm: fall back to service spec port if none on DaemonDescription
mgr/cephadm: fix redeploy when daemons have ip:port
mgr/cephadm/schedule: add test case
qa/suites/rados/cephadm/smoke-roleless: add rgw test on many ports
doc/cephadm/rgw: update docs to show count-per-host
mgr/cephadm: add support for rgw_frontend_type (beast or civetweb)
mgr/cephadm: remove ssl_frontend_ssl_key from RGWSpec
mgr/cephadm: fix beast private key config option
mgr/cephadm: fix rgw ssl cert/key config-key path
mgr/cephadm/schedule: dynamically assign ports for rgw
mgr/cephadm/schedule: only 1 port in DaemonPlacement
mgr/cephadm: move rgw frontend logic into RgwService
mgr/cephadm/schedule: return DaemonPlacement instead of HostPlacementSpec
mgr/cephadm/schedule: remove unused methods
mgr/cephadm: propagate ip:port from CephadmDaemoNDeploySpec to deployment
cephadm: populate ports if known and not included in unit.meta
mgr/cephadm: gather and report ports in 'orch ps' output

Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoMerge pull request #40120 from zdover23/wip-doc-cephadm-mon-service-1-16-Mar-2021
zdover23 [Tue, 16 Mar 2021 16:44:33 +0000 (02:44 +1000)]
Merge pull request #40120 from zdover23/wip-doc-cephadm-mon-service-1-16-Mar-2021

doc/cephadm: break mon section into sections

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agodoc/cephadm: break mon section into sections 40120/head
Zac Dover [Mon, 15 Mar 2021 15:03:06 +0000 (01:03 +1000)]
doc/cephadm: break mon section into sections

This PR breaks the "Deploy Additional Monitors" section
of the cephadm documentation into several subsections
whose titles spotlight the matter under discussion in
those respective subsections.

inb4: Another PR is on deck that rewrites the sentences
in this chapter of the cephadm documentation. I'd like
to get this chapter broken up into these subsections before
I rewrite those sentences. So I'm hoping for no grammatical
mission creep on this one. The grammar and clarity updates
are coming.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agodoc: update osd-class-path error messages 40152/head
Jonas Jelten [Tue, 16 Mar 2021 10:45:50 +0000 (11:45 +0100)]
doc: update osd-class-path error messages

the return value is "95 Operation not supported"

Signed-off-by: Jonas Jelten <jj@sft.lol>
4 years agoMerge pull request #40141 from zdover23/wip-doc-cephadm-mon-deploying-add-rewrite...
zdover23 [Tue, 16 Mar 2021 15:10:01 +0000 (01:10 +1000)]
Merge pull request #40141 from zdover23/wip-doc-cephadm-mon-deploying-add-rewrite-2021-Mar-16

doc/cephadm: rewrite "deploying add. mons"

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agoMerge pull request #40153 from rzarzynski/wip-osd-obclock-cleanup
Kefu Chai [Tue, 16 Mar 2021 14:04:32 +0000 (22:04 +0800)]
Merge pull request #40153 from rzarzynski/wip-osd-obclock-cleanup

osd: clean obc locking and repop creation as a sidequest of crimson's InternalClientRequest

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39996 from tchaikov/wip-crimson-cleanups
Kefu Chai [Tue, 16 Mar 2021 14:01:58 +0000 (22:01 +0800)]
Merge pull request #39996 from tchaikov/wip-crimson-cleanups

crimson: cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoosd: move PrimaryLogPG::get_rw_locks() from header to .cc. 40153/head
Radoslaw Zarzynski [Tue, 16 Mar 2021 12:45:06 +0000 (12:45 +0000)]
osd: move PrimaryLogPG::get_rw_locks() from header to .cc.

For the sake of symmetry with `release_object_locks()`
and easier grepping for e.g. `ctx->lock_manager`.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoosd: PrimaryLogPG::new_repop() doesn't depend on obc anymore.
Radoslaw Zarzynski [Mon, 15 Mar 2021 19:14:44 +0000 (19:14 +0000)]
osd: PrimaryLogPG::new_repop() doesn't depend on obc anymore.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge PR #40130 into master
Sage Weil [Tue, 16 Mar 2021 12:43:06 +0000 (08:43 -0400)]
Merge PR #40130 into master

* refs/pull/40130/head:
mgr/cephadm: less noise about refreshing hosts

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agoMerge PR #40140 into master
Sage Weil [Tue, 16 Mar 2021 12:42:46 +0000 (08:42 -0400)]
Merge PR #40140 into master

* refs/pull/40140/head:
mgr/cephadm: fix 'auth caps' fallback

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agoceph.spec.in: use ninja instead of ninja-build for openSUSE 40136/head
Kyr Shatskyy [Mon, 15 Mar 2021 21:08:27 +0000 (22:08 +0100)]
ceph.spec.in: use ninja instead of ninja-build for openSUSE

And since the dependency is now distro-conditional, moving it down
to the 'distro-conditional make check dependencies' section.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
4 years agocrimson/os: do not include unused headers 39996/head
Kefu Chai [Wed, 10 Mar 2021 09:13:22 +0000 (17:13 +0800)]
crimson/os: do not include unused headers

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agotest/crimson: use GTEST_FAIL() for printing failure message
Kefu Chai [Wed, 10 Mar 2021 09:07:25 +0000 (17:07 +0800)]
test/crimson: use GTEST_FAIL() for printing failure message

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: drop duplicated __attribute__
Kefu Chai [Wed, 10 Mar 2021 09:07:02 +0000 (17:07 +0800)]
crimson/os: drop duplicated __attribute__

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agomgr: enhance the rados service 39290/head
Xiubo Li [Thu, 4 Feb 2021 06:14:13 +0000 (14:14 +0800)]
mgr: enhance the rados service

For some use cases, like the tcmu-runner, there maybe handreds or
thousands of LUNs, and then for each LUN it will register one service
daemon, then in the `ceph -s` output will be full of useless info.

This will allow to classify the sevices service daemons in one
specified format by adding two pairs in metadata:

  "daemon_type"   : "${TYPE}"
  "daemon_prefix" : "${PREFIX}"

TYPE: will be used to replace the default "daemon(s)"
showed in `ceph -s`. If absent, the "daemon" will be used.
PREFIX: if present the active members will be classified
by the prefix instead of "daemon_name".

For exmaple for iscsi gateways, it will be something likes:
  "daemon_type"   : "portal"
  "daemon_prefix" : "gw${N}"

Then the `ceph -s` output will be:

  ...
  services:
    mon:   3 daemons, quorum a,b,c (age 50m)
    mgr:   x(active, since 49m)
    mds:   a:1 {0=c=up:active} 2 up:standby
    osd:   3 osds: 3 up (since 49m), 3 in (since 49m)
    iscsi: 8 portals active (gw0, gw1, gw2, gw3, gw4, gw5, gw6, gw7)
  ...

Fixes: https://tracker.ceph.com/issues/49057
Signed-off-by: Xiubo Li <xiubli@redhat.com>
4 years agodoc/cephadm: rewrite "deploying add. mons" 40141/head
Zac Dover [Mon, 15 Mar 2021 15:03:06 +0000 (01:03 +1000)]
doc/cephadm: rewrite "deploying add. mons"

This rewrites the section "Deploying Additional
Monitors (Beyond the Default Three)" for elegance
and clarity.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agomgr/cephadm: fall back to service spec port if none on DaemonDescription 39931/head
Sage Weil [Mon, 15 Mar 2021 22:58:15 +0000 (18:58 -0400)]
mgr/cephadm: fall back to service spec port if none on DaemonDescription

For an RGW instance that is deployed in an older version, we won't have
IP or port metadata in the DaemonDescription.  In that case, fall back
to the port in the ServiceSpec.  This should be safe since any such
instance will only have 1 daemon per host.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: fix redeploy when daemons have ip:port
Sage Weil [Mon, 15 Mar 2021 19:34:13 +0000 (15:34 -0400)]
mgr/cephadm: fix redeploy when daemons have ip:port

The _daemon_action() method can be called directly by upgrade and by
the 'orch daemon <action> <name>' commands.  When this happens, construct
a CephadmDaemonDeploySpec from the DaemonDescription that incldes the
metadata we assigned when teh service was created: the IP and port(s).
This fixes upgrade and the CLI.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/schedule: add test case
Sage Weil [Thu, 11 Mar 2021 18:47:37 +0000 (13:47 -0500)]
mgr/cephadm/schedule: add test case

Matches wonky cephadm.py placement (+1 on the count).

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/suites/rados/cephadm/smoke-roleless: add rgw test on many ports
Sage Weil [Wed, 10 Mar 2021 21:54:28 +0000 (16:54 -0500)]
qa/suites/rados/cephadm/smoke-roleless: add rgw test on many ports

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agodoc/cephadm/rgw: update docs to show count-per-host
Sage Weil [Wed, 10 Mar 2021 21:48:58 +0000 (16:48 -0500)]
doc/cephadm/rgw: update docs to show count-per-host

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: add support for rgw_frontend_type (beast or civetweb)
Sage Weil [Wed, 10 Mar 2021 22:51:49 +0000 (17:51 -0500)]
mgr/cephadm: add support for rgw_frontend_type (beast or civetweb)

Previously we were assuming beast.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: remove ssl_frontend_ssl_key from RGWSpec
Sage Weil [Wed, 10 Mar 2021 19:58:09 +0000 (14:58 -0500)]
mgr/cephadm: remove ssl_frontend_ssl_key from RGWSpec

Since this didn't work anyway, stop collecting and passing through the
private key portion of the certificate.  Instead, users should include
both in the first option.  This is simpler, and provides consistency
across civetweb and beast rgw backends (for whatever that is worth).

NOTE: dashboard changes are not included here.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: fix beast private key config option
Sage Weil [Wed, 10 Mar 2021 19:50:30 +0000 (14:50 -0500)]
mgr/cephadm: fix beast private key config option

This has always been broken.  However, beast SSL will also accept a pem
(cert + key) via the ssl_certificate option, so any existing non-broken
users (if they exist) must be using that.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: fix rgw ssl cert/key config-key path
Sage Weil [Wed, 10 Mar 2021 22:50:22 +0000 (17:50 -0500)]
mgr/cephadm: fix rgw ssl cert/key config-key path

I broke this when making the realm and zone optional.

Fixes: af0216d93fb522d239b10b5d7d18208ee02fed77
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/schedule: dynamically assign ports for rgw
Sage Weil [Wed, 10 Mar 2021 19:25:23 +0000 (14:25 -0500)]
mgr/cephadm/schedule: dynamically assign ports for rgw

Dynamically number ports for RGW instances, with the start port being
the one configured on the service (or the default of 80 or 443).

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/schedule: only 1 port in DaemonPlacement
Sage Weil [Wed, 10 Mar 2021 18:59:18 +0000 (13:59 -0500)]
mgr/cephadm/schedule: only 1 port in DaemonPlacement

It would be weird to dynamically number multiple ports (although doable).
But until we have plans to support something like that, no need to handle
it here.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: move rgw frontend logic into RgwService
Sage Weil [Wed, 10 Mar 2021 18:48:10 +0000 (13:48 -0500)]
mgr/cephadm: move rgw frontend logic into RgwService

This doesn't belong in the ServiceSpec/RGWSpec

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/schedule: return DaemonPlacement instead of HostPlacementSpec
Sage Weil [Wed, 10 Mar 2021 17:24:32 +0000 (12:24 -0500)]
mgr/cephadm/schedule: return DaemonPlacement instead of HostPlacementSpec

Create a new type for the result of scheduling/place().  Include new fields
like ip:port.  Introduce a matches_daemon() to see whether an existing
daemon matches a scheduling slot.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm/schedule: remove unused methods
Sage Weil [Wed, 10 Mar 2021 16:00:04 +0000 (11:00 -0500)]
mgr/cephadm/schedule: remove unused methods

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: propagate ip:port from CephadmDaemoNDeploySpec to deployment
Sage Weil [Mon, 8 Mar 2021 21:07:20 +0000 (16:07 -0500)]
mgr/cephadm: propagate ip:port from CephadmDaemoNDeploySpec to deployment

Pass the port through to the CephadmDaemonDeploySpec, and unit.meta.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agocephadm: populate ports if known and not included in unit.meta
Sage Weil [Mon, 8 Mar 2021 19:18:04 +0000 (14:18 -0500)]
cephadm: populate ports if known and not included in unit.meta

This is only helpful for daemons deployed by older versions of cephadm
that have known ports based on their type.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: gather and report ports in 'orch ps' output
Sage Weil [Mon, 8 Mar 2021 19:14:49 +0000 (14:14 -0500)]
mgr/cephadm: gather and report ports in 'orch ps' output

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge PR #40122 into master
Patrick Donnelly [Mon, 15 Mar 2021 22:46:24 +0000 (15:46 -0700)]
Merge PR #40122 into master

* refs/pull/40122/head:
qa/tests: reduced octopus-x jobs on master and pacific (--subset 1/10)

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agomgr/cephadm: fix 'auth caps' fallback 40140/head
Sage Weil [Mon, 15 Mar 2021 22:34:57 +0000 (17:34 -0500)]
mgr/cephadm: fix 'auth caps' fallback

The first get-or-create attempt also needs to tolerate failure.

Fixes: 8ceea1961f818dc2d07edf9c256ebe5150b6b133
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge PR #40078 into master
Sage Weil [Mon, 15 Mar 2021 20:11:24 +0000 (16:11 -0400)]
Merge PR #40078 into master

* refs/pull/40078/head:
mgr: wait for ~3 beacons on startup if mons are pre-pacific
mon/MgrMonitor: populate available_modules from promote_standby()

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #40071 into master
Sage Weil [Mon, 15 Mar 2021 20:10:34 +0000 (16:10 -0400)]
Merge PR #40071 into master

* refs/pull/40071/head:
mgr/cephadm: tolerate failure to update daemon caps
mgr/cephadm: fix get_keyring_with_caps

Reviewed-by: Adam King <adking@redhat.com>
4 years agoMerge pull request #40043 from cbodley/wip-49741
Casey Bodley [Mon, 15 Mar 2021 19:12:00 +0000 (15:12 -0400)]
Merge pull request #40043 from cbodley/wip-49741

rgw: rgw::sal::RGWBucket initializes creation_time

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #39820 from cbodley/wip-qa-rgw-ignore-pg-avail
Casey Bodley [Mon, 15 Mar 2021 19:09:29 +0000 (15:09 -0400)]
Merge pull request #39820 from cbodley/wip-qa-rgw-ignore-pg-avail

qa/rgw: put PG_AVAILABILITY ignorelist override in its own file

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge PR #39979 into master
Sage Weil [Mon, 15 Mar 2021 18:49:24 +0000 (14:49 -0400)]
Merge PR #39979 into master

* refs/pull/39979/head:
python-common: fix PlacementSpec target size method
python-common: count-per-host must be combined with label or hosts or host_pattern
mgr/cephadm: handle bare 'count-per-host:NNN', fix comments
mgr/cephadm/schedule: remove Scheduler abstraction (for now at least)
mgr/cephadm/schedule: calculate additions/removals in place()
mgr/cephadm/schedule: allow colocation of certain daemon types
mgr/cephadm/schedule: shuffle candidates, not final placements
mgr/cephadm/schedule: pass per-type allow_colo to the scheduler
mgr/cephadm/services/cephadmservice: fix typo
mgr/cephadm/schedule: pass daemons, not get_daemons_func
mgr/cephadm: use local var
mgr/cephadm/schedule: move host filtering into get_candidates()
python-common/ceph/deployment/service_spec: disallow max-per-host + explicit placement
mgr/cephadm/schedule: respect count-per-host
mgr/cephadm: adjust deployment logic to allow multiple daemons per host
python-common: add count-per-host to PlacementSpec
mgr/cephadm: do not worry about even # of monitors

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoqa/tests: reduced octopus-x jobs on master and pacific (--subset 1/10) 40122/head
Yuri Weinstein [Mon, 15 Mar 2021 16:52:42 +0000 (09:52 -0700)]
qa/tests: reduced octopus-x jobs on master and pacific (--subset 1/10)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
4 years agomgr/cephadm: less noise about refreshing hosts 40130/head
Sage Weil [Mon, 15 Mar 2021 18:47:39 +0000 (13:47 -0500)]
mgr/cephadm: less noise about refreshing hosts

These happen every ~10 minutes and will obscure any real messages of
interest.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #40049 from gregsfortytwo/wip-stretch-fixes-2
Gregory Farnum [Mon, 15 Mar 2021 18:33:35 +0000 (11:33 -0700)]
Merge pull request #40049 from gregsfortytwo/wip-stretch-fixes-2

osd: Fix a bunch of stretch peering issues

Reviewed-by: Sam Just <sjust@redhat.com>
4 years agoMerge pull request #39858 from sseshasa/wip-dmclock-hdd-misc-imprvmts
Neha Ojha [Mon, 15 Mar 2021 17:27:56 +0000 (10:27 -0700)]
Merge pull request #39858 from sseshasa/wip-dmclock-hdd-misc-imprvmts

osd: Incorporate cost per io & cost per byte for cost scaling calculations in mclock scheduler.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agorbd/cache/pwl: update wait_buffer state and add wake_up 39915/head
Yin Congmin [Mon, 15 Mar 2021 07:34:35 +0000 (15:34 +0800)]
rbd/cache/pwl: update wait_buffer state and add wake_up

Signed-off-by: Yin Congmin <congmin.yin@intel.com>
4 years agoqa/suites/rados/cephadm/upgrade: add centos upgrade on latest octopus 40075/head
Sage Weil [Fri, 12 Mar 2021 16:49:10 +0000 (10:49 -0600)]
qa/suites/rados/cephadm/upgrade: add centos upgrade on latest octopus

centos currently has the semi-broken podman 2.2.1 (no json function
for inspect), but the latest octopus (15.2.10) works around it.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agomgr/cephadm: tolerate failure to update daemon caps 40071/head
Sage Weil [Mon, 15 Mar 2021 16:55:36 +0000 (11:55 -0500)]
mgr/cephadm: tolerate failure to update daemon caps

If we're upgrading from 15.2.0, we may fail to update caps.  Instead of
failing the upgrade hard, warn to the log and continue.  This is less
than ideal, but the caps will get corrected the next time the daemon is
redeployed on the next upgrade, and most likely the previous caps will
continue to work (given they were presumably working before the upgrade).

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agodoc: reorganize Windows docs 40114/head
Lucian Petrut [Mon, 15 Mar 2021 07:38:00 +0000 (07:38 +0000)]
doc: reorganize Windows docs

Most of the Windows documentation is currently included in the
README.windows.rst file.

To make it more accessible, we're moving most of it to the
"doc/" folder, adding the following pages:

* Installing Ceph on Windows
* RBD on Windows
* Windows troubleshooting

We'll keep the build and manual install instructions in
README.windows.rst. Note that ceph-dokan already has a separate
doc page.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
4 years agoMerge pull request #40098 from tchaikov/wip-qa-powertools
Kefu Chai [Mon, 15 Mar 2021 15:05:47 +0000 (23:05 +0800)]
Merge pull request #40098 from tchaikov/wip-qa-powertools

qa/workunits/rados/test_envlibrados_rocksdb: s/PowerTools/powertools/

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #40111 from tchaikov/wip-ceph-argparse-no-option-name
Kefu Chai [Mon, 15 Mar 2021 14:46:32 +0000 (22:46 +0800)]
Merge pull request #40111 from tchaikov/wip-ceph-argparse-no-option-name

pybind/ceph_argparse: do not print flag name before CephChoices in he…

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Adam King <adking@redhat.com>
4 years agoMerge pull request #39916 from zdover23/wip-doc-cephadm-adoption-preparation-2021...
Sebastian Wagner [Mon, 15 Mar 2021 14:25:39 +0000 (15:25 +0100)]
Merge pull request #39916 from zdover23/wip-doc-cephadm-adoption-preparation-2021-Mar-08

doc/cephadm: rewrite "preparation" in adoption.rst

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #40115 from tchaikov/wip-mgr-prometheus
Kefu Chai [Mon, 15 Mar 2021 14:17:24 +0000 (22:17 +0800)]
Merge pull request #40115 from tchaikov/wip-mgr-prometheus

pybind/mgr/prometheus: add typing annotations

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #40002 from orozery/rbd-encryption-api
Jason Dillaman [Mon, 15 Mar 2021 12:28:33 +0000 (08:28 -0400)]
Merge pull request #40002 from orozery/rbd-encryption-api

librbd: crypto format api semantics change

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agopybind/mgr/prometheus: add typing annotations 40115/head
Kefu Chai [Mon, 15 Mar 2021 11:35:16 +0000 (19:35 +0800)]
pybind/mgr/prometheus: add typing annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/ceph_argparse: do not print flag name before CephChoices in help descs 40111/head
Kefu Chai [Mon, 15 Mar 2021 10:27:44 +0000 (18:27 +0800)]
pybind/ceph_argparse: do not print flag name before CephChoices in help descs

This reverts commit 332abcd25059c3ed07ca5df54110a83e07f28f86.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agomgr/cephadm: extend disk selector test which disk size may be decimal 40051/head
jianglong01 [Mon, 15 Mar 2021 10:09:19 +0000 (18:09 +0800)]
mgr/cephadm:  extend disk selector test which disk size may be decimal

Signed-off-by: jianglong01 <jianglong01@qianxin.com>
4 years agoMerge pull request #40101 from dzafman/wip-49779
Kefu Chai [Mon, 15 Mar 2021 09:38:41 +0000 (17:38 +0800)]
Merge pull request #40101 from dzafman/wip-49779

standalone: osd-recovery-scrub.sh: Recovery never started

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
4 years agoMerge pull request #40056 from tchaikov/wip-no-more-MOSDPGNotify
Kefu Chai [Mon, 15 Mar 2021 09:21:39 +0000 (17:21 +0800)]
Merge pull request #40056 from tchaikov/wip-no-more-MOSDPGNotify

osd: do not send/handle MOSDPGNotify anymore

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #36298 from p-se/mgr-prom-collect-time-metric
Kefu Chai [Mon, 15 Mar 2021 09:11:55 +0000 (17:11 +0800)]
Merge pull request #36298 from p-se/mgr-prom-collect-time-metric

mgr/prometheus: introduce metric for collection time

Reviewed-by: Mykola Golub <mgolub@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>