]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agocrimson/osd: drop the unused handle_failed_op() from PG. 41526/head
Radoslaw Zarzynski [Tue, 25 May 2021 10:10:07 +0000 (10:10 +0000)]
crimson/osd: drop the unused handle_failed_op() from PG.

It became unused after the `InternalClientRequest` rework.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #40328 from rhcs-dashboard/fix-49262-master
Ernesto Puerta [Thu, 20 May 2021 09:33:39 +0000 (11:33 +0200)]
Merge pull request #40328 from rhcs-dashboard/fix-49262-master

mgr/dashboard: provide the service events when showing a service in the UI

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Paul Cuzner <pcuzner@redhat.com>
Reviewed-by: sebastian-philipp <NOT@FOUND>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41396 from tchaikov/wip-crimson-metrics
Kefu Chai [Thu, 20 May 2021 06:34:43 +0000 (14:34 +0800)]
Merge pull request #41396 from tchaikov/wip-crimson-metrics

crimson/os: replace perfcounter with seastar::metrics

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
4 years agoMerge pull request #41258 from ronen-fr/wip-ronenf-repair-stat
Kefu Chai [Thu, 20 May 2021 06:29:05 +0000 (14:29 +0800)]
Merge pull request #41258 from ronen-fr/wip-ronenf-repair-stat

osd/scrub: separate between PG state flags and internal scrubber operation

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #41264 from mgfritch/cephadm-sysctl-osd
Kefu Chai [Thu, 20 May 2021 06:25:18 +0000 (14:25 +0800)]
Merge pull request #41264 from mgfritch/cephadm-sysctl-osd

cephadm: apply sysctl settings

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #41368 from idryomov/wip-monc-rotating-timeout
Kefu Chai [Thu, 20 May 2021 06:24:07 +0000 (14:24 +0800)]
Merge pull request #41368 from idryomov/wip-monc-rotating-timeout

mon/MonClient: tolerate a rotating key that is slightly out of date

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #41277 from david-caro/consider_dev_root_mounted
Kefu Chai [Thu, 20 May 2021 06:20:21 +0000 (14:20 +0800)]
Merge pull request #41277 from david-caro/consider_dev_root_mounted

ceph-volume: Consider /dev/root as mounted

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #41287 from adk3798/daemon-for-service-warn
Kefu Chai [Thu, 20 May 2021 06:19:15 +0000 (14:19 +0800)]
Merge pull request #41287 from adk3798/daemon-for-service-warn

mgr/cephadm: warn users if they try to pass a daemon to 'orch rm'

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agoMerge pull request #41356 from tchaikov/wip-options-mgr-mon
Kefu Chai [Thu, 20 May 2021 06:18:43 +0000 (14:18 +0800)]
Merge pull request #41356 from tchaikov/wip-options-mgr-mon

common/options: extract mgr and mon options out

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #41373 from myoungwon/wip-fix-50806
Kefu Chai [Thu, 20 May 2021 06:16:59 +0000 (14:16 +0800)]
Merge pull request #41373 from myoungwon/wip-fix-50806

 osd: fix wrong input when calling recover_object()

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agotest/crimson: remove unnecessary local variable 41396/head
Kefu Chai [Wed, 19 May 2021 10:21:05 +0000 (18:21 +0800)]
test/crimson: remove unnecessary local variable

to return the return value of "make_unique<>()" is more readable, and more
consistent with other part of this source file.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agotest/crimson/seastore: teardown in reactor
Kefu Chai [Wed, 19 May 2021 10:02:59 +0000 (18:02 +0800)]
test/crimson/seastore: teardown in reactor

otherwise, we rely on the destructor of TMTestState to teardown the
fixuture created in TMTestState::_init(), but TMTestState::_init() is
called in reactor. the objects like seastar::metric_groups are
supposed to be destroyed on the same thread where they are created.
because they use thread local storage of storing persisting their status.
if we destroy objects like seastar::metric_groups on different reactor
or thread where they are created, we would have memory leak and
unexpected behavior.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: drop seastore_perf_counters.h
Kefu Chai [Wed, 19 May 2021 04:00:19 +0000 (12:00 +0800)]
crimson/os: drop seastore_perf_counters.h

it is not used anymore after the seastar::metrics migration

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/tools/store_nbd: remove PerfService
Kefu Chai [Wed, 19 May 2021 04:40:10 +0000 (12:40 +0800)]
crimson/tools/store_nbd: remove PerfService

since PerfCounter is not used in seastore, there is no need to keep an
instance of PerfService.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: do not create PerfService for SeaStore
Kefu Chai [Wed, 19 May 2021 04:32:39 +0000 (12:32 +0800)]
crimson/os: do not create PerfService for SeaStore

because we are replacing PerfCounter with seastart::metrics in crimson,
and the former has already a sharded service builtin in seastar, there
is no need to create PerfService or its counterpart for SeaStore
anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: move ctor of SeaStore into .cc
Kefu Chai [Wed, 19 May 2021 04:26:15 +0000 (12:26 +0800)]
crimson/os: move ctor of SeaStore into .cc

so the ctor and dtor are colocated in the same file. for better
readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: replace perfcounter with metrics in SegmentCleaner
Kefu Chai [Wed, 19 May 2021 04:01:56 +0000 (12:01 +0800)]
crimson/os: replace perfcounter with metrics in SegmentCleaner

for better support of m:n mapping, and better maintainability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: register metrics() in TransactionManager's ctor
Kefu Chai [Wed, 19 May 2021 04:03:27 +0000 (12:03 +0800)]
crimson/os: register metrics() in TransactionManager's ctor

this change should have been a part of
2356da856784e78750ba7f3b381b31b1c8ad5672

also do not include the unused header file

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: move ctor of SegmentCleaner into .cc
Kefu Chai [Wed, 19 May 2021 03:56:34 +0000 (11:56 +0800)]
crimson/os: move ctor of SegmentCleaner into .cc

will add more stuff into it. and presumably, smaller header file, faster
compilation.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41423 from athanatos/sjust/wip-lba-merge-fix
Samuel Just [Thu, 20 May 2021 04:15:40 +0000 (21:15 -0700)]
Merge pull request #41423 from athanatos/sjust/wip-lba-merge-fix

crimson/.../lba_btree_node_impl: handle relative addr in merge

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
4 years agoMerge pull request #41392 from tchaikov/wip-librados
Kefu Chai [Thu, 20 May 2021 01:04:33 +0000 (09:04 +0800)]
Merge pull request #41392 from tchaikov/wip-librados

include/rados/librados.h: use UINT64_C to define constants

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #41409 from zdover23/wip-doc-security-2021-05-19-fifth-item
Josh Durgin [Thu, 20 May 2021 00:42:21 +0000 (17:42 -0700)]
Merge pull request #41409 from zdover23/wip-doc-security-2021-05-19-fifth-item

doc/security: updating fifth listitem

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #41410 from zdover23/wip-doc-security-2021-05-19-seventh-item
Josh Durgin [Thu, 20 May 2021 00:41:38 +0000 (17:41 -0700)]
Merge pull request #41410 from zdover23/wip-doc-security-2021-05-19-seventh-item

doc/security: enrich seventh listitem

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #41424 from athanatos/sjust/wip-interruptor-unittest-name
Samuel Just [Thu, 20 May 2021 00:11:02 +0000 (17:11 -0700)]
Merge pull request #41424 from athanatos/sjust/wip-interruptor-unittest-name

test/crimson/CMakeLists.txt: fix name of unittest-interruptible-future

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@360.cn>
4 years agoMerge pull request #41425 from athanatos/sjust/wip-seastore-build
Kefu Chai [Thu, 20 May 2021 00:10:11 +0000 (08:10 +0800)]
Merge pull request #41425 from athanatos/sjust/wip-seastore-build

crimson: fix misc build issues

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/tools/store_nbd/tm_driver: remove perfcounter argument 41425/head
Samuel Just [Wed, 19 May 2021 21:41:06 +0000 (14:41 -0700)]
crimson/tools/store_nbd/tm_driver: remove perfcounter argument

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/logging: fix log() argument
Samuel Just [Wed, 19 May 2021 21:40:42 +0000 (14:40 -0700)]
crimson/os/seastore/logging: fix log() argument

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agotest/crimson/CMakeLists.txt: fix name of unittest-interruptible-future 41424/head
Samuel Just [Wed, 19 May 2021 21:07:46 +0000 (14:07 -0700)]
test/crimson/CMakeLists.txt: fix name of unittest-interruptible-future

unittest-interruptible-future matches the other unit tests here.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/.../lba_btree_node_impl: handle relative addr in merge 41423/head
Samuel Just [Sat, 8 May 2021 02:42:47 +0000 (02:42 +0000)]
crimson/.../lba_btree_node_impl: handle relative addr in merge

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #40836 from kamoltat/wip-autoscaler-new-alg-turn-on-off
Josh Durgin [Wed, 19 May 2021 20:15:37 +0000 (13:15 -0700)]
Merge pull request #40836 from kamoltat/wip-autoscaler-new-alg-turn-on-off

pybind/mgr/pg_autoscaler: Added autoscale-profile feature

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agopybind/mgr/pg_autoscaler: Added autoscale-profile feature 40836/head
Kamoltat [Tue, 13 Apr 2021 21:12:40 +0000 (21:12 +0000)]
pybind/mgr/pg_autoscaler: Added autoscale-profile feature

The autoscaler by default will start out each pool with minimal
pgs and `scale-up` the pgs when there is more usage in each pool.

Users can now use the commands:

`osd pool set autoscale-profile scale-down` to make the pools
start out with a full complement of pgs and only `scale-down`
when usage ratio across the pools are not even.

`osd pool set autoscale-profile scale-up` (by default) to make the pools
start out with minimal pgs and `scale-up` the pgs when there
is more usage in each pool.

Edited KVMonitor.cc file to make the `autoscale_profile` variable
persistent.

Edited tests/test_cal_final_pg_target.py so that it takes into account
the new `profile` argument when calling cal_final_pg_target(). Also,
added some new test cases for when profile is `scale-up`

Renamed tests/test_autoscaler.py to a more appropriate name:
tests/test_cal_ratio.py

Signed-off-by: Kamoltat <ksirivad@redhat.com>
4 years agodoc/security: enrich seventh listitem 41410/head
Zac Dover [Wed, 19 May 2021 14:44:00 +0000 (00:44 +1000)]
doc/security: enrich seventh listitem

This PR improves the language of the seventh listitem
in the Vulnerability Management Process in the security
documentation.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #41406 from zdover23/wip-doc-security-2021-05-19-first-item
Josh Durgin [Wed, 19 May 2021 14:40:59 +0000 (07:40 -0700)]
Merge pull request #41406 from zdover23/wip-doc-security-2021-05-19-first-item

doc/security: removing pleonasm

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/security: updating fifth listitem 41409/head
Zac Dover [Wed, 19 May 2021 14:37:52 +0000 (00:37 +1000)]
doc/security: updating fifth listitem

This PR enriches the text in the fifth listitem
of the Vulnerability Management Process.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #41391 from tchaikov/wip-libcephfs
Kefu Chai [Wed, 19 May 2021 14:13:15 +0000 (22:13 +0800)]
Merge pull request #41391 from tchaikov/wip-libcephfs

libcephfs: include <sys/time.h>

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #41359 from tchaikov/wip-cmake-cleanup
Kefu Chai [Wed, 19 May 2021 14:12:24 +0000 (22:12 +0800)]
Merge pull request #41359 from tchaikov/wip-cmake-cleanup

cmake: cleanups related to file(GLOB_RECURSE..) call

Reviewed-by: Sebastian Wagner <sewagner@redhat.com>
4 years agodoc/security: removing pleonasm 41406/head
Zac Dover [Wed, 19 May 2021 14:07:45 +0000 (00:07 +1000)]
doc/security: removing pleonasm

s/three business days or less/three business days/

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #41355 from tchaikov/wip-doc-rgw-conf
Kefu Chai [Wed, 19 May 2021 13:19:11 +0000 (21:19 +0800)]
Merge pull request #41355 from tchaikov/wip-doc-rgw-conf

doc/radosgw: use confval directive to define options

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #41399 from idryomov/wip-mrun-if-posix
Kefu Chai [Wed, 19 May 2021 12:38:07 +0000 (20:38 +0800)]
Merge pull request #41399 from idryomov/wip-mrun-if-posix

mrun: avoid == in [ command

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41393 from rosinL/fix-50835
Kefu Chai [Wed, 19 May 2021 12:37:44 +0000 (20:37 +0800)]
Merge pull request #41393 from rosinL/fix-50835

common/crc32c_aarch64: fix crc32c unittest failed on aarch64

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/radosgw: use confval directive to define options 41355/head
Kefu Chai [Sun, 16 May 2021 03:54:19 +0000 (11:54 +0800)]
doc/radosgw: use confval directive to define options

less repeating this way

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41388 into master
Sage Weil [Wed, 19 May 2021 11:57:20 +0000 (07:57 -0400)]
Merge PR #41388 into master

* refs/pull/41388/head:
cephadm: write config files as utf-8

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
4 years agoMerge PR #41286 into master
Sage Weil [Wed, 19 May 2021 11:55:30 +0000 (07:55 -0400)]
Merge PR #41286 into master

* refs/pull/41286/head:
qa/suites/orch/rook: disable centos for now
qa/suites/orch/rook/smoke: initial smoke suite
qa/tasks/rook: ROOK_HOSTPATH_REQUIRES_PRIVILEGED=true on centos
qa/tasks/rook: simplify shutdown
qa/tasks/rook: archive logs
qa/tasks/rook: more orderly cluster teardown
qa/tasks/rook: deploy ceph via rook on top of kubernetes
qa/tasks/kubeadm: install kubernetes with kubeadm
qa/suites: move rados/cephadm -> orch/cephadm; symlink
qa/tasks/cephadm: add whitespace between functions
qa/tasks/cephadm: clean up ctx.manager setup

Reviewed-by: Sébastien Han <seb@redhat.com>
4 years agoMerge PR #41370 into master
Sage Weil [Wed, 19 May 2021 11:52:17 +0000 (07:52 -0400)]
Merge PR #41370 into master

* refs/pull/41370/head:
mgr/cephadm: pin haproxy image to 2.3.z

Reviewed-by: Daniel Pivonka <dpivonka@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #41345 from rhcs-dashboard/fix-CVE-2021-3509-master
Ernesto Puerta [Wed, 19 May 2021 11:43:42 +0000 (13:43 +0200)]
Merge pull request #41345 from rhcs-dashboard/fix-CVE-2021-3509-master

mgr/dashboard: fix cookie injection issue (CVE-2021-3509)

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agomrun: avoid == in [ command 41399/head
Ilya Dryomov [Wed, 19 May 2021 11:00:16 +0000 (13:00 +0200)]
mrun: avoid == in [ command

== operator is not supported in POSIX sh (e.g. dash):

  ./mrun: 24: [: unexpected operator

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agoMerge pull request #41104 from rhcs-dashboard/fix-ESOCKETTIMEDOUT-e2e-failure
Ernesto Puerta [Wed, 19 May 2021 10:41:33 +0000 (12:41 +0200)]
Merge pull request #41104 from rhcs-dashboard/fix-ESOCKETTIMEDOUT-e2e-failure

mgr/dashboard: fix ESOCKETTIMEDOUT E2E failure

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge pull request #41273 from rhcs-dashboard/update-frontend-deps
Ernesto Puerta [Wed, 19 May 2021 10:40:02 +0000 (12:40 +0200)]
Merge pull request #41273 from rhcs-dashboard/update-frontend-deps

mgr/dashboard: update frontend deps due to security vulnerabilities

Reviewed-by: Waad Alkhoury <walkhour@redhat.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agocommon/crc32c_aarch64: fix crc32c unittest failed on aarch64 41393/head
luo rixin [Wed, 19 May 2021 02:27:18 +0000 (10:27 +0800)]
common/crc32c_aarch64: fix crc32c unittest failed on aarch64

On centos 8.2 for aarch64 with gcc 8.3, the complier will use
register v0 conflicting with the register v0 be usded in inline
asm code. Adding the related registers into clobber list to inform
complier avoiding the confict.

Fixes: https://tracker.ceph.com/issues/50835
Signed-off-by: luo rixin <luorixin@huawei.com>
4 years agocephadm: write config files as utf-8 41388/head
Sage Weil [Tue, 18 May 2021 12:23:36 +0000 (08:23 -0400)]
cephadm: write config files as utf-8

If we don't specify the encoding, python may use something like 'latin-1',
which cannot handle unicode characters.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoinclude/rados/librados.h: use UINT64_C to define constants 41392/head
Kefu Chai [Wed, 19 May 2021 01:30:08 +0000 (09:30 +0800)]
include/rados/librados.h: use UINT64_C to define constants

UINT64_C add the appropriate suffix to literals, this is more readable
than casting the literal to the expected type.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agolibcephfs: include <sys/time.h> 41391/head
Kefu Chai [Wed, 19 May 2021 01:18:33 +0000 (09:18 +0800)]
libcephfs: include <sys/time.h>

header files should be self-contained. we are using `timeval` in the header,
but failed to include the header defining it.

this change addresses the build failures on Darwin like:

int ceph_lutimes(struct ceph_mount_info *cmount, const char *path, struct timeval times[2]);
                                                                          ^
src/include/cephfs/libcephfs.h:1080:78: error: array has incomplete element type 'struct timeval'
int ceph_futimes(struct ceph_mount_info *cmount, int fd, struct timeval times[2]);
                                                                             ^
src/include/cephfs/libcephfs.h:1080:65: note: forward declaration of 'struct timeval'
int ceph_futimes(struct ceph_mount_info *cmount, int fd, struct timeval times[2]);
                                                                ^
3 errors generated.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41390 from batrick/i50865
Kefu Chai [Wed, 19 May 2021 00:47:34 +0000 (08:47 +0800)]
Merge pull request #41390 from batrick/i50865

doc: use sphinx .dot include

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41357 into master
Patrick Donnelly [Tue, 18 May 2021 20:43:28 +0000 (13:43 -0700)]
Merge PR #41357 into master

* refs/pull/41357/head:
mds: do not assert when receiving a unknow metric type

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
4 years agoMerge PR #41347 into master
Patrick Donnelly [Tue, 18 May 2021 20:42:47 +0000 (13:42 -0700)]
Merge PR #41347 into master

* refs/pull/41347/head:
mon,doc: deprecate CephFS min_compat_client
doc: add alternate_name cephfs feature to table

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
4 years agoMerge PR #41332 into master
Patrick Donnelly [Tue, 18 May 2021 20:42:09 +0000 (13:42 -0700)]
Merge PR #41332 into master

* refs/pull/41332/head:
mds: place the journaler pointer under the mds_lock

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #41171 into master
Patrick Donnelly [Tue, 18 May 2021 20:41:28 +0000 (13:41 -0700)]
Merge PR #41171 into master

* refs/pull/41171/head:
test: disable mirroring module for certain tests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #41097 into master
Patrick Donnelly [Tue, 18 May 2021 20:40:45 +0000 (13:40 -0700)]
Merge PR #41097 into master

* refs/pull/41097/head:
cephfs-mirror: allow connecting to local cluster using mon address

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Sébastien Han <seb@redhat.com>
4 years agoMerge PR #40885 into master
Patrick Donnelly [Tue, 18 May 2021 20:36:05 +0000 (13:36 -0700)]
Merge PR #40885 into master

* refs/pull/40885/head:
doc: document cephfs-mirror configuration options
cephfs-mirror: use sensible mount timeout when mounting local/remote fs
test: add tests for settting mount timeout
pybind/cephfs: add interface to set mount timeout
libcephfs: add interface to set mount timeout

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #41371 into master
Patrick Donnelly [Tue, 18 May 2021 20:20:20 +0000 (13:20 -0700)]
Merge PR #41371 into master

* refs/pull/41371/head:
mds/Mutation: initialize MutationImpl object's initiated_at

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoqa/suites/orch/rook: disable centos for now 41286/head
Sage Weil [Tue, 18 May 2021 18:37:32 +0000 (13:37 -0500)]
qa/suites/orch/rook: disable centos for now

We are hitting an selinux denial:

type=AVC msg=audit(1621353663.595:6016): avc: denied { write } for pid=28812 comm="cephcsi" name="pids.max" dev="cgroup" ino=2933 scontext=system_u:system_r:container_t:s0:c43,c177 tcontext=system_u:object_r:cgroup_t:s0 tclass=file permissive=1

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/suites/orch/rook/smoke: initial smoke suite
Sage Weil [Fri, 14 May 2021 15:45:00 +0000 (10:45 -0500)]
qa/suites/orch/rook/smoke: initial smoke suite

- os: 18.04, 20.04, centos, centos+kubic
- nodes: 1 or 3
- rook: 1.6.2 or master
- k8s: 1.21 always (for now)
- net: calico always (flannel + centos is wonky)
- workload: none, rados bench

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/tasks/rook: ROOK_HOSTPATH_REQUIRES_PRIVILEGED=true on centos
Sage Weil [Tue, 18 May 2021 14:54:42 +0000 (09:54 -0500)]
qa/tasks/rook: ROOK_HOSTPATH_REQUIRES_PRIVILEGED=true on centos

This fixes OSD creation on centos.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/tasks/rook: simplify shutdown
Sage Weil [Tue, 18 May 2021 14:54:15 +0000 (09:54 -0500)]
qa/tasks/rook: simplify shutdown

For some reason deleting common.yaml sometimes fails.  Not really
sure why, but since we will tear down kubernetes anyway this
cleanup isn't really needed.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/tasks/rook: archive logs
Sage Weil [Tue, 18 May 2021 14:53:39 +0000 (09:53 -0500)]
qa/tasks/rook: archive logs

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/tasks/rook: more orderly cluster teardown
Sage Weil [Tue, 18 May 2021 13:57:12 +0000 (08:57 -0500)]
qa/tasks/rook: more orderly cluster teardown

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/tasks/rook: deploy ceph via rook on top of kubernetes
Sage Weil [Thu, 13 May 2021 14:26:35 +0000 (09:26 -0500)]
qa/tasks/rook: deploy ceph via rook on top of kubernetes

This assumes that k8s is installed and kubectl works.

The ceph container to use is selected the same way the cephadm
task does it.

All scratch devices are consumed as OSDs.

A ceph.conf and client.admin keyring are deployed on all test
nodes, so normal tasks should work (if/when packages are installed).

Fixes: https://tracker.ceph.com/issues/47507
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agodoc: use sphinx .dot include 41390/head
Patrick Donnelly [Tue, 18 May 2021 18:13:00 +0000 (11:13 -0700)]
doc: use sphinx .dot include

To compile svg automatically into docs.

Fixed: https://tracker.ceph.com/issues/50865
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge pull request #40921 from kamoltat/wip-pool-start-out-with-1-pg
Josh Durgin [Tue, 18 May 2021 17:56:05 +0000 (10:56 -0700)]
Merge pull request #40921 from kamoltat/wip-pool-start-out-with-1-pg

mon/OSDMonitor: prepare_new_pool() starts out with 1 pg

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge PR #41383 into master
Sage Weil [Tue, 18 May 2021 16:23:48 +0000 (12:23 -0400)]
Merge PR #41383 into master

* refs/pull/41383/head:
doc/security: Add single GPG key in lieu of existing three GPG keys

Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoqa/tasks/kubeadm: install kubernetes with kubeadm
Sage Weil [Fri, 7 May 2021 18:24:31 +0000 (13:24 -0500)]
qa/tasks/kubeadm: install kubernetes with kubeadm

- install k8s with kubeadm
- initial support for flannel only
- remove taint from bootstrap/master node
- create PVs for all scratch_devs + a 'scratch' SC
- kubeadm.kubectl task

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoqa/suites: move rados/cephadm -> orch/cephadm; symlink
Sage Weil [Fri, 14 May 2021 15:40:02 +0000 (10:40 -0500)]
qa/suites: move rados/cephadm -> orch/cephadm; symlink

Move cephadm under orch/ top-level suite.  Symlink so that we
still include it in a rados run.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #41352 from tchaikov/wip-crimson-metrics
Kefu Chai [Tue, 18 May 2021 15:37:22 +0000 (23:37 +0800)]
Merge pull request #41352 from tchaikov/wip-crimson-metrics

crimson/os/seastore: use seastar::metrics for metrics reporting

Reviewed-by: Amnon Hanuhov <ahanukov@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #41377 from rhcs-dashboard/change-password-preserve-url-fix
Ernesto Puerta [Tue, 18 May 2021 15:22:02 +0000 (17:22 +0200)]
Merge pull request #41377 from rhcs-dashboard/change-password-preserve-url-fix

mgr/dashboard: Fix for query params resetting on change-password

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
4 years agoqa/tasks/cephadm: add whitespace between functions
Sage Weil [Mon, 17 May 2021 21:23:48 +0000 (16:23 -0500)]
qa/tasks/cephadm: add whitespace between functions

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agotest: recovery_scrub: do not display 'repair' status on auto-repair deep-scrub 41258/head
Ronen Friedman [Sat, 15 May 2021 19:14:38 +0000 (22:14 +0300)]
test: recovery_scrub: do not display 'repair' status on auto-repair deep-scrub

A new test: auto_repair_bluestore_tag.

Based on auto_repair_bluestore_basic. Sets auto-repair, starts a periodic
deep-scrub, then verifies that the PG state, while scrubbing, is 'scrubbing+deep'
and not 'scrubbing+deep+repair'.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
4 years agodoc/security: Add single GPG key in lieu of existing three GPG keys 41383/head
Hardik Vyas [Tue, 18 May 2021 12:13:53 +0000 (17:43 +0530)]
doc/security: Add single GPG key in lieu of existing three GPG keys

Replace existing three GPG keys with new Ceph Security Team GPG key

Signed-off-by: Hardik Vyas <hvyas@redhat.com>
4 years agoMerge PR #40640 into master
Sage Weil [Tue, 18 May 2021 13:32:52 +0000 (09:32 -0400)]
Merge PR #40640 into master

* refs/pull/40640/head:
common: send SYSLOG_IDENTIFIER to journald
cephadm: enable log to journald by default

Reviewed-by: Juan Miguel Olmo <jolmomar@redhat.com>
4 years agoMerge pull request #39580 from ifed01/wip-ifed-migrate
Rishabh Dave [Tue, 18 May 2021 12:50:43 +0000 (18:20 +0530)]
Merge pull request #39580 from ifed01/wip-ifed-migrate

ceph-volume: implement bluefs volume migration.

4 years agomgr/cephadm: pin haproxy image to 2.3.z 41370/head
Sage Weil [Mon, 17 May 2021 19:40:10 +0000 (15:40 -0400)]
mgr/cephadm: pin haproxy image to 2.3.z

The 2.4 version fails for reasons that aren't immediate clear:

May 17 15:28:42 dael conmon[3007013]: [NOTICE]   (8) : haproxy version is 2.4.0-6cbbecf
May 17 15:28:42 dael conmon[3007013]: [NOTICE]   (8) : path to executable is /usr/local/sbin/haproxy
May 17 15:28:42 dael conmon[3007013]: [ALERT]    (8) : Cannot open configuration file/directory /var/lib/haproxy/haproxy.cfg : Permission denied

Even with mode 777 on those files, there is an error about the chroot to
/var/lib/haproxy.

For now, just stick with a working version.

Fixes: https://tracker.ceph.com/issues/50830
Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #41282 from cbodley/wip-rgw-rm-civetweb
Matt Benjamin [Tue, 18 May 2021 12:19:09 +0000 (08:19 -0400)]
Merge pull request #41282 from cbodley/wip-rgw-rm-civetweb

rgw: remove the civetweb and fcgi frontends

4 years agocrimson/admin: reindent SeastarMetricsHook 41352/head
Kefu Chai [Sat, 15 May 2021 08:55:49 +0000 (16:55 +0800)]
crimson/admin: reindent SeastarMetricsHook

we don't use 1-space indent.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: add optional prefix arg to "perf dump_seastar"
Kefu Chai [Sat, 15 May 2021 06:35:31 +0000 (14:35 +0800)]
crimson/admin: add optional prefix arg to "perf dump_seastar"

so we can filter the dumped metrics by their prefix

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: do not construct AdminSocketHook with osd
Kefu Chai [Sat, 15 May 2021 06:00:33 +0000 (14:00 +0800)]
crimson/admin: do not construct AdminSocketHook with osd

osd is not used at all.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: add desc for DumpPerfCountersHook
Kefu Chai [Sat, 15 May 2021 05:58:33 +0000 (13:58 +0800)]
crimson/admin: add desc for DumpPerfCountersHook

logger and counter are used when executing the hook

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os/seastore: use seastar::metrics for metrics reporting
Kefu Chai [Sat, 15 May 2021 05:44:49 +0000 (13:44 +0800)]
crimson/os/seastore: use seastar::metrics for metrics reporting

to be better prepeare for the m:n mapping and to attach more labels
to a metric for better off-line data analysis using ETL tools in future.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: print different metric type in different way
Kefu Chai [Sat, 15 May 2021 05:43:33 +0000 (13:43 +0800)]
crimson/admin: print different metric type in different way

and use registered_metric::operator() for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: use better variable names
Kefu Chai [Sat, 15 May 2021 05:21:48 +0000 (13:21 +0800)]
crimson/admin: use better variable names

for better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: define SeastoreCollection as a class
Kefu Chai [Sat, 15 May 2021 04:18:07 +0000 (12:18 +0800)]
crimson/os: define SeastoreCollection as a class

to be consistent with the forward declaration. C++ standard does not
differentiate class from struct in this perspective. but Clang warngs at
seeing it. so silence the warning.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41364 from rzarzynski/wip-crimson-monc-pending_messages-assert
Kefu Chai [Tue, 18 May 2021 09:45:28 +0000 (17:45 +0800)]
Merge pull request #41364 from rzarzynski/wip-crimson-monc-pending_messages-assert

crimson/monc: fix send_message() racing with reopen_session().

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #41366 from tchaikov/wip-crimson-os-debug
Kefu Chai [Tue, 18 May 2021 09:05:15 +0000 (17:05 +0800)]
Merge pull request #41366 from tchaikov/wip-crimson-os-debug

crimson/os: use compile-time validation

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agomgr/dashboard: Fix for query params resetting on change-password 41377/head
Nizamudeen A [Tue, 18 May 2021 08:48:38 +0000 (14:18 +0530)]
mgr/dashboard: Fix for query params resetting on change-password

Fixes: https://tracker.ceph.com/issues/50857
Signed-off-by: Nizamudeen A <nia@redhat.com>
4 years agoMerge pull request #41354 from idryomov/wip-rbd-pwl-ssd-recovery
Ilya Dryomov [Tue, 18 May 2021 08:25:42 +0000 (10:25 +0200)]
Merge pull request #41354 from idryomov/wip-rbd-pwl-ssd-recovery

librbd/cache/pwl/ssd: fix some crash recovery issues

Reviewed-by: Yin Congmin <congmin.yin@intel.com>
Reviewed-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
4 years agoosd: fix wrong input when calling recover_object() 41373/head
myoungwon oh [Tue, 18 May 2021 05:38:05 +0000 (14:38 +0900)]
osd: fix wrong input when calling recover_object()

Iterating the obc' clones, recover_object() should check correspoding
adjacent clones.

fixes: https://tracker.ceph.com/issues/50806

Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>
4 years agoMerge pull request #39772 from xxhdx1985126/wip-crimson-client-req-pipeline-parallelism
Kefu Chai [Tue, 18 May 2021 05:44:53 +0000 (13:44 +0800)]
Merge pull request #39772 from xxhdx1985126/wip-crimson-client-req-pipeline-parallelism

crimson/osd: optimize crimson-osd's client requests process parallelism

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #41362 from Aran85/crimson-diagrams
Kefu Chai [Tue, 18 May 2021 05:29:13 +0000 (13:29 +0800)]
Merge pull request #41362 from Aran85/crimson-diagrams

crimson/seastore: add string_kv_node_layout diagrams

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agoMerge PR #40234 into master
Patrick Donnelly [Tue, 18 May 2021 02:50:45 +0000 (19:50 -0700)]
Merge PR #40234 into master

* refs/pull/40234/head:
client: always register callbacks before mount()
client: move SnapRealm methods to ClientSnapRealm.cc

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoMerge PR #40842 into master
Patrick Donnelly [Tue, 18 May 2021 02:49:40 +0000 (19:49 -0700)]
Merge PR #40842 into master

* refs/pull/40842/head:
qa: update the ffsb.sh to clone it from git://git.ceph.com/ffsb.git

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41235 into master
Patrick Donnelly [Tue, 18 May 2021 02:47:57 +0000 (19:47 -0700)]
Merge PR #41235 into master

* refs/pull/41235/head:
mds: PurgeQueue.cc fix for 32bit compilation

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #41239 into master
Patrick Donnelly [Tue, 18 May 2021 02:46:56 +0000 (19:46 -0700)]
Merge PR #41239 into master

* refs/pull/41239/head:
librbd: use uint64_t instead of size_t for SparseExtent::length
mgr/PyModule: use Py_ssize_t for the PyList index
os/bluestore: print size_t using %xz
client: print int64_t using PRId64

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>