]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agocrimson/osd: rename execute_{osd,pg}_op() in OpsExecuters. 36520/head
Radoslaw Zarzynski [Sun, 9 Aug 2020 22:46:25 +0000 (00:46 +0200)]
crimson/osd: rename execute_{osd,pg}_op() in OpsExecuters.

The reason for renaming is the duplicating the information
already present in the classes' names.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: simplify PgOpsExecuter and impove const-correctness.
Radoslaw Zarzynski [Fri, 7 Aug 2020 16:50:24 +0000 (18:50 +0200)]
crimson/osd: simplify PgOpsExecuter and impove const-correctness.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: reorder definitions in ops_executer.cc.
Radoslaw Zarzynski [Fri, 7 Aug 2020 16:27:54 +0000 (18:27 +0200)]
crimson/osd: reorder definitions in ops_executer.cc.

The idea here is to move `OpsExecuter::execute_osd_op()`
closer to other method definitions of this class

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: dissect PgOpsExecuter from OpsExecuter.
Radoslaw Zarzynski [Fri, 7 Aug 2020 15:55:32 +0000 (17:55 +0200)]
crimson/osd: dissect PgOpsExecuter from OpsExecuter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #36500 from tchaikov/wip-crimson-checksum
Kefu Chai [Sat, 8 Aug 2020 01:51:46 +0000 (09:51 +0800)]
Merge pull request #36500 from tchaikov/wip-crimson-checksum

crimson/osd: support checksum op and chop off consumed chunk from indata.

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #36492 from ricardoasmarques/prevent-unnecessary-ssh-reconfiguration
Sebastian Wagner [Fri, 7 Aug 2020 19:42:45 +0000 (21:42 +0200)]
Merge pull request #36492 from ricardoasmarques/prevent-unnecessary-ssh-reconfiguration

mgr/cephadm: Prevent unnecessary SSH reconfiguration

4 years agoMerge pull request #36481 from tchaikov/wip-remove-BatchWorkQueue
Kefu Chai [Fri, 7 Aug 2020 16:57:58 +0000 (00:57 +0800)]
Merge pull request #36481 from tchaikov/wip-remove-BatchWorkQueue

common/WorkQueue: remove unused BatchWorkQueue

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agoMerge pull request #36488 from lxbsz/librados
Jason Dillaman [Fri, 7 Aug 2020 12:42:29 +0000 (08:42 -0400)]
Merge pull request #36488 from lxbsz/librados

librados: add LIBRADOS_SUPPORTS_GETADDRS support

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #36480 from wjwithagen/wjw-fix-scope-ParentCacheObjectDispatch.cc
Jason Dillaman [Fri, 7 Aug 2020 12:40:08 +0000 (08:40 -0400)]
Merge pull request #36480 from wjwithagen/wjw-fix-scope-ParentCacheObjectDispatch.cc

librbd/cache: Fix scoping issue with lambda capture renaming

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #36423 from bk201/wip-46812
Lenz Grimmer [Fri, 7 Aug 2020 11:53:06 +0000 (13:53 +0200)]
Merge pull request #36423 from bk201/wip-46812

mgr/dashboard: fix tasks.mgr.dashboard.test_rbd.RbdTest.test_move_image_to_trash error

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
4 years agoMerge pull request #36427 from bk201/wip-46761
Lenz Grimmer [Fri, 7 Aug 2020 11:48:44 +0000 (13:48 +0200)]
Merge pull request #36427 from bk201/wip-46761

mgr/dashboard: Fix host attributes like labels are not returned

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agocrimson/osd: support sparse_read op 36500/head
Kefu Chai [Fri, 7 Aug 2020 10:47:52 +0000 (18:47 +0800)]
crimson/osd: support sparse_read op

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: support getxattrs op
Kefu Chai [Fri, 7 Aug 2020 05:02:11 +0000 (13:02 +0800)]
crimson/osd: support getxattrs op

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: consume indata
Kefu Chai [Thu, 6 Aug 2020 17:14:54 +0000 (01:14 +0800)]
crimson/osd: consume indata

we should chop off the consumed chunk from indata, otherwise the op
consuming data from indata will always read from the beginning of
indata.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: support checksum op
Kefu Chai [Thu, 6 Aug 2020 16:28:59 +0000 (00:28 +0800)]
crimson/osd: support checksum op

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36373 from varshar16/wip-update-orch-doc-str
Sebastian Wagner [Fri, 7 Aug 2020 09:30:49 +0000 (11:30 +0200)]
Merge pull request #36373 from varshar16/wip-update-orch-doc-str

mgr/orchestrator: Update docstring of service_action method

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #36447 from sebastian-philipp/mgr-cephadm-kill--refresh
Sebastian Wagner [Fri, 7 Aug 2020 09:29:23 +0000 (11:29 +0200)]
Merge pull request #36447 from sebastian-philipp/mgr-cephadm-kill--refresh

mgr/cephadm: Make `ceph orch * --refresh` asynchronous

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agomgr/orchestrator: Update service_action() docstring 36373/head
Varsha Rao [Thu, 30 Jul 2020 08:27:26 +0000 (13:57 +0530)]
mgr/orchestrator: Update service_action() docstring

Signed-off-by: Varsha Rao <varao@redhat.com>
4 years agoMerge PR #36501 into master
Patrick Donnelly [Fri, 7 Aug 2020 05:12:42 +0000 (22:12 -0700)]
Merge PR #36501 into master

* refs/pull/36501/head:
qa: add tests for mds_min_caps_working_set
mds: add working set minimum for caps
qa: use config_set/config_get
qa: do not append file names to dirname
qa: add exception for test timeouts

Reviewed-by: Douglas Fuller <dfuller@redhat.com>
4 years agoMerge PR #36504 into master
Patrick Donnelly [Fri, 7 Aug 2020 05:02:19 +0000 (22:02 -0700)]
Merge PR #36504 into master

* refs/pull/36504/head:
qa: log entrance/exit into the volumes plugin
qa: add debug client for volumes plugin qa

Reviewed-by: Venky Shankar <vshankar@redhat.com>
4 years agoMerge PR #36503 into master
Patrick Donnelly [Fri, 7 Aug 2020 04:51:59 +0000 (21:51 -0700)]
Merge PR #36503 into master

* refs/pull/36503/head:
client: use non-static dirent for thread-safety

Reviewed-by: Venky Shankar <vshankar@redhat.com>
4 years agoMerge pull request #36361 from tchaikov/wip-common-config-refactor
Kefu Chai [Fri, 7 Aug 2020 02:32:51 +0000 (10:32 +0800)]
Merge pull request #36361 from tchaikov/wip-common-config-refactor

common/config: refactor md_config_t::parse_config_files()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoqa: add tests for mds_min_caps_working_set 36501/head
Patrick Donnelly [Thu, 6 Aug 2020 17:57:20 +0000 (10:57 -0700)]
qa: add tests for mds_min_caps_working_set

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agomds: add working set minimum for caps
Patrick Donnelly [Thu, 6 Aug 2020 16:04:23 +0000 (09:04 -0700)]
mds: add working set minimum for caps

A client may hold many inodes pinned in its cache for open files. That
client may be unable to release those caps to respond to cache pressure
from the MDS (or quiescent client cap recall). We should not complain if
that number of capabilities is reasonable (< 10k by default).

Fixes: https://tracker.ceph.com/issues/46830
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoqa: use config_set/config_get
Patrick Donnelly [Thu, 6 Aug 2020 17:56:57 +0000 (10:56 -0700)]
qa: use config_set/config_get

It's simpler and does not require MDS restarts.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoqa: do not append file names to dirname
Patrick Donnelly [Thu, 6 Aug 2020 17:55:21 +0000 (10:55 -0700)]
qa: do not append file names to dirname

Otherwise the files generated are not actually under the sub-directory!
This is correcting a confusing aspect of the test infrastructure but
doesn't actually require any changes to the tests.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoqa: log entrance/exit into the volumes plugin 36504/head
Patrick Donnelly [Wed, 5 Aug 2020 17:50:03 +0000 (10:50 -0700)]
qa: log entrance/exit into the volumes plugin

Fixes: https://tracker.ceph.com/issues/46851
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoqa: add debug client for volumes plugin qa
Patrick Donnelly [Wed, 5 Aug 2020 16:03:54 +0000 (09:03 -0700)]
qa: add debug client for volumes plugin qa

Fixes: https://tracker.ceph.com/issues/46851
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoclient: use non-static dirent for thread-safety 36503/head
Patrick Donnelly [Thu, 6 Aug 2020 21:04:57 +0000 (14:04 -0700)]
client: use non-static dirent for thread-safety

Fixes: https://tracker.ceph.com/issues/46832
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agoqa: add exception for test timeouts
Patrick Donnelly [Thu, 6 Aug 2020 17:53:58 +0000 (10:53 -0700)]
qa: add exception for test timeouts

To make this easier to catch. It is still a RuntimeError so it should
not affect current tests by default.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
4 years agolibrbd/cache: Fix scoping issue with lambda capture renaming 36480/head
Willem Jan Withagen [Wed, 5 Aug 2020 19:04:11 +0000 (19:04 +0000)]
librbd/cache: Fix scoping issue with lambda capture renaming

Clang complains:
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:97:66: error: 'object_off' in capture list does not name a variable
   ([this, read_data, dispatch_result, on_dispatched, object_no, object_off,
                                                                 ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:98:6: error: 'object_len' in capture list does not name a variable
     object_len, snap_id, &parent_trace](ObjectCacheRequest* ack) {
     ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:99:41: error: reference to local binding 'object_off' declared in enclosing function 'librbd::cache::ParentCacheObjectDispatch::read'
      handle_read_cache(ack, object_no, object_off, object_len, snap_id,
                                        ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:81:9: note: 'object_off' declared here
  auto [object_off, object_len] = extents.front();
        ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:99:53: error: reference to local binding 'object_len' declared in enclosing function 'librbd::cache::ParentCacheObjectDispatch::read'
      handle_read_cache(ack, object_no, object_off, object_len, snap_id,
                                                    ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:81:21: note: 'object_len' declared here
  auto [object_off, object_len] = extents.front();
                    ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:99:41: error: reference to local binding 'object_off' declared in enclosing function 'librbd::cache::ParentCacheObjectDispatch<librbd::ImageCtx>::read'
      handle_read_cache(ack, object_no, object_off, object_len, snap_id,
                                        ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:242:31: note: in instantiation of member function 'librbd::cache::ParentCacheObjectDispatch<librbd::ImageCtx>::read' requested here
template class librbd::cache::ParentCacheObjectDispatch<librbd::ImageCtx>;
                              ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:81:9: note: 'object_off' declared here
  auto [object_off, object_len] = extents.front();
        ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:99:53: error: reference to local binding 'object_len' declared in enclosing function 'librbd::cache::ParentCacheObjectDispatch<librbd::ImageCtx>::read'
      handle_read_cache(ack, object_no, object_off, object_len, snap_id,
                                                    ^
/home/jenkins/workspace/ceph-master-compile/src/librbd/cache/ParentCacheObjectDispatch.cc:81:21: note: 'object_len' declared here
  auto [object_off, object_len] = extents.front();
                    ^
6 errors generated.

fixes: https://github.com/ceph/ceph/pull/36145
Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
4 years agoMerge pull request #36426 from sebastian-philipp/cephadm-redeploy-image
Sebastian Wagner [Thu, 6 Aug 2020 16:00:35 +0000 (18:00 +0200)]
Merge pull request #36426 from sebastian-philipp/cephadm-redeploy-image

 mgr/cephadm, Add --image to `ceph orch redeploy <what>`

Reviewed-by: Joshua Schmid <jschmid@suse.de>
4 years agomgr/cephadm: Prevent unnecessary SSH reconfiguration 36492/head
Ricardo Marques [Thu, 6 Aug 2020 09:47:29 +0000 (10:47 +0100)]
mgr/cephadm: Prevent unnecessary SSH reconfiguration

When 'set-user', 'set-pub-key' or 'set-priv-key' are called two
times with the same value, we can ignore the second call.

Signed-off-by: Ricardo Marques <rimarques@suse.com>
4 years agolibrados: add LIBRADOS_SUPPORTS_GETADDRS support 36488/head
Xiubo Li [Thu, 6 Aug 2020 03:56:26 +0000 (11:56 +0800)]
librados: add LIBRADOS_SUPPORTS_GETADDRS support

This will be very helpful when release ceph package(like RPM) with
backporting rados_getaddrs() api to previous version.

In tcmu-runner we need to check whether the rados_getaddrs() is
support or not.

Fixes: https://tracker.ceph.com/issues/46842
Signed-off-by: Xiubo Li <xiubli@redhat.com>
4 years agoMerge pull request #36445 from rhcs-dashboard/crushmap_fix
Lenz Grimmer [Thu, 6 Aug 2020 11:48:54 +0000 (13:48 +0200)]
Merge pull request #36445 from rhcs-dashboard/crushmap_fix

mgr/dashboard: Fix for CrushMap viewer items getting compressed vertically

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agomgr/cephadm: Refactor upgrade to call _daemon_action with new image 36426/head
Sebastian Wagner [Mon, 3 Aug 2020 09:35:41 +0000 (11:35 +0200)]
mgr/cephadm: Refactor upgrade to call _daemon_action with new image

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agomgr/cephadm, Add --image to `ceph orch daemon redeploy <what>`
Sebastian Wagner [Fri, 31 Jul 2020 16:02:34 +0000 (18:02 +0200)]
mgr/cephadm, Add --image to `ceph orch daemon redeploy <what>`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36490 from tchaikov/wip-crimson-zero-op
Kefu Chai [Thu, 6 Aug 2020 10:16:58 +0000 (18:16 +0800)]
Merge pull request #36490 from tchaikov/wip-crimson-zero-op

crimson/osd: support CEPH_OSD_OP_ZERO op and cls_cxx_write_zero()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: support cls_cxx_write_zero() 36490/head
Kefu Chai [Thu, 6 Aug 2020 06:31:42 +0000 (14:31 +0800)]
crimson/osd: support cls_cxx_write_zero()

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: support CEPH_OSD_OP_ZERO op
Kefu Chai [Thu, 6 Aug 2020 06:31:25 +0000 (14:31 +0800)]
crimson/osd: support CEPH_OSD_OP_ZERO op

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: drop stale comment
Kefu Chai [Wed, 5 Aug 2020 03:36:14 +0000 (11:36 +0800)]
crimson/admin: drop stale comment

the UNIX domain socket file was removed in
71c08cb02eb6c0f8220c0cf4b82332398b25de2c

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36455 from rzarzynski/wip-crimson-ops_executer-cleanups
Kefu Chai [Thu, 6 Aug 2020 09:39:52 +0000 (17:39 +0800)]
Merge pull request #36455 from rzarzynski/wip-crimson-ops_executer-cleanups

crimson/osd: don't reset osd_op_params.clean_regions. Clean-up OpsExecuter

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: spawn osd_op_params in do_write_op(). Fix overriding. 36455/head
Radoslaw Zarzynski [Tue, 4 Aug 2020 14:32:42 +0000 (16:32 +0200)]
crimson/osd: spawn osd_op_params in do_write_op(). Fix overriding.

This commit deduplicates the `OpsExexuter::do_osd_op()` by moving
`std::optional`-typed `osd_op_params` instantiation to `do_write_op()`.
It also fixes an issue where `clean_regions` of `osd_op_params` were
reset on every write, writefull or truncate.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: instantiate osd_op_params for every write.
Radoslaw Zarzynski [Tue, 4 Aug 2020 14:24:19 +0000 (16:24 +0200)]
crimson/osd: instantiate osd_op_params for every write.

This simplifies the logic of `OpsExecuter::flush_changes()` and
enables for moving the instantiation code to `do_write_op()` in
the future.

The performance impact seems negligible as the affected ops are
not the ones dominating in our expected workload (read/write, omap).

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/osd: delegate the txn.empty() dispatch to OpsExecuter.
Radoslaw Zarzynski [Tue, 4 Aug 2020 13:59:55 +0000 (15:59 +0200)]
crimson/osd: delegate the txn.empty() dispatch to OpsExecuter.

The benefit is no need to instantiate `osd_op_params` when only
non-modifying operation had been handled.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #36478 from zdover23/wip-doc-removing-auth-disable-warning
zdover23 [Thu, 6 Aug 2020 00:10:48 +0000 (10:10 +1000)]
Merge pull request #36478 from zdover23/wip-doc-removing-auth-disable-warning

doc/rados: remove upgrade auth disable

This commit removes a recommendation to disable
authentication before performing an upgrade.

Reviewed-by: Joao Eduardo Luis
Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #35947 from orozery/crypto_object_dispatch_layer
Jason Dillaman [Wed, 5 Aug 2020 18:48:41 +0000 (14:48 -0400)]
Merge pull request #35947 from orozery/crypto_object_dispatch_layer

librbd: add a crypto object dispatch layer

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #36474 from linuxbox2/wip-rgwlc-structv
Matt Benjamin [Wed, 5 Aug 2020 18:06:25 +0000 (14:06 -0400)]
Merge pull request #36474 from linuxbox2/wip-rgwlc-structv

rgwlc:  fix conditional decode of legacy lc op structures

4 years agocommon/WorkQueue: remove unused BatchWorkQueue 36481/head
Kefu Chai [Wed, 5 Aug 2020 17:00:33 +0000 (01:00 +0800)]
common/WorkQueue: remove unused BatchWorkQueue

we are using ShardedWQ for handling peering events. and PeeringWQ
was the only user of BatchWorkQueue. so let's remove the latter.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agodoc/rados: remove upgrade auth disable 36478/head
Zac Dover [Wed, 5 Aug 2020 15:48:11 +0000 (01:48 +1000)]
doc/rados: remove upgrade auth disable

This commit removes a recommendation to disable
authentication before performing an upgrade.

Fixes: https://tracker.ceph.com/issues/30630
Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #36444 from sebastian-philipp/cephadm-upgrade-upstradestate-class
Sebastian Wagner [Wed, 5 Aug 2020 15:02:06 +0000 (17:02 +0200)]
Merge pull request #36444 from sebastian-philipp/cephadm-upgrade-upstradestate-class

mgr/cephadm: Add UpgradeState class

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agomgr/cephadm: module.py: PEP8tify `ceph orch * ls` 36447/head
Sebastian Wagner [Wed, 5 Aug 2020 10:50:01 +0000 (12:50 +0200)]
mgr/cephadm: module.py: PEP8tify `ceph orch * ls`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agomgr/cephadm: Make `ceph orch * --refresh` asynchronous
Sebastian Wagner [Tue, 4 Aug 2020 12:01:49 +0000 (14:01 +0200)]
mgr/cephadm: Make `ceph orch * --refresh` asynchronous

There is just no way we can allow those calls to be synchornous:

* All CLI commands are blocked, till the refresh completes.
* ^C doesn't help at all, as it just kills the client

Therefore they have to be scheduled in the bg.

Fixes: https://tracker.ceph.com/issues/46813
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36463 from SUSE/wip-fix-46833
Sebastian Wagner [Wed, 5 Aug 2020 14:46:30 +0000 (16:46 +0200)]
Merge pull request #36463 from SUSE/wip-fix-46833

cephadm: don't add `ceph-volume lvm activate` for adopted simple OSDs

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agorgwlc: fix conditional decode of legacy lc op structures 36474/head
Matt Benjamin [Wed, 5 Aug 2020 14:45:45 +0000 (10:45 -0400)]
rgwlc:  fix conditional decode of legacy lc op structures

Fixes the case where an upgraded radosgw operates on an unupgraded
OSD, so likely these cases do not execute.

Fixes: https://tracker.ceph.com/issues/46838
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
4 years agoMerge pull request #36349 from sebastian-philipp/cephadm-check_deamons_except
Sebastian Wagner [Wed, 5 Aug 2020 14:44:05 +0000 (16:44 +0200)]
Merge pull request #36349 from sebastian-philipp/cephadm-check_deamons_except

mgr/cephadm: guard _check_daemons

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #36246 from linuxbox2/wip-rgwlc-decode
Casey Bodley [Wed, 5 Aug 2020 14:30:10 +0000 (10:30 -0400)]
Merge pull request #36246 from linuxbox2/wip-rgwlc-decode

rgwlc: fix error return from backward-compat decode

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
4 years agoMerge pull request #36345 from tspmelo/wip-action-disable
Lenz Grimmer [Wed, 5 Aug 2020 14:26:56 +0000 (16:26 +0200)]
Merge pull request #36345 from tspmelo/wip-action-disable

mgr/dashboard: Merge disable and disableDesc

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agomgr/caphadm/tests: Add `test_daemon_check_post` 36349/head
Sebastian Wagner [Thu, 30 Jul 2020 11:47:26 +0000 (13:47 +0200)]
mgr/caphadm/tests: Add `test_daemon_check_post`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36396 from tspmelo/wip-doc-component
Lenz Grimmer [Wed, 5 Aug 2020 13:11:54 +0000 (15:11 +0200)]
Merge pull request #36396 from tspmelo/wip-doc-component

mgr/dashboard: Extract documentation link to a component

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #36372 from jan--f/c-v-fix-package-common-dep
Jan Fajerski [Wed, 5 Aug 2020 11:37:50 +0000 (13:37 +0200)]
Merge pull request #36372 from jan--f/c-v-fix-package-common-dep

ceph-volume: dependency on python-ceph-common

4 years agomgr/cephadm: Don't run `daemons_post` if _create_daemon fail
Sebastian Wagner [Thu, 30 Jul 2020 11:38:45 +0000 (13:38 +0200)]
mgr/cephadm: Don't run `daemons_post` if _create_daemon fail

We can't run daemon_check_post for the type, if a single daemon failed.
Mainly cause `daemon_check_post` is run by service type.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agomgr/cephadm: Add test_daemon_action_fail
Sebastian Wagner [Wed, 29 Jul 2020 12:23:06 +0000 (14:23 +0200)]
mgr/cephadm: Add test_daemon_action_fail

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agomgr/cephadm/tests: Split `test_daemon_action`
Sebastian Wagner [Wed, 29 Jul 2020 12:09:00 +0000 (14:09 +0200)]
mgr/cephadm/tests: Split `test_daemon_action`

Into:

* `test_list_daemons`
* `test_daemon_action` (now without listing daemons)

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agomgr/cephadm: guard _check_daemons
Sebastian Wagner [Wed, 29 Jul 2020 10:55:04 +0000 (12:55 +0200)]
mgr/cephadm: guard _check_daemons

Continue with other daemons, if one fails

Fixes: https://tracker.ceph.com/issues/46748
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36446 from sebastian-philipp/mgr-cephadm-pep8-remotes.py
Sebastian Wagner [Wed, 5 Aug 2020 09:46:28 +0000 (11:46 +0200)]
Merge pull request #36446 from sebastian-philipp/mgr-cephadm-pep8-remotes.py

mgr/cephadm: PEP8tify a few files

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agolibrbd: add a crypto object dispatch layer 35947/head
Or Ozeri [Thu, 2 Jul 2020 14:50:16 +0000 (17:50 +0300)]
librbd: add a crypto object dispatch layer

This commit adds a new object dispatch layer to enable encryption of RBD images.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
4 years agoMerge pull request #36148 from athanatos/sjust/wip-lba-pin
Kefu Chai [Wed, 5 Aug 2020 07:34:28 +0000 (15:34 +0800)]
Merge pull request #36148 from athanatos/sjust/wip-lba-pin

crimson: pin LogicalExtentRef lba tree ancestors

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocephadm: don't add `ceph-volume lvm activate` for adopted simple OSDs 36463/head
Tim Serong [Wed, 5 Aug 2020 06:34:20 +0000 (16:34 +1000)]
cephadm: don't add `ceph-volume lvm activate` for adopted simple OSDs

This changes the logic in deploy_daemon_units() to add either `chown` calls
for simple (ceph-disk style) OSDs, or to add `ceph-volume lvm activate` calls
for LVM OSDs, rather than always adding both.  When I was working on
https://github.com/ceph/ceph/pull/34703, I'd originally added an "osd_simple"
flag to figure out what type of OSD was being adopted/deployed, but passing
that around was kinda ugly, so was removed from that PR.  This time around
I'm checking if /etc/ceph/osd/$OSD_ID-$OSD_FSID.json.adopted-by-cephadm
exists, which seems pretty safe IMO.  My only concern with this method is:
what happens if someone adopts a simple OSD, then later wants to migrate it
to LVM.  Presumably that's a destroy and recreate, keeping the same OSD ID?
If that's true, then the JSON file probably still exists, so the subsequent
create will do the wrong thing, i.e. will add `chown` calls, not `ceph-volume
lvm activate` calls.  Any/all feedback appreciated...

Fixes: https://tracker.ceph.com/issues/46833
Signed-off-by: Tim Serong <tserong@suse.com>
4 years agocrimson/os/seastore: link in LBAPin after read 36148/head
Samuel Just [Wed, 15 Jul 2020 23:47:08 +0000 (16:47 -0700)]
crimson/os/seastore: link in LBAPin after read

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore: initialize dirty extents after replay
Samuel Just [Wed, 15 Jul 2020 23:43:32 +0000 (16:43 -0700)]
crimson/os/seastore: initialize dirty extents after replay

lba_manager needs to preload BtreeLBAPin and parent nodes.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/lba_manager/btree: improved debugging
Samuel Just [Wed, 15 Jul 2020 23:41:12 +0000 (16:41 -0700)]
crimson/os/seastore/lba_manager/btree: improved debugging

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/lba_manager/btree: wire in btree pin set
Samuel Just [Thu, 16 Jul 2020 22:25:45 +0000 (15:25 -0700)]
crimson/os/seastore/lba_manager/btree: wire in btree pin set

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/lba_manager/btree: add lba pin structure
Samuel Just [Tue, 7 Jul 2020 21:23:20 +0000 (14:23 -0700)]
crimson/os/seastore/lba_manager/btree: add lba pin structure

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #36458 from drunkard/master
Kefu Chai [Wed, 5 Aug 2020 03:18:53 +0000 (11:18 +0800)]
Merge pull request #36458 from drunkard/master

doc: fix typo in output of osdmaptool example

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36145 from orozery/object_dispatch_read_extents_with_version
Jason Dillaman [Tue, 4 Aug 2020 20:53:29 +0000 (16:53 -0400)]
Merge pull request #36145 from orozery/object_dispatch_read_extents_with_version

librbd: add read extents with version to object dispatch interface

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agocrimson/os/seastore: actually use LBAManager::complete_transaction
Samuel Just [Mon, 13 Jul 2020 22:45:58 +0000 (15:45 -0700)]
crimson/os/seastore: actually use LBAManager::complete_transaction

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/: add a meta field to FixedKVNodeLayout and use in LBANode*
Samuel Just [Tue, 7 Jul 2020 18:41:23 +0000 (11:41 -0700)]
crimson/: add a meta field to FixedKVNodeLayout and use in LBANode*

Also removes LBANode::depth.

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/os/seastore/lba_manager/btree: move BtreeLBAPin to btree_range_pin.h
Samuel Just [Tue, 7 Jul 2020 18:38:53 +0000 (11:38 -0700)]
crimson/os/seastore/lba_manager/btree: move BtreeLBAPin to btree_range_pin.h

Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #36371 from votdev/issue_46775_alertmanagerspec
Sebastian Wagner [Tue, 4 Aug 2020 14:38:28 +0000 (16:38 +0200)]
Merge pull request #36371 from votdev/issue_46775_alertmanagerspec

mgr/cephadm: Enhance AlertManagerSpec to allow adding additional webhook receiver URLs

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36433 from votdev/issue_46814_alertmanager
Sebastian Wagner [Tue, 4 Aug 2020 13:41:45 +0000 (15:41 +0200)]
Merge pull request #36433 from votdev/issue_46814_alertmanager

cephadm: Deploying alertmanager image is broken

Reviewed-by: Patrick Seidensal <pseidensal@suse.com>
4 years agoMerge pull request #36441 from Daniel-Pivonka/cephadm-46819
Sebastian Wagner [Tue, 4 Aug 2020 13:40:04 +0000 (15:40 +0200)]
Merge pull request #36441 from Daniel-Pivonka/cephadm-46819

mgr/orchestrator: fix placement of '1' activating dry-run bug

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #36391 from jschmid1/followup_fix_36151
Sebastian Wagner [Tue, 4 Aug 2020 13:38:20 +0000 (15:38 +0200)]
Merge pull request #36391 from jschmid1/followup_fix_36151

mgr/cephadm: correct to_json output from str->dict

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #36388 from sebastian-philipp/cephadm-inventory-get_daemons_by_service
Sebastian Wagner [Tue, 4 Aug 2020 13:37:53 +0000 (15:37 +0200)]
Merge pull request #36388 from sebastian-philipp/cephadm-inventory-get_daemons_by_service

mgr/cephadm: get_daemons_by_service should use dd.service_name

4 years agoMerge pull request #36383 from mgfritch/orch-nfs-rados-url
Sebastian Wagner [Tue, 4 Aug 2020 13:37:15 +0000 (15:37 +0200)]
Merge pull request #36383 from mgfritch/orch-nfs-rados-url

python-common: don't assert when building rados url

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #36405 from tchaikov/wip-heartbeat-atomic
Kefu Chai [Tue, 4 Aug 2020 13:27:31 +0000 (21:27 +0800)]
Merge pull request #36405 from tchaikov/wip-heartbeat-atomic

common/HeartbeatMap: use std::atomic<time_point> for timeout

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agomgr/dashboard: Extract documentation link to a component 36396/head
Tiago Melo [Thu, 30 Jul 2020 20:31:13 +0000 (20:31 +0000)]
mgr/dashboard: Extract documentation link to a component

Fixes: https://tracker.ceph.com/issues/36565
Fixes: https://tracker.ceph.com/issues/43375
Signed-off-by: Tiago Melo <tmelo@suse.com>
4 years agomgr/cephadm: PEP8tify 36446/head
Sebastian Wagner [Tue, 4 Aug 2020 10:25:33 +0000 (12:25 +0200)]
mgr/cephadm: PEP8tify

Those files should generate fewer conflicts for now.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agomgr/dashboard: Fix for CrushMap viewer items getting compressed vertically 36445/head
Nizamudeen A [Tue, 4 Aug 2020 08:45:45 +0000 (14:15 +0530)]
mgr/dashboard: Fix for CrushMap viewer items getting compressed vertically

Intents to fix the crushmap viewer tree children getting compressed vertically when more osd's are added.

Fixes: https://tracker.ceph.com/issues/46826
Signed-off-by: Nizamudeen A <nia@redhat.com>
4 years agomgr/cephadm: PEP8tify upgrade.py 36444/head
Sebastian Wagner [Tue, 4 Aug 2020 09:13:49 +0000 (11:13 +0200)]
mgr/cephadm: PEP8tify  upgrade.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agomgr/cephadm: Add UpgradeState class
Sebastian Wagner [Mon, 3 Aug 2020 13:51:53 +0000 (15:51 +0200)]
mgr/cephadm: Add UpgradeState class

My resons are:

* This increase mypy coverage
* Makes the code more discoverable.
* Allowes us to have intelligent UpgradeState properties

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36408 from tchaikov/wip-33725
Kefu Chai [Tue, 4 Aug 2020 08:47:14 +0000 (16:47 +0800)]
Merge pull request #36408 from tchaikov/wip-33725

tools/rados: Set locator key when exporting or importing a pool

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #36443 from tchaikov/wip-crimson-asok
Kefu Chai [Tue, 4 Aug 2020 08:45:51 +0000 (16:45 +0800)]
Merge pull request #36443 from tchaikov/wip-crimson-asok

crimson/admin: read non-string settings as well

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agocrimson/admin: do not validate params of ConfigSetHook 36443/head
Kefu Chai [Tue, 4 Aug 2020 07:28:00 +0000 (15:28 +0800)]
crimson/admin: do not validate params of ConfigSetHook

validate_cmd() should have taken care of this

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: read non-string settings as well
Kefu Chai [Tue, 4 Aug 2020 07:20:46 +0000 (15:20 +0800)]
crimson/admin: read non-string settings as well

before this change, we were using

local_conf().get_val<std::string>(var);

for reading a setting when serving a "config get" command even if the
setting being queried is a non-string. so without this change, a failure
is returned complaining "unrecognized option..."

in this change:

* use get_val(string,string*) for querying the string representation
  of the queried setting
* drop the check for existence of "var" parameter, validate_cmd()
  always take care of this.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: drop prefix from cmd desc
Kefu Chai [Tue, 4 Aug 2020 07:08:02 +0000 (15:08 +0800)]
crimson/admin: drop prefix from cmd desc

there is no need to repeat the prefix in the command's description,
validate_cmd() skips it anyway.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/admin: move config hooks into admin_socket.cc
Kefu Chai [Tue, 4 Aug 2020 06:27:35 +0000 (14:27 +0800)]
crimson/admin: move config hooks into admin_socket.cc

they are not specific to OSD, let's move them into admin_socket.cc

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36438 from rhcs-dashboard/wip-pull_request_template_api-master
Lenz Grimmer [Tue, 4 Aug 2020 07:14:15 +0000 (09:14 +0200)]
Merge pull request #36438 from rhcs-dashboard/wip-pull_request_template_api-master

github: replace trigger phrase for api tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-By: Patrick Donelly <pdonell@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agodoc: typo fix on output of osdmaptool example 36458/head
Drunkard Zhang [Tue, 4 Aug 2020 05:50:03 +0000 (13:50 +0800)]
doc: typo fix on output of osdmaptool example

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>
4 years agotools/rados: Set locator key when exporting or importing a pool 36408/head
Iain Buclaw [Wed, 4 Mar 2020 14:32:43 +0000 (15:32 +0100)]
tools/rados: Set locator key when exporting or importing a pool

Fixes the following error when exporting a pool that contains objects
with a locator key set:

        error getting xattr set [object name]: (2) No such file or directory
        error from export: (2) No such file or directory

Fixes: https://tracker.ceph.com/issues/46824
Signed-off-by: Iain Buclaw <iain.buclaw@dunnhumby.com>