]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agocrimson/osd/object_context: s/rwlock/lock/ 37172/head
Kefu Chai [Wed, 16 Sep 2020 01:12:52 +0000 (09:12 +0800)]
crimson/osd/object_context: s/rwlock/lock/

so it's less confusing.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/common/tri_mutex: update the class comment
Kefu Chai [Wed, 16 Sep 2020 01:10:10 +0000 (09:10 +0800)]
crimson/common/tri_mutex: update the class comment

to explain the reason why we have tri_mutex, and how it is related to
pipelined read / write. and the mutual exclusion between read, write
and rmw operations.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37131 from tchaikov/wip-47311
Kefu Chai [Wed, 16 Sep 2020 01:01:32 +0000 (09:01 +0800)]
Merge pull request #37131 from tchaikov/wip-47311

crimson/object_context: make obc interruptible

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/osd: replace "ceph_abort_msg()" with assert() 37131/head
Kefu Chai [Tue, 15 Sep 2020 01:55:47 +0000 (09:55 +0800)]
crimson/osd: replace "ceph_abort_msg()" with assert()

these are programming errors, and are easy to detect. also assert() does
not return, so compiler won't complain at seeing a branch does not
return value in a function that returns value.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/object_context: make obc interruptible
Xuehan Xu [Fri, 11 Sep 2020 11:01:52 +0000 (19:01 +0800)]
crimson/object_context: make obc interruptible

* add tri_mutex::abort() to pass given exception to all waiters
* add ObjectContext::interrupt() to abort all pending consumers
  of current object context

Fixes: https://tracker.ceph.com/issues/47311
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson: use tri_mutex for guarding ObjectContext
Kefu Chai [Mon, 14 Sep 2020 07:23:17 +0000 (15:23 +0800)]
crimson: use tri_mutex for guarding ObjectContext

before this change, a seastar::shared_mutex, a RWState and a
shared_promise are used for tracking the consumers of ObjectContext.
and all of the consumers are put into writers if the predicate function
evaluates to "false", and is awaken if the predicate function evaluates
to "true" afterwards in a polling loop waiting on the shared_promise,
which is in turn fulfilled once the last consumer of the given category
relinquishes the lock.

this approach has couple issues:

* it is heavy weighted. seastar::shared_mutex already tracks each of
  the waiters' continuation using separate promise<>, and it does try
  to reschedule them once a given consumer releases the last lock.
  so it's like a design of a customized shared_mutex over a
  shared_mutex.
* it is complicated. 3 variables for tracking the different
  consumers of ObjectContext.

in this change,

* `tri_mutex` is introduced as a variant of the original
  `seastar::shared_mutex` to track two different shared users in
  addition to an exclusive user.
* replace `shared_mutex` with `tri_mutex` in `ObjectContext`, to
  simplify the design.
* move recovery_read_marker into `ObjectContext`. assuming all
  pending actions will be added as a waiter for the related
  object context before they acquire the lock.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: add ObjectContext::wait_recovery_read()
Kefu Chai [Mon, 14 Sep 2020 06:22:40 +0000 (14:22 +0800)]
crimson/osd: add ObjectContext::wait_recovery_read()

instead of reusing ObjectContext::get_recovery_read() for both
sync call and async call. just add a new method for the async call
for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37113 from dillaman/wip-47390
Mykola Golub [Tue, 15 Sep 2020 07:29:47 +0000 (10:29 +0300)]
Merge pull request #37113 from dillaman/wip-47390

librbd: ensure local site is included in mirror image global status

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #37126 from tchaikov/wip-crimson-test
Kefu Chai [Tue, 15 Sep 2020 01:21:57 +0000 (09:21 +0800)]
Merge pull request #37126 from tchaikov/wip-crimson-test

test/crimson: cleanups

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #35167 from BryceCao/wip-inspect-zonegroup-when-create-existing...
Casey Bodley [Mon, 14 Sep 2020 21:11:08 +0000 (17:11 -0400)]
Merge pull request #35167 from BryceCao/wip-inspect-zonegroup-when-create-existing-bucket

rgw : add check for zonegroup when create existing bucket

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #36978 from cbodley/wip-47028
Casey Bodley [Mon, 14 Sep 2020 21:09:59 +0000 (17:09 -0400)]
Merge pull request #36978 from cbodley/wip-47028

rgw: add check for ACL when create existing bucket

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge PR #36125 into master
Patrick Donnelly [Mon, 14 Sep 2020 20:49:50 +0000 (13:49 -0700)]
Merge PR #36125 into master

* refs/pull/36125/head:
doc: cephfs/fs-nfs-exports: more cleanups
doc: cephfs/fs-nfs-exports: command requires enabled orch module
doc: cephfs/fs-nfs-exports.rst: write "RADOS" in ALL-CAPS
doc: cephfs/fs-nfs-exports: use "NFS Ganesha" everywhere
doc: cephfs: clarify that pseudo root path must be absolute
doc: cephfs: fix misleading "ceph fs volume create" command
doc: cephfs: fix misleading "ceph nfs create" command

Reviewed-by: Varsha Rao <varao@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge pull request #37086 from trociny/wip-qa-rbd-nbd-unmap
Jason Dillaman [Mon, 14 Sep 2020 18:32:30 +0000 (14:32 -0400)]
Merge pull request #37086 from trociny/wip-qa-rbd-nbd-unmap

qa/workunits/rbd: yet another attempt to improve rbd-nbd unmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #37038 from trociny/wip-rbd-nbd-quiesce-error
Jason Dillaman [Mon, 14 Sep 2020 18:31:58 +0000 (14:31 -0400)]
Merge pull request #37038 from trociny/wip-rbd-nbd-quiesce-error

rbd-nbd: return quiesce hook exit code

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #37107 from dillaman/wip-47405
Mykola Golub [Mon, 14 Sep 2020 15:13:28 +0000 (18:13 +0300)]
Merge pull request #37107 from dillaman/wip-47405

qa/workunits/mon: fixed excessively large pool PG count

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #37115 from sebastian-philipp/cephadm-refactor-cephamservice-create
Sebastian Wagner [Mon, 14 Sep 2020 14:43:45 +0000 (16:43 +0200)]
Merge pull request #37115 from sebastian-philipp/cephadm-refactor-cephamservice-create

mgr/cephadm: CephadmService.create -> CephadmService.prepare_create

Reviewed-by: Stephan Müller <smueller@suse.com>
4 years agoMerge pull request #35945 from jschmid1/osdspec_limit_fixes
Sebastian Wagner [Mon, 14 Sep 2020 14:41:57 +0000 (16:41 +0200)]
Merge pull request #35945 from jschmid1/osdspec_limit_fixes

mgr/cephadm: allow to map existing daemons to osdspecs

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #36997 from linuxbox2/wip-rgwlc-mtime
Casey Bodley [Mon, 14 Sep 2020 14:19:25 +0000 (10:19 -0400)]
Merge pull request #36997 from linuxbox2/wip-rgwlc-mtime

rgwlc:  fix (post-parallel) non-current expiration

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agoMerge pull request #37125 from tchaikov/wip-rgw-const-char
Casey Bodley [Mon, 14 Sep 2020 14:15:31 +0000 (10:15 -0400)]
Merge pull request #37125 from tchaikov/wip-rgw-const-char

rgw: convert string constant to string_view

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoqa/workunits/mon: fixed excessively large pool PG count 37107/head
Jason Dillaman [Mon, 14 Sep 2020 12:58:52 +0000 (08:58 -0400)]
qa/workunits/mon: fixed excessively large pool PG count

Fixes: https://tracker.ceph.com/issues/47405
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agomgr/cephadm: CephadmService.create -> CephadmService.prepare_create 37115/head
Sebastian Wagner [Fri, 11 Sep 2020 17:49:57 +0000 (19:49 +0200)]
mgr/cephadm: CephadmService.create -> CephadmService.prepare_create

Refactored this to simplify the control flow. It became obvious that

* `CephadmService.make_daemon_spec`
* `CephadmService.prepare_create`
* `CephadmService.generate_config`

are basically the same thing. They're all called directly
before the deployment of a daemon. All of them should be
unified. This PR makes this refactorization simpler

By renaming `create` to `prepare_create`, we make `create`
no longer being the entrypoint to call
`create_daemon`. Thus all the functions above
return some data structures.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #37044 from votdev/update_on_demand
Lenz Grimmer [Mon, 14 Sep 2020 09:50:19 +0000 (11:50 +0200)]
Merge pull request #37044 from votdev/update_on_demand

mgr/dashboard: Update datatable only when necessary

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
4 years agoMerge pull request #36845 from bk201/wip-44803
Lenz Grimmer [Mon, 14 Sep 2020 09:48:22 +0000 (11:48 +0200)]
Merge pull request #36845 from bk201/wip-44803

mgr/dashboard: allow getting fresh inventory data from the orchestrator

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
4 years agoMerge pull request #36365 from bk201/wip-44637
Lenz Grimmer [Mon, 14 Sep 2020 08:56:41 +0000 (10:56 +0200)]
Merge pull request #36365 from bk201/wip-44637

mgr/dashboard: add e2e tests for Orchestrator components

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
4 years agoMerge pull request #37127 from tchaikov/wip-crimson-errorator
Kefu Chai [Mon, 14 Sep 2020 05:19:47 +0000 (13:19 +0800)]
Merge pull request #37127 from tchaikov/wip-crimson-errorator

crimson/common: add specialization for futurize::invoke(Func, monostate)

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agocrimson/common: add comment to explain the partial specialization 37127/head
Kefu Chai [Mon, 14 Sep 2020 02:14:01 +0000 (10:14 +0800)]
crimson/common: add comment to explain the partial specialization

it might be confusing why we don't use explicit specialization for
defining errorator::futurize::stored_to_future.

quote from item 16, § 17.7.3, n4659:

In an explicit specialization declaration for a member of a class
template or a member template that appears in namespace scope, the
member template and some of its enclosing class templates may remain
unspecialized, except that the declaration shall not explicitly
specialize a class member template if its enclosing class templates are
not explicitly specialized as well.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37098 from tchaikov/wip-47179
Kefu Chai [Mon, 14 Sep 2020 01:44:35 +0000 (09:44 +0800)]
Merge pull request #37098 from tchaikov/wip-47179

common/admin_socket: always validate the parameters

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agocrimson/common: add specialization for futurize::invoke(Func, monostate)
Kefu Chai [Mon, 14 Sep 2020 00:36:12 +0000 (08:36 +0800)]
crimson/common: add specialization for futurize::invoke(Func, monostate)

this is a leftover of 260a702ba983f1bca29d4c8d1e28f3eef46c6699. where we
bumped up the Seastar API level to 5, in which seastar::internal::monostate
is used to represent the stored state of a future instead of a tuple<>.

to address FTBFS with GCC-9.2.1

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge PR #36987 into master
Patrick Donnelly [Sun, 13 Sep 2020 20:24:13 +0000 (13:24 -0700)]
Merge PR #36987 into master

* refs/pull/36987/head:
mgr: PyModuleRegistry::unregister_client() can run endlessly

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37069 from tchaikov/wip-seastar-lv5
Kefu Chai [Sun, 13 Sep 2020 16:37:38 +0000 (00:37 +0800)]
Merge pull request #37069 from tchaikov/wip-seastar-lv5

crimson/common: use seastar API level 5

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agotest/crimson: use ASSERT_EQ() for verify critial results 37126/head
Kefu Chai [Sun, 13 Sep 2020 15:59:24 +0000 (23:59 +0800)]
test/crimson: use ASSERT_EQ() for verify critial results

the following EXPECT_EQ() statements actually depends on this test.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agotest/crimson: use better names than ret
Kefu Chai [Sun, 13 Sep 2020 15:57:21 +0000 (23:57 +0800)]
test/crimson: use better names than ret

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agotest/crimson: use range-based loop when appropriate
Kefu Chai [Sun, 13 Sep 2020 15:52:00 +0000 (23:52 +0800)]
test/crimson: use range-based loop when appropriate

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agotest/crimson: remove unused variables
Kefu Chai [Sun, 13 Sep 2020 15:46:33 +0000 (23:46 +0800)]
test/crimson: remove unused variables

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agorgw: convert string constant to string_view 37125/head
Kefu Chai [Sun, 13 Sep 2020 15:12:02 +0000 (23:12 +0800)]
rgw: convert string constant to string_view

instead of converting string constant to char*, construct string_views
from string constants

to silence GCC warnings like:

src/rgw/services/svc_sys_obj_cache.cc:512:7: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
  512 |     { "cache list name=filter,type=CephString,req=false",
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoseastar: pickup alien fixes 37069/head
Kefu Chai [Wed, 9 Sep 2020 13:14:57 +0000 (21:14 +0800)]
seastar: pickup alien fixes

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agorbd-nbd: initialize quiesce state variable 37038/head
Mykola Golub [Sat, 12 Sep 2020 09:20:40 +0000 (10:20 +0100)]
rbd-nbd: initialize quiesce state variable

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #37097 from tchaikov/wip-buffer-offsetof
Kefu Chai [Sat, 12 Sep 2020 15:13:39 +0000 (23:13 +0800)]
Merge pull request #37097 from tchaikov/wip-buffer-offsetof

common/buffer: use homebrew BUF_OFFSETOF to replace offsetof()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #37117 from bstillwell/zstd-upgrade-v1_4_5
Kefu Chai [Sat, 12 Sep 2020 15:12:36 +0000 (23:12 +0800)]
Merge pull request #37117 from bstillwell/zstd-upgrade-v1_4_5

zstd: upgrade to v1.4.5

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37096 from majianpeng/osd-pg-remove-unused-ocde
Kefu Chai [Sat, 12 Sep 2020 11:31:13 +0000 (19:31 +0800)]
Merge pull request #37096 from majianpeng/osd-pg-remove-unused-ocde

osd/PG: remove unused function.

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #36918 from tchaikov/wip-pybind-type-annotations
Kefu Chai [Fri, 11 Sep 2020 22:49:58 +0000 (06:49 +0800)]
Merge pull request #36918 from tchaikov/wip-pybind-type-annotations

pybind/rados: use PEP484 style type annotations

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #37105 from tchaikov/wip-47404
Kefu Chai [Fri, 11 Sep 2020 22:27:51 +0000 (06:27 +0800)]
Merge pull request #37105 from tchaikov/wip-47404

mgr/zabbix: indent the output of "zabbix config-show"

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #37012 from trociny/wip-46803
Jason Dillaman [Fri, 11 Sep 2020 22:14:46 +0000 (18:14 -0400)]
Merge pull request #37012 from trociny/wip-46803

librbd: track complete async operation requests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agozstd: upgrade to v1.4.5 37117/head
Bryan Stillwell [Fri, 11 Sep 2020 21:49:39 +0000 (15:49 -0600)]
zstd: upgrade to v1.4.5

Since the v1.4.0 release there have been a few improvements to Zstandard
including improved compression ratios, faster compression, and faster
decompression.

Signed-off-by: Bryan Stillwell <bstillwell@godaddy.com>
4 years agoMerge pull request #37114 from yuriw/wip-yuriw-crontab-master
Josh Durgin [Fri, 11 Sep 2020 21:42:24 +0000 (14:42 -0700)]
Merge pull request #37114 from yuriw/wip-yuriw-crontab-master

qa/tests: running rados with -p 80 to produce better test results on …

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agorgw: return success on existing bucket 36978/head
Casey Bodley [Fri, 11 Sep 2020 19:39:30 +0000 (15:39 -0400)]
rgw: return success on existing bucket

RGWRadosStore::create_bucket() only returns EEXIST errors when a
conflict is detected and the recreation should fail. in other cases,
return success and use the 'bool *existed' flag to notify the caller of
its prior existence

Signed-off-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #35543 from sebastian-philipp/qa-cephadm-iscsi
Sebastian Wagner [Fri, 11 Sep 2020 19:10:37 +0000 (21:10 +0200)]
Merge pull request #35543 from sebastian-philipp/qa-cephadm-iscsi

qa/cephadm: Add iSCSI

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Georgios Kyratsas <gkyratsas@suse.com>
Reviewed-by: Matthew Oliver <moliver@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #36936 from mgfritch/cephadm-keyring-rm
Sebastian Wagner [Fri, 11 Sep 2020 19:09:19 +0000 (21:09 +0200)]
Merge pull request #36936 from mgfritch/cephadm-keyring-rm

mgr/cephadm: remove keyring during daemon post-remove

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #32581 from rishabh-d-dave/wip-djf-15070
Ramana Raja [Fri, 11 Sep 2020 18:30:52 +0000 (00:00 +0530)]
Merge pull request #32581 from rishabh-d-dave/wip-djf-15070

mon, cephfs: Add auth caps for CephFS fsids

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
4 years agoqa/tests: running rados with -p 80 to produce better test results on master 37114/head
Yuri Weinstein [Fri, 11 Sep 2020 17:49:35 +0000 (10:49 -0700)]
qa/tests: running rados with -p 80 to produce better test results on master

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
4 years agoMerge pull request #37021 from tchaikov/wip-osd-pass-by-ref
Kefu Chai [Fri, 11 Sep 2020 17:38:09 +0000 (01:38 +0800)]
Merge pull request #37021 from tchaikov/wip-osd-pass-by-ref

osd: pass pg_log_t by lvalue reference

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agoMerge pull request #37011 from tchaikov/wip-rgw-cache-list
Casey Bodley [Fri, 11 Sep 2020 17:04:00 +0000 (13:04 -0400)]
Merge pull request #37011 from tchaikov/wip-rgw-cache-list

rgw: set command desc for "cache list"

Reviewed-by: Casey Bodley <cbodley@redhat.com>
4 years agoMerge pull request #37002 from tchaikov/wip-alpine-cleanup
Kefu Chai [Fri, 11 Sep 2020 16:43:36 +0000 (00:43 +0800)]
Merge pull request #37002 from tchaikov/wip-alpine-cleanup

alpine: remove .editorconfig

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
4 years agolibrbd: ensure local site is included in mirror image global status 37113/head
Jason Dillaman [Fri, 11 Sep 2020 16:41:44 +0000 (12:41 -0400)]
librbd: ensure local site is included in mirror image global status

Ensure we always return local status for a mirrored image. The Python
bindings expect it to be included for backwards compatibility. Previously
the local status was not included if at least one remote site was reporting
status.

Fixes: https://tracker.ceph.com/issues/47390
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agopybind/rados: do not encode bytes 36918/head
Kefu Chai [Fri, 4 Sep 2020 03:44:38 +0000 (11:44 +0800)]
pybind/rados: do not encode bytes

the annotations ensures the type of input parameter, so no need to
encode a bytes object anymore

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoceph.in: read bytes from stdin
Kefu Chai [Fri, 4 Sep 2020 02:21:35 +0000 (10:21 +0800)]
ceph.in: read bytes from stdin

the data read from stdin is used as the input parameter for calls like
`cluster.osd_command(...)` and `cluster.mon_command`. all of them
expect a bytes `inbuf`. in Python2, this sys.stdin.read() returns a str,
and we don't differentiate str from byte back then. but we need enforce
the type now for better readablity and type correctness.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agotest/pybind: pass a str to "ping_monitor()"
Kefu Chai [Thu, 3 Sep 2020 16:26:09 +0000 (00:26 +0800)]
test/pybind: pass a str to "ping_monitor()"

monitors' names are not numbers, they are referenced by names in
general. their ranks are numbers though, but that's different story.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/rados: define unicode alias
Kefu Chai [Tue, 1 Sep 2020 13:53:10 +0000 (21:53 +0800)]
pybind/rados: define unicode alias

for silencing warnings like:

WARNING: Cannot resolve forward reference in type annotations of "rados.Rados.conf_get": name 'unicode' is not defined

because cython < 3.0 with language_level = "3", translates "str" to "unicode"
to be python2 compatible, but we've migrated to python3. and the specified
"language_level" is "3'. see also
https://github.com/cython/cython/issues/1370

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/rados: drop redundant type in docstring
Kefu Chai [Tue, 1 Sep 2020 12:54:28 +0000 (20:54 +0800)]
pybind/rados: drop redundant type in docstring

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/rados: remove duplicated statement
Kefu Chai [Tue, 1 Sep 2020 12:46:09 +0000 (20:46 +0800)]
pybind/rados: remove duplicated statement

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind: s/unicode/str/
Kefu Chai [Tue, 1 Sep 2020 12:38:46 +0000 (20:38 +0800)]
pybind: s/unicode/str/

we've migrated to python3, so no "unicode" anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/rados: use PEP484 style type annotations
Kefu Chai [Tue, 1 Sep 2020 08:16:31 +0000 (16:16 +0800)]
pybind/rados: use PEP484 style type annotations

* use PEP484 style typo annotations
* drop homebrew `requires()` decorator
* drop tests for `requires()` decorator
* add test for type annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/ceph_argparse.py: pass a str not a list to mon_command()
Kefu Chai [Tue, 1 Sep 2020 08:10:54 +0000 (16:10 +0800)]
pybind/ceph_argparse.py: pass a str not a list to mon_command()

no need to construct a list as the cmd. and more importantly,

the document of Rados.mon_command() reads:

  Rados.mon_command(self, cmd, inbuf, timeout=0, target=None):
  ...
  :param cmd: JSON formatted string.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/rados: construct cmd as an array of a single entry
Kefu Chai [Tue, 1 Sep 2020 05:07:17 +0000 (13:07 +0800)]
pybind/rados: construct cmd as an array of a single entry

otherwise what monitor receives would be something like

mon_command({ " p r e f i x " :   " o s d   m a p " ...

difficult to parse/grep by machine/human.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/rados: add type annotations to Rados.mon_command()
Kefu Chai [Tue, 1 Sep 2020 05:02:15 +0000 (13:02 +0800)]
pybind/rados: add type annotations to Rados.mon_command()

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37052 from cbodley/wip-rgw-rest-op-err
Casey Bodley [Fri, 11 Sep 2020 15:33:11 +0000 (11:33 -0400)]
Merge pull request #37052 from cbodley/wip-rgw-rest-op-err

rgw: RGWRESTOp no longer tracks separate error code

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agomgr/cephadm: merge update_keyring_caps -> create_keyring 36936/head
Michael Fritch [Fri, 11 Sep 2020 14:10:44 +0000 (08:10 -0600)]
mgr/cephadm: merge update_keyring_caps -> create_keyring

keyring is removed during daemon remove, which obsoletes the need to
update the keyring caps after get-or-create

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agomgr/cephadm: disallow removing the mon keyring
Michael Fritch [Fri, 11 Sep 2020 14:10:38 +0000 (08:10 -0600)]
mgr/cephadm: disallow removing the mon keyring

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agomgr/cephadm: remove keyring during daemon post-remove
Michael Fritch [Fri, 11 Sep 2020 14:10:32 +0000 (08:10 -0600)]
mgr/cephadm: remove keyring during daemon post-remove

Fixes: https://tracker.ceph.com/issues/47298
Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agomgr/cephadm: pass `DaemonDescription` during pre/post remove
Michael Fritch [Fri, 11 Sep 2020 14:10:26 +0000 (08:10 -0600)]
mgr/cephadm: pass `DaemonDescription` during pre/post remove

add more context than simply passing the `daemon_id`

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agomgr/cephadm: add `post_remove` daemon action
Michael Fritch [Fri, 11 Sep 2020 14:10:17 +0000 (08:10 -0600)]
mgr/cephadm: add `post_remove` daemon action

invoked after a daemon has been removed

Signed-off-by: Michael Fritch <mfritch@suse.com>
4 years agorbd-nbd: return quiesce hook exit code
Mykola Golub [Mon, 7 Sep 2020 15:16:11 +0000 (16:16 +0100)]
rbd-nbd: return quiesce hook exit code

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agorbd-nbd: fix potential race when waiting unqiesce
Mykola Golub [Mon, 7 Sep 2020 13:05:21 +0000 (14:05 +0100)]
rbd-nbd: fix potential race when waiting unqiesce

There was a window between issuing quiesce_complete and checking
the quiesce variable unset by the unqiesce callback, when the lock
was not holding. If during that window the unquiesce following the
next quiesce callbacks were called we would miss unquiesce event.

Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agolibrbd: track complete async operation requests 37012/head
Mykola Golub [Sun, 6 Sep 2020 12:48:53 +0000 (13:48 +0100)]
librbd: track complete async operation requests

to prevent duplicate maintenance operations due to RPC hiccups.

Fixes: https://tracker.ceph.com/issues/46803
Signed-off-by: Mykola Golub <mgolub@suse.com>
4 years agoMerge pull request #36825 from trociny/wip-47068
Jason Dillaman [Fri, 11 Sep 2020 13:25:57 +0000 (09:25 -0400)]
Merge pull request #36825 from trociny/wip-47068

librbd: task finisher should distinguish quiesce tasks with different ID

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoqa/cephfs: add tests for "fs authorize" subcommand 32581/head
Rishabh Dave [Mon, 24 Aug 2020 18:52:20 +0000 (00:22 +0530)]
qa/cephfs: add tests for "fs authorize" subcommand

Make caps FS-specific affects "fs authorize" subcommand. Let's add few
tests to verify its behaviour.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
4 years agoqa/cephfs: backup mount object before running tests
Rishabh Dave [Mon, 24 Aug 2020 18:34:40 +0000 (00:04 +0530)]
qa/cephfs: backup mount object before running tests

Right now, only client IDs are stashed and restored but with the recent
changes (addition of more attributes to mount objects, specifically),
this is not enough. Saving and restoring these details before and after
tests respectively ensures that mount commands rus smoothly. Not doing
this typically leads to mount command failure for the second test in the
testsuite under execution since the client IDs are saved and restored in
CephFSTestCase.setUp and CephFSTestCase.tearDown respectively but the
rest of the details are not.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
4 years agoqa/cephfs: add tests for multi-FS auth tests
Rishabh Dave [Mon, 27 Jan 2020 06:01:40 +0000 (11:31 +0530)]
qa/cephfs: add tests for multi-FS auth tests

Add testsuite for testing authorization on Ceph cluster with multiple
file systems and enable it to be executable with Teuthology framework.

Also add helper methods required to setup the test environment for
multi-FS tests.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
4 years agomgr/zabbix: indent the output of "zabbix config-show" 37105/head
Kefu Chai [Fri, 11 Sep 2020 12:29:56 +0000 (20:29 +0800)]
mgr/zabbix: indent the output of "zabbix config-show"

Fixes: https://tracker.ceph.com/issues/47404
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agomgr/cephadm: allow iscsi to be named by teuthology 35543/head
Sebastian Wagner [Fri, 11 Sep 2020 12:18:54 +0000 (14:18 +0200)]
mgr/cephadm: allow iscsi to be named by teuthology

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoqa/cephadm: Also fetch iscsi logs
Sebastian Wagner [Wed, 1 Jul 2020 09:03:20 +0000 (11:03 +0200)]
qa/cephadm: Also fetch iscsi logs

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoqa/cephadm: Add iSCSI
Sebastian Wagner [Thu, 11 Jun 2020 14:11:29 +0000 (16:11 +0200)]
qa/cephadm: Add iSCSI

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #37094 from adk3798/iscsi-m
Sebastian Wagner [Fri, 11 Sep 2020 12:09:52 +0000 (14:09 +0200)]
Merge pull request #37094 from adk3798/iscsi-m

cephadm: remove z option from iSCSI dev and configfs mount

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #37084 from tchaikov/wip-crimson-fmt
Kefu Chai [Fri, 11 Sep 2020 11:37:12 +0000 (19:37 +0800)]
Merge pull request #37084 from tchaikov/wip-crimson-fmt

crimson/osd: fix fmt::format() errors and cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #37099 from tchaikov/wip-crimson-cause-sv
Kefu Chai [Fri, 11 Sep 2020 10:55:04 +0000 (18:55 +0800)]
Merge pull request #37099 from tchaikov/wip-crimson-cause-sv

crimson/osd: pass the the cause by string_view

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoMerge pull request #37031 from sseshasa/wip-pgpush-recovery-class
Sridhar Seshasayee [Fri, 11 Sep 2020 09:36:04 +0000 (15:06 +0530)]
Merge pull request #37031 from sseshasa/wip-pgpush-recovery-class

osd: Introduce new PGOpQueueable class for recovery push/reply messages.

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
4 years agoMerge pull request #36745 from ifed01/wip-ifed-bluefs-remove-obsolete
Kefu Chai [Fri, 11 Sep 2020 09:13:25 +0000 (17:13 +0800)]
Merge pull request #36745 from ifed01/wip-ifed-bluefs-remove-obsolete

os/bluestore: get rid of obsolete stuff in bluefs.

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>
4 years agoMerge pull request #36822 from sebastian-philipp/cephadm-container-init
Sebastian Wagner [Fri, 11 Sep 2020 09:12:21 +0000 (11:12 +0200)]
Merge pull request #36822 from sebastian-philipp/cephadm-container-init

cephadm: Add --container-init

4 years agoMerge pull request #36211 from kshtsk/wip-remote-write-file
Kefu Chai [Fri, 11 Sep 2020 09:11:10 +0000 (17:11 +0800)]
Merge pull request #36211 from kshtsk/wip-remote-write-file

qa/tasks: use remote write file

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37029 from tchaikov/wip-pybind-finer-grainer-test
Kefu Chai [Fri, 11 Sep 2020 09:10:43 +0000 (17:10 +0800)]
Merge pull request #37029 from tchaikov/wip-pybind-finer-grainer-test

test/pybind: split aio_test into smaller pieces

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #32169 from zjcmszh/cmpext_op
Kefu Chai [Fri, 11 Sep 2020 09:09:47 +0000 (17:09 +0800)]
Merge pull request #32169 from zjcmszh/cmpext_op

pybind: add aio_cmpext()

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #37036 from changchengx/unused_var
Kefu Chai [Fri, 11 Sep 2020 09:06:44 +0000 (17:06 +0800)]
Merge pull request #37036 from changchengx/unused_var

common: remove data_dir_option in func global_init

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: pass the the cause by string_view 37099/head
Kefu Chai [Fri, 11 Sep 2020 08:43:37 +0000 (16:43 +0800)]
crimson/osd: pass the the cause by string_view

in future, string_view can be used almost every where string can be
used. in this change, `const char*` is instead passed to the constructor
of system_error, as we can ensure that the string_view instances are
always constructed from a `const char*` ended with `\0`.

we need this change for two reasons:

* better performance
* prefer for the world where string_view rules.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/admin_socket: always validate the parameters 37098/head
Kefu Chai [Fri, 11 Sep 2020 08:42:24 +0000 (16:42 +0800)]
common/admin_socket: always validate the parameters

this change addresses the regression introduced by
65267d55cd3714c436b188aaa8b2049ad2b21225.

before this change, we reply the client with a failure only if
both the prefix and the arguments do not match.

after this change, both the prefix and argument are checked.

Fixes: https://tracker.ceph.com/issues/47179
Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agociommon/admin_socket: extract find_matched_hook() out
Kefu Chai [Fri, 11 Sep 2020 08:35:29 +0000 (16:35 +0800)]
ciommon/admin_socket: extract find_matched_hook() out

this function is complicated and isolated enough to be a separated
method.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocommon/buffer: use homebrew BUF_OFFSETOF to replace offsetof() 37097/head
Kefu Chai [Fri, 11 Sep 2020 04:53:28 +0000 (12:53 +0800)]
common/buffer: use homebrew BUF_OFFSETOF to replace offsetof()

because buffer::raw is not a standard layout type. so `offsetof()`
does not always work. let's use a homebrew macro to calculate the
address of `bptr_storage`.

this change silences the warning like:

src/common/buffer.cc:2156:15: warning: ‘offsetof’ within non-standard-layout type ‘ceph::buffer::v15_2_0::raw’ is conditionally-supported [-Winvalid-offsetof]
 2156 |      offsetof(buffer::raw, bptr_storage));

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoosd/PG: remove unused function. 37096/head
Jianpeng Ma [Fri, 11 Sep 2020 03:08:17 +0000 (11:08 +0800)]
osd/PG: remove unused function.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
4 years agocephadm: remove z option from iSCSI dev and configfs mount 37094/head
Adam King [Fri, 4 Sep 2020 12:06:37 +0000 (08:06 -0400)]
cephadm: remove z option from iSCSI dev and configfs mount

selinux complains if you try to deploy iSCSI with the z option
on the /dev and /sys/kernel/config mounts

Signed-off-by: Adam King <adking@redhat.com>
4 years agoMerge pull request #36935 from mgfritch/cephadm-ceph-service
Sebastian Wagner [Thu, 10 Sep 2020 22:13:10 +0000 (00:13 +0200)]
Merge pull request #36935 from mgfritch/cephadm-ceph-service

mgr/cephadm: introduce `CephService` class

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #37070 from zdover23/wip-doc-3-hacking-into-docs
zdover23 [Thu, 10 Sep 2020 22:04:24 +0000 (08:04 +1000)]
Merge pull request #37070 from zdover23/wip-doc-3-hacking-into-docs

doc/dev: add HACKING.rst to Dev Guide

Reviewed-by: Josh Durgin <jdurgin@redhat.com>