]> git.apps.os.sepia.ceph.com Git - ceph.git/log
ceph.git
4 years agodoc/dev: rewrite "about suites and filters" 39434/head
Zac Dover [Thu, 11 Feb 2021 23:28:06 +0000 (09:28 +1000)]
doc/dev: rewrite "about suites and filters"

This improves the English in the section
"About Suites and Filters" in the new
Teuthology Guide in the Developer Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoMerge PR #39404 into master
Sage Weil [Thu, 11 Feb 2021 22:10:44 +0000 (17:10 -0500)]
Merge PR #39404 into master

* refs/pull/39404/head:
qa/suites/upgrade/octopus-x/stress-split: cephadm-based stress-split upgrade

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
4 years agoMerge pull request #39375 from idryomov/wip-rbd-rm-exclusive
Jason Dillaman [Thu, 11 Feb 2021 19:32:08 +0000 (14:32 -0500)]
Merge pull request #39375 from idryomov/wip-rbd-rm-exclusive

librbd: refuse to release exclusive lock when removing

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #39368 from MahatiC/wip-ssd-bug-fix
Jason Dillaman [Thu, 11 Feb 2021 19:31:24 +0000 (14:31 -0500)]
Merge pull request #39368 from MahatiC/wip-ssd-bug-fix

librbd/cache/pwl: Fix user request completion

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Xiaoyan Li <xiaoyan.li@intel.com>
4 years agoMerge pull request #39037 from liu-chunmei/seastore_collection
Samuel Just [Thu, 11 Feb 2021 01:51:37 +0000 (17:51 -0800)]
Merge pull request #39037 from liu-chunmei/seastore_collection

crimson/seastore: add collection implementation

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #39398 from ktdreyer/cephadm-sys-fs-selinux
Ken Dreyer [Thu, 11 Feb 2021 01:34:12 +0000 (18:34 -0700)]
Merge pull request #39398 from ktdreyer/cephadm-sys-fs-selinux

cephadm: make /sys/fs/selinux empty

4 years agoqa/suites/upgrade/octopus-x/stress-split: cephadm-based stress-split upgrade 39404/head
Sage Weil [Thu, 4 Feb 2021 18:56:49 +0000 (12:56 -0600)]
qa/suites/upgrade/octopus-x/stress-split: cephadm-based stress-split upgrade

Apply stress during upgrade, and also pause partway and do some
thrashing.

Signed-off-by: Sage Weil <sage@newdream.net>
4 years agoMerge pull request #39384 from neha-ojha/wip-cephadm-compress
Neha Ojha [Wed, 10 Feb 2021 23:17:27 +0000 (15:17 -0800)]
Merge pull request #39384 from neha-ojha/wip-cephadm-compress

Revert "qa/tasks/cephadm: use cluster.sh and remote.sh for better readibility"

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
4 years agoMerge pull request #39155 from dillaman/wip-49037
Mykola Golub [Wed, 10 Feb 2021 16:37:34 +0000 (18:37 +0200)]
Merge pull request #39155 from dillaman/wip-49037

librbd: correct incremental deep-copy object-map inconsistencies

Reviewed-by: Mykola Golub <mgolub@suse.com>
4 years agocephadm: make /sys/fs/selinux empty 39398/head
Ken Dreyer [Wed, 10 Feb 2021 15:08:51 +0000 (08:08 -0700)]
cephadm: make /sys/fs/selinux empty

When the following conditions are true:

  1) A host has selinux-policy-targeted,
  2) We mount the host's /sys into a privileged container,
  3) The container has SELINUXTYPE=targeted in /etc/selinux/config,
  4) The container does not have an selinux-policy-targeted package,

then SELinux-enabled applications like restorecon or DNF do not work inside
the container.

Resolve this by making /sys/fs/selinux an empty directory.

Fixes: https://tracker.ceph.com/issues/49239
Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
4 years agoMerge pull request #38963 from rhcs-dashboard/refresh-crushmap
Ernesto Puerta [Wed, 10 Feb 2021 14:03:09 +0000 (15:03 +0100)]
Merge pull request #38963 from rhcs-dashboard/refresh-crushmap

mgr/dashboard: Automatically refresh crush map metadata table

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
4 years agoMerge pull request #39355 from sebastian-philipp/doc-install-rm-deepsea
Sebastian Wagner [Wed, 10 Feb 2021 12:29:30 +0000 (13:29 +0100)]
Merge pull request #39355 from sebastian-philipp/doc-install-rm-deepsea

doc/install: cephadm is the successor of DeepSea

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #38505 from sebastian-philipp/doc-dev-cephadm-scalability-notes
Sebastian Wagner [Wed, 10 Feb 2021 12:17:00 +0000 (13:17 +0100)]
Merge pull request #38505 from sebastian-philipp/doc-dev-cephadm-scalability-notes

doc/dev/cephadm: scalability notes

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
4 years agoMerge pull request #39287 from david-z/wip-fix-join-fscid
Venky Shankar [Wed, 10 Feb 2021 12:00:14 +0000 (17:30 +0530)]
Merge pull request #39287 from david-z/wip-fix-join-fscid

mon/MDSMonitor.cc: fix join fscid not applied with pending fsmap at boot

4 years agoMerge PR #38318 into master
Jan Fajerski [Wed, 10 Feb 2021 10:50:09 +0000 (11:50 +0100)]
Merge PR #38318 into master

* refs/pull/38318/head:
common: Add Windows ETW support

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/seastore: add collection manager implementation 39037/head
chunmei-liu [Sat, 6 Feb 2021 02:13:37 +0000 (18:13 -0800)]
crimson/seastore: add collection manager implementation

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>
Signed-off-by: Samuel Just <sjust@redhat.com>
4 years agoMerge pull request #39387 from xxhdx1985126/wip-client-request-pg-ref
Kefu Chai [Wed, 10 Feb 2021 05:00:13 +0000 (13:00 +0800)]
Merge pull request #39387 from xxhdx1985126/wip-client-request-pg-ref

crimson/osd: capture pgref by value if it is to be referenced asynchronously

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39388 from ivzhh/master
Kefu Chai [Wed, 10 Feb 2021 04:56:46 +0000 (12:56 +0800)]
Merge pull request #39388 from ivzhh/master

cmake: don't pass ccache argument to RocksDB build

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocmake: don't pass ccache argument to RocksDB build 39388/head
Sheng Mao [Wed, 10 Feb 2021 03:14:58 +0000 (20:14 -0700)]
cmake: don't pass ccache argument to RocksDB build

RocksDB's cmake has a similar logic to turn on ccache as the main cmake.
Currently BuildRocksDB.cmake explicitly passes ccache to RocksDB cmake
arguments as CMAKE_CXX_COMPILER_LAUNCHER. This causes a compilation error
on ccache 4.0, especially in make -j mode.

ccache: error: Recursive invocation (the name of the ccache binary must
be "ccache")

This commit lets cmake from RocksDB decide RULE_LAUNCH_COMPILE by itself.

Signed-off-by: Sheng Mao <shngmao@gmail.com>
4 years agocrimson/osd: capture pgref by value if it is to be referenced asynchronously 39387/head
Xuehan Xu [Wed, 10 Feb 2021 02:37:29 +0000 (10:37 +0800)]
crimson/osd: capture pgref by value if it is to be referenced asynchronously

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agoMerge pull request #39351 from zdover23/wip-doc-dev-getting-binaries-rewrite-08-Feb...
zdover23 [Wed, 10 Feb 2021 02:00:15 +0000 (12:00 +1000)]
Merge pull request #39351 from zdover23/wip-doc-dev-getting-binaries-rewrite-08-Feb-2021

doc/dev: rewrite "getting binaries" section

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39383 from badone/wip-unmask-deep-scrub-stdout-lazy_omap_stats_test
Brad Hubbard [Wed, 10 Feb 2021 01:37:17 +0000 (11:37 +1000)]
Merge pull request #39383 from badone/wip-unmask-deep-scrub-stdout-lazy_omap_stats_test

test/lazy-omap-stats: Immediate deep-scrub output

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39333 from cyx1231st/wip-seastore-onode-tree-empty-oid-ns
Samuel Just [Tue, 9 Feb 2021 23:39:13 +0000 (15:39 -0800)]
Merge pull request #39333 from cyx1231st/wip-seastore-onode-tree-empty-oid-ns

crimson/onode-staged-tree: support empty ns and oid

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agoRevert "qa/tasks/cephadm: use cluster.sh and remote.sh for better readibility" 39384/head
Neha Ojha [Tue, 9 Feb 2021 22:10:19 +0000 (22:10 +0000)]
Revert "qa/tasks/cephadm: use cluster.sh and remote.sh for better readibility"

This reverts commit 6d7ba1180e66df2d6dee3a01fe3e12cafaa6fe8c.

Fixes: https://tracker.ceph.com/issues/49228
Signed-off-by: Neha Ojha <nojha@redhat.com>
4 years agotest/lazy-omap-stats: Immediate deep-scrub output 39383/head
Brad Hubbard [Tue, 9 Feb 2021 03:21:56 +0000 (13:21 +1000)]
test/lazy-omap-stats: Immediate deep-scrub output

Show stdout from the child process as it happens.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
4 years agoMerge pull request #38871 from cbodley/wip-qa-rgw-valgrind-on
Casey Bodley [Tue, 9 Feb 2021 21:22:42 +0000 (16:22 -0500)]
Merge pull request #38871 from cbodley/wip-qa-rgw-valgrind-on

qa/rgw: reenable valgrind in rgw/verify suite

Reviewed-by: Ali Maredia <amaredia@redhat.com>
4 years agodoc/dev: rewrite "getting binaries" section 39351/head
Zac Dover [Mon, 8 Feb 2021 09:04:56 +0000 (19:04 +1000)]
doc/dev: rewrite "getting binaries" section

This rewrites the "Getting Binaries" section
of the "Workflow" section of the "Tests:
Integration Tests" section of the Developer Guide.

Signed-off-by: Zac Dover <zac.dover@gmail.com>
4 years agoqa/rgw: test_rgw_reshard.py just tries common ports 38871/head
Casey Bodley [Mon, 18 Jan 2021 21:59:05 +0000 (16:59 -0500)]
qa/rgw: test_rgw_reshard.py just tries common ports

the 'sudo netstat | grep radosgw' command is causing failures:

  File "qa/workunits/rgw/test_rgw_reshard.py", line 53, in get_radosgw_port
    x = out.decode('utf8').split(" ")
AttributeError: 'bool' object has no attribute 'decode'

because when valgrind is enabled, the process name is 'valgrind' instead
of 'radosgw'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
4 years agotest/rgw: re-enable valgrind in rgw verify suite
Casey Bodley [Tue, 12 Jan 2021 13:19:06 +0000 (08:19 -0500)]
test/rgw: re-enable valgrind in rgw verify suite

this only enables it for radosgw itself, not the other daemons

Signed-off-by: Casey Bodley <cbodley@redhat.com>
4 years agoqa/rgw: move valgrind config into valgrind.yaml
Casey Bodley [Tue, 12 Jan 2021 19:36:01 +0000 (14:36 -0500)]
qa/rgw: move valgrind config into valgrind.yaml

Signed-off-by: Casey Bodley <cbodley@redhat.com>
4 years agomgr/dashboard: Automatically refresh the crush map metadata table 38963/head
Nizamudeen A [Tue, 19 Jan 2021 12:35:43 +0000 (18:05 +0530)]
mgr/dashboard: Automatically refresh the crush map metadata table

If we make any change to the osd crush map like do an osd crush reweight from cli, for that change to be reflected on metadata table we need to reload the entire page. Instead this PR takes care of auto refreshing the tree view.

Fixes: https://tracker.ceph.com/issues/48922
Signed-off-by: Nizamudeen A <nia@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
4 years agolibrbd: journal::StandardPolicy template defaults to ImageCtx 39375/head
Ilya Dryomov [Tue, 9 Feb 2021 13:04:39 +0000 (14:04 +0100)]
librbd: journal::StandardPolicy template defaults to ImageCtx

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agolibrbd: don't hold owner_lock for validate_image_removal()
Ilya Dryomov [Mon, 8 Feb 2021 16:01:47 +0000 (17:01 +0100)]
librbd: don't hold owner_lock for validate_image_removal()

handle_exclusive_lock() and handle_shut_down_exclusive_lock() call
validate_image_removal() without owner_lock held, so holding it in
shut_down_exclusive_lock() appears to be redundant.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agolibrbd: treat EROFS as expected in handle_acquire_lock()
Ilya Dryomov [Sun, 7 Feb 2021 14:09:24 +0000 (15:09 +0100)]
librbd: treat EROFS as expected in handle_acquire_lock()

If the peer refuses to release exclusive lock (e.g. in case automatic
exclusive lock transitions are disabled), EROFS is retured.  Suppress
a rather confusing "Read-only file system" error message -- this case
is no different from EBUSY or EAGAIN.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agolibrbd: refuse to release exclusive lock when removing
Ilya Dryomov [Sun, 7 Feb 2021 12:46:15 +0000 (13:46 +0100)]
librbd: refuse to release exclusive lock when removing

Commit 25c2ffe145be ("librbd: acquire exclusive lock from peer when
removing") changed PreRemoveRequest to request exclusive lock from the
peer instead of giving up and proceeding without exclusive lock.  This
caused one of the test cases that sometimes runs concurrent "rbd rm"
against the same image to fail intermittently, most often on assert

  ceph_assert(image_ctx.exclusive_lock == nullptr ||
              image_ctx.exclusive_lock->is_lock_owner());

because exclusive lock is now automatically transitioned to another
"rbd rm" on its request.

The root cause is older and probably goes back to when synchronous
librbd::remove() which held owner_lock across all operations including
trim_image() was converted to a set of state machines.  Since then, any
peer that requests exclusive lock (instead of trying once and backing
off) is able to mess with image removal.

Install StandardPolicy to disable automatic exclusive lock transitions
during image removal.

Fixes: https://tracker.ceph.com/issues/49226
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agolibrbd: templatize exclusive_lock::StandardPolicy
Ilya Dryomov [Mon, 8 Feb 2021 15:49:35 +0000 (16:49 +0100)]
librbd: templatize exclusive_lock::StandardPolicy

This will be used by expect_set_exclusive_lock_policy() in
test_mock_PreRemoveRequest.cc.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
4 years agoMerge pull request #39211 from cbodley/wip-rgw-admin-formatter
Casey Bodley [Tue, 9 Feb 2021 12:31:05 +0000 (07:31 -0500)]
Merge pull request #39211 from cbodley/wip-rgw-admin-formatter

radosgw-admin: fix leaks with make_unique()

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
4 years agoMerge pull request #39301 from cbodley/wip-49169
Casey Bodley [Tue, 9 Feb 2021 12:30:40 +0000 (07:30 -0500)]
Merge pull request #39301 from cbodley/wip-49169

rgw: fix snprintf() truncate warning in ETagVerifier

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
4 years agodoc/dev/cephadm: scalability notes 38505/head
Sebastian Wagner [Wed, 9 Dec 2020 10:59:59 +0000 (11:59 +0100)]
doc/dev/cephadm: scalability notes

Add a few notes about possible future scalability improvements

Co-authored-by: Joshua Schmid <jschmid@suse.de>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agodoc/install: cephadm is the successor of DeepSea 39355/head
Sebastian Wagner [Mon, 8 Feb 2021 12:35:02 +0000 (13:35 +0100)]
doc/install: cephadm is the successor of DeepSea

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agocommon: Add Windows ETW support 38318/head
Alin Gabriel Serdean [Mon, 16 Nov 2020 22:36:36 +0000 (00:36 +0200)]
common: Add Windows ETW support

This patch adds generic ETW logging on top of syslog calls.

The events are described in the message compiler file `event_logging.mc`.

Using the cross platform utility `windmc`, we will generate the header file,
`event_logging.h`, needed by the ETW implementation, and the resource file
`event_logging.rc`.

Over the generated resource file we will run another utility called `windres`.
this tool is used to compile the binary objects needed by the OS utilities
(i.e. `Event Viewer`) to view the logged events.

On usage, a registry key needs to be added/removed on the target computer.
The registry looks like the following:
[HKLM\SYSTEM\CurrentControlSet\Services\EventLog\Application\<get_process_name>]
"EventMessageFile"="<Folder_location_to>\\event_logging.dll"

I.E.:
[HKLM\SYSTEM\CurrentControlSet\Services\EventLog\Application\rbd-wnbd]
"EventMessageFile"="C:\\Program Files\\Ceph\\bin\\event_logging.dll"

Signed-off-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
Co-authored-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>
4 years agolibrbd/cache/pwl: Fix user request completion in 39368/head
Mahati Chamarthy [Wed, 3 Feb 2021 15:03:50 +0000 (20:33 +0530)]
librbd/cache/pwl: Fix user request completion in

... persist_on_flush mode

Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>
4 years agoMerge pull request #39345 from tchaikov/wip-crimson-move
Kefu Chai [Tue, 9 Feb 2021 05:51:01 +0000 (13:51 +0800)]
Merge pull request #39345 from tchaikov/wip-crimson-move

crimson/osd: do not pass oop as a const ref and cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agoMerge pull request #39353 from lxbsz/minor
Venky Shankar [Tue, 9 Feb 2021 05:13:10 +0000 (10:43 +0530)]
Merge pull request #39353 from lxbsz/minor

client: use scoped_lock instead of unique_lock

4 years agocrimson/onode-staged-tree: be explicit about string_key markers 39333/head
Yingxin Cheng [Tue, 9 Feb 2021 04:48:59 +0000 (12:48 +0800)]
crimson/onode-staged-tree: be explicit about string_key markers

To be explicit that string_kew_view_t::MIN and MAX are not valid
string lengths. They are reserved as markers.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agoMerge pull request #39367 from runsisi/wip-fix-spec
Kefu Chai [Tue, 9 Feb 2021 04:23:50 +0000 (12:23 +0800)]
Merge pull request #39367 from runsisi/wip-fix-spec

ceph.spec.in: fix missing backslash for line continuation

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/onode-staged-tree: support empty ns and oid
Yingxin Cheng [Sun, 7 Feb 2021 03:09:14 +0000 (11:09 +0800)]
crimson/onode-staged-tree: support empty ns and oid

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
4 years agocrimson/os: disable code block only executed by Debug build 39345/head
Kefu Chai [Mon, 8 Feb 2021 16:35:59 +0000 (00:35 +0800)]
crimson/os: disable code block only executed by Debug build

silences warning like:

warning: lambda capture 'this' is not used [-Wunused-lambda-capture]
  ).safe_then([this, target_size] {
               ^~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: capture labels using new variables
Kefu Chai [Mon, 8 Feb 2021 16:34:25 +0000 (00:34 +0800)]
crimson/osd: capture labels using new variables

structured binding does not define variables, let's capture the bound
"variables" using variables defined in the capture list.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/common: mark ExitBarrier final
Kefu Chai [Mon, 8 Feb 2021 16:27:27 +0000 (00:27 +0800)]
crimson/common: mark ExitBarrier final

to silence -Wfinal-dtor-non-final-class warnings like

operation.h:461:20: warning: class with destructor marked 'final' cannot
be inherited from [-Wfinal-dtor-non-final-class]
    ~ExitBarrier() final {
                   ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: refactor ClientRequest::do_process()
Kefu Chai [Mon, 8 Feb 2021 05:11:37 +0000 (13:11 +0800)]
crimson/osd: refactor ClientRequest::do_process()

instead of using a local lambda which tries to have a single point
for handling both the happy path and failure path, it'd be simpler
to just early return if we fail to handle a request,

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: do not pass oop as a const ref
Kefu Chai [Sun, 7 Feb 2021 15:23:33 +0000 (23:23 +0800)]
crimson/osd: do not pass oop as a const ref

before this change, std::move(oop) will create a copy of oop.
after this change, std::move(oop) actually moves oop.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: do not pass ops to submit_transaction() separately
Kefu Chai [Sun, 7 Feb 2021 15:12:14 +0000 (23:12 +0800)]
crimson/osd: do not pass ops to submit_transaction() separately

the order of evaluation of function arguments is unspecified with some
exceptions. but in this case, the reason why ops is still valid is that
the parameter type of `osd_op_p` is `const osd_op_params_t&`. so a copy
is passed to it.

to enable us to actually move `osd_op_p`, in this change, the
osd_op_p.req->ops is used instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: extract methods out of ClientRequest::process_op()
Kefu Chai [Mon, 8 Feb 2021 10:19:25 +0000 (18:19 +0800)]
crimson/osd: extract methods out of ClientRequest::process_op()

* do_recover_missing()
* do_process()

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: do not move op_params_p in fill_op_params_bump_pg_version()
Kefu Chai [Sun, 7 Feb 2021 06:46:12 +0000 (14:46 +0800)]
crimson/osd: do not move op_params_p in fill_op_params_bump_pg_version()

it's confusing to move the parameter for updating it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: do not add parentheses unless necessary
Kefu Chai [Sun, 7 Feb 2021 06:35:50 +0000 (14:35 +0800)]
crimson/osd: do not add parentheses unless necessary

do not use the notation of

[]() -> return_type {

unless it helps with readability or we have to do so.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: remove no-op func passed to OpsExecuter::flush_changes()
Kefu Chai [Sun, 7 Feb 2021 05:53:11 +0000 (13:53 +0800)]
crimson/osd: remove no-op func passed to OpsExecuter::flush_changes()

no need to create an empty func for an request which creates no
transactions.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: rename PG::rep_repair_primary_object() to repair_object()
Kefu Chai [Sun, 7 Feb 2021 04:24:21 +0000 (12:24 +0800)]
crimson/osd: rename PG::rep_repair_primary_object() to repair_object()

as we should expose the internals using the name of interface, and there
is no need to differentiate rep_repair_primary_object() from another
repair method.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39361 from sebastian-philipp/orch-fix-flake8
Kefu Chai [Tue, 9 Feb 2021 02:15:36 +0000 (10:15 +0800)]
Merge pull request #39361 from sebastian-philipp/orch-fix-flake8

mgr/orch: Fix flake8 error

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoceph.spec.in: fix missing backslash for line continuation 39367/head
luo.runbing [Tue, 9 Feb 2021 01:18:18 +0000 (09:18 +0800)]
ceph.spec.in: fix missing backslash for line continuation

it errors while building without lua_packages:

+ -DBOOST_J=102 -DWITH_GRAFANA=ON
/var/tmp/rpm-tmp.aq2X3J: line 91: -DBOOST_J=102: command not found

Signed-off-by: luo.runbing <luo.runbing@zte.com.cn>
4 years agoMerge pull request #39332 from cyx1231st/wip-seastore-range-query
Samuel Just [Mon, 8 Feb 2021 20:04:31 +0000 (12:04 -0800)]
Merge pull request #39332 from cyx1231st/wip-seastore-range-query

crimson/onode-staged-tree: implement Cursor::get_next() and comparators for range query

Reviewed-by: Samuel Just <sjust@redhat.com>
4 years agorbd-mirror: don't prune older mirror snapshots when pruning incomplete snapshot 39155/head
Jason Dillaman [Mon, 8 Feb 2021 16:53:28 +0000 (11:53 -0500)]
rbd-mirror: don't prune older mirror snapshots when pruning incomplete snapshot

Since we normally prune in order, we need to ensure that we don't prune older
snapshots when we need to delete an incomplete mirror snapshot since the
older snapshot might be the only remaining mirror snapshot.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agomgr/orch: Fix flake8 error 39361/head
Sebastian Wagner [Mon, 8 Feb 2021 16:37:52 +0000 (17:37 +0100)]
mgr/orch: Fix flake8 error

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #37729 from mgfritch/cephadm-podman-logdriver
Sebastian Wagner [Mon, 8 Feb 2021 16:34:10 +0000 (17:34 +0100)]
Merge pull request #37729 from mgfritch/cephadm-podman-logdriver

cephadm: configure journald as the logdriver

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dimitri Savineau <dsavinea@redhat.com>
4 years agoMerge pull request #37764 from mgfritch/cephadm-no-container-init
Sebastian Wagner [Mon, 8 Feb 2021 16:33:16 +0000 (17:33 +0100)]
Merge pull request #37764 from mgfritch/cephadm-no-container-init

cephadm: run containers using `--init` by default

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
4 years agoMerge pull request #39153 from mgfritch/cephadm-flake8
Sebastian Wagner [Mon, 8 Feb 2021 16:32:04 +0000 (17:32 +0100)]
Merge pull request #39153 from mgfritch/cephadm-flake8

mgr/cephadm: introduce flake8

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #39331 from ceph/revert-37954-wip-F46226-log-dq-throttle-limit-new
Kefu Chai [Mon, 8 Feb 2021 16:04:51 +0000 (00:04 +0800)]
Merge pull request #39331 from ceph/revert-37954-wip-F46226-log-dq-throttle-limit-new

Revert "msg,mon,common: log when DispatchQueue throttle limit is reached"

Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoqa/workunits/rbd: show snapshot deltas during stress test failure
Jason Dillaman [Mon, 8 Feb 2021 15:13:35 +0000 (10:13 -0500)]
qa/workunits/rbd: show snapshot deltas during stress test failure

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agoMerge pull request #39359 from tchaikov/wip-crimson-clang-build
Kefu Chai [Mon, 8 Feb 2021 15:03:16 +0000 (23:03 +0800)]
Merge pull request #39359 from tchaikov/wip-crimson-clang-build

crimson/os: fix build with clang

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
4 years agolibrbd/deep-copy: compute parent overlap before computing zero extents
Jason Dillaman [Fri, 5 Feb 2021 18:33:16 +0000 (13:33 -0500)]
librbd/deep-copy: compute parent overlap before computing zero extents

An object might not overlap with its parent so avoid injecting an empty
object to hide the non-existent parent.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agoqa/suites/rbd: add snapshot-based mirroring stress test
Jason Dillaman [Tue, 2 Feb 2021 20:28:48 +0000 (15:28 -0500)]
qa/suites/rbd: add snapshot-based mirroring stress test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agolibrbd/deep_copy: skip snap list if object is known to be clean
Jason Dillaman [Fri, 29 Jan 2021 15:44:38 +0000 (10:44 -0500)]
librbd/deep_copy: skip snap list if object is known to be clean

If the fast-diff indicates that the destination object should exist
and that it hasn't changed, there shouldn't be a need to issue the
snap list operation. Instead, just update the destination object map
to indicate the existence of the object.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
4 years agocrimson/os: capture labels using a new variable 39359/head
Kefu Chai [Mon, 8 Feb 2021 14:04:32 +0000 (22:04 +0800)]
crimson/os: capture labels using a new variable

structured-binding does not define variables, so we cannot use "[=]" to
implicitly capture them. define variables in the capture list instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: mark unused variable [[maybe_unused]]
Kefu Chai [Mon, 8 Feb 2021 13:54:46 +0000 (21:54 +0800)]
crimson/os: mark unused variable [[maybe_unused]]

to silence warning like

key_layout.h:516:10: warning: unused variable 'nspace_masked'
[-Wunused-variable]
    auto nspace_masked = string_view_masked_t::decode(nspace, delta);
         ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: do not define assignment operator for Value
Kefu Chai [Mon, 8 Feb 2021 13:52:07 +0000 (21:52 +0800)]
crimson/os: do not define assignment operator for Value

we cannot assign a reference.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/os: do not capture unused variable
Kefu Chai [Mon, 8 Feb 2021 13:45:02 +0000 (21:45 +0800)]
crimson/os: do not capture unused variable

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/common: mark ExitBarrier final
Kefu Chai [Mon, 8 Feb 2021 13:43:09 +0000 (21:43 +0800)]
crimson/common: mark ExitBarrier final

to silence warnings like:

crimson/common/operation.h:394:20: warning: class with destructor marked
'final' cannot be inherited from [-Wfinal-dtor-non-final-class]
    ~ExitBarrier() final {
                   ^

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoclient: use scoped_lock instead of unique_lock 39353/head
Xiubo Li [Mon, 8 Feb 2021 11:02:17 +0000 (19:02 +0800)]
client: use scoped_lock instead of unique_lock

Signed-off-by: Xiubo Li <xiubli@redhat.com>
4 years agoMerge pull request #39350 from tchaikov/wip-cephadm-unicode
Sebastian Wagner [Mon, 8 Feb 2021 10:47:11 +0000 (11:47 +0100)]
Merge pull request #39350 from tchaikov/wip-cephadm-unicode

cephadm: do not cast subnet to unicode

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #39266 from sebastian-philipp/cephadm-rm-keyring-on-failed-deploy
Sebastian Wagner [Mon, 8 Feb 2021 10:46:35 +0000 (11:46 +0100)]
Merge pull request #39266 from sebastian-philipp/cephadm-rm-keyring-on-failed-deploy

mgr/cephadm: cleanup daemon keyring on failed daemon deploy

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
4 years agoMerge pull request #38695 from pcuzner/orch-status-changes
Sebastian Wagner [Mon, 8 Feb 2021 10:15:19 +0000 (11:15 +0100)]
Merge pull request #38695 from pcuzner/orch-status-changes

mgr/orchestrator: minor improvements to orch status

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #39347 from xxhdx1985126/wip-crimson-pg-bug-fix
Kefu Chai [Mon, 8 Feb 2021 10:08:57 +0000 (18:08 +0800)]
Merge pull request #39347 from xxhdx1985126/wip-crimson-pg-bug-fix

crimson/osd: make sure ox be removed after all continuations are finished in PG::do_osd_ops

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
4 years agoRevert "msg,mon,common: log when DispatchQueue throttle limit is reached" 39331/head
Kefu Chai [Sat, 6 Feb 2021 06:47:56 +0000 (14:47 +0800)]
Revert "msg,mon,common: log when DispatchQueue throttle limit is reached"

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocephadm: do not cast subnet to unicode 39350/head
Kefu Chai [Mon, 8 Feb 2021 08:27:44 +0000 (16:27 +0800)]
cephadm: do not cast subnet to unicode

this change addresses a regression introduced by
fe4f4402fbcd87667613640f2808d5d0e07e749d, which was tested before
abd9287db0e4f4f7873864119f5ce62519af1d48 got merged. and the former was
merged after abd9287db0e4f4f7873864119f5ce62519af1d48. hence the
regression.

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agocrimson/osd: make sure ox be removed after all continuations are finished in PG:... 39347/head
Xuehan Xu [Mon, 8 Feb 2021 06:59:48 +0000 (14:59 +0800)]
crimson/osd: make sure ox be removed after all continuations are finished in PG::do_osd_ops

Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agoMerge pull request #39193 from tchaikov/wip-cephadm-py3
Kefu Chai [Mon, 8 Feb 2021 04:20:03 +0000 (12:20 +0800)]
Merge pull request #39193 from tchaikov/wip-cephadm-py3

cephadm: drop support to python<3.3

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #38897 from Daniel-Pivonka/mgroktostop
Kefu Chai [Mon, 8 Feb 2021 03:57:21 +0000 (11:57 +0800)]
Merge pull request #38897 from Daniel-Pivonka/mgroktostop

mgr/cephadm: ok-to-stop for mgr

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
4 years agoMerge pull request #39082 from ifed01/wip-ifed-meta-in-objectstore-tool
Kefu Chai [Mon, 8 Feb 2021 03:55:17 +0000 (11:55 +0800)]
Merge pull request #39082 from ifed01/wip-ifed-meta-in-objectstore-tool

tools/ceph-objectstore-tool: allow object operations on meta pgid

Reviewed-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39273 from tchaikov/wip-timer-mono
Kefu Chai [Mon, 8 Feb 2021 03:54:03 +0000 (11:54 +0800)]
Merge pull request #39273 from tchaikov/wip-timer-mono

common/Timer: use mono_clock for clock_t

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
4 years agoMerge pull request #39295 from tchaikov/wip-mgr-balancer-ann
Kefu Chai [Mon, 8 Feb 2021 03:52:21 +0000 (11:52 +0800)]
Merge pull request #39295 from tchaikov/wip-mgr-balancer-ann

pybind/mgr/balancer: add type annotations

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
4 years agoMerge pull request #39164 from tchaikov/wip-crimson-push-mov
Kefu Chai [Mon, 8 Feb 2021 01:39:35 +0000 (09:39 +0800)]
Merge pull request #39164 from tchaikov/wip-crimson-push-mov

crimson/osd: consume the push payload

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Xuehan Xu <xxhdx1985126@gmail.com>
4 years agoMerge pull request #39204 from TRYTOBE8TME/wip-rgw-create-bucket-dpp
Ali Maredia [Sun, 7 Feb 2021 17:07:14 +0000 (12:07 -0500)]
Merge pull request #39204 from TRYTOBE8TME/wip-rgw-create-bucket-dpp

src/rgw: Dpp addition to create bucket log

4 years agorgw: Dpp addition to create bucket log 39204/head
root [Wed, 13 Jan 2021 07:19:08 +0000 (12:49 +0530)]
rgw: Dpp addition to create bucket log

This commit adds dpp to the create bucket log.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
4 years agopybind/mgr/balancer: flake8 cleanups 39295/head
Kefu Chai [Thu, 4 Feb 2021 13:31:24 +0000 (21:31 +0800)]
pybind/mgr/balancer: flake8 cleanups

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/mgr/balancer: define Plan.{dump,show}()
Kefu Chai [Thu, 4 Feb 2021 13:15:27 +0000 (21:15 +0800)]
pybind/mgr/balancer: define Plan.{dump,show}()

as they are called by the commands

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/mgr/alerts: flake8 cleanups
Kefu Chai [Thu, 4 Feb 2021 05:00:57 +0000 (13:00 +0800)]
pybind/mgr/alerts: flake8 cleanups

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/mgr/balancer: define options using Option
Kefu Chai [Thu, 4 Feb 2021 07:32:35 +0000 (15:32 +0800)]
pybind/mgr/balancer: define options using Option

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/mgr/balancer: add type annotations
Kefu Chai [Thu, 4 Feb 2021 07:22:47 +0000 (15:22 +0800)]
pybind/mgr/balancer: add type annotations

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agopybind/mgr/balancer: define commands using CLICommand
Kefu Chai [Wed, 3 Feb 2021 05:44:10 +0000 (13:44 +0800)]
pybind/mgr/balancer: define commands using CLICommand

Signed-off-by: Kefu Chai <kchai@redhat.com>
4 years agoMerge pull request #39336 from AlexWangD/patch-4
Kefu Chai [Sun, 7 Feb 2021 12:07:01 +0000 (20:07 +0800)]
Merge pull request #39336 from AlexWangD/patch-4

doc: fix the spelling mistake of “Topicd”

Reviewed-by: Kefu Chai <kchai@redhat.com>