os/bluestore: tune deferred_batch_ops separately for hdd and ssd

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14420 from optimistyzy/329

os/bluestore/NVMEDevice: Add multiple thread support for SPDK I/O thread

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #14458 from linuxbox2/wip-rgw_file-readahead

rgw_file: support readdir cb type hints (plus fixes)

Merge pull request #13966 from jcsp/wip-client-logmsg

client: specify inode in get_caps log message

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #14087 from jcsp/wip-doc-fscid

doc: how to specify filesystem for cephfs clients

Reviewed-by: Yan, Zheng <zyan@redhat.com>

rgw_file: don't expire directories being read

If a readdir expire event turns out to be older than last_readdir,
just reschedule it (but actually, we should just discard it, as
another expire event must be in queue.

Fixes: http://tracker.ceph.com/issues/19625
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file:  chunked readdir

Adjust readdir callback path for new nfs-ganesha chunked readdir,
including changes to respect the result of callback to not
continue.

Pending introduction of offset name hint, our caller will just be
completely enumerating, so it is possible to remove the offset map
and just keep a last offset.

Fixes: http://tracker.ceph.com/issues/19624
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file: introduce rgw_lookup type hints

The new type hints optimize object type deduction, when the
rgw_lookup is called from an rgw_readdir callback.

Fixes: http://tracker.ceph.com/issues/19623
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #14468 from tchaikov/wip-show-me-fake-plugin

qa/workunits/ceph-helpers: display rejected string

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14234 from jcsp/wip-19406-dirhash

tools/cephfs: set dir_layout when injecting inodes

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #14472 from joscollin/wip-warning-test

test: add explicit braces to avoid ambiguous ‘else’ and to silence warnings

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14495 from tchaikov/wip-cmake-ctags-without-repo

cmake: do not try to add submodule to exclude list if .git is not around

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14404 from tchaikov/wip-another-mgr-command

mon,mgr: move reweight-by-* to mgr

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: do not try to add submodule to exclude list if .git is not around

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14445 from dillaman/wip-rbd-import-diff

rbd: import-diff should discard any zeroed extents

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14474 from Wilhelmshaven/fix_non-exist_string

rgw: avoid using null pointer in rgw_file.cc

Merge pull request #14482 from trociny/wip-qa-diff

qa/workunits/rbd: diff.sh failed removing nonexistent file

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tools/cephfs: set dir_layout when injecting inodes

When we left this as zero, the MDS would interpret it was HASH_LINUX
rather than the default HASH_RJENKINS.  Potentially that
could cause problems if there perhaps were already dirfrags in
the metadata pool that were set up using rjenkins.  Mainly
it just seems more appropriate to explicitly set this field
rather than hit the fallback behaviour.

Related: http://tracker.ceph.com/issues/19406
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #14463 from trociny/wip-19451

pybind/rbd: add image metadata methods

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14471 from trociny/wip-19588

librbd: fix rbd_metadata_list and rbd_metadata_get

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14484 from dillaman/wip-librados-test-stub

test/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14375 from cbodley/wip-rgw-fetch-versioned-obj

qa/suites: add mon-reweight-min-pgs-per-osd = 4

so we can run reweight commands in a small cluster with 3 OSD and 27
PGs.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14476 from wjwithagen/wip-wjw-freebsd-ceph-disk-call-rc.d

ceph-disk/ceph_disk/main.py: fix calling of the bsdrc init scripts

Reviewed-by: Loic Dachary <ldachary@redhat.com>

test/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value

Fixes: http://tracker.ceph.com/issues/19597
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14399 from ifed01/wip-bluestore-better-better-reuse

os/bluestore: refactor small write handling to reuse blob more effect…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14422 from tchaikov/wip-19427

mon/MonClient: make get_mon_log_message() atomic

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14439 from adamemerson/wip-warning-sign

misc: Warning Elimination

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/workunits/rbd: diff.sh failed removing nonexistent file

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14347 from ukernel/wip-19501

mds: make C_MDSInternalNoop::complete() delete 'this'

Reviewed-by: John Spray <john.spray@redhat.com>

bluestore, NVMEDevice: Add multiple thread support for SPDK I/O thread

Previously, we only have one thread to do SPDK I/O, and this patch
adds the multiple thread support.

In this first version, we use the tid of the thread to map the I/Os
of this thread to the corresponding SPDK I/O thread.

Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

rbd: import-diff should discard any zeroed extents

Sparse (zeroed) extents cannot be safely skipped. Instead, the
zeroed extent should be discarded from the image to ensure
the import remains consistent with the export.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14460 from dillaman/wip-qa-rbd-cli

qa/workunits: corrected issues with RBD cli test

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

ceph-disk/ceph_disk/main.py: fix calling of the bsdrc init scripts

 - rc.d/ceph does not really like the extra stop/start as las cmd.
 - And make Start and Stop look similar.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

test: add explicit braces to avoid ambiguous ‘else’ and to silence warnings

The following warning appears during make for several files in the test submodule:
warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wdangling-else]

Fixed the review comment too.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #14467 from badone/wip-debug-mon-features-api-removal-cleanup

mon: Tidy up removal of debug mon features

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/workunits/ceph-helpers: do not error out if is_clean

it would be a race otherwise, because we cannot be sure that the cluster
pgs are not all clean or not when run_osd() returns, but we can be sure
that they are expected to active+clean after a while. that's what
wait_for_clean() does.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: avoid using null pointer in rgw_file.cc

When rgw_fh doesn't exist, we should not use rgw_fh->full_object_name().

Signed-off-by: lihongjie <lihongjie@cmss.chinamobile.com>

pybind/rbd: add image metadata methods

Fixes: http://tracker.ceph.com/issues/19451
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: fix rbd_metadata_list and rbd_metadata_get

- properly check for val_len in rbd_metadata_list
- don't expect output buffers are zero pre-filled

Fixes: http://tracker.ceph.com/issues/19588
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

qa/workunits/ceph-helpers: display rejected string

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14361 from ShiqiCooperation/master

mailmap: shiqi affiliation

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14446 from badone/wip-cppcheck-errors

Wip cppcheck errors

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: Suppress cppcheck error

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14419 from badone/wip-doc-pg-states-min_size

docs: Clarify the relationship of min_size to EC pool recovery

Reviewed-by: Sage Weil <sage@redhat.com>

mon: Tidy up removal of debug mon features

545bc83 removed most of the plumbing for the debug mon features admin
socket commands but failed to remove the register/unregister command
pairs. This means the monitor asserts if an attempt is made to use any
of these commands.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14290 from wjwithagen/wip-wjw-freebsd-procprefix

core: Prefix /proc/ with FreeBSD emulation

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14362 from majianpeng/fs-remove-redundant-code

os/fs/FS.cc: remove the redundant code.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14462 from pdinnen/patch-2

doc: correct arguments for ceph tell osd.N bench

Reviewed-by: Sage Weil <sage@redhat.com>

doc: correct arguments for ceph tell osd.N bench

Signed-off-by: Patrick Dinnen pdinnen@gmail.com

Merge pull request #14461 from dmick/master

stop.sh: boilerplate error (don't stop mon when stopping mgr)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

stop.sh: boilerplate error (don't stop mon when stopping mgr)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #14376 from cbodley/wip-rgw-forward-canned-acl

rgw: fix for EINVAL errors on forwarded bucket put_acl requests
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #13105 from rjfd/wip-18481

librbd: deferred image deletion

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits: corrected issues with RBD cli test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14355 from cbodley/wip-19505

rgw: allow larger payload for period commit

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14319 from cbodley/wip-19488

rgw: multisite: fixes for zonegroup redirect

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14343 from Liuchang0812/wip-rgw-read-user-buckets

rgw: update is_truncated in function rgw_read_user_buckets

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14438 from theanalyst/fix/build-doc

script: build-doc/serve-doc fixes

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14455 from optimistyzy/411_1

os/bluestore/NVMEDevice: fix the compilation issue for collect_metadata

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13698 from ukernel/wip-multimds-misc

mds: miscellaneous multimds fixes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14453 from trociny/wip-test_mock_InstanceWatcher

test/rbd_mirror: race in TestMockInstanceWatcher on destroy

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

pybind: librbd: added trash API to python RBD binding

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/workunits/rbd: cli testing of rbd trash commands

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: added image-id optional to Info command

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: Added image-id optional to Snap commands

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: added image_id param to Utils::init_and_open_image

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: added group image remove by id to CLI

Signed-off-by: Ricardo Dias <rdias@suse.com>

librbd: added group image remove by id to API

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: added CLI support for trash operations

Signed-off-by: Ricardo Dias <rdias@suse.com>

librbd: added trash methods to API

Signed-off-by: Ricardo Dias <rdias@suse.com>

librbd: open image gets name from directory or trash

Signed-off-by: Ricardo Dias <rdias@suse.com>

librbd: added open an image by id to API

Signed-off-by: Ricardo Dias <rdias@suse.com>

cls_rbd: methods to manipulate the rbd_trash object

Signed-off-by: Ricardo Dias <rdias@suse.com>

bluestore/NVMEDevice: fix the compilation issue for collect_metadata

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

test/rbd_mirror: race in TestMockInstanceWatcher on destroy

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14295 from theanalyst/release/10.2.7/notes

doc: release notes for v10.2.7 Jewel

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14441 from theanalyst/doc/changelog/v10.2.7

doc: add changelog for v10.2.7

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #14426 from tchaikov/wip-13562-kefu

misc: SCA fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: Silence cppcheck memory leak warnings in ceph_test_rados_api_io

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

denc: Silence cppcheck uninitialised variable errors

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

common: Restructure cppcheck suppresions

Some of the suppressions are currently redundant and some need to be
changed.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

crypto: Suppress cppcheck error

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

blkdev: Suppress cppcheck error

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

librados: Since a pragma push was removed, get rid of that pop

Don't you think the plural of pragma should be pragmata?

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

cmake: Make -Wno-unknown-pragmas common to all compilers

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

bluestore: Remove unused expression in statement

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

filestore: Fix uninitialized return

In some error conditions we were returning the value of an
uninitialized variable.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rbd: Remove pessimizing move

Do not std::move returned values.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

cls: Fix order of operations of std::string constructor

Presumably the intent was to create a string with 128 '1's, rather than
a string with 49 '\200's.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Remove pessimizing move of return values

C++ performs copy elision of return values if:

1. The value is a local declared in the function (i.e. not a
   parameter)
2. The type of the value in the return statement is, modulo
   const-value qualification, identical to the return type of the
   function.

An RValue reference to a type is not identical, modulo const-value
qualification, to the type it is a reference to.

Do not std::move local values in the return statement unless the value
is a parameter or you will make the compiler perform a copy it would
iotherwise avoid.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rgw: Make AuthResult::get_completer return rvalue reference

Since we are explicitly moving the completer out of the AuthResult
and allowing the client to have ownership of it, simply return an
rvalue reference rather than moving into a temporary that is itself
returned.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

msg: Remove pessimizing move

Do not std::move the result of a function returning a value.

(Once could reasonably std::move the result of a function returning an
lvalue reference, but *not* that of one returning a value.)

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

osd: The shard_id_t constructor takes an int8_t

So pass it -1 rather than 255 to avoid an implicit conversion warning.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

monc: Remove pessimizing move

Calling std::move on on a function returning a value can interfere
with RVO. At best it has no effect, but can actively interfere with
Copy/Move Elision and make code less efficient.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #13940 from Adirl/out

msg/async/rdma: Update fix broken compilation

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #14409 from Adirl/conn_info

msg/async/rdma: RDMA-CM, Pass specific ConnMgr info in constructor

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #14410 from Adirl/get_device

msg/async/rdma: RDMA-CM, get_device() by ibv_context

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13073 from chardan/jfw-wip-15475-admin_heap_command

osd: add "heap *" admin command

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #14235 from nvedant07/bug_fix_3

rados: more info added to pool deletion error

Reviewed-by: Sage Weil <sage@redhat.com>