osd: pg-remap -> pg-upmap

'remap' is to non-specific a name.  In particular, it
sounds like it is related to the 'remapped' PG state
but in reality it is not related.

'upmap' or 'pg-upmap' is more specific: it maps a pgid
to the 'up' set value (or item)

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14435 from liewegas/wip-bluestore-deferred

os/bluestore: tune deferred_batch_ops separately for hdd and ssd

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #14489 from liewegas/wip-bluestore-fix-flush

os/bluestore: fix narrow osr->flush() race

Merge pull request #13859 from jcsp/wip-19204

mds: shut down finisher before objecter

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #14505 from jcsp/wip-19566

mds: fix mgrc shutdown

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #14398 from jcsp/wip-19551

mon: emit cluster log messages on MDS health changes

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add log whitelists for MDS health messages

Now that we send these to the cluster log, we must
whitelist them in the tests that exercise those
unhealthy states.

Fixes: http://tracker.ceph.com/issues/19551
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #10851 from atheism/rgw-s3-doc-update

doc/radosgw/s3/cpp.rst: update usage of libs3 APIs to make the examples work

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14469 from badone/wip-return-error-on-missing-client-id

mds: Return error message instead of asserting

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14508 from smithfarm/wip-doc-dev

doc: dev guide: how to run s3-tests locally against vstart

Reviewed-by: Loic Dachary <ldachary@redhat.com>

os/bluestore: tune deferred_batch_ops separately for hdd and ssd

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14420 from optimistyzy/329

os/bluestore/NVMEDevice: Add multiple thread support for SPDK I/O thread

Reviewed-by: Haomai Wang <haomai@xsky.com>

doc: dev guide: how to run s3-tests locally against vstart

Add a bunch of verbiage to the Developer Guide

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #14458 from linuxbox2/wip-rgw_file-readahead

rgw_file: support readdir cb type hints (plus fixes)

mds: shut down finisher before objecter

Some of the finisher contexts would try to call into Objecter.
We mostly are protected from this by mds_lock+the stopping
flag, but at the Filer level there's no mds_lock, so in the
case of file size probing we have a problem.

Fixes: http://tracker.ceph.com/issues/19204
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #13966 from jcsp/wip-client-logmsg

client: specify inode in get_caps log message

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #14087 from jcsp/wip-doc-fscid

doc: how to specify filesystem for cephfs clients

Reviewed-by: Yan, Zheng <zyan@redhat.com>

rgw_file: don't expire directories being read

If a readdir expire event turns out to be older than last_readdir,
just reschedule it (but actually, we should just discard it, as
another expire event must be in queue.

Fixes: http://tracker.ceph.com/issues/19625
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

mon: emit cluster log messages on MDS health changes

Previously, when we got a beacon that updated the health
metrics for an MDS, the user would just see mysterious-looking
cluster log messages indicating a rising fsmap epoch number.

It would be good to do this for health messages in general at
some point, but for now just do it for the MDS ones.

Fixes: http://tracker.ceph.com/issues/19551
Signed-off-by: John Spray <john.spray@redhat.com>

mds: fix mgrc shutdown

Were previously only tearing MgrClient down when not
holding a rank, leading to it trying to continue
to run after monclient was shut down.

Fixes: http://tracker.ceph.com/issues/19566
Signed-off-by: John Spray <john.spray@redhat.com>

rgw_file:  chunked readdir

Adjust readdir callback path for new nfs-ganesha chunked readdir,
including changes to respect the result of callback to not
continue.

Pending introduction of offset name hint, our caller will just be
completely enumerating, so it is possible to remove the offset map
and just keep a last offset.

Fixes: http://tracker.ceph.com/issues/19624
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file: introduce rgw_lookup type hints

The new type hints optimize object type deduction, when the
rgw_lookup is called from an rgw_readdir callback.

Fixes: http://tracker.ceph.com/issues/19623
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #14468 from tchaikov/wip-show-me-fake-plugin

qa/workunits/ceph-helpers: display rejected string

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14234 from jcsp/wip-19406-dirhash

tools/cephfs: set dir_layout when injecting inodes

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #14472 from joscollin/wip-warning-test

test: add explicit braces to avoid ambiguous ‘else’ and to silence warnings

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14495 from tchaikov/wip-cmake-ctags-without-repo

cmake: do not try to add submodule to exclude list if .git is not around

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14404 from tchaikov/wip-another-mgr-command

mon,mgr: move reweight-by-* to mgr

Reviewed-by: Sage Weil <sage@redhat.com>

cmake: do not try to add submodule to exclude list if .git is not around

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14445 from dillaman/wip-rbd-import-diff

rbd: import-diff should discard any zeroed extents

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14474 from Wilhelmshaven/fix_non-exist_string

rgw: avoid using null pointer in rgw_file.cc

mds: Return error message instead of asserting

For "session evict" admin socket command return an error message when we
receive an invalid/missing client_id parameter rather than asserting.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14482 from trociny/wip-qa-diff

qa/workunits/rbd: diff.sh failed removing nonexistent file

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

tools/cephfs: set dir_layout when injecting inodes

When we left this as zero, the MDS would interpret it was HASH_LINUX
rather than the default HASH_RJENKINS.  Potentially that
could cause problems if there perhaps were already dirfrags in
the metadata pool that were set up using rjenkins.  Mainly
it just seems more appropriate to explicitly set this field
rather than hit the fallback behaviour.

Related: http://tracker.ceph.com/issues/19406
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #14463 from trociny/wip-19451

pybind/rbd: add image metadata methods

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14471 from trociny/wip-19588

librbd: fix rbd_metadata_list and rbd_metadata_get

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

os/bluestore: fix narrow osr->flush() race

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14484 from dillaman/wip-librados-test-stub

test/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14375 from cbodley/wip-rgw-fetch-versioned-obj

qa/suites: add mon-reweight-min-pgs-per-osd = 4

so we can run reweight commands in a small cluster with 3 OSD and 27
PGs.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14476 from wjwithagen/wip-wjw-freebsd-ceph-disk-call-rc.d

ceph-disk/ceph_disk/main.py: fix calling of the bsdrc init scripts

Reviewed-by: Loic Dachary <ldachary@redhat.com>

test/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value

Fixes: http://tracker.ceph.com/issues/19597
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14399 from ifed01/wip-bluestore-better-better-reuse

os/bluestore: refactor small write handling to reuse blob more effect…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14422 from tchaikov/wip-19427

mon/MonClient: make get_mon_log_message() atomic

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14439 from adamemerson/wip-warning-sign

misc: Warning Elimination

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/workunits/rbd: diff.sh failed removing nonexistent file

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14347 from ukernel/wip-19501

mds: make C_MDSInternalNoop::complete() delete 'this'

Reviewed-by: John Spray <john.spray@redhat.com>

bluestore, NVMEDevice: Add multiple thread support for SPDK I/O thread

Previously, we only have one thread to do SPDK I/O, and this patch
adds the multiple thread support.

In this first version, we use the tid of the thread to map the I/Os
of this thread to the corresponding SPDK I/O thread.

Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

rbd: import-diff should discard any zeroed extents

Sparse (zeroed) extents cannot be safely skipped. Instead, the
zeroed extent should be discarded from the image to ensure
the import remains consistent with the export.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14460 from dillaman/wip-qa-rbd-cli

qa/workunits: corrected issues with RBD cli test

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

ceph-disk/ceph_disk/main.py: fix calling of the bsdrc init scripts

 - rc.d/ceph does not really like the extra stop/start as las cmd.
 - And make Start and Stop look similar.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

test: add explicit braces to avoid ambiguous ‘else’ and to silence warnings

The following warning appears during make for several files in the test submodule:
warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wdangling-else]

Fixed the review comment too.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #14467 from badone/wip-debug-mon-features-api-removal-cleanup

mon: Tidy up removal of debug mon features

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/workunits/ceph-helpers: do not error out if is_clean

it would be a race otherwise, because we cannot be sure that the cluster
pgs are not all clean or not when run_osd() returns, but we can be sure
that they are expected to active+clean after a while. that's what
wait_for_clean() does.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: avoid using null pointer in rgw_file.cc

When rgw_fh doesn't exist, we should not use rgw_fh->full_object_name().

Signed-off-by: lihongjie <lihongjie@cmss.chinamobile.com>

pybind/rbd: add image metadata methods

Fixes: http://tracker.ceph.com/issues/19451
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: fix rbd_metadata_list and rbd_metadata_get

- properly check for val_len in rbd_metadata_list
- don't expect output buffers are zero pre-filled

Fixes: http://tracker.ceph.com/issues/19588
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

qa/workunits/ceph-helpers: display rejected string

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14361 from ShiqiCooperation/master

mailmap: shiqi affiliation

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14446 from badone/wip-cppcheck-errors

Wip cppcheck errors

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: Suppress cppcheck error

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14419 from badone/wip-doc-pg-states-min_size

docs: Clarify the relationship of min_size to EC pool recovery

Reviewed-by: Sage Weil <sage@redhat.com>

mon: Tidy up removal of debug mon features

545bc83 removed most of the plumbing for the debug mon features admin
socket commands but failed to remove the register/unregister command
pairs. This means the monitor asserts if an attempt is made to use any
of these commands.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #14290 from wjwithagen/wip-wjw-freebsd-procprefix

core: Prefix /proc/ with FreeBSD emulation

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14362 from majianpeng/fs-remove-redundant-code

os/fs/FS.cc: remove the redundant code.

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #14462 from pdinnen/patch-2

doc: correct arguments for ceph tell osd.N bench

Reviewed-by: Sage Weil <sage@redhat.com>

doc: correct arguments for ceph tell osd.N bench

Signed-off-by: Patrick Dinnen pdinnen@gmail.com

Merge pull request #14461 from dmick/master

stop.sh: boilerplate error (don't stop mon when stopping mgr)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

stop.sh: boilerplate error (don't stop mon when stopping mgr)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #14376 from cbodley/wip-rgw-forward-canned-acl

rgw: fix for EINVAL errors on forwarded bucket put_acl requests
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #13105 from rjfd/wip-18481

librbd: deferred image deletion

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits: corrected issues with RBD cli test

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #14355 from cbodley/wip-19505

rgw: allow larger payload for period commit

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14319 from cbodley/wip-19488

rgw: multisite: fixes for zonegroup redirect

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #14343 from Liuchang0812/wip-rgw-read-user-buckets

rgw: update is_truncated in function rgw_read_user_buckets

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14438 from theanalyst/fix/build-doc

script: build-doc/serve-doc fixes

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14455 from optimistyzy/411_1

os/bluestore/NVMEDevice: fix the compilation issue for collect_metadata

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13698 from ukernel/wip-multimds-misc

mds: miscellaneous multimds fixes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #14453 from trociny/wip-test_mock_InstanceWatcher

test/rbd_mirror: race in TestMockInstanceWatcher on destroy

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

pybind: librbd: added trash API to python RBD binding

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/workunits/rbd: cli testing of rbd trash commands

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: added image-id optional to Info command

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: Added image-id optional to Snap commands

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: added image_id param to Utils::init_and_open_image

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: added group image remove by id to CLI

Signed-off-by: Ricardo Dias <rdias@suse.com>

librbd: added group image remove by id to API

Signed-off-by: Ricardo Dias <rdias@suse.com>

rbd: added CLI support for trash operations

Signed-off-by: Ricardo Dias <rdias@suse.com>

librbd: added trash methods to API

Signed-off-by: Ricardo Dias <rdias@suse.com>

librbd: open image gets name from directory or trash

Signed-off-by: Ricardo Dias <rdias@suse.com>

librbd: added open an image by id to API

Signed-off-by: Ricardo Dias <rdias@suse.com>

cls_rbd: methods to manipulate the rbd_trash object

Signed-off-by: Ricardo Dias <rdias@suse.com>

bluestore/NVMEDevice: fix the compilation issue for collect_metadata

Signed-off-by: optimistyzy <optimistyzy@gmail.com>

test/rbd_mirror: race in TestMockInstanceWatcher on destroy

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #14295 from theanalyst/release/10.2.7/notes

doc: release notes for v10.2.7 Jewel

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #14441 from theanalyst/doc/changelog/v10.2.7

doc: add changelog for v10.2.7

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #14426 from tchaikov/wip-13562-kefu

misc: SCA fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

test: Silence cppcheck memory leak warnings in ceph_test_rados_api_io

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

denc: Silence cppcheck uninitialised variable errors

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

common: Restructure cppcheck suppresions

Some of the suppressions are currently redundant and some need to be
changed.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

crypto: Suppress cppcheck error

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

blkdev: Suppress cppcheck error

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>