osd: Fix assert when checking missing version

Fixes: http://tracker.ceph.com/issues/21218
Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #19939 from liewegas/wip-osd-new-class

mon,osd: do not use crush_device_class file to initalize class for new osds

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20363 from dongbula/e-rados-get-put-command

make rados get/put/append command help txt clear

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

tools/rados: make rados get/put/append command help txt clearer

Fixes: http://tracker.ceph.com/issues/22958
Signed-off-by: lvshuhua <lvshuhua@cmss.chinamobile.com>

Merge pull request #20392 from adamemerson/wip-clangtastic-voyage

cmake, test/fs, client: fix build with clang

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20379 from tchaikov/wip-test-fix-freebsd

test/librados: s/invoke_result_t/result_of_t/

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #20360 from ifed01/wip-ifed-extents-cleanup

os/bluestore: cleanup around ExtentList, AllocExtent and bluestore_extent_t classes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20320 from rzarzynski/wip-common-md_config_cacher_t

common: introduce md_config_cacher_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20198 from wido/zabbix-health-checks

mgr/zabbix: Implement health checks

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20378 from tchaikov/wip-os-perf-stat-in-ns

osd: check feature bits when encoding objectstore_perf_stat_t

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20275 from badone/wip-osdmonitor-unused-function

mon/OSDMonitor: Comment out unused function

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20371 from tchaikov/wip-admin_socket_output

test/admin_socket_output: add --vstart=path/to/asok option

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #20227 from rzarzynski/wip-bs-final-aiocontexts

os/bluestore: mark derivatives of AioContext as final.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20376 from dillaman/wip-22873

librbd: auto-remove trash snapshots when image is deleted

Reviewed-by: Mykola Golub <mgolub@suse.com>

build: Use the right experimental library

Test whether we're building with libstdc++ or libc++.

Use stdc++filesystem if the former and c++experimental if the later.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

{client,test}: Explicitly initialize empty inode_t

Using an explicit {} initializer. This unbreaks Clang.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

librbd: auto-remove trash snapshots when image is deleted

Fixes: http://tracker.ceph.com/issues/22873
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librados: s/invoke_result_t/result_of_t/

to workaround buggy libc++ implementation. see
https://cplusplus.github.io/LWG/issue2807 . we should switch to
invoke_result_t once issue2807 is fixed in the latest release of
libc++.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: cleanup around ExtentList class.

Also move it out of bluestore_types.h.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: kill AllocExtent[Vector] to simplify code a bit

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #20387 from ifed01/wip-ifed-fio-build-fix

test/fio: fix build failure caused by sequencer replacement.

Merge pull request #20043 from Rubab-Syed/configuration_setting_browser

mgr/dashboard: add configuration setting browser

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20389 from trociny/wip-cli_generic

qa/workunits/rbd: fix cli_generic test_purge for rbd default format 1

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd/cli_generic: fix test_purge for rbd default format 1

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20364 from Songweibin/wip-fix-test-rbd

test/rbd: cli_generic fails if v1 image format or deep-flatten disabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

tools/ceph-objectstore-tool: do not fail on lack of OI_ATTR for an
object.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

test/fio: create superblock on mkfs to provide proper tools functioning

E.g. ceph-objectstore-tool requires that

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

test/fio: fix build failure caused by sequencer replacement.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #20372 from tchaikov/wip-mgr-doc

doc/mgr: remove non user-facing code from doc

Reviewed-by: John Spray <john.spray@redhat.com>

mgr/zabbix: Implement health checks in Zabbix module

Should make it easier for users to see why the module is failing.

Signed-off-by: Wido den Hollander <wido@42on.com>

test/rbd: cli_generic fails if v1 image format or deep-flatten disabled

Fixes: http://tracker.ceph.com/issues/22950
Signed-off-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #20382 from tchaikov/wip-fix-ftbfs-store-test

test/store_test: fix FTBFS as Sequencer is removed

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

check feature bits when encoding objectstore_perf_stat_t

OS_PERF_STAT_NS feature bit is added. it's sharing the same bit with
QOS_DMC.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/store_test: fix FTBFS as Sequencer is removed

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20294 from rzarzynski/wip-bs-drop-std_function

os/bluestore: avoid overhead of std::function in blob_t.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19232 from socketpair/precision

mgr: increase time resolution of Commit/Apply OSD latencies.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18804 from majianpeng/bluestore-collection-prealloc

os/bluestore: Prealloc memory avoid realloc in list_collection.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18494 from ifed01/wip-stupidalloc-fix2

os/bluestore: do not assert if BlueFS rebalance is unable to allocate sufficient space

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18343 from shinobu-x/sk-remove-osdmap

mon/PGMap: Remove unnecessary header

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #19954 into master

* refs/pull/19954/head:
test/encoding: refactor to avoid escaping shell magic
mds: minor refactor of SimpleLock
mds: track Capability in mempool
mds: move CInode container members to mempool
mds: move CDentry container members to mempool
mds: move CDir container members to mempool
mds: put MDSCacheObject compact_map in mempool
common: use size_t for object size
mds: convert to allocator agnostic string_view
mds: simplify initialization
compact_*: support mempool allocated containers

Reviewed-by: Zheng Yan <zyan@redhat.com>

osd: PrimaryLogPG::do_osd_ops() uses md_config_cacher_t.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #20007 from mogeb/steady-clock-librbd

librbd: use steady clock to measure elapsed time in AioCompletion

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20008 from mogeb/steady-clock-tools-rbd

tools/rbd: use steady clock in bencher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20218 from shun-s/wip-speedup-diskusage-resize

librbd: speed up object map disk usage and resize

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20311 from Songweibin/wip-group-snap-ls

rbd: do not show title if there is no group snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20349 from trociny/wip-22932

rbd-mirror: fix potential infinite loop when formatting status message

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #20310 into master

* refs/pull/20310/head:
qa: adjust cephfs full test for kclient

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20148 into master

* refs/pull/20148/head:
mds: reset connection's priv when marking down connection
mds: fix session reference leak

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20155 into master

* refs/pull/20155/head:
osdc/Journaler: make sure flush() writes enough data

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20190 into master

* refs/pull/20190/head:
mon: allow removal of tier of ec overwritable pool

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge PR #20200 into master

* refs/pull/20200/head:
client: add cap_dirtier_uid/gid to CapSnap

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #20246 into master

* refs/pull/20246/head:
mds: remove extra 0x in ino prints
mds: print inode number not CInode ptr

mon/OSDMonitor: accept crush_device_class in 'osd new' json input

If provided, set the OSD device_class at OSD creation time.  This is
simpler than writing a file that the OSD has to read in and use to
set its initial device class, and also avoids a bit of sticky state
at the OSD that will make it keep trying to reset its device class on
startup if it ever gets cleared.

Note that we now ignore json input fields we don't understand, so remove
a test case.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/mgr: remove none user-facing code from doc

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/test_admin_socket_output: allow different asok directory

the asok files are not necessary located in ./out, actually, they are
put in $TMPDIR/ceph-asok.$random_stuff .

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/admin_socket_output: no need to create a copy of fs::path

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20367 from ceph/simple-custom-cluster

ceph-volume: adds custom cluster name support to simple

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: use a custom cluster name in simple functional tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20305 from xiexingguo/wip-more-balancer-fixes

pybind/mgr/balancer: more specific command outputs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20355 from luomuyao/typo

rgw: fix a typo in comment

Reviewed-by: Jos Collin <jcollin@redhat.com>

test/encoding: refactor to avoid escaping shell magic

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: minor refactor of SimpleLock

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: track Capability in mempool

Partial-fix: http://tracker.ceph.com/issues/21402

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: move CInode container members to mempool

Partial-fix: http://tracker.ceph.com/issues/21402

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

os/bluestore: avoid overhead of std::function in blob_t.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: fix a typo in comment

Signed-off-by: luomuyao <luo.muyao@zte.com.cn>

mon/OSDMonitor: Remove unused functions

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #20161 from tchaikov/wip-20086

test/librados: be more tolerant with timed lock tests

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #20173 from liewegas/wip-os-ch

os: unify Sequencer and CollectionHandle

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph-volume: set conf.cluster name during simple activate

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20144 from adamemerson/wip-cache-register

Cache Register!

Provide the ability to examine and delete elements from the cache.

rgw: Add cache introspection and manipulation

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20176 from dillaman/wip-librbd-clone-v2-1

librbd: initial hooks for clone v2 support

Reviewed-by: Mykola Golub <mgolub@suse.com>

rgw: Add cache introspection and manipulation

Provide the ability to examine and delete elements from the cache.

Fixes: http://tracker.ceph.com/issues/22603
Fixes: http://tracker.ceph.com/issues/22604
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

rbd-mirror: fix potential infinite loop when formatting status message

The improvements include:

- tag_tid values should always be increasing, so loop only if
  master.tag_tid > mirror_tag_tid in calculate_behind_master_or_send_update;
- in send_update_tag_cache don't refetch a tag if it is already in the
  cache;
- make fake tags with tag_data.predecessor.tag_tid set to zero;
- make sure the new tag is inserted to the cache if an old entry
  with this id happens to exist.

Fixes: http://tracker.ceph.com/issues/22932
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20318 from ovh/bp-limit-normalize-key

common/config: limit calls to normalize_key_name

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19958 from renhwztetecs/renhw-wip-osd-sighup

osd: Sign in early SIGHUP signal

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mds: move CDentry container members to mempool

Partial-fix: http://tracker.ceph.com/issues/21402

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: move CDir container members to mempool

Partial-fix: http://tracker.ceph.com/issues/21402

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: put MDSCacheObject compact_map in mempool

Partial-fix: http://tracker.ceph.com/issues/21402

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

common: use size_t for object size

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: convert to allocator agnostic string_view

This is necessary to allow many interfaces to take mempool allocated strings.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #20264 from ceph/wip-rm22620

ceph-volume dmcrypt support for simple

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20307 from tchaikov/wip-std-filesystem

test/admin_socket_output: switch to std::experimental::filesystem

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #20324 from tchaikov/wip-cmake-build-boost

cmake: fix the include dir for building boost::python

Reviewed-by: Piotr Dałek <piotr.dalek@corp.ovh.com>

pybind/mgr/balancer: cancel plan if distribution is already perfect

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

pybind/mgr/balancer: allow customized optimization threshold

In practice it is hard to achieve a 100% (perfect) balanced distribution.
This patch introduces a user visiable threshold configuration here, which
allow user to define the minimal deviation to trigger an optimization.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

pybind/mgr/balancer: be more specific on creating an optimization plan

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #20347 from Songweibin/wip-osd-cleanup

osd: drop redundant comment

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: drop redundant comment

Signed-off-by: songweibin <song.weibin@zte.com.cn>

Merge PR #20288 into master

* refs/pull/20288/head:
ptl-tool: open githubmap with utf-8 code

Reviewed-by: Jos Collin <jcollin@redhat.com>

mds: remove extra 0x in ino prints

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

test/objectstore/TestObjectStoreState: make objects match collection

- start pool at 1
- make hash fall within pg (just match it for now)

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: remove void *osr from Transaction

No longer used!  (It was only needed for an ugly FileStore hack.)

Signed-off-by: Sage Weil <sage@redhat.com>

os/filestore: do not rely on ObjectStore::Transaction::osr

This was used only to name the osr for the purposes of the tracepoint. We
can do that ourselves!

Signed-off-by: Sage Weil <sage@redhat.com>

osd: drop explicit calls to osr->flush

These are no longer needed as the implementations do it themselves.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: drop unnecessary flush() on shutdown

The ObjectStore implementation should handle this.

Signed-off-by: Sage Weil <sage@redhat.com>

os: make implementation flush before collection_{list,empty}

This takes the (awkward) burden off the caller.

Signed-off-by: Sage Weil <sage@redhat.com>

os/ObjectStore: make all read operations CollectionHandle-based

Drop the coll_t shims, and convert the last few that weren't already
handle-based.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20313 from guzhongyan/fix-22350

osd/OSDMap: misleading message in print_oneline_summary()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20317 from liewegas/wip-pg-scrub-priority

osd/PG: pass scrub priority to replica

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>