Merge pull request #20423 from dillaman/wip-22945

librbd: new tag should use on-disk committed position

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20442 from JordanHus/master

doc: Fixes a spelling error and a broken hyperlink.

Reviewed-by: John Wilkins

doc: Fixes a spelling error and a broken hyperlink.

Signed-off-by: Jordan Hus <jordan.hus@dicecorp.com>

journal: flush commit positions should wait for refresh

Fixes: http://tracker.ceph.com/issues/22945
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: support concurrent refresh requests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19859 from yaozongyou/rgw-remove-register-warning

rgw: silence not allow register storage class specifier warning

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20187 from wido/influxdb-connection-error

mgr/influx: Various fixes and improvements

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20265 from ceph/wip-rm22876

ceph-volume: lvm zap will unmount osd paths used by zapped devices

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #20406 from dillaman/wip-22979

librbd: fix issues discovered in clone v2 during upgrade tests

Reviewed-by: Mykola Golub <mgolub@suse.com>

ceph-volume: use terminal.MultiLogger in `lvm zap`

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: use api helper methods for pvs and lvs in lvm zap

This is just a cleaner way of doing this instead of using PVolumes
and Volumes directly.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: fix documentation typos for lvm zap

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: when zapping unmount osd directories

If you zap an lv, device or partition and it's currently mounted as a
ceph osd directory then it will be unmounted so the zap can complete.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume: adds a util.system.unmount function

This function will unmount the given path

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20427 from ceph/wip-rm22988

ceph-volume use realpath when checking mounts

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #19263 into master

* refs/pull/19263/head:
qa: ignore bad backtrace cluster wrn
qa/cephfs: Add tests to validate scrub functionality
cephfs: Add option to load invalid metadata from disk
cephfs: Reset scrub data when inodes move

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa: ignore bad backtrace cluster wrn

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

ceph-volume tests verify realpath usage in mounts

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.system optionally use realpath on devices

To check a mounted device it is needed to verify on a combination of
realpath and plain devices against realpath and plain paths. In LVM, two
different paths might refer to the same devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge PR #19957 into master

* refs/pull/19957/head:
client: fixup parallel calls to ceph_ll_lookup_inode() in NFS FASL

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #20336 into master

* refs/pull/20336/head:
client: re-use free snap tag

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20373 into master

* refs/pull/20373/head:
client: clamp I/O sizes to INT_MAX when we can't return larger values
test: new testcase for ceph_ll_readv and ceph_ll_writev
client: hook up ceph_ll_readv and ceph_ll_writev
client: type safety cleanup for _read and _write codepaths

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #20386 into master

* refs/pull/20386/head:
mds: optimize getattr file size

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/cephfs: Add tests to validate scrub functionality

Add tests to ensure the scrub operation is not adversly affected
by certain metadata pathologies.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>

Merge pull request #20147 from smithfarm/wip-15904

 tools: rados: make -f be --format for consistency with ceph tool

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20422 from cbodley/wip-check-cls-rgw

cmake: 'make check' builds radosgw and its cls dependencies

Reviewed-by: Ali Maredia <amaredia@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

cmake: 'make check' builds radosgw and its cls dependencies

Signed-off-by: Casey Bodley <cbodley@redhat.com>

librbd: flush journal commit position after stopping replay

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librbd: disable clone v2 for negative test cases

Fixes: http://tracker.ceph.com/issues/22979
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls/rbd: don't clear CLONE_CHILD op feature if still a child

After removing the last snapshot linked to a parent image,
don't clear the CLONE_CHILD op feature bit if the image HEAD
is still linked to the parent.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19939 from liewegas/wip-osd-new-class

mon,osd: do not use crush_device_class file to initalize class for new osds

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20363 from dongbula/e-rados-get-put-command

make rados get/put/append command help txt clear

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

client: clamp I/O sizes to INT_MAX when we can't return larger values

We have several API functions that allow the caller to request I/Os
larger than INT_MAX bytes, but that return an int. Ensure that we don't
try to do more I/O than we can represent in the return value.

Tracker: http://tracker.ceph.com/issues/22948
Signed-off-by: Jeff Layton <jlayton@redhat.com>

test: new testcase for ceph_ll_readv and ceph_ll_writev

Just copy the PreadvPwritev test and alter accordingly.

Tracker: http://tracker.ceph.com/issues/22948
Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: hook up ceph_ll_readv and ceph_ll_writev

Break the core of _preadv_pwritev out into a function that takes a Fh.
Make _preadv_pwritev into a wrapper around that.

Then add in plumbing for ceph_ll_readv and ceph_ll_writev.

Tracker: http://tracker.ceph.com/issues/22948
Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: type safety cleanup for _read and _write codepaths

In the read codepath, bl->length() returns an unsigned value, and that
could end up looking negative when cast to int. On the write side,
totalwritten is a uint64_t, which could look negative when cast to int.

Have the underlying layers pass back an int64_t and convert them to
int at a higher level. This prepares the underlying infrastructure for
ceph_ll_readv and ceph_ll_writev support.

Tracker: http://tracker.ceph.com/issues/22948
Signed-off-by: Jeff Layton <jlayton@redhat.com>

tools/rados: make rados get/put/append command help txt clearer

Fixes: http://tracker.ceph.com/issues/22958
Signed-off-by: lvshuhua <lvshuhua@cmss.chinamobile.com>

Merge pull request #20392 from adamemerson/wip-clangtastic-voyage

cmake, test/fs, client: fix build with clang

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20379 from tchaikov/wip-test-fix-freebsd

test/librados: s/invoke_result_t/result_of_t/

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #20360 from ifed01/wip-ifed-extents-cleanup

os/bluestore: cleanup around ExtentList, AllocExtent and bluestore_extent_t classes

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20320 from rzarzynski/wip-common-md_config_cacher_t

common: introduce md_config_cacher_t

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20198 from wido/zabbix-health-checks

mgr/zabbix: Implement health checks

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20378 from tchaikov/wip-os-perf-stat-in-ns

osd: check feature bits when encoding objectstore_perf_stat_t

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20275 from badone/wip-osdmonitor-unused-function

mon/OSDMonitor: Comment out unused function

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20371 from tchaikov/wip-admin_socket_output

test/admin_socket_output: add --vstart=path/to/asok option

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #20227 from rzarzynski/wip-bs-final-aiocontexts

os/bluestore: mark derivatives of AioContext as final.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20376 from dillaman/wip-22873

librbd: auto-remove trash snapshots when image is deleted

Reviewed-by: Mykola Golub <mgolub@suse.com>

build: Use the right experimental library

Test whether we're building with libstdc++ or libc++.

Use stdc++filesystem if the former and c++experimental if the later.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

{client,test}: Explicitly initialize empty inode_t

Using an explicit {} initializer. This unbreaks Clang.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>

librbd: auto-remove trash snapshots when image is deleted

Fixes: http://tracker.ceph.com/issues/22873
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librados: s/invoke_result_t/result_of_t/

to workaround buggy libc++ implementation. see
https://cplusplus.github.io/LWG/issue2807 . we should switch to
invoke_result_t once issue2807 is fixed in the latest release of
libc++.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: cleanup around ExtentList class.

Also move it out of bluestore_types.h.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: kill AllocExtent[Vector] to simplify code a bit

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #20387 from ifed01/wip-ifed-fio-build-fix

test/fio: fix build failure caused by sequencer replacement.

Merge pull request #20043 from Rubab-Syed/configuration_setting_browser

mgr/dashboard: add configuration setting browser

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20389 from trociny/wip-cli_generic

qa/workunits/rbd: fix cli_generic test_purge for rbd default format 1

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/workunits/rbd/cli_generic: fix test_purge for rbd default format 1

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20364 from Songweibin/wip-fix-test-rbd

test/rbd: cli_generic fails if v1 image format or deep-flatten disabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

tools/ceph-objectstore-tool: do not fail on lack of OI_ATTR for an
object.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

test/fio: create superblock on mkfs to provide proper tools functioning

E.g. ceph-objectstore-tool requires that

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

test/fio: fix build failure caused by sequencer replacement.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

mds: optimize getattr file size

Fixes: http://tracker.ceph.com/issues/22925
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #20372 from tchaikov/wip-mgr-doc

doc/mgr: remove non user-facing code from doc

Reviewed-by: John Spray <john.spray@redhat.com>

mgr/zabbix: Implement health checks in Zabbix module

Should make it easier for users to see why the module is failing.

Signed-off-by: Wido den Hollander <wido@42on.com>

test/rbd: cli_generic fails if v1 image format or deep-flatten disabled

Fixes: http://tracker.ceph.com/issues/22950
Signed-off-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #20382 from tchaikov/wip-fix-ftbfs-store-test

test/store_test: fix FTBFS as Sequencer is removed

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

check feature bits when encoding objectstore_perf_stat_t

OS_PERF_STAT_NS feature bit is added. it's sharing the same bit with
QOS_DMC.

Signed-off-by: Kefu Chai <kchai@redhat.com>

client: fixup parallel calls to ceph_ll_lookup_inode() in NFS FASL

fixup parallel calls to ceph_ll_lookup_inode() in NFS FASL,
Because the original fuction does not have a global Lock to
ensure the atomicity of the lookup inode process,It eventually
caused an exception when the NFS-Ganesha FSAL module called
this interface in parallel,see the Bug report.

Fixes: http://tracker.ceph.com/issues/22683
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

test/store_test: fix FTBFS as Sequencer is removed

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20294 from rzarzynski/wip-bs-drop-std_function

os/bluestore: avoid overhead of std::function in blob_t.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19232 from socketpair/precision

mgr: increase time resolution of Commit/Apply OSD latencies.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #18804 from majianpeng/bluestore-collection-prealloc

os/bluestore: Prealloc memory avoid realloc in list_collection.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18494 from ifed01/wip-stupidalloc-fix2

os/bluestore: do not assert if BlueFS rebalance is unable to allocate sufficient space

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18343 from shinobu-x/sk-remove-osdmap

mon/PGMap: Remove unnecessary header

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #19954 into master

* refs/pull/19954/head:
test/encoding: refactor to avoid escaping shell magic
mds: minor refactor of SimpleLock
mds: track Capability in mempool
mds: move CInode container members to mempool
mds: move CDentry container members to mempool
mds: move CDir container members to mempool
mds: put MDSCacheObject compact_map in mempool
common: use size_t for object size
mds: convert to allocator agnostic string_view
mds: simplify initialization
compact_*: support mempool allocated containers

Reviewed-by: Zheng Yan <zyan@redhat.com>

osd: PrimaryLogPG::do_osd_ops() uses md_config_cacher_t.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #20007 from mogeb/steady-clock-librbd

librbd: use steady clock to measure elapsed time in AioCompletion

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20008 from mogeb/steady-clock-tools-rbd

tools/rbd: use steady clock in bencher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20218 from shun-s/wip-speedup-diskusage-resize

librbd: speed up object map disk usage and resize

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20311 from Songweibin/wip-group-snap-ls

rbd: do not show title if there is no group snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20349 from trociny/wip-22932

rbd-mirror: fix potential infinite loop when formatting status message

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge PR #20310 into master

* refs/pull/20310/head:
qa: adjust cephfs full test for kclient

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20148 into master

* refs/pull/20148/head:
mds: reset connection's priv when marking down connection
mds: fix session reference leak

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20155 into master

* refs/pull/20155/head:
osdc/Journaler: make sure flush() writes enough data

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #20190 into master

* refs/pull/20190/head:
mon: allow removal of tier of ec overwritable pool

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge PR #20200 into master

* refs/pull/20200/head:
client: add cap_dirtier_uid/gid to CapSnap

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge PR #20246 into master

* refs/pull/20246/head:
mds: remove extra 0x in ino prints
mds: print inode number not CInode ptr

mon/OSDMonitor: accept crush_device_class in 'osd new' json input

If provided, set the OSD device_class at OSD creation time.  This is
simpler than writing a file that the OSD has to read in and use to
set its initial device class, and also avoids a bit of sticky state
at the OSD that will make it keep trying to reset its device class on
startup if it ever gets cleared.

Note that we now ignore json input fields we don't understand, so remove
a test case.

Signed-off-by: Sage Weil <sage@redhat.com>

doc/mgr: remove none user-facing code from doc

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/test_admin_socket_output: allow different asok directory

the asok files are not necessary located in ./out, actually, they are
put in $TMPDIR/ceph-asok.$random_stuff .

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/admin_socket_output: no need to create a copy of fs::path

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20367 from ceph/simple-custom-cluster

ceph-volume: adds custom cluster name support to simple

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: use a custom cluster name in simple functional tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20305 from xiexingguo/wip-more-balancer-fixes

pybind/mgr/balancer: more specific command outputs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20355 from luomuyao/typo

rgw: fix a typo in comment

Reviewed-by: Jos Collin <jcollin@redhat.com>

test/encoding: refactor to avoid escaping shell magic

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: minor refactor of SimpleLock

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: track Capability in mempool

Partial-fix: http://tracker.ceph.com/issues/21402

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: move CInode container members to mempool

Partial-fix: http://tracker.ceph.com/issues/21402

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

os/bluestore: avoid overhead of std::function in blob_t.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>