Hint argument in alloc_blocks + test case changes

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

remove wrap argument and handle in wrappers

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

hint in extent_alloc code

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #10996 from jcsp/wip-16973

mds: log path with CDir damage messages

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10928 from stiopaa1/rbd_mirror_imagesyncthrottler

rbd_mirror/ImageSynceThrottler: move struct to .cc

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10947 from atheism/radosgw-admin-help

rgw: right parenthesis is missing in radosgw-admin help message on caps

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10954 from stiopaa1/rgw_rgwrados_removeUnneededClassCTick

rgw/rgw_rados.h: remove unneeded class C_Tick

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10967 from stiopaa1/osd_replicatedbackend_moveClasses

osd/ReplicatedBackend: move classes to cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10971 from stiopaa1/osd_replicatedpg_moveContextClasses

osd/ReplicatedPG: move classes to .cc file

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10979 from wjwithagen/wip-wjw-cephtool-test

qa/workunits/cephtool/test.sh: fix omission of ceph-command

Merge pull request #10988 from jazeltq/fix-compile-error

autogen: Fix rocksdb error when make dist

mds: log path with CDir damage messages

Previously you just got the inode number, which
wasn't terribly useful for e.g. a missing fragment
object, as you couldn't readily resolve the parent
path.

Fixes: http://tracker.ceph.com/issues/16973
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #10930 from atheism/rgw-lc-fix-sleep-time

rgw/rgw_lc.cc: fix sleep time according to the error message

Makfile: Fix rocksdb error when make dist

Rocksdb delete some files, so we should change another way
to get rocksdb's files.

Currently, if we do not run configure in googletest/googlemock,
we also will get a error when make dist. So fix it too.

Signed-off-by: tianqing <tianqing@unitedstack.com>

Merge pull request #10972 from wjwithagen/wip-wjw-cmake-resolv

CmakeLists.txt: use LIB_RESOLV instead of resolv.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10378 from trociny/wip-14738

librbd: optionally unregister "laggy" journal clients

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10870 from ceph/wip_build_cls_rbd

cmake: add dependency from ceph_smalliobenchrbd to cls libraries

Reviewed-by: Jason Dillaman <jdillama@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10912 from stiopaa1/journal_journaltimer_moveCRemoveSet

journal: move JournalTrimmer::C_RemoveSet struct

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Add dependencies from ceph_smalliobenchrbd on cls_rbd, cls_journal,
and cls_lock.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #10984 from ceph/wip-kfsx-exclusive-lock

test/librbd/fsx: enable exclusive-lock feature in krbd mode

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10964 from liewegas/wip-bluefs-compaction

os/bluestore/BlueFS: fix async compaction logging bug

Merge pull request #10966 from liewegas/wip-bluefs-runway

os/bluestore/BlueFS: factor unflushed log into runway calculation

Reviewed-by: Varada Kari <varada.kari@sandisk.com>
Reviewed-by: Somnath Roy <somnath.roy@sandisk.com>

Merge pull request #10970 from xiexingguo/xxg-wip-honour-alloweio-flag

os/bluestore: honour allow-eio flag; use global compressor if possible

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10974 from dillaman/wip-17210

librbd: potential double-unwatch of watch handle upon error

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

mds: s/used_ions/used_inos/

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #10774 from renhwztetecs/renhw-wip-cephfs-python-umount

libcephfs: add unmount function in cephfs.pyx

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #10914 from batrick/mds-no-maxmds

mds: remove max_mds config option

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #10922 from ceph/wip-jlayton-statx

First pile of statx patches

test/cephfs: add mount&&unmount test

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

Merge pull request #10981 from SUSE/wip-fix-install-deps

build/ops: fix /etc/os-release parsing in install-deps.sh

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

build/ops: fix /etc/os-release parsing in install-deps.sh

85a370e35fc42031a7f7e24dea9e50a649c0f309 introduced the DISTRO variable whose
value was obtained by parsing /etc/os-release like this:

DISTRO=$(grep  "^ID=" /etc/os-release | sed "s/ID=//")

This unfortunately picks up the double-quotes, so on a CentOS system DISTRO
will be equal to '"centos"'.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

os/bluestore: use global compressor if possible

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #10969 from xiexingguo/xxg-wip-fix-cache-logger

os/bluestore: replace store with logger in Cache

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

qa/workunits/cephtool/test.sh: fix omision of ceph-command

 - The script fails with these commands, but not for the correct reason.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

rbd-mirror: option to automatically resync after journal client disconnect

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: stop replay when client is disconnected

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: resync was possible only when image replayer start had succeeded

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: decode_client_meta should return false on error

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

rbd: new command to disconnect journal client

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

librbd: optionally flag "laggy" journal clients disconnected

Fixes: http://tracker.ceph.com/issues/14738
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

journal: allow to trim journal for "laggy" clients

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

cls/journal: add async client_update_state method

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9592 from trociny/wip-15632

librbd: API methods to directly acquire and release the exclusive lock

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: potential double-unwatch of watch handle upon error

Fixes: http://tracker.ceph.com/issues/17210
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9758 from SUSE/wip-btrfs-not-ideal

doc: remove btrfs contradiction

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #10469 from wjwithagen/wip-wjw-compileflag

cmake: supress more warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/ReplicatedPG: move classes to .cc file

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

CmakeLists.txt: use LIB_RESOLV instead of resolv.

 - FreeBSD does not have a separate libresolv, but keeps it in
   libc

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

os/bluestore: honour allow-eio flag during read

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: check iterator before using during fsck

Also drop redundant checking.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: replace store with logger in Cache

As what we really want is to log cache-related statistics.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/ReplicatedBackend: move classes to cc file

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

os/bluestore/BlueFS: factor unflushed log into runway calculation

Include the size of the in-memory buffer we haven't yet flushed
into the runway calculation when deciding whether to allocate
more log space.

Reported-by: Somnath Roy <somnath.roy@sandisk.com>
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10945 from dillaman/wip-17188

librbd: deadlock when replaying journal during image open

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

SQUASH: client: request all shared caps if ctime was requested

Much like the change_attr, ctime changes can potentially be cached on
clients. Request all shared caps if the want mask specifies the ctime.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #10746 from ceph/wip-async-default

msg: make async backend default

scripts/bdev_grep: parse bluefs style extents too

(they use + instead of ~)

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: fix async compaction txn

Our transaction writes are labeled with a seq and uuid
to avoid replaying over garbage.

Two bugs, one real, one potential.

1) The second async compaction transactoin didn't have
its seq and uuid set, so replay always stopped.

2) We were writing two separate transactions, one with
all the new metadata, and the next one with a jump to
the new log offset.  If the first write completed but
it was torn and the second transaction didn't hit disk,
we might see an old transaction with seq == 2 and the
same uuid and replay that instead.

Fix both of these by making the async log txn one single
transaction that jumps directly to the new log offset.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10938 from liewegas/wip-bluestore-csum-error

os/bluestore: fix/improve csum error message

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #10961 from xiexingguo/xxg-wip-bluestore-09-02

os/bluestore: add cache-related stats

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #10953 from liewegas/wip-bluestore-fixes

os/bluestore: misc fixes

Reviewed- by: Igor Fedotov ifedotov@mirantis.com

test/librbd/fsx: enable exclusive-lock feature in krbd mode

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

SQUASH: mds/client: add routines to copy to/from ceph_mds_request_head_legacy

Signed-off-by: Jeff Layton <jlayton@redhat.com>

os/bluestore: trivial cleanups

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: don't overwrite type if it does already exist and at desired type

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

src/CMakeLists.txt: -Woverloaded-virtual is already included in -Wall on Clang

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #10956 from ceph/wip-cmake

cmake: pass -DINTEL* to gf-complete cflags

Reviewed-by: Loic Dachary <ldachary@redhat.com>

os/bluestore: conditional checking label of bluefs devices

As NVMe devices may not support to do so.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

cmake: pass -DINTEL* to gf-complete cflags

* in addition to the -march=foobar, we also need to pass -DFOOBAR to
  gf-complete sources' cflags, so that the SIMD instructions can be
  generated as expected.
* also extract the compiler support for instruction extensions detections
  into SIMDExt.cmake.
* and only check the compiler support if CMAKE_SYSTEM_PROCESSOR matches
  with the instruction-set/arch to be checked.

Signed-off-by: tone.zhang <tone.zhang@linaro.org>
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10934 from tchaikov/wip-kill-warnings

Wip kill warnings

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

os/bluestore: end scope of std::hex properly

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: add onode/buffer cache statistics

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

SQUASH: client: querying for stx_version field requires getting all shared caps

Otherwise, someone could potentially query for just CEPH_STATX_VERSION,
and see it as unchanged, even when there are changes buffered up on
other clients.

By doing this, I don't think we'll incur any perf hit in the common use
case which is ganesha querying for all attributes. We are adding Xs
here, but unless there is a lot of xattr activity I don't think that
will generally cause a lot of exclusive cap recalls.

Also, we don't actually need AUTH caps to fetch CEPH_STATX_RDEV, remove
that from statx_to_mask. PIN is sufficient there.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

rgw/rgw_rados.h: remove unneeded class C_Tick

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

SQUASH: mds/client: don't get clever with ceph_mds_request_head_legacy encode/decode

As requested by Greg...

Declare a legacy object on the stack, and do an extra copy to/from it.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

SQUASH: client: always take a change_attr update from the server if it's newer

I think this is more correct in the client. If we see a newer
change_attr, then we always want to take note of that fact, even if
we have the right caps.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

SQUASH: Revert "mds: ensure that change_attr reflects metadata changes on clients that hold CAP_FILE_EXCL"

This reverts commit 26ab2fa10041ce34b25aa1535715131a2709a27f.

I don't think this is necessary. We don't necessarily need to increment
the change attribute on every change, as long as we ensure that it gets
bumped iff there were changes since the last time you queried for it.

IOW: it's ok to have a single change_attr change encompass a large set
of changes, as long as you ensure that that it is larger by at least 1
after all of those changes.

In order to look at (and potentially cache) other attributes under that
change_attribute (e.g. owner or mode), you need to recall any outstanding
exclusive caps for those attrs. That causes their change_attrs to be
synched to the largest, which is enough to ensure that it changed in
some way.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

SQUASH: client: request AUTH caps on the directory during create if we'll need to check perms locally

Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #10517 from wjwithagen/wip-wjw-freebsd-cmake-execludes

cmake: FreeBSD specific excludes in CMakeLists.txt files

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph_test_objectstore: fix TryMoveRename test

Put the objects in the right pool.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: clean up Rename test a bit

Drop useless remove calls.

Leave in the no-op setattr call since it triggered a
bluestore bug :)

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: assert umount succeeds

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: make object existence check robust

We may have the OnodeRef defined but the object may have
exists==false.  Move the check out of the lookup block
so that we get the right behavior.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: use _setattr for SETATTR

This is a bit faster.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #10843 from liewegas/wip-osdmap-reencode

messages/MOSDMap: mark as enlighten OSDMap encoder

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #10950 from xiexingguo/xxg-wip-blue-09-01

os/bluestore: gifting bluefs more carefully

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: don't over-gift if bluefs free is smaller than bluestore_bluefs_min

- If free space of bluefs is smaller than bluestore_bluefs_min, then raise it
  exactly up to bluestore_bluefs_min.

- Pre-check if bluestore_bluefs_min will exceed bluestore_bluefs_max_ratio * bluefs_total,
  so if free space of bluefs is smaller than bluestore_bluefs_min_ratio * bluefs_total,
  we'll still be able to gift.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rgw: right parenthesis is missing in radosgw-admin help message on caps

Signed-off-by: Weibing Zhang <zhangweibing@unitedstack.com>

os/bluestore: end scope of std::hex properly

So it does not extend to the following output stream and
causes unexpected behaviour.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

vstart.sh: add option to turn on multimds

This makes the necessary `ceph fs` calls to get it running.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: remove max_mds config option

It is now required that all changes to max_mds use the run-time `ceph fs
set max_mds` command. The rationale for this change is that it is
confusing to have a configuration for max_mds which is only observed at
file system creation.

Fixes: http://tracker.ceph.com/issues/17105
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

librbd: deadlock when replaying journal during image open

Fixes: http://tracker.ceph.com/issues/17188
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: improve image state machine debug log messages

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10404 from ilc/build_fix

make-dist/ceph.spec.in: Fix srpm build breakage.

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #10846 from stiopaa1/mds_mdstable_addConstToMemFun

mds/MDSTable: add const to member functions

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10781 from stiopaa1/osdc_objectcacher_moveClassesToCC

osdc/ObjectCacher: move C_ReadFinish, C_RetryRead

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10744 from stiopaa1/osdc_journaler_moveCDelayFlushClass

osdc/Journaler: move C_DelayFlush class to .cc

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10668 from stiopaa1/mon_mdsmonitor_moveCUpdatedClass

mon/MDSMonitor: move C_Updated class to .cc file

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10628 from jcsp/wip-16909

mon: tear down standby replays on MDS rank stop

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10937 from ceph/dang-build

Allow compressor build without YASM

Reviewed-by: Ali Maredia <amaredia@redhat.com>