Set ms_async_send_inline to false to improve small randread iops.

Signed-off-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #11262 from jcsp/wip-17270-master

Revert "osdc: After write try merge bh."

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #11211 from ceph/wip-no-recordlock-test

test: temporarily remove fork()ing flock tests

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11440 from ceph/wip-jlayton-linkref

client: don't take extra target inode reference in ll_link

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11421 from ceph/wip-jlayton-cephfs

Small pile of random cephfs fixes and cleanup

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11515 from badone/wip-find_best_info-typos

osd: Fix typos in PG::find_best_info

Reviewed-by: xiexingguo <xie.xingguo@zte.com.cn>

osd: Fix typos in PG::find_best_info

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #11513 from badone/clear_primary_state-typo

osd: fix typo in PG::clear_primary_state

Merge pull request #11514 from xiexingguo/xxg-wip-bluestore-1015

os/bluestore: formatting nits

Reviewed-by: Sage Weil <sage@redhat.com>

osd: fix typo in PG::clear_primary_state

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #11443 from wjwithagen/wip-wjw-freebsd-cmake-options

do_freebsd.sh: Set options for debug building.

Reviewed-by: Kefu Chai <kchai@redhat.com>

./do_freebsd.sh: remove pipe to get correct return value

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #11430 from wjwithagen/wip-wjw-freebsd-EventKqueue

FreeBSD/EventKqueue.{h,cc} Added code to restore events on (thread)fork

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #11284 from liewegas/wip-mon-osdmap-features

mon/OSDMonitor: encode full osdmaps with features all OSDs can understand

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11401 from liewegas/wip-mgr-keys

auth: tolerate missing MGR keys during upgrade

Reviewed-by: Kefu Chai <kchai@redhat.com>

auth/cephx: do not re-request *only* the MGR key

If we request a bunch of service keys, we may not get
back a MGR key because of an in-progress upgrade.  If we
have everything we need except for just the MGR key, do
not bother re-requesting it.  Instead just continue and
we'll re-request it later when the secrets rotate.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/MonClient: do not request MGR key from pre-kraken mons

If we do, the older mon will return EPERM.

Signed-off-by: Sage Weil <sage@redhat.com>

client: don't take extra target inode reference in ll_link

For the life of me, I can't figure out where this reference is ever put.
We usually take a reference like this when there is an Inode **
parameter that we'll return to the caller. ll_link doesn't have one of
those, so as best I can tell this reference is just leaked.

Zheng however says that FUSE will eventually put this reference via
ll_forget. I still don't quite get how that works, but the other callers
clearly do not handle this correctly. Change the code to only make the
fuse code take this extra reference.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

src/msg/async/EventKqueue: refactor and add test_thread_change()

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

src/msg/async/EventKqueue.cc: adjust logging levels.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

src/msg/async/EventKqueue.{h,cc} Added code to restore events on (thread)fork

According the FreeBSD man page of kqueue(), the kq-descriptors become invalid
upon fork. It looks like the same happens when a kq-handle is created and then
a thread is created.

So we keep a list of assigned events with each kq-handle and when the handle
has beccome invalid, we recreated the kq-handle and the events that go with it.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #11276 from tchaikov/wip-17400

tools/ceph_monstore_tool: update pgmap_meta also when rebuilding store.db

Tested-by: Huawen Ren <ren.huanwen@zte.com.cn>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

os/bluestore: narrow scope of local variables

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: formatting nits

Remove dupilicated semicolon, 80 chars etc.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11339 from liewegas/wip-mon-stuck

mon/PGMap: PGs can be stuck more than one thing

Reviewed-by: Kefu Chai <kchai@redhat.com>

auth/cephx: tolerate missing rotating keys

During an upgrade, we may have a client requesting an
MGR service key but not have one in the database yet,
either because we *just* upgraded and haven't generated
one yet, or because the leader mon hasn't been upgraded
yet.

Fix this by silently tolerating a missing key as long as
one or more other service keys were present and we have
something to give to the client.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11398 from ifed01/wip-bluestore-rmcoll-fix

os/bluestore: fix remove_collection to properly detect collection e…

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11493 from atheism/rgw-rgw_rados-fix-specifier

rgw: rgw_rados.cc fix shard_num format for snprintf

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11496 from stiopaa1/mds_infotableAddOverride

mds/InfoTable.h: add override to virtual functs

mds/InfoTable.h: add override to virtual functs

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

os/bluestore: fix remove_collection to properly detect collection's empty

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

client: no need to check mask before calling _getattr in statx

If the mask is zero, then the cap check will succeed, "yes" will be
true and we'll end up returning quickly from _getattr anyway.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #11483 from liewegas/wip-bluestore-cleanups

os/bluestore: a few cleanups

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11445 from ifed01/wip-fix-isal

compressor/ZLibCompressor: fix broken isal-l

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11495 from yuyuyu101/wip-fix-list

bluestore: set next object as ghobject_t::get_max() when start.hobj.i…

Reviewed-by: Sage Weil <sage@redhat.com>

bluestore: set next object as ghobject_t::get_max() when start.hobj.is_max()

Signed-off-by: Haomai Wang <haomai@xsky.com>
Signed-off-by: Xinze Chi <xinze@xsky.com>

Merge pull request #11491 from xiexingguo/xxg-wip-fs-1014

os/filestore: fix potential result code overwriting

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11482 from ifed01/wip-bluestore-gc-fixes2

os/bluestore: GC infra refactor, more UTs and GC range calculation fixes

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: fix warning

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11490 from xiexingguo/xxg-wip-bluestore-1014

os/bluestore: use iterator for erase() method directly

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: use iterator for erase() method directly

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

common/compressor/ZLibCompressor: add a config option to specify compression level

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/blustore: fix compile warning

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: fix GC gc_end_offset miscalculation

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: isolate GC stuff to be able to cover it with UT

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #11487 from batrick/cephfs-gid

cephfs: add gid to asok status

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11357 from batrick/i17518

mds: use parse_filesystem in parse_role to handle exceptions and reuse parsing code

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11380 from stiopaa1/mds_snapserver_addOverrideToVirt

mds/SnapServer.h: add override to virtual functs

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #11402 from stiopaa1/mds_fsmapRemoveUnneededUseCount

mds/FSMap.cc: remove unneeded use of count

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #11435 from stiopaa1/mds_damageTableMoveSharePtr

mds/DamageTable.cc: move shared ptrs

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: rgw_rados.cc fix shard_num format for snprintf

Signed-off-by: Weibing Zhang <zhangweibing@unitedstack.com>

test: temporarily disable fork()'ing tests

Fixes: http://tracker.ceph.com/issues/16556
Signed-off-by: John Spray <john.spray@redhat.com>

os/filestore: fix potential result code overwriting

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11484 from ceph/wip-move-ranges-tp

tracing/objectstore.tp: add missing move_ranges_... tp

tracing/objectstore.tp: add missing move_ranges_... tp

Signed-off-by: Sage Weil <sage@redhat.com>

cephfs: add gid to asok status

This is useful e.g. for building a database with a unique ID for each
entity.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

os/bluestore: simplify blob print if !compressed

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11468 from xiexingguo/xxg-wip-bluestore-1013

os/bluestore: traverse buffer_map in reverse order when spliting BufferSpace

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11296 from liewegas/wip-fixes

osd: drop stray debug message

Merge pull request #11237 from liewegas/wip-ec-move

os: move_ranges_destroy_src

Reviewed-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: set REQUIRE_JEWEL when setting REQUIRE_KRAKEN

Lots of checks look for the jewel flag; setting the
kraken flag should also set this one.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: health warn if require_{jewel,kraken} flags aren't set

We want to prompt users to set these flags as soon as their
upgrades complete.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11426 from ceph/wip-rgw-17371

rgw: loses realm/period/zonegroup/zone data: period overwritten if somewhere in the cluster is still running Hammer

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11391 from agraf/fix-aarch64-crc-v3

AArch64: Detect crc32 extension support from assembler

Reviewed-by: Yazen Ghannam <yazen.ghannam@linaro.org>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11451 from liewegas/wip-bluestore-reshard-fixes

os/bluestore: fix writes that span existing shard boundaries

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: force reshard if new extent spans shard boundary

A new write may create a new lextent that spans an
existing shard boundary.  If that happens, set a flag
so that we force a reshard when the onode is written.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11452 from liewegas/wip-bluestore-keys

os/bluestore: fix onode vs extent key suffix
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: add checking to catch unspecified compression algorithm

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: traverse buffer_map in reverse order

Which is more efficient.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #11454 from zealoussnow/wip-1013

doc: fix dead link of "os-recommendations" in troubleshooting-osd

Reviewed-by: Kefu Chai <kchai@redhat.com>

common/config_opts: comment nit

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: kill bluestore_shared_blob_hash_table_size_ratio

Which is depreciated by https://github.com/ceph/ceph/commit/7260166da20bfe31469498e55265b40a12f80652#diff-6519c151b74b5a9355a697be278a4382

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: fix tips when adding buffer to cache

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

doc: fix dead link of "os-recommendations" in troubleshooting-osd

Signed-off-by: Leo Zhang <nguzcf@gmail.com>

Merge pull request #11439 from xiexingguo/xxg-wip-bluestore-1012

os/bluestore: avoid polluting shard info if need resharding

Reviewed-by: Sage Weil <sage@redhat.com>

os/bluestore: fix onode vs extent key suffix

I set the extent key suffix as 'x', thinking that was
not a valid hex character and would let us quickly
identify extent keys (vs onode keys, which end with an
encoded u64). But that doesn't work: the keys encode
integer values in raw form--not in hex
(pretty_binary_string just prints it out that way in
the debug log).

Fix by appending 'o' to the onode keys, so that we will
always have a trailing 'o' or 'x' and can use the last
char reliably to determine the type.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11269 from yehudasa/wip-multipart-uploads-cp

rgw: multipart upload copy

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #11403 from bassamtabbara/wip-cmake-allocator-fix

cmake: find gperftools package for tcmalloc_minimal too

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11442 from wjwithagen/wip-wjw-freebsd-readme

README.FreeBSD: update to match the bimonthly FreeBSD status report

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #11449 from dzafman/wip-scrub-prep

test/osd-scrub-repair.sh: Use test case specific object names to help…

Reviewed-by: Kefu Chai <kchai@redhat.com>

README.FreeBSD: update to match the bimonthly FreeBSD status report

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

test/osd-scrub-repair.sh: Use test case specific object names to help with diagnostics

Signed-off-by: David Zafman <dzafman@redhat.com>

compressor/ZLibCompressor: fix broken isal-l

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

client: lseek shouldn't do permission checking

With the addition of the UserPerm changes, ceph_lseek started
checking for open permissions using the default mount perms. lseek
should not require those permissions though, as it operates on an
already-opened file descriptor. Remove the perms argument from most
of these functions.

Fixes: be9e43e2da41e91ffd8577dfd0b890cdb361a08e
Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: convert ceph_setattr callers to ceph_setattrx

Signed-off-by: Jeff Layton <jlayton@redhat.com>

pybind: fix the libcephfs bindings to use ceph_statx and ceph_fstatx

Signed-off-by: Jeff Layton <jlayton@redhat.com>

jni: convert libcephfs java bindings to use ceph_statx based APIs

The java calls themselves are still the same for now. Eventually, it
would be good to covert the java API over to something more statx
like.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: convert some ceph_fstat callers to ceph_fstatx

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: convert some ceph_stat and ceph_lstat calls to ceph_statx

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: fix caps handling when calling _ll_getattr

The _ll_getattr was hardcoded to request CEPH_STAT_CAP_INODE_ALL, even
when the caller didn't need that many caps. Have it take a caps mask
as a parameter and pass that through to _getattr.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: fix handling of ctime and change_attr in fill_statx

Technically, we need all the shared caps to fill those fields out, to
force synchronization from all the clients. Ensure that we have the
right caps before filling out those fields.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: small cleanup in fill_statx()

Make the setting of stx_ino more compact, and add comment explaining
why we set the type bits there.

Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: fix AT_SYMLINK_NOFOLLOW handling

We're passing the reverse to path_walk. It should be true if we want
to follow links.

Fixes: f7c885e1f9cefc359f8b6fe7e51e212387432529
Fixes: bce221c1b99a564d2f4827d2f28cbe939184d308
Signed-off-by: Jeff Layton <jlayton@redhat.com>

client: remove vestigial uid and gid params from path_walk

Signed-off-by: Jeff Layton <jlayton@redhat.com>

do_freebsd.sh: Set options for debug building.

And that requires that the options need to be passed on completely
to the cmake program as well. Thus adding """'s around the option
argument with do_cmake.sh as well

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

os/bluestore: avoid polluting shard info if need resharding

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: remove dead code from reshard()

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #11422 from xiexingguo/xxg-wip-bluestore-1011

os/bluestore: fix race condtion during blob spliting

Reviewed-by: Sage Weil <sage@redhat.com>

mds/DamageTable.cc: move shared ptrs

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #11394 from liewegas/wip-bluestore-shared-blob-intrusive

os/bluestore: use std::unordered_map for SharedBlob lookup