os/bluestore: print alloc hint flags as a string

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: add ceph_osd_alloc_hint_flag_string helper

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: _setallochint -> _set_alloc_hint

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: do not print bnode key to debug

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9607 from ifed01/wip-bluestore-statfs-fsck

os/bluestore: add statfs result consistency checking to fsck

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9524 from liewegas/wip-ctz

common: add cbits/ctz/clz bit op helpers, replace open-coded calc_bits_for helpers

Reviewed-by: Samuel Just <sjust@redhat.com>

os/bluestore/StupidAllocator: use cbits instead of calculating shift manually

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: pg_pool_t::calc_bits_of -> cbits

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9696 from ifed01/wip-bluestore-csum-fixes

os/bluestore: fix for unaligned writes and improve test coverage

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9604 from ifed01/wip-bluestore-maxalloc

os/bluestore: add an option to limit max allocation size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9378 from xiexingguo/xxg-wip-fix-piderrorcode

global: negative error return code for pidfh::write()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9397 from xiexingguo/xxg-wip-fixsocketfd-leak

common/admin_socket: fix socket fd leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8679 from prallabh/master

rgw: Have a flavor of bucket deletion to bypass GC.

it gets the OSDs substantially hotter (CPU), but is a large speedup (I measured 54% on a small setup, some distortion from debugging)

os/bluestore: fix exceeding blob number after random 4K writes

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

test/store_test: add an additional bluestore test case to verify for exceeding blob number

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #9666 from ceph/wip-ceph_test_objectstore

test: ceph_test_objectstore: do not override plugin-dir if not necessary

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

vstart.sh: set plugin-dir for cmake

test: ceph_test_objectstore: do not override plugin-dir if not necessary

there is chance that we launch the test with the plugins installed in
`/usr/${lib}/ceph'. and we don't have ".lib" or $CEPH_LIB for
ceph_test_objectstore, in this case, we should leave plugin-dir
unchanged. it will work just fine if ceph-base or ceph-common is
installed.

Fixes: http://tracker.ceph.com/issues/16254
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9709 from trociny/wip-16289

qa/workunits/rbd: respect RBD_CREATE_ARGS environment variable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

os/bluestore: fixes assert on unaligned writes when csum verification is disabled

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #9442 from houlinfei/wip-modify-parlistseq

ceph-disk: modify the disk partition sequence of ceph-disk list

Reviewed-by: Loic Dachary <ldachary@redhat.com>

objectstore/store_test: improves test coverage for Bluestore - adds no csum and many 4K writes cases. Fix corresponding issues in testing framework.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #9354 from dreamhost/wip-15975

rgw: Fallback to Host header for bucket name.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9681 from xiexingguo/xxg-wip-server_ts

server: set terminating_sessions flag correctly

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9693 from xiexingguo/xxg-wip-server_hco2

server: negative error code when responding to client

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9472 from xiexingguo/xxg-wip-fix-divisionbyzero

mds: fix potential division-by-zero error

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9445 from xiexingguo/xxg-wip-fix-mdsmemoryleak

mds: avoid duplicated call of context; fix potential memory leak

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9494 from renhwztetecs/renhw-wip-cdir-total

mds/cdir: fix the type of dentry in the judgment; cleanup undefined functions

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9649 from xiexingguo/xxg-wip-server_hco

server: return after respond_to_request() for EROFS

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8218 from jupiturliu/kstore_r/w_flush

os/kstore: we have flush_txns is to make sure read after write

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9639 from jjhuo/rocksdb-directio

os/bluestore: Add an option to choose bluefs direct or buffered io mode

Reviewed-by: Sage Weil <sage@redhat.com>

doc/release-notes: v10.2.2

Signed-off-by: Sage Weil <sage@redhat.com>

qa/workunits/rbd: respect RBD_CREATE_ARGS environment variable

Fixes: http://tracker.ceph.com/issues/16289
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9697 from jcsp/wip-16298

mds: fix MDLog recovery rare case

Merge pull request #9705 from dillaman/wip-librados-compat

librados: restore ability to compile against librados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librados: restore ability to compile against librados

Partial revert of 62be9268de5e9c9a08bdb977a7dab1ab9c55b2be which
added a new header dependency which was is not part of the librados
API.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9093 from zhouyuan/journaling_flush_skip_cache

librbd: do not flush rbd cache if journaling is enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: Skip rbd cache flush if journaling is enabled under aio_flush

With journaling rbd writes will be persisteted on rbd journal objects.
The journal will be replayed if crash happen. So it's not necessary to
flush rbd_cache in this case. This will improve the flush latency.

This patch adds checking on handling aio_flush: if journaling is
enabled, rbd cache flushing is skipped.
In a system flush(ImageCtx::flush) the cache is flushed even w/ journaling
where we truly do need to flush all IO out to disk.

Fixes: http://tracker.ceph.com/issues/15761
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Merge pull request #7741 from rjfd/wip-14527

Lookup monitors through DNS, feature #7741

verified in teuthology (we had 8 failures that appear unrelated to this change)

Merge pull request #9602 from tchaikov/wip-ceph_test_cls_hello

test: fix ceph_test_cls_hello test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9572 from xiexingguo/xxg-wip-fix-ll_read_block

client: fix wrong lock logic of ll_read_block() method

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9586 from xiexingguo/xxg-wip-fix_write

client: add check for _lseek() during _write() process

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9583 from xiexingguo/xxg-wip-fix_preadv_pwritev

client: fix potential access violation

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5521 from ceph/wip-12653

ceph-fuse: fuse_disable_pagecache

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9590 from xiexingguo/xxg-wip-fix-make_request

client: fix error process if we fail to choose a target mds

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9537 from xiexingguo/xxg-wip-fix-clientmemoryleak

client: fix potential memory leak

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9447 from ukernel/wip-mds-snap-fix

mds: fix SnapRealm::have_past_parents_open()

Reviewed-by: John Spray <john.spray@redhat.com>

mds: fix MDLog recovery rare case

This handled the case where one daemon
saw a log that was partially rewritten
by another.  It was calling the context
incorrectly (without mds_lock held)

Fixes: http://tracker.ceph.com/issues/16298
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #9540 from Yan-waller/yj-wip-rgwsystemobj

rgw/rados: remove meaningless assignment operation

since the removed variable (manifest_bl) is indeed unused, this change cannot have any side effect

Merge pull request #9689 from jmunhoz/aws4-streaming-fix-bs

rgw: aws4: fix buffer sharing issue with chunked uploads

verfified (by hand)

mds/MDSMap: change test_flag() return type from int to bool

which better matches its behaviour.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: fix potential memory leak

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mds: cancel context correctly if we already have required osdmap in hand

By simply deleting the fin, the callbacks will be requeued into
the specified finisher and recalled later, which is not the expected
behaviour.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9669 from ceph/wip-selinux

selinux: allow chown for self and setattr for /var/run/ceph

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9346 from jcsp/wip-enable-quota

client: enable client_quota by default

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9591 from ukernel/wip-16066

client: kill QuotaTree

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9651 from renhwztetecs/renhw-wip-mdcache

mds: fix mdsmap->get_metadata_pool() return to int64_t

Reviewed-by: John Spray <john.spray@redhat.com>

server: avoid duplicated call to utime_t

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: negative error code when responding to client

As the comment suggests. Also a zero or positive return code
shall indicates a success, which does not match our intention here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9672 from dillaman/wip-16260

librbd: do not shut down exclusive lock while acquiring

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9653 from trociny/wip-16245

rbd-mirror: FAILED assert(!m_status_watcher)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9675 from dillaman/wip-16268

librbd: remove should ignore mirror errors from older OSDs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rgw: aws4: fix buffer sharing issue with chunked uploads

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

Merge pull request #9684 from chhabaramesh/master

os/bluestore: Fix for bit_alloc unit test case stack size and handle device that has less than block size at end.

client: kill QuotaTree

Multiple clients can modify cephfs at the same time. It is
very tricky to keep QuotaTree consistant with the global FS
hiberarchy. This patch kills the quota tree.

After removing the quota tree, we traverse inode's path to
find quota root.

Fixes: http://tracker.ceph.com/issues/16066
Fixes: http://tracker.ceph.com/issues/16067
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9668 from ceph/wip-rocksdb-wo-rdb

rocksdb: remove rdb source files from dist tarball

Reviewed-by: Haomai Wang <haomai@xsky.com>

test: fix ceph_test_cls_hello test

cls_register_cxx_filter() is exercised by cls_hello, so add the missing
stub for cls_register_cxx_filter() call.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mds/SessionMap: avoid potential memory leak

This is safer and clean.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: Reduce stack size for bit alloc test case

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

os/bluestore: fix few calculations in bit alloc

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #9678 from dzafman/wip-16273

test: Explicitly use tempfile delete= feature instead of os.unlink()

Reviewed-by: Kefu Chai <kefu@redhat.com>

mds/SessionMap: explicitly require importing_count to be postive

The importing_count is of type int, thus it can go underflow and
assert(importing_count) can still return false positive while it
should not supposed to do so.

Also this can make the assert itself more sensitive, e.g., can
catch underflow errors.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: remove redundant semicolon

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: set terminating_sessions flag correctly

So caller won't do this multiple times.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

librbd: track in-flight flush events for journaling

The tracking for flush events was accidentally dropped. This
can result in the journal being closed too early since it
might miss in-flight flush ops.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: Explicitly use tempfile delete= feature instead of os.unlink()

The warning messages are caused by new delete= tempfile feature
trying to unlink a file after test has used os.unlink().

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #9484 from jmunhoz/aws4-streaming

rgw: aws4: add STREAMING-AWS4-HMAC-SHA256-PAYLOAD support

verified (by hand)

librbd: remove should ignore mirror errors from older OSDs

Fixes: http://tracker.ceph.com/issues/16268
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not shut down exclusive lock while acquiring

Fixes: http://tracker.ceph.com/issues/16260
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9664 from dillaman/wip-16253

rbd-mirror: do not re-use image id from mirror directory if creating image

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9588 from renhwztetecs/renhw-wip-mdsboot

mds: Ignore create_root_inode's access value

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9502 from tchaikov/wip-buffer-noexcept

common/buffer: add noexcept to ensure move ctor is used

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9658 from runsisi/ss-wip-fix-rbd-mirror-doc

doc: fix typo for rbd-mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9657 from dillaman/wip-16235

librbd: avoid holding owner_lock waiting for exclusive lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9661 from dillaman/wip-16248

qa/workunits: ensure replay has started before checking position

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

selinux: allow chown for self and setattr for /var/run/ceph

Fixes: http://tracker.ceph.com/issues/16126
Signed-off-by: Boris Ranto <branto@redhat.com>

mds/Mutation: remove redundant _dump method

The original passed in parameter "now" is useless, so
_dump method is identical to dump and shall be considered
as redundant.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rocksdb: remove rdb source files from dist tarball

rdb is based on NodeJS, and we don't build rdb to use rocksdb as
a keyvaluestore backend.

Fixes: http://tracker.ceph.com/issues/13554
Signed-off-by: Kefu Chai <kchai@redhat.com>

mds: remove useless code

As scrub_dentry_next() never returns a negative result code,
it always returns 0, EAGAIN, ENOENT.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rbd-mirror: reset local image id when creating new image

Fixes: http://tracker.ceph.com/issues/16253
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: handle an attempt to delete a non-mirrored image

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: disallow setting an empty mirror image entry to disabling

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9642 from yuyuyu101/wip-16237

msg/async/AsyncConnection: make verify auth callback without connection lock

Reviewed-by: Kefu Chai <kefu@redhat.com>

Merge pull request #9213 from stiopaa1/replicatedpg_movetoforward

osd/ReplicatedPG.h: remove incorrect use of move

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6967 from majianpeng/osd-full-skip-readop

osd: bypass readonly ops when osd full.

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8913 from vumrao/wip-vumrao-15704

rados: Add cleanup message with time to rados bench output

Reviewed-by: David Zafman <dzafman@redhat.com>

qa/workunits: ensure replay has started before checking position

Fixes: http://tracker.ceph.com/issues/16248
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: fix typo for rbd-mirroring

Signed-off-by: runsisi <runsisi@zte.com.cn>

librbd: avoid holding owner_lock waiting for exclusive lock

Fixes: http://tracker.ceph.com/issues/16235
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9374 from batrick/i16024

client: use fh min readahead to enable readahead

Reviewed-by: Greg Farnum <gfarnum@redhat.com>