rgw/user: remove needless bufferlist encoding

info encoded but never be used.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

Merge pull request #9708 from tchaikov/wip-make_rocksdb_makefile-without-rdb

script/make_rocksdb_makefile.sh: do not include rocksdb/tools/rdb/*

Reviewed-by: Dan Mick <dmick@redhat.com>
Reviewed-by: Sage Weil <sweil@redhat.com>

Merge pull request #9713 from xiexingguo/xxg-wip-server_droppin

server: drop locks and auth pins before waiting for trunc

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9710 from xiexingguo/xxg-wip-server_duplock

Server: avoid duplicated call to acquire_locks()

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9706 from xiexingguo/xxg-wip-server_dcr

mds/server: avoid side-effect of more() method

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9647 from xiexingguo/xxg-wip-server_hcr

server: fix potential access violation

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8737 from david-z/wip-mds-slow-req-output

mds: ceph status outputs mds slow request for better monitoring

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9526 from liewegas/wip-bluestore-csum2

os/bluestore: vary csum chunk sizes based on hints

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

os/bluestore: add crc32c_16 and crc32c_8

This is much faster than a slice-by-8 crc16, perhaps even without the
intel instructions.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: whitespace

Signed-off-by: Sage Weil <sage@redhat.com>

compressor/snappy: style cleanup

Signed-off-by: Sage Weil <sage@redhat.com>

compressor/snappy: compress to page-aligned memory buffer

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: remove unused ext_offset region_t field

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: simplify spare read path, remove pextent constraint

The previous read code had the constraint that a physical extent had to
be a multiple of the csum chunk size.  This isn't needed: we might have
a csum_block of 1MB and min_alloc_size of 4KB and that's okay.

Collapse the two helpers into a single loop that uses the blob_t::map()
method to do the pextent part of the read.  This is simpler and avoids
the temporary extents2read structure.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: csum_block -> csum_chunk

Be consistent with terminology.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: ensure blob_t::put_ref respects csum chunk size

We can't deallocate part of a csum chunk.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: test larger writes from Synthetic

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: randomly vary object size and write size hints

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: choose better csum_order in _do_alloc_write

Try to use the wctx hint, but set a floor based on the buffer length.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: make preferred csum order a function of expected_write_size

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: use larger csum blocks for sequential writes

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: fix _do_alloc_write compress condition

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: set csum_order via WriteContext

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: maintain min_alloc_size_order

Signed-off-by: Sage Weil <sage@redhat.com>

ceph_test_objectstore: test alloc hints

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: print alloc hint flags as a string

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: add ceph_osd_alloc_hint_flag_string helper

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: _setallochint -> _set_alloc_hint

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: do not print bnode key to debug

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9607 from ifed01/wip-bluestore-statfs-fsck

os/bluestore: add statfs result consistency checking to fsck

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9524 from liewegas/wip-ctz

common: add cbits/ctz/clz bit op helpers, replace open-coded calc_bits_for helpers

Reviewed-by: Samuel Just <sjust@redhat.com>

os/bluestore/StupidAllocator: use cbits instead of calculating shift manually

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: pg_pool_t::calc_bits_of -> cbits

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9696 from ifed01/wip-bluestore-csum-fixes

os/bluestore: fix for unaligned writes and improve test coverage

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9604 from ifed01/wip-bluestore-maxalloc

os/bluestore: add an option to limit max allocation size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9378 from xiexingguo/xxg-wip-fix-piderrorcode

global: negative error return code for pidfh::write()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9397 from xiexingguo/xxg-wip-fixsocketfd-leak

common/admin_socket: fix socket fd leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #8679 from prallabh/master

rgw: Have a flavor of bucket deletion to bypass GC.

it gets the OSDs substantially hotter (CPU), but is a large speedup (I measured 54% on a small setup, some distortion from debugging)

os/bluestore: fix exceeding blob number after random 4K writes

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

test/store_test: add an additional bluestore test case to verify for exceeding blob number

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #9666 from ceph/wip-ceph_test_objectstore

test: ceph_test_objectstore: do not override plugin-dir if not necessary

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

vstart.sh: set plugin-dir for cmake

test: ceph_test_objectstore: do not override plugin-dir if not necessary

there is chance that we launch the test with the plugins installed in
`/usr/${lib}/ceph'. and we don't have ".lib" or $CEPH_LIB for
ceph_test_objectstore, in this case, we should leave plugin-dir
unchanged. it will work just fine if ceph-base or ceph-common is
installed.

Fixes: http://tracker.ceph.com/issues/16254
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9709 from trociny/wip-16289

qa/workunits/rbd: respect RBD_CREATE_ARGS environment variable

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

os/bluestore: fixes assert on unaligned writes when csum verification is disabled

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #9442 from houlinfei/wip-modify-parlistseq

ceph-disk: modify the disk partition sequence of ceph-disk list

Reviewed-by: Loic Dachary <ldachary@redhat.com>

objectstore/store_test: improves test coverage for Bluestore - adds no csum and many 4K writes cases. Fix corresponding issues in testing framework.

Signed-off-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #9354 from dreamhost/wip-15975

rgw: Fallback to Host header for bucket name.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9681 from xiexingguo/xxg-wip-server_ts

server: set terminating_sessions flag correctly

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9693 from xiexingguo/xxg-wip-server_hco2

server: negative error code when responding to client

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9472 from xiexingguo/xxg-wip-fix-divisionbyzero

mds: fix potential division-by-zero error

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9445 from xiexingguo/xxg-wip-fix-mdsmemoryleak

mds: avoid duplicated call of context; fix potential memory leak

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9494 from renhwztetecs/renhw-wip-cdir-total

mds/cdir: fix the type of dentry in the judgment; cleanup undefined functions

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9649 from xiexingguo/xxg-wip-server_hco

server: return after respond_to_request() for EROFS

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8218 from jupiturliu/kstore_r/w_flush

os/kstore: we have flush_txns is to make sure read after write

Reviewed-by: Sage Weil <sage@redhat.com>

server: drop locks and auth pins before waiting for trunc

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9639 from jjhuo/rocksdb-directio

os/bluestore: Add an option to choose bluefs direct or buffered io mode

Reviewed-by: Sage Weil <sage@redhat.com>

doc/release-notes: v10.2.2

Signed-off-by: Sage Weil <sage@redhat.com>

Server: avoid duplicated call to acquire_locks()

For snapped case we may call acquire_locks() twice,
which is not necessary.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/workunits/rbd: respect RBD_CREATE_ARGS environment variable

Fixes: http://tracker.ceph.com/issues/16289
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Server: use req directly

We have req pointed to mdr->client_request already, so use
it directly to keep the bothering of a long dereference series of
pointer(which also results in a longer line) away.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mds/Mutation: avoid side-effect of slave_did_prepare() method

We shall check _more does exist first.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mds/Mutation: use c++11-style null pointer check

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Ceph status outputs mds slow request for better monitoring

Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>

script/make_rocksdb_makefile.sh: do not include rocksdb/tools/rdb/*

Fixes: http://tracker.ceph.com/issues/13554
Signed-off-by: Kefu Chai <kchai@redhat.com>

server: avoid side-effect of assert

The more() method will implicitly allocate a new internal _more memeber
if it does not have and we shall avoid this.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9697 from jcsp/wip-16298

mds: fix MDLog recovery rare case

Merge pull request #9705 from dillaman/wip-librados-compat

librados: restore ability to compile against librados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librados: restore ability to compile against librados

Partial revert of 62be9268de5e9c9a08bdb977a7dab1ab9c55b2be which
added a new header dependency which was is not part of the librados
API.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9093 from zhouyuan/journaling_flush_skip_cache

librbd: do not flush rbd cache if journaling is enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: Skip rbd cache flush if journaling is enabled under aio_flush

With journaling rbd writes will be persisteted on rbd journal objects.
The journal will be replayed if crash happen. So it's not necessary to
flush rbd_cache in this case. This will improve the flush latency.

This patch adds checking on handling aio_flush: if journaling is
enabled, rbd cache flushing is skipped.
In a system flush(ImageCtx::flush) the cache is flushed even w/ journaling
where we truly do need to flush all IO out to disk.

Fixes: http://tracker.ceph.com/issues/15761
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Merge pull request #7741 from rjfd/wip-14527

Lookup monitors through DNS, feature #7741

verified in teuthology (we had 8 failures that appear unrelated to this change)

Merge pull request #9602 from tchaikov/wip-ceph_test_cls_hello

test: fix ceph_test_cls_hello test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9572 from xiexingguo/xxg-wip-fix-ll_read_block

client: fix wrong lock logic of ll_read_block() method

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9586 from xiexingguo/xxg-wip-fix_write

client: add check for _lseek() during _write() process

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9583 from xiexingguo/xxg-wip-fix_preadv_pwritev

client: fix potential access violation

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5521 from ceph/wip-12653

ceph-fuse: fuse_disable_pagecache

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9590 from xiexingguo/xxg-wip-fix-make_request

client: fix error process if we fail to choose a target mds

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9537 from xiexingguo/xxg-wip-fix-clientmemoryleak

client: fix potential memory leak

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9447 from ukernel/wip-mds-snap-fix

mds: fix SnapRealm::have_past_parents_open()

Reviewed-by: John Spray <john.spray@redhat.com>

mds: fix MDLog recovery rare case

This handled the case where one daemon
saw a log that was partially rewritten
by another.  It was calling the context
incorrectly (without mds_lock held)

Fixes: http://tracker.ceph.com/issues/16298
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #9540 from Yan-waller/yj-wip-rgwsystemobj

rgw/rados: remove meaningless assignment operation

since the removed variable (manifest_bl) is indeed unused, this change cannot have any side effect

Merge pull request #9689 from jmunhoz/aws4-streaming-fix-bs

rgw: aws4: fix buffer sharing issue with chunked uploads

verfified (by hand)

mds/MDSMap: change test_flag() return type from int to bool

which better matches its behaviour.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: fix potential memory leak

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mds: cancel context correctly if we already have required osdmap in hand

By simply deleting the fin, the callbacks will be requeued into
the specified finisher and recalled later, which is not the expected
behaviour.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9669 from ceph/wip-selinux

selinux: allow chown for self and setattr for /var/run/ceph

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9346 from jcsp/wip-enable-quota

client: enable client_quota by default

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9591 from ukernel/wip-16066

client: kill QuotaTree

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9651 from renhwztetecs/renhw-wip-mdcache

mds: fix mdsmap->get_metadata_pool() return to int64_t

Reviewed-by: John Spray <john.spray@redhat.com>

server: avoid duplicated call to utime_t

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: negative error code when responding to client

As the comment suggests. Also a zero or positive return code
shall indicates a success, which does not match our intention here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9672 from dillaman/wip-16260

librbd: do not shut down exclusive lock while acquiring

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9653 from trociny/wip-16245

rbd-mirror: FAILED assert(!m_status_watcher)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9675 from dillaman/wip-16268

librbd: remove should ignore mirror errors from older OSDs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rgw: aws4: fix buffer sharing issue with chunked uploads

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

Merge pull request #9684 from chhabaramesh/master

os/bluestore: Fix for bit_alloc unit test case stack size and handle device that has less than block size at end.

client: kill QuotaTree

Multiple clients can modify cephfs at the same time. It is
very tricky to keep QuotaTree consistant with the global FS
hiberarchy. This patch kills the quota tree.

After removing the quota tree, we traverse inode's path to
find quota root.

Fixes: http://tracker.ceph.com/issues/16066
Fixes: http://tracker.ceph.com/issues/16067
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9668 from ceph/wip-rocksdb-wo-rdb

rocksdb: remove rdb source files from dist tarball

Reviewed-by: Haomai Wang <haomai@xsky.com>

test: fix ceph_test_cls_hello test

cls_register_cxx_filter() is exercised by cls_hello, so add the missing
stub for cls_register_cxx_filter() call.

Signed-off-by: Kefu Chai <kchai@redhat.com>