qa/workunits: rados cppool now requires --yes-i-really-really-mean-it

Fixes: http://tracker.ceph.com/issues/16202
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9442 from houlinfei/wip-modify-parlistseq

ceph-disk: modify the disk partition sequence of ceph-disk list

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #9354 from dreamhost/wip-15975

rgw: Fallback to Host header for bucket name.
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #9681 from xiexingguo/xxg-wip-server_ts

server: set terminating_sessions flag correctly

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9693 from xiexingguo/xxg-wip-server_hco2

server: negative error code when responding to client

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9472 from xiexingguo/xxg-wip-fix-divisionbyzero

mds: fix potential division-by-zero error

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9445 from xiexingguo/xxg-wip-fix-mdsmemoryleak

mds: avoid duplicated call of context; fix potential memory leak

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9494 from renhwztetecs/renhw-wip-cdir-total

mds/cdir: fix the type of dentry in the judgment; cleanup undefined functions

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9649 from xiexingguo/xxg-wip-server_hco

server: return after respond_to_request() for EROFS

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #8218 from jupiturliu/kstore_r/w_flush

os/kstore: we have flush_txns is to make sure read after write

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #9639 from jjhuo/rocksdb-directio

os/bluestore: Add an option to choose bluefs direct or buffered io mode

Reviewed-by: Sage Weil <sage@redhat.com>

doc/release-notes: v10.2.2

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #9697 from jcsp/wip-16298

mds: fix MDLog recovery rare case

Merge pull request #9705 from dillaman/wip-librados-compat

librados: restore ability to compile against librados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librados: restore ability to compile against librados

Partial revert of 62be9268de5e9c9a08bdb977a7dab1ab9c55b2be which
added a new header dependency which was is not part of the librados
API.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9093 from zhouyuan/journaling_flush_skip_cache

librbd: do not flush rbd cache if journaling is enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: Skip rbd cache flush if journaling is enabled under aio_flush

With journaling rbd writes will be persisteted on rbd journal objects.
The journal will be replayed if crash happen. So it's not necessary to
flush rbd_cache in this case. This will improve the flush latency.

This patch adds checking on handling aio_flush: if journaling is
enabled, rbd cache flushing is skipped.
In a system flush(ImageCtx::flush) the cache is flushed even w/ journaling
where we truly do need to flush all IO out to disk.

Fixes: http://tracker.ceph.com/issues/15761
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Merge pull request #7741 from rjfd/wip-14527

Lookup monitors through DNS, feature #7741

verified in teuthology (we had 8 failures that appear unrelated to this change)

Merge pull request #9602 from tchaikov/wip-ceph_test_cls_hello

test: fix ceph_test_cls_hello test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9572 from xiexingguo/xxg-wip-fix-ll_read_block

client: fix wrong lock logic of ll_read_block() method

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9586 from xiexingguo/xxg-wip-fix_write

client: add check for _lseek() during _write() process

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9583 from xiexingguo/xxg-wip-fix_preadv_pwritev

client: fix potential access violation

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5521 from ceph/wip-12653

ceph-fuse: fuse_disable_pagecache

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9590 from xiexingguo/xxg-wip-fix-make_request

client: fix error process if we fail to choose a target mds

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9537 from xiexingguo/xxg-wip-fix-clientmemoryleak

client: fix potential memory leak

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9447 from ukernel/wip-mds-snap-fix

mds: fix SnapRealm::have_past_parents_open()

Reviewed-by: John Spray <john.spray@redhat.com>

mds: fix MDLog recovery rare case

This handled the case where one daemon
saw a log that was partially rewritten
by another.  It was calling the context
incorrectly (without mds_lock held)

Fixes: http://tracker.ceph.com/issues/16298
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #9540 from Yan-waller/yj-wip-rgwsystemobj

rgw/rados: remove meaningless assignment operation

since the removed variable (manifest_bl) is indeed unused, this change cannot have any side effect

Merge pull request #9689 from jmunhoz/aws4-streaming-fix-bs

rgw: aws4: fix buffer sharing issue with chunked uploads

verfified (by hand)

mds/MDSMap: change test_flag() return type from int to bool

which better matches its behaviour.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: fix potential memory leak

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mds: cancel context correctly if we already have required osdmap in hand

By simply deleting the fin, the callbacks will be requeued into
the specified finisher and recalled later, which is not the expected
behaviour.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9669 from ceph/wip-selinux

selinux: allow chown for self and setattr for /var/run/ceph

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9346 from jcsp/wip-enable-quota

client: enable client_quota by default

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9591 from ukernel/wip-16066

client: kill QuotaTree

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9651 from renhwztetecs/renhw-wip-mdcache

mds: fix mdsmap->get_metadata_pool() return to int64_t

Reviewed-by: John Spray <john.spray@redhat.com>

server: avoid duplicated call to utime_t

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: negative error code when responding to client

As the comment suggests. Also a zero or positive return code
shall indicates a success, which does not match our intention here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9672 from dillaman/wip-16260

librbd: do not shut down exclusive lock while acquiring

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9653 from trociny/wip-16245

rbd-mirror: FAILED assert(!m_status_watcher)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9675 from dillaman/wip-16268

librbd: remove should ignore mirror errors from older OSDs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rgw: aws4: fix buffer sharing issue with chunked uploads

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

Merge pull request #9684 from chhabaramesh/master

os/bluestore: Fix for bit_alloc unit test case stack size and handle device that has less than block size at end.

client: kill QuotaTree

Multiple clients can modify cephfs at the same time. It is
very tricky to keep QuotaTree consistant with the global FS
hiberarchy. This patch kills the quota tree.

After removing the quota tree, we traverse inode's path to
find quota root.

Fixes: http://tracker.ceph.com/issues/16066
Fixes: http://tracker.ceph.com/issues/16067
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9668 from ceph/wip-rocksdb-wo-rdb

rocksdb: remove rdb source files from dist tarball

Reviewed-by: Haomai Wang <haomai@xsky.com>

test: fix ceph_test_cls_hello test

cls_register_cxx_filter() is exercised by cls_hello, so add the missing
stub for cls_register_cxx_filter() call.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mds/SessionMap: avoid potential memory leak

This is safer and clean.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: Reduce stack size for bit alloc test case

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

os/bluestore: fix few calculations in bit alloc

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #9678 from dzafman/wip-16273

test: Explicitly use tempfile delete= feature instead of os.unlink()

Reviewed-by: Kefu Chai <kefu@redhat.com>

mds/SessionMap: explicitly require importing_count to be postive

The importing_count is of type int, thus it can go underflow and
assert(importing_count) can still return false positive while it
should not supposed to do so.

Also this can make the assert itself more sensitive, e.g., can
catch underflow errors.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: remove redundant semicolon

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: set terminating_sessions flag correctly

So caller won't do this multiple times.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

librbd: track in-flight flush events for journaling

The tracking for flush events was accidentally dropped. This
can result in the journal being closed too early since it
might miss in-flight flush ops.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: Explicitly use tempfile delete= feature instead of os.unlink()

The warning messages are caused by new delete= tempfile feature
trying to unlink a file after test has used os.unlink().

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #9484 from jmunhoz/aws4-streaming

rgw: aws4: add STREAMING-AWS4-HMAC-SHA256-PAYLOAD support

verified (by hand)

librbd: remove should ignore mirror errors from older OSDs

Fixes: http://tracker.ceph.com/issues/16268
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not shut down exclusive lock while acquiring

Fixes: http://tracker.ceph.com/issues/16260
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9664 from dillaman/wip-16253

rbd-mirror: do not re-use image id from mirror directory if creating image

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9588 from renhwztetecs/renhw-wip-mdsboot

mds: Ignore create_root_inode's access value

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9502 from tchaikov/wip-buffer-noexcept

common/buffer: add noexcept to ensure move ctor is used

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9658 from runsisi/ss-wip-fix-rbd-mirror-doc

doc: fix typo for rbd-mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9657 from dillaman/wip-16235

librbd: avoid holding owner_lock waiting for exclusive lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9661 from dillaman/wip-16248

qa/workunits: ensure replay has started before checking position

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

selinux: allow chown for self and setattr for /var/run/ceph

Fixes: http://tracker.ceph.com/issues/16126
Signed-off-by: Boris Ranto <branto@redhat.com>

mds/Mutation: remove redundant _dump method

The original passed in parameter "now" is useless, so
_dump method is identical to dump and shall be considered
as redundant.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rocksdb: remove rdb source files from dist tarball

rdb is based on NodeJS, and we don't build rdb to use rocksdb as
a keyvaluestore backend.

Fixes: http://tracker.ceph.com/issues/13554
Signed-off-by: Kefu Chai <kchai@redhat.com>

mds: remove useless code

As scrub_dentry_next() never returns a negative result code,
it always returns 0, EAGAIN, ENOENT.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

rbd-mirror: reset local image id when creating new image

Fixes: http://tracker.ceph.com/issues/16253
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: handle an attempt to delete a non-mirrored image

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: disallow setting an empty mirror image entry to disabling

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9642 from yuyuyu101/wip-16237

msg/async/AsyncConnection: make verify auth callback without connection lock

Reviewed-by: Kefu Chai <kefu@redhat.com>

Merge pull request #9213 from stiopaa1/replicatedpg_movetoforward

osd/ReplicatedPG.h: remove incorrect use of move

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6967 from majianpeng/osd-full-skip-readop

osd: bypass readonly ops when osd full.

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8913 from vumrao/wip-vumrao-15704

rados: Add cleanup message with time to rados bench output

Reviewed-by: David Zafman <dzafman@redhat.com>

qa/workunits: ensure replay has started before checking position

Fixes: http://tracker.ceph.com/issues/16248
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: fix typo for rbd-mirroring

Signed-off-by: runsisi <runsisi@zte.com.cn>

librbd: avoid holding owner_lock waiting for exclusive lock

Fixes: http://tracker.ceph.com/issues/16235
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9374 from batrick/i16024

client: use fh min readahead to enable readahead

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #8739 from ukernel/wip-14271

mds,client: sort dentries in hash order

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9640 from dillaman/wip-16235

librbd: recursive lock possible when disabling journaling

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: FAILED assert(!m_status_watcher)

Fixes: http://tracker.ceph.com/issues/16245
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9645 from trociny/rbd-mirror-teuthology-fix

qa/workunits/rbd: specify source path for rbd-mirror helper functions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mds/mdsrank: fix MDSRank::get_metadata_pool return value

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds/tools: fix mds->mdsmap->get_metadata_pool() return int->int64_t

mds->mdsmap->get_metadata_pool() return should be int64_t not int,
fix up

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds/mdcache: fix get poolid in open_remote_dentry

if dnl->get_remote_d_type() == DT_DIR not establish, then pool = -1
so the type of pool should be int64_t not uint64_t

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds/cdir: fix the type of dentry in the judgment

Perfect is_primary/is_remote/is_null type of dentry in the judgment

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds/MDSRank: Ignore create_root_inode's access value

add (void), Ignore create_root_inode's access value

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

server: don't get mdr hang for unknown lock type

It's better to respond and do the cleanup jobs anyway.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

server: return after respond_to_request() for EROFS

Respond_to_request will do cleanup jobs, thus we shall
stall and exit under this case.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

qa/workunits/rbd: specify source path

otherwise it looks in $PATH for the script

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8436 from dreamhost/wip-python-prefix

python: Pass prefix/sbindir from autoconf to distutils.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9364 from H3C/H3C

osd: sparse_read offset may not be zero for ecpool

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9387 from stiopaa1/osd_osd_constiter

osd/OSD.h: change iterator to const_iterator

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9508 from x11562/auth_rm

mon: fix typo of 'ceph auth rm'

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9395 from tchaikov/wip-16010

buffer: handle integer underflow in iterator::copy(large_int, dest)

Reviewed-by: Sage Weil <sage@redhat.com>

msg/async/AsyncConnection: make verify auth callback without connection lock

Fixes: http://tracker.ceph.com/issues/16237
Signed-off-by: Haomai Wang <haomai@xsky.com>

client: move readahead ref get/put to obj constr

The readahead context was incrementing the file handle reference count but only
decreasing in finish which is not called if readahead is a no-op.

This fixes a bug caught in testing where readahead was causing inode to become
disconnected:

    2016-06-10 19:46:48.953018 7f2a4351be80  1 client.4125 dump_inode: DISCONNECTED inode 10000000502 #10000000502 ref 110000000502.head(faked_ino=2307 ref=1 ll_ref=0 cap_refs={1024=0,2048=0,4096=0,8192=0} open={1=0,2=0} mode=100666 size=4194304/0 mtime=2016-06-10 19:29:45.107417 caps=-(0=pAsLsXsFscr) objectset[10000000502 ts 2/4012653 objects 0 dirty_or_tx 0] 0x7f2a24300d00)
    2016-06-10 19:46:48.953032 7f2a4351be80  2 client.4125 cache still has 0+155 items, waiting (for caps to release?)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #9637 from stiopaa1/osdc_ObjectCacher_deleteCopyAssign

osdc/ObjectCacher.h: delete copy constr and assign

Reviewed-by: Sage Weil <sage@redhat.com>

client: use fh min readahead to enable readahead

The client was using the configured maximum readahead of 0 which was changed in
95ee69989129750fddce6a3b5644238c4b88ed74. This prevented readahead from ever
running (without setting a different default from 0).

Fixes: http://tracker.ceph.com/issues/16024
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>