script/make_rocksdb_makefile.sh: do not include rocksdb/tools/rdb/*

Fixes: http://tracker.ceph.com/issues/13554
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9697 from jcsp/wip-16298

mds: fix MDLog recovery rare case

Merge pull request #9705 from dillaman/wip-librados-compat

librados: restore ability to compile against librados

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

librados: restore ability to compile against librados

Partial revert of 62be9268de5e9c9a08bdb977a7dab1ab9c55b2be which
added a new header dependency which was is not part of the librados
API.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9093 from zhouyuan/journaling_flush_skip_cache

librbd: do not flush rbd cache if journaling is enabled

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rbd: Skip rbd cache flush if journaling is enabled under aio_flush

With journaling rbd writes will be persisteted on rbd journal objects.
The journal will be replayed if crash happen. So it's not necessary to
flush rbd_cache in this case. This will improve the flush latency.

This patch adds checking on handling aio_flush: if journaling is
enabled, rbd cache flushing is skipped.
In a system flush(ImageCtx::flush) the cache is flushed even w/ journaling
where we truly do need to flush all IO out to disk.

Fixes: http://tracker.ceph.com/issues/15761
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Merge pull request #7741 from rjfd/wip-14527

Lookup monitors through DNS, feature #7741

verified in teuthology (we had 8 failures that appear unrelated to this change)

Merge pull request #9602 from tchaikov/wip-ceph_test_cls_hello

test: fix ceph_test_cls_hello test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9572 from xiexingguo/xxg-wip-fix-ll_read_block

client: fix wrong lock logic of ll_read_block() method

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9586 from xiexingguo/xxg-wip-fix_write

client: add check for _lseek() during _write() process

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9583 from xiexingguo/xxg-wip-fix_preadv_pwritev

client: fix potential access violation

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #5521 from ceph/wip-12653

ceph-fuse: fuse_disable_pagecache

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9590 from xiexingguo/xxg-wip-fix-make_request

client: fix error process if we fail to choose a target mds

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9537 from xiexingguo/xxg-wip-fix-clientmemoryleak

client: fix potential memory leak

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9447 from ukernel/wip-mds-snap-fix

mds: fix SnapRealm::have_past_parents_open()

Reviewed-by: John Spray <john.spray@redhat.com>

mds: fix MDLog recovery rare case

This handled the case where one daemon
saw a log that was partially rewritten
by another.  It was calling the context
incorrectly (without mds_lock held)

Fixes: http://tracker.ceph.com/issues/16298
Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #9540 from Yan-waller/yj-wip-rgwsystemobj

rgw/rados: remove meaningless assignment operation

since the removed variable (manifest_bl) is indeed unused, this change cannot have any side effect

Merge pull request #9689 from jmunhoz/aws4-streaming-fix-bs

rgw: aws4: fix buffer sharing issue with chunked uploads

verfified (by hand)

Merge pull request #9669 from ceph/wip-selinux

selinux: allow chown for self and setattr for /var/run/ceph

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9346 from jcsp/wip-enable-quota

client: enable client_quota by default

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9591 from ukernel/wip-16066

client: kill QuotaTree

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9651 from renhwztetecs/renhw-wip-mdcache

mds: fix mdsmap->get_metadata_pool() return to int64_t

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9672 from dillaman/wip-16260

librbd: do not shut down exclusive lock while acquiring

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9653 from trociny/wip-16245

rbd-mirror: FAILED assert(!m_status_watcher)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9675 from dillaman/wip-16268

librbd: remove should ignore mirror errors from older OSDs

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rgw: aws4: fix buffer sharing issue with chunked uploads

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

Merge pull request #9684 from chhabaramesh/master

os/bluestore: Fix for bit_alloc unit test case stack size and handle device that has less than block size at end.

client: kill QuotaTree

Multiple clients can modify cephfs at the same time. It is
very tricky to keep QuotaTree consistant with the global FS
hiberarchy. This patch kills the quota tree.

After removing the quota tree, we traverse inode's path to
find quota root.

Fixes: http://tracker.ceph.com/issues/16066
Fixes: http://tracker.ceph.com/issues/16067
Signed-off-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #9668 from ceph/wip-rocksdb-wo-rdb

rocksdb: remove rdb source files from dist tarball

Reviewed-by: Haomai Wang <haomai@xsky.com>

test: fix ceph_test_cls_hello test

cls_register_cxx_filter() is exercised by cls_hello, so add the missing
stub for cls_register_cxx_filter() call.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: Reduce stack size for bit alloc test case

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

os/bluestore: fix few calculations in bit alloc

Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #9678 from dzafman/wip-16273

test: Explicitly use tempfile delete= feature instead of os.unlink()

Reviewed-by: Kefu Chai <kefu@redhat.com>

librbd: track in-flight flush events for journaling

The tracking for flush events was accidentally dropped. This
can result in the journal being closed too early since it
might miss in-flight flush ops.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: Explicitly use tempfile delete= feature instead of os.unlink()

The warning messages are caused by new delete= tempfile feature
trying to unlink a file after test has used os.unlink().

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #9484 from jmunhoz/aws4-streaming

rgw: aws4: add STREAMING-AWS4-HMAC-SHA256-PAYLOAD support

verified (by hand)

librbd: remove should ignore mirror errors from older OSDs

Fixes: http://tracker.ceph.com/issues/16268
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: do not shut down exclusive lock while acquiring

Fixes: http://tracker.ceph.com/issues/16260
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9664 from dillaman/wip-16253

rbd-mirror: do not re-use image id from mirror directory if creating image

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9588 from renhwztetecs/renhw-wip-mdsboot

mds: Ignore create_root_inode's access value

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9502 from tchaikov/wip-buffer-noexcept

common/buffer: add noexcept to ensure move ctor is used

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9658 from runsisi/ss-wip-fix-rbd-mirror-doc

doc: fix typo for rbd-mirroring

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9657 from dillaman/wip-16235

librbd: avoid holding owner_lock waiting for exclusive lock

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9661 from dillaman/wip-16248

qa/workunits: ensure replay has started before checking position

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

selinux: allow chown for self and setattr for /var/run/ceph

Fixes: http://tracker.ceph.com/issues/16126
Signed-off-by: Boris Ranto <branto@redhat.com>

rocksdb: remove rdb source files from dist tarball

rdb is based on NodeJS, and we don't build rdb to use rocksdb as
a keyvaluestore backend.

Fixes: http://tracker.ceph.com/issues/13554
Signed-off-by: Kefu Chai <kchai@redhat.com>

rbd-mirror: reset local image id when creating new image

Fixes: http://tracker.ceph.com/issues/16253
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd-mirror: handle an attempt to delete a non-mirrored image

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: disallow setting an empty mirror image entry to disabling

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9642 from yuyuyu101/wip-16237

msg/async/AsyncConnection: make verify auth callback without connection lock

Reviewed-by: Kefu Chai <kefu@redhat.com>

Merge pull request #9213 from stiopaa1/replicatedpg_movetoforward

osd/ReplicatedPG.h: remove incorrect use of move

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #6967 from majianpeng/osd-full-skip-readop

osd: bypass readonly ops when osd full.

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #8913 from vumrao/wip-vumrao-15704

rados: Add cleanup message with time to rados bench output

Reviewed-by: David Zafman <dzafman@redhat.com>

qa/workunits: ensure replay has started before checking position

Fixes: http://tracker.ceph.com/issues/16248
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: fix typo for rbd-mirroring

Signed-off-by: runsisi <runsisi@zte.com.cn>

librbd: avoid holding owner_lock waiting for exclusive lock

Fixes: http://tracker.ceph.com/issues/16235
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9374 from batrick/i16024

client: use fh min readahead to enable readahead

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #8739 from ukernel/wip-14271

mds,client: sort dentries in hash order

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #9640 from dillaman/wip-16235

librbd: recursive lock possible when disabling journaling

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: FAILED assert(!m_status_watcher)

Fixes: http://tracker.ceph.com/issues/16245
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9645 from trociny/rbd-mirror-teuthology-fix

qa/workunits/rbd: specify source path for rbd-mirror helper functions

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mds/mdsrank: fix MDSRank::get_metadata_pool return value

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds/tools: fix mds->mdsmap->get_metadata_pool() return int->int64_t

mds->mdsmap->get_metadata_pool() return should be int64_t not int,
fix up

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds/mdcache: fix get poolid in open_remote_dentry

if dnl->get_remote_d_type() == DT_DIR not establish, then pool = -1
so the type of pool should be int64_t not uint64_t

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds/MDSRank: Ignore create_root_inode's access value

add (void), Ignore create_root_inode's access value

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

qa/workunits/rbd: specify source path

otherwise it looks in $PATH for the script

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #8436 from dreamhost/wip-python-prefix

python: Pass prefix/sbindir from autoconf to distutils.

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9364 from H3C/H3C

osd: sparse_read offset may not be zero for ecpool

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9387 from stiopaa1/osd_osd_constiter

osd/OSD.h: change iterator to const_iterator

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9508 from x11562/auth_rm

mon: fix typo of 'ceph auth rm'

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #9395 from tchaikov/wip-16010

buffer: handle integer underflow in iterator::copy(large_int, dest)

Reviewed-by: Sage Weil <sage@redhat.com>

msg/async/AsyncConnection: make verify auth callback without connection lock

Fixes: http://tracker.ceph.com/issues/16237
Signed-off-by: Haomai Wang <haomai@xsky.com>

client: move readahead ref get/put to obj constr

The readahead context was incrementing the file handle reference count but only
decreasing in finish which is not called if readahead is a no-op.

This fixes a bug caught in testing where readahead was causing inode to become
disconnected:

    2016-06-10 19:46:48.953018 7f2a4351be80  1 client.4125 dump_inode: DISCONNECTED inode 10000000502 #10000000502 ref 110000000502.head(faked_ino=2307 ref=1 ll_ref=0 cap_refs={1024=0,2048=0,4096=0,8192=0} open={1=0,2=0} mode=100666 size=4194304/0 mtime=2016-06-10 19:29:45.107417 caps=-(0=pAsLsXsFscr) objectset[10000000502 ts 2/4012653 objects 0 dirty_or_tx 0] 0x7f2a24300d00)
    2016-06-10 19:46:48.953032 7f2a4351be80  2 client.4125 cache still has 0+155 items, waiting (for caps to release?)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #9637 from stiopaa1/osdc_ObjectCacher_deleteCopyAssign

osdc/ObjectCacher.h: delete copy constr and assign

Reviewed-by: Sage Weil <sage@redhat.com>

client: use fh min readahead to enable readahead

The client was using the configured maximum readahead of 0 which was changed in
95ee69989129750fddce6a3b5644238c4b88ed74. This prevented readahead from ever
running (without setting a different default from 0).

Fixes: http://tracker.ceph.com/issues/16024
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

client: update comment

Default was changed to unlimited in 11e4680e7394e6e02612441c964e05207b97737b.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

client: use layout.get_period() helper

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #9593 from Yan-waller/yj-hwip-rgwbasictype

rgw/basic_type: use the standard usage of string::npos

verified

osdc/ObjectCacher.h: delete copy constr and assign

Delete copy constructor and assignment operator, so the checking can
be moved from linking to compile time - as it was when the above
where only declared, but not defined.

Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

Merge pull request #9636 from linuxbox2/xio-entity

msg/xio: fix entity_addr_t encoding

Merge pull request #9625 from ceph/ldap-build

Allow building RGW with LDAP disabled

validated

librbd: recursive lock possible when disabling journaling

If pool-level mirroring is enabled, deleting the journal
could cause a deadlock attempting to delete remote peer
sync-point snapshots.

Fixes: http://tracker.ceph.com/issues/16235
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

xio: fix build (entity_addr_t encoding)

Currently XioMessenger encodes an entity_addr_t in its wire header.
The new 'features' argument to entity_addr_t::encode should have
the actual XioMessenger::local_features value when actually encoding
(and some value otherwise).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

cmake:  fix mis-matched arguments to if warning

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #9534 from shehbazj/master

os/bluestore: benchmark Host Aware SMR Drives for linear writes

Merge pull request #9615 from stiopaa1/osdc_objectCacher_removeUnneededPublic

osdc/ObjectCacher.h: remove public access specifier

Reviewed-by: Sage Weil <sage@redhat.com>

Allow building RGW with LDAP disabled

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #8718 from stiopaa1/avoid_copying_for_loop

rgw/rgw_op.cc: avoid copying in for loops

I'm going with, if it compiles, it must be correct (because const).

Merge pull request #9622 from ceph/wip-ceph-disk-test

cmake, test: enable ceph-disk.sh to run with cmake

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #9515 from liewegas/wip-kraken-defaults

default to async messenger, wpq

Reviewed-by: Samuel Just <sjust@redhat.com>

test: ceph-disk.sh: use ceph-helper.sh helpers

so we can use the utlitiy functions offered by ceph-helper.sh, and enable
cmake to run this test.

* ceph-disk/tests/ceph-disk.sh: it checks ../ for the executables for which
  fixtures are set up. but this assumption does not hold when we are using
  cmake for out-of-source build. so we need to pass the build directory to
  ceph-disk.sh. ${CEPH_BIN} is used for this purpose in this change, it is
  ".." by default, so this change is backward compatible. and one can run
  tox without setting up the ${CEPH_BIN} env variable.
* ceph-disk/test-requirements.txt: bump up tox to 2.0+
  in the case of "make check", both cmake and automake pass ${CEPH_BIN} env
  variables to the "make check" tests, so we can simpy pass it down via tox.ini's
  "passenv" setting. per
  https://testrun.org/tox/latest/example/basic.html#passing-down-environment-variables,
  this feature was introduce in tox 2.0. so bump up the minimum required
  tox version to 2.0 in ceph-disk/test-requirements.txt.
* ceph-disk/tox.ini: pass CEPH_BIN, CEPH_BIN, and CEPH_ROOT down to ceph-disk.sh

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: respect CEPH_*_VIRTUALENV when running run-tox-ceph-*

so ceph-{detect-init,disk}/run-tox.sh is able to pick it up

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9621 from dillaman/wip-16229

rbd-mirror: do not propagate deletions when pool unavailable

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9619 from dillaman/wip-16197

test: improve rbd-mirror test case coverage

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9616 from liewegas/wip-osd-markdown-test

test/osd/osd-markdown: make test more reliable

Reviewed-by: Loic Dachary <ldachary@redhat.com>

qa/workunits/rbd: additional rbd-mirror stress tests

Fixes: http://tracker.ceph.com/issues/16197
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

vstart: add --nolockdep option

rbd-mirror stress test cases need a way to disable lockdep when
initiated in a vstart environment.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9618 from dillaman/wip-16226

rbd-mirror: fix deletion propagation edge cases

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #9620 from trociny/rbd-mirror-error-formatting

rbd-mirror: fix error message formatting

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #9464 from vumrao/wip-vumrao-15807

rbd: add error message "snapshot is already protected"

Reviewed-by: Jason Dillaman <dillaman@redhat.com>