libcephfs: kill compiling warning

As below:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/include/cephfs/libcephfs.h:1499:20: warning: ‘struct flock’ declared inside parameter list [enabled by default]
     Fh *fh, struct flock *fl, uint64_t owner);
                    ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/include/cephfs/libcephfs.h:1499:20: warning: its scope is only this definition or declaration, which is probably not what you want [enabled by default]
/home/jenkins-build/build/workspace/ceph-pull-requests/src/include/cephfs/libcephfs.h:1501:20: warning: ‘struct flock’ declared inside parameter list [enabled by default]
     Fh *fh, struct flock *fl, uint64_t owner, int sleep);

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #10526 from xiexingguo/xxg-wip-misc-2016-07-30

osd: minor performance improvements and fixes

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #10548 from xiexingguo/xxg-wip-osd-2016-08-02

osd: more cleanups

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #10559 from xiexingguo/xxg-wip-osd-2016-08-03

osd: drop unused variables/methods

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #10212 from rzarzynski/wip-rgw-cleanup-auth

rgw: clean-up in the authentication infrastructure

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10233 from weiqiaomiao/wqm-wip-ver_check

rgw: fix wrong  variable definition in cls_version_check func

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10594 from michelmno/forpull2

rpm: ExclusiveArch for suse_version

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #10122 from Yan-waller/yj-wiph-rgwradoscc-0704

rgw: fix the missing return value

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10194 from Yan-waller/yj-wip-rgwrados-0708

rgw: remove unused bufferlist variable

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10203 from Yan-waller/yj-wip-rgwcache-0708

rgw: merge setting flags operation together and cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10214 from theanalyst/rgw/multisite-fwd-log-cleanup

rgw: drop create_bucket in fwd_request log message

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10226 from Yan-waller/yj-wip-rgwrests3-0709

rgw: use the standard usage of string.find

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10299 from Yan-waller/yj-wiph-rgwmetadata-0715

rgw: miscellaneous cleanups

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10408 from weiqiaomiao/wqm-wip-rgw-lc-set

rgw: fix wrong variable definition in rgw_cls_lc_set_entry function

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10562 from aclamk/wip-rgw-s3-keystone-cache-mutex

rgw: Got rid of recursive mutex.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10424 from theanalyst/rgw/mp-complete-error-msg

rgw: log mp upload failures due to parts mismatch

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10475 from yonghengdexin735/wip-zzz-rgw

rgw/rgw_main: fix unnecessary variables defined

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10589 from Yan-waller/yj-wip-rgwcls-0805

rgw: remove unused variables

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #10611 from linuxbox2/rgw-nfs-28

Rgw nfs 28

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw nfs: fix write continuations

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw nfs: don't leak fh->mtx in lookup_handle()

This change fixes a serious latent locking problem, noticed after
updating the ganesha/rgw driver invalidation after renames.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw file: refuse to rename directories

The representation of paths as an aggregate involving any
number of objects argues against ever permitting such
operations.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file: refuse ops on deleted targets

Detect these illegal cases, as they are indicate various incorrect
behaviors/bugs.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #10598 from dillaman/wip-16882

cmake: correct ceph_test_librbd/ceph_test_rbd_mirror linkage

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9566 from ffilz/master

Add ceph_ll_setlk and ceph_ll_getlk

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #10452 from jcsp/wip-16556

test: break out librados-using cephfs test

Merge pull request #10574 from dillaman/wip-16923

librbd: delay acquiring lock if image watch has failed

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10581 from dillaman/wip-16887

librbd: prevent creation of v2 image ids that are too large

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

cmake: correct ceph_test_librbd/ceph_test_rbd_mirror linkage

This now matches the autotools configuration and fixes the failed
assertion during test execution.

Fixes: http://tracker.ceph.com/issues/16882
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10579 from dreamhost/wip-16928

rgw: Fix civetweb IPv6

verified:  autobuild

Merge pull request #10597 from pritha-srivastava/wip_ldap_fix

rgw: Fix for incorrect access key id in reworked ldap auth code.

verified non-regression (MS AD)

rgw: Fix for incorrect access key id reworked ldap auth code.

The reworked ldap code does not use the correct access key id in get_policy.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>

Merge pull request #10595 from tchaikov/wip-rocksdb

rocksdb: revert the change introduced by dc41731

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rocksdb: revert the change introduced by dc41731

dc41731 reverts the submodule reference of rocksdb, and point it to an
older version which fails to support ppc64le and AArch64.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10262 from dachary/wip-16580-ceph-disk-timeout

ceph-disk: timeout ceph-disk to avoid blocking forever

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Ken Dreyer (Red Hat) <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.cz>

Merge pull request #10491 from xiexingguo/xxg-wip-osdmon-2016-07-28

osd/OSDMonitor: misc. fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #9259 from xiexingguo/xxg-wip-fix-monclientpinger

mon: MonClient may hang on pinging an unresponsive monitor

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10593 from linuxbox2/ldap-bindv3-again

rgw ldap: enforce simple_bind w/LDAPv3

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #9718 from songbaisen/a8

mon: clear list in better way

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10392 from stiopaa1/mon_monclient_moveCCancelMonCommand

mon/MonClient: move C_CancelMonCommand to cc file

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #10470 from xiexingguo/xxg-wip-osd-2016-07-28

osd: minor performance improvements

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #10539 from jcsp/wip-mds-counters

mds: add perf counters for MDLog replay and SessionMap

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

rgw ldap: enforce simple_bind w/LDAPv3

Found by Harald Klein <hklein@redhat.com>.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

ExclusiveArch for suse_version

for SLES supports only x86_64 and aarch64 targets
for openSUSE (Tumbleweed and Leap) add ppc64/ppc64le targets.

fixes: http://tracker.ceph.com/issues/16936
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>

Merge pull request #10542 from ktdreyer/doc-rbd-libvirt-logging

doc: describe libvirt client logging

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10488 from lande1234/wip-lan-testing

rbd-mirror: add additional test scenarios

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10573 from renhwztetecs/renhw-wip-doc-cephfs

doc/mds: fixup mds doc

Reviewed-by: John Spray <john.spray@redhat.com>

mds: add perf counter for rate of replay

Useful for watching the progress of a daemon
in standby-replay or replay.

Signed-off-by: John Spray <john.spray@redhat.com>

mds: expose perf counters for SessionMap

So that we can see how many sessions exist via
the counters rather than only by looking at status.

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #10512 from renhwztetecs/renhw-wip-mds-balancer

mds/MDBalancer: cleanup

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #10485 from gregsfortytwo/wip-client-dump-complete

client: include COMPLETE and ORDERED states in cache dump

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #10533 from gregsfortytwo/wip-mds-dispatch-refcount

mds: fix up _dispatch ref-counting semantics

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #10540 from jcsp/wip-mds-multimds-standby

mds: fix daemon selection when starting ranks

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

rgw: remove unused variables

remove the unused bufferlist variables

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>

librbd: prevent creation of v2 image ids that are too large

The librbd API is capped at 24 characters for expressing the
object prefix for data blocks (including trailing null byte).

Fixes: http://tracker.ceph.com/issues/16887
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rgw: Fix civetweb IPv6

Commit c38e3cbb6f7c6221209f2b512cba18c564c895a4 introduced a second
compiled version of src/civetweb/src/civetweb.c, but did not pass the
configuration header (civetweb/include/civetweb_conf.h).

As a result, USE_IPV6 was not defined when it was compiled, and that
copy was included into the radosgw binary. This caused breakage for the
civetweb frontend when used with IPv6:
  rgw frontends = civetweb port=[::]:7480

Reintroduce the header so that civetweb is compiled correctly again.

Fixes: http://tracker.ceph.com/issues/16928
Backport: jewel
Signed-off-by: Robin H. Johnson <robin.johnson@dreamhost.com>

Merge pull request #10565 from dzafman/wip-16672

OSD crash with Hammer to Jewel Upgrade: void FileStore::init_temp_collections()

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #10545 from trociny/fixup-rbd-mirror-test

qa/workunits/rbd: wait for image deleted before checking health

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: describe libvirt client logging

Describe how to configure logging and the admin socket for debugging RBD
libvirt clients.

Thanks to Jason Dillman <jdillama@redhat.com> on ceph-devel for this
information.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

librbd: delay acquiring exclusive lock if watch has failed

Fixes: http://tracker.ceph.com/issues/16923
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: convert ImageWatcher class to template

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #10566 from batrick/mds-daemon-unused

mds: remove unused MDSDaemon::objecter

Reviewed-by: John Spray <john.spray@redhat.com>

doc/mds: add generated files into the /etc/ceph/

Fixes: 16906
Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

osd: add a comment for the purpose of pinned_maps

For a better understanding since it can confuse newcommers.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

mds: remove unused MDSDaemon::objecter

Leftover from a2682edb76ceac428c686e0ac21c06511e27693e.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

osd: use osd_target_transaction_size when batch removing objects

Hard code transaction size to be 300 is not good. Since we already
introduce osd_target_transaction_size for the same purpose, we can
use it directly here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: fix rare race while looking for a pg

When _have_pg() returns true, we drop the pg_map_lock
and pg->lock() simultaneously. So theoretically the
subsequent call to _lookup_lock_pg() can return NULL,
although the chance is rare.

Also, since we are going to call _lookup_lock_pg() anyway,
we can call it directly, thus the _have_pg() method is
totally unnecessary.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: kill _lookup_pg() method

It is never used and I see no chance it will be used.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: don't get transaction too big during clear_temp_objects()

Basically we don't allow a transaction to get too big, e.g., to
avoid contiguously occupation of CPU.

We break and queue the transaction here whenever we have collected
osd_target_transaction_size temp objects.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #10307 from pritha-srivastava/wip_s3_auth_rework

rgw: Rework of s3 LDAP Authentication code.

verified: MS AD

Got rid of recursive mutex.

Signed-off-by: Adam Kupczyk <akupczyk@mirantis.com>

os: Fix HashIndex::recursive_remove() to remove everything but original path

Fixes: http://tracker.ceph.com/issues/16672
Signed-off-by: David Zafman <dzafman@redhat.com>

filestore: Clear objects before calling collection_list() again

Signed-off-by: David Zafman <dzafman@redhat.com>

filestore: Improve logging

Signed-off-by: David Zafman <dzafman@redhat.com>

mds/MDBalancer: change OPT_FLOAT to OPT_DOUBLE

utime_t convert double in MDBalancer::tick(),
so g_conf->mds_bal_sample_interval should be set OPT_DOUBLE

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds/MDBalancer: redundance cleanup

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds/MDBalancer: some types of definitions are cleanup

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

mds/MDBalancer: add err info when ifstream open failure

Signed-off-by: huanwen ren <ren.huanwen@zte.com.cn>

osd: initialize class_handler of OSD properly

The destructor of OSD will delete class_handler arbitrarily
and there are cases we'll fail to start an osd before we
load the class_handler correctly.

So it is safe to initialize class_handler to NULL as to
avoid access violation.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd/ReplicatedPG: fix typo(from ECANCELLED to ECANCELED)

According to Linux Man pages:

  ECANCELED       Operation canceled (POSIX.1)

So ECANCELED is the preferred one.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: add sanity check for scrubs_active counter

To make sure there is no underflow and the scrub logic
works fine.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

osd: kill redundant call of cancel_all_events()

The agent_timer.shutdown() method will call cancel_all_events()
within in a more safe and elegant way, so we don't have to
call it explicitly here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge remote-tracking branch 'upstream/pull/10543/head'

* upstream/pull/10543/head:
  vstart.sh: only delete conf if -n

Merge pull request #10492 from chhabaramesh/master

os/bluestore: remove cmake warning from extent alloc functions

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

vstart.sh: only delete conf if -n

src/test/ceph_objectstore_tool.py depends on an implicit "-k" when not
creating a new cluster. (Deleting the conf file without -n makes no
sense anyway.)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #10462 from dillaman/wip-16478

rbd-mirror: non-primary image is recording journal events during image sync

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10484 from dillaman/wip-16538

rbd-mirror: image deleter should use pool id + global image uuid for key

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

qa/workunits/rbd: wait for image deleted before checking health

This is a fixup to the previous commit.

Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #10458 from gregsfortytwo/wip-doc-troubleshooting

doc: extend the CephFS troubleshooting guide

Reviewed-by: Zack Cerza <zack@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #9695 from ktdreyer/doc-tarballs-url

doc: update canonical tarballs URL

Reviewed-by: Alfredo Deza <adeza@redhat.com>

os/bluestore: remove cmake warning from extent alloc functions.
link: http://tracker.ceph.com/issues/16766
Signed-off-by: Ramesh Chander <Ramesh.Chander@sandisk.com>

Merge pull request #10386 from ceph/wip-jlayton-nlink

Fix attribute handling at lookup time

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #10541 from tchaikov/wip-cmake

cmake: install ceph_test_rados_striper_api_*

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

cmake: install ceph_test_rados_striper_api_*

so we can test them in ceph-qa-suite

Signed-off-by: Kefu Chai <kchai@redhat.com>

test: unit test cases for disabling librbd journal by policy

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: utilize factory methods to create AioObjectRequest objects

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: convert AioObjectRequest/AioObjectRead classes to templates

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: move read callback helpers to narrowest scope

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: convert AioImageRequest derived classes to templates

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: removed namespace indentation from legacy classes

Better follows the Ceph C++ style guide

Signed-off-by: Jason Dillaman <dillaman@redhat.com>