librbd: Add tests for metadata get operation

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Add metadata_get method to support single metadata key retrieve

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Make metadata support clone operation

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

rbd.cc: add metadata cli operations

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

test_librbd: Add librbd interface tests for metadata

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

test_cls_rbd: add metadata cls tests

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Add librbd interface impl and tracing for metadata

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd/internal: Add internal implementation for metatdata ops

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

include/librbd: Add metadata interface to librbd

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

cls_rbd: Add cls rbd implementation for librbd metadata

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

cls_rbd_client: Add cls rbd client interface to support rbd metadata

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #3986 from Al3xis-N/master

doc: How to generate an object corpus : should be a subsection

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: How to generate an object corpus : should be a subsection

Fixes: #11099
Signed-off-by: Alexis Normand <n.al3xis@gmail.com>

Merge pull request #3987 from RobinDehu/master

doc: extra \ in CEPH\_AUTH\_UNKNOWN

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: extra \ in CEPH\_AUTH\_UNKNOWN

Suppress all \

fixes: #11097

Signed-off-by: DEHU Robin <robindehu@gmail.com>

Merge pull request #3979 from ceph/wip-rbd-map-options

rbd: default map options, new options, misc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rbd: fix rw option skipping logic

We slip in an extraneous comma if { "rw": "rw" } happens to be the
first map_options pair:

    # ./rbd map -o rw,share foo
    /dev/rbd0

    -> 127.0.0.1:6789 name=admin,key=client.admin,,share rbd foo -

The kernel mount options parser can handle it, but fix it nonetheless.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: recognize cephx_require_signatures and tcp_nodelay options

Support for these went into 3.19 and 4.0 kernels respectively.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: add rbd default map options config option

Allow supplying rbd map -o / --options through 'rbd default map
options' config option.  Options specified on the command line take
precedence on a per-option basis, i.e. default and cli options are
merged with a preference for the latter.  For example:

    # grep 'rbd default map options' ceph.conf
            rbd default map options = "ro,nocrc"
    # ./rbd map -o noshare,rw foo
    /dev/rbd0

results in a rw mapping with nocrc,noshare.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: drop redefining map option warning

For example:

    # ./rbd map -o share,noshare foo
    rbd: warning: redefining map option share: 'share' -> 'noshare'
    /dev/rbd0

mount(8) doesn't do this and it's a bit silly to begin with.  Drop this
in preparation for adding rbd default map options config option.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: regenerate rbd(8) man page

merge-diff and status commands, --image-features and --image-shared
options, other stuff.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #3924 from ktdreyer/wip-doc-rm-old-ceph-deploy-reference

rm old ceph-deploy reference

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #3965 from linuxbox2/xio-fixes

xio: Xio fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Fix XioLoopbackConnection Lifecycle.

Since XioLoopbackConnection is a RefCountedObject, it can't be
an expanded member of XioMessenger.

Fixes cleanup/shutdown errors.

Signed-off-by: Matt Benjamin <matt@cohortfs.com>

Fix CMake build w/o RADOSGW.

Just fixes linkage of dencoder in test.  Fix the false assumption
that cls is only used if WITH_RADOSGW.

Signed-off-by: Matt Benjamin <matt@cohortfs.com>

Merge pull request #3625 from majianpeng/objectcacher

Bugs fix about ObjectCacher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #3881 from ceph/wip-da-SCA-20150304

SCA fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

tools/rados/rados.cc: init some vars in constructor

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados: don't assign value to parameter

Fix for cppcheck issue:
Assignment of function parameter has no effect outside the function.

Remove assignment to need_to_wait before exit.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/msg/xio/*: reduce scope of some vars

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_quota.cc: init variables in constructor

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec_thread.cc: reduce scope of var

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ObjectStoreTransactionBenchmark.cc: reduce scope of var

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

objectstore/store_test.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados.cc: remove unused string vars

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/internal.cc: reduce scope of some variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/ObjectMap.cc: reduce scope of variable, prevent redef

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeShec::minimum_to_decode: reduce scope of variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_op.cc: fix potential null-deref in strcmp()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_main.cc: fix null-deref in case get_storage() fails

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDS.cc: add missing asserts to check return values

Change existing asserts in set_up_admin_socket() to
check (VAR == 0).

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec_all.cc: fix sprintf specifier

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw: use static_cast instead of c-style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedPG.cc: use static_cast instead of c-style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec_thread.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

itest/msgr/test_msgr.cc: prefer ++operator for non-primitive iter

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #3666 from xroche/master

Added ceph_flock() to libcephfs.

Merge pull request #3920 from rajukv/xio-bugfix

XIO: Handle requeue case of XIO messages

Reviewed-by: Kefu Chai <kchai@redhat.com>

XIO: Handle requeue case of XIO messages

If we are not able to send the XIO message using xio_send_msg(),
remove the XIO message from the send Q, before queuing it to the resend
Q. Otherwise, boost will generate a assert.

Signed-off-by: Raju Kurunkad <raju.kurunkad@sandisk.com>

doc/release-notes: v0.80.8 and .9 changelogs

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3871 from dachary/wip-install-deps

tests: install-deps.sh install EPEL and RHEL Optional

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

rm old ceph-deploy reference

There's no need to refer to this old version of ceph-deploy. v1.1.3 is
circa June 2013, and since that time, we have shipped much newer version
of ceph-deploy (1.5.22 at the time of this writing). We should simply
state the behavior of modern ceph-deploys here.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #3887 from stiopaa1/feature10891

mon: add "osd pool get $pool all" command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'gh/hammer'

Conflicts:
src/gmock

tests: install-deps.sh install EPEL and RHEL Optional

On CentOS and RHEL, some dependencies come from EPEL.

Always enable RHEL Optional repository for RHEL as packages such as
packages such as libatomic_ops-devel or libedit-devel are only found
there in RHEL 6.5.

http://tracker.ceph.com/issues/11061 Refs: #11061

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3901 from dachary/wip-10922-ceph-deploy-documentation

doc: preparing a device implies activate

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #3907 from dachary/wip-test-delays

tests: increase the delay waiting for a cluster event

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #3914 from ktdreyer/wip-doc-rgw-apache-log-perms

doc: rm Apache log permissions instructions

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #3917 from dachary/wip-rhel6-doc

doc: rhel6.5 is rhel6

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #3890 from XunleiGFS/fix_C_TwoContexts

osdc: fix a memory leak in C_TwoContexts

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #3845 from ceph/wip-wn-rgw-hammer

Wip wn rgw hammer

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3913 from yuyuyu101/wip-async-fix-throttle-deadlock

Wip async fix throttle deadlock

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3911 from athanatos/wip-11057

ReplicatedPG: block write on degraded object if there are waiters

Reviewed-by: Sage Weil <sage@redhat.com>

doc: rhel6.5 is rhel6

RHEL 6.5 is actually  http://ceph.com/rpm-giant/rhel6/

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3909 from dachary/wip-11068-debian-jessie

deb: add zlib1g-dev to Build-Depends for Debian/jessie

Reviewed-by: Yann Dupont <yann@objoo.org>

mon/OSDMonitor: refactor and new key "all" for osd pool get command

This will only output all the values applicable to a given type of pool.
So for example for a pool that is not a tier pool values like HIT_SET_TYPE,
HIT_SET_PERIOD, HIT_SET_COUNT etc. will be ignored.

Fixes: #10891
Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

doc: rm Apache log permissions instructions

On all distros, Apache should be able to write to its own log directory.
If not, that is a severe distro bug that should be reported/fixed.

Remove the references to running "chown" on /var/log/httpd or
/var/log/apache2.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

doc/release-notes: update notes for v0.80.9

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3608 from guangyy/wip-rgw-quota-threads

rgw: make quota/gc thread configurable for starting

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3795 from rzarzynski/wip-10938

rgw: improve support for X-Timestamp header of Swift API

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Event: Let external events ran without lock/unlock

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Don't block process when throttle is full

If this connection is blocking on throttler and another worker thread
try to acquire this connection's lock, it will deadlock.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #3668 from ceph/wip-compact-container

Optimize memory usage of MDS

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #3908 from gravitystorm/patch-2

fix typo

Reviewed-by: Loic Dachary <ldachary@redhat.com>

deb: add zlib1g-dev to Build-Depends for Debian/jessie

The zlib1g-dev is installed indirectly for Ubuntu 12.04 or Ubuntu 14.04
but it is only suggested in Debian/jessie. Adding it to the
Build-depends is redundant and harmless for Ubuntu and resolves the
missing dependency for Debian.

http://tracker.ceph.com/issues/11068 Fixes: #11068

Signed-off-by: Loic Dachary <ldachary@redhat.com>

doc: fix typo

Signed-off-by: Andy Allan <github@gravitystorm.co.uk>

Merge pull request #3882 from dachary/wip-install-deps-alternatives

install-deps.sh: strip | in the list of packages

Reviewed-by: Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3897 from ceph/wip-11015

crush: fix compilation of strwa2 buckets (fixes 11015)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3899 from yuyuyu101/hammer-test

TestAsyncDriver: Fix typo in ceph_test_async_driver

Merge pull request #3906 from dachary/wip-mailmap

mailmap: David Zhang affiliation

Reviewed-by: Zhi (David) Zhang <zhangz@yahoo-inc.com>

tests: increase the delay waiting for a cluster event

In some cases the machine running tests may be slow enough that it takes
more than a minute for an OSD to come up. It only happens rarely and
changing the wait period from 60 seconds to 120 seconds. Is so slow
that it takes more than 2 minutes to bring an OSD up, chances are a lot
more will go wrong anyway.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Zhi (David) Zhang affiliation

And name normalization

Reviewed-by: Zhi (David) Zhang <zhangz@yahoo-inc.com>
Signed-off-by: Loic Dachary <ldachary@redhat.com>

doc: preparing a device implies activate

http://tracker.ceph.com/issues/10922 Fixes: #10922

Signed-off-by: Loic Dachary <ldachary@redhat.com>

librbd: Add a paramter:purge_on_error in ImageCtx::invalidate_cache().

If bh_write met error, it will try again. For closing image, if met this
issue, it will trigger a assert:
>>2015-02-03 15:22:49.198292 7ff62d537800 -1 osdc/ObjectCacher.cc: In
function 'ObjectCacher::~ObjectCacher()' thread 7ff62d537800 time
>>2015-02-03 15:22:49.195927osdc/ObjectCacher.cc: 551: FAILED
>>assert(i->empty())

Now add purge_on_error, when shutdown_cache it set true.
In ImageCtx::invalidate_cache, if met error and purge_on_error is true,
purge the dirty bh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

librbd: Remvoe unused func ImageCtx::read_from_cache.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: clean up code in ObjectCacher::Object::map_write

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: Don't pass mutex into ObjectCacher::_wait_for_write.

Because the mutex is the same as ObjectCacher::lock.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: After write try merge bh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: Make last missing bh to wake up the reader.

Avoid wakeup early and wait again.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: For trust_enoent is true, there is only one extent.

Now the judgement only in conditon which will return -ENOENT.
But o->exists don't depend on the extent size. It only depend on
trust_enoent. So move this judgement at the first of _readx().
Make this bug ASAP occur.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: In realease(Object *) make the error bh as clean and later remove.

Avoid met this assert error:
  osdc/ObjectCacher.cc: 551: FAILED assert(i->empty())

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: In _readx() only no error can tidy read result.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #3874 from ceph/wip-10828

objecter: fix linger cancel link bug 10828

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3900 from dachary/wip-recursive-submodules

autogen.sh: do not submodule --force if git does not support it

autogen.sh: do not submodule --force if git does not support it

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3895 from dachary/wip-recursive-submodules

doc,tests: force checkout of submodules

TestAsyncDriver: Fix typo in ceph_test_async_driver

Fix: #11058
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

ReplicatedPG: block write on degraded object if there are waiters

Suppose we have min_size of 2 and size of 3, foo exists only on the
primary.
- block op 1 on foo due to < min_size
- start recovery on foo for replicas 1 and 2
- complete push to replica 1 (2 copies now)
- allow op 2 on foo through since we have 2 copies
- complete recovery on foo, requeue op 1

Fixes: 11057
Signed-off-by: Samuel Just <sjust@redhat.com>

crushtool: test compilation of a rule with straw2 buckets

Signed-off-by: Sage Weil <sage@redhat.com>