mon: add perf counters description

Sign-off-by: Alyona Kiseleva <akiselyova@mirantis.com>

osdc: perf counters description added

Sign-off-by: Alyona Kiseleva <akiselyova@mirantis.com>

Merge pull request #3979 from ceph/wip-rbd-map-options

rbd: default map options, new options, misc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rbd: fix rw option skipping logic

We slip in an extraneous comma if { "rw": "rw" } happens to be the
first map_options pair:

    # ./rbd map -o rw,share foo
    /dev/rbd0

    -> 127.0.0.1:6789 name=admin,key=client.admin,,share rbd foo -

The kernel mount options parser can handle it, but fix it nonetheless.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: recognize cephx_require_signatures and tcp_nodelay options

Support for these went into 3.19 and 4.0 kernels respectively.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: add rbd default map options config option

Allow supplying rbd map -o / --options through 'rbd default map
options' config option.  Options specified on the command line take
precedence on a per-option basis, i.e. default and cli options are
merged with a preference for the latter.  For example:

    # grep 'rbd default map options' ceph.conf
            rbd default map options = "ro,nocrc"
    # ./rbd map -o noshare,rw foo
    /dev/rbd0

results in a rw mapping with nocrc,noshare.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: drop redefining map option warning

For example:

    # ./rbd map -o share,noshare foo
    rbd: warning: redefining map option share: 'share' -> 'noshare'
    /dev/rbd0

mount(8) doesn't do this and it's a bit silly to begin with.  Drop this
in preparation for adding rbd default map options config option.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: regenerate rbd(8) man page

merge-diff and status commands, --image-features and --image-shared
options, other stuff.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #3924 from ktdreyer/wip-doc-rm-old-ceph-deploy-reference

rm old ceph-deploy reference

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #3965 from linuxbox2/xio-fixes

xio: Xio fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Fix XioLoopbackConnection Lifecycle.

Since XioLoopbackConnection is a RefCountedObject, it can't be
an expanded member of XioMessenger.

Fixes cleanup/shutdown errors.

Signed-off-by: Matt Benjamin <matt@cohortfs.com>

Fix CMake build w/o RADOSGW.

Just fixes linkage of dencoder in test.  Fix the false assumption
that cls is only used if WITH_RADOSGW.

Signed-off-by: Matt Benjamin <matt@cohortfs.com>

Merge pull request #3625 from majianpeng/objectcacher

Bugs fix about ObjectCacher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #3881 from ceph/wip-da-SCA-20150304

SCA fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

tools/rados/rados.cc: init some vars in constructor

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados: don't assign value to parameter

Fix for cppcheck issue:
Assignment of function parameter has no effect outside the function.

Remove assignment to need_to_wait before exit.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/msg/xio/*: reduce scope of some vars

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_quota.cc: init variables in constructor

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec_thread.cc: reduce scope of var

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ObjectStoreTransactionBenchmark.cc: reduce scope of var

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

objectstore/store_test.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados.cc: remove unused string vars

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/internal.cc: reduce scope of some variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/ObjectMap.cc: reduce scope of variable, prevent redef

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeShec::minimum_to_decode: reduce scope of variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_op.cc: fix potential null-deref in strcmp()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_main.cc: fix null-deref in case get_storage() fails

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDS.cc: add missing asserts to check return values

Change existing asserts in set_up_admin_socket() to
check (VAR == 0).

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec_all.cc: fix sprintf specifier

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw: use static_cast instead of c-style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedPG.cc: use static_cast instead of c-style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec_thread.cc: use static_cast instead of c-style

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

itest/msgr/test_msgr.cc: prefer ++operator for non-primitive iter

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #3666 from xroche/master

Added ceph_flock() to libcephfs.

Merge pull request #3920 from rajukv/xio-bugfix

XIO: Handle requeue case of XIO messages

Reviewed-by: Kefu Chai <kchai@redhat.com>

XIO: Handle requeue case of XIO messages

If we are not able to send the XIO message using xio_send_msg(),
remove the XIO message from the send Q, before queuing it to the resend
Q. Otherwise, boost will generate a assert.

Signed-off-by: Raju Kurunkad <raju.kurunkad@sandisk.com>

doc/release-notes: v0.80.8 and .9 changelogs

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3871 from dachary/wip-install-deps

tests: install-deps.sh install EPEL and RHEL Optional

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

rm old ceph-deploy reference

There's no need to refer to this old version of ceph-deploy. v1.1.3 is
circa June 2013, and since that time, we have shipped much newer version
of ceph-deploy (1.5.22 at the time of this writing). We should simply
state the behavior of modern ceph-deploys here.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #3887 from stiopaa1/feature10891

mon: add "osd pool get $pool all" command

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge remote-tracking branch 'gh/hammer'

Conflicts:
src/gmock

tests: install-deps.sh install EPEL and RHEL Optional

On CentOS and RHEL, some dependencies come from EPEL.

Always enable RHEL Optional repository for RHEL as packages such as
packages such as libatomic_ops-devel or libedit-devel are only found
there in RHEL 6.5.

http://tracker.ceph.com/issues/11061 Refs: #11061

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3901 from dachary/wip-10922-ceph-deploy-documentation

doc: preparing a device implies activate

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #3907 from dachary/wip-test-delays

tests: increase the delay waiting for a cluster event

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #3914 from ktdreyer/wip-doc-rgw-apache-log-perms

doc: rm Apache log permissions instructions

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #3917 from dachary/wip-rhel6-doc

doc: rhel6.5 is rhel6

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #3890 from XunleiGFS/fix_C_TwoContexts

osdc: fix a memory leak in C_TwoContexts

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #3845 from ceph/wip-wn-rgw-hammer

Wip wn rgw hammer

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3913 from yuyuyu101/wip-async-fix-throttle-deadlock

Wip async fix throttle deadlock

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3911 from athanatos/wip-11057

ReplicatedPG: block write on degraded object if there are waiters

Reviewed-by: Sage Weil <sage@redhat.com>

doc: rhel6.5 is rhel6

RHEL 6.5 is actually  http://ceph.com/rpm-giant/rhel6/

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3909 from dachary/wip-11068-debian-jessie

deb: add zlib1g-dev to Build-Depends for Debian/jessie

Reviewed-by: Yann Dupont <yann@objoo.org>

mon/OSDMonitor: refactor and new key "all" for osd pool get command

This will only output all the values applicable to a given type of pool.
So for example for a pool that is not a tier pool values like HIT_SET_TYPE,
HIT_SET_PERIOD, HIT_SET_COUNT etc. will be ignored.

Fixes: #10891
Signed-off-by: Michal Jarzabek <stiopa@gmail.com>

doc: rm Apache log permissions instructions

On all distros, Apache should be able to write to its own log directory.
If not, that is a severe distro bug that should be reported/fixed.

Remove the references to running "chown" on /var/log/httpd or
/var/log/apache2.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

doc/release-notes: update notes for v0.80.9

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #3608 from guangyy/wip-rgw-quota-threads

rgw: make quota/gc thread configurable for starting

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3795 from rzarzynski/wip-10938

rgw: improve support for X-Timestamp header of Swift API

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Event: Let external events ran without lock/unlock

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

AsyncConnection: Don't block process when throttle is full

If this connection is blocking on throttler and another worker thread
try to acquire this connection's lock, it will deadlock.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #3668 from ceph/wip-compact-container

Optimize memory usage of MDS

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #3908 from gravitystorm/patch-2

fix typo

Reviewed-by: Loic Dachary <ldachary@redhat.com>

deb: add zlib1g-dev to Build-Depends for Debian/jessie

The zlib1g-dev is installed indirectly for Ubuntu 12.04 or Ubuntu 14.04
but it is only suggested in Debian/jessie. Adding it to the
Build-depends is redundant and harmless for Ubuntu and resolves the
missing dependency for Debian.

http://tracker.ceph.com/issues/11068 Fixes: #11068

Signed-off-by: Loic Dachary <ldachary@redhat.com>

doc: fix typo

Signed-off-by: Andy Allan <github@gravitystorm.co.uk>

Merge pull request #3882 from dachary/wip-install-deps-alternatives

install-deps.sh: strip | in the list of packages

Reviewed-by: Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3897 from ceph/wip-11015

crush: fix compilation of strwa2 buckets (fixes 11015)

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3899 from yuyuyu101/hammer-test

TestAsyncDriver: Fix typo in ceph_test_async_driver

Merge pull request #3906 from dachary/wip-mailmap

mailmap: David Zhang affiliation

Reviewed-by: Zhi (David) Zhang <zhangz@yahoo-inc.com>

tests: increase the delay waiting for a cluster event

In some cases the machine running tests may be slow enough that it takes
more than a minute for an OSD to come up. It only happens rarely and
changing the wait period from 60 seconds to 120 seconds. Is so slow
that it takes more than 2 minutes to bring an OSD up, chances are a lot
more will go wrong anyway.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

mailmap: Zhi (David) Zhang affiliation

And name normalization

Reviewed-by: Zhi (David) Zhang <zhangz@yahoo-inc.com>
Signed-off-by: Loic Dachary <ldachary@redhat.com>

doc: preparing a device implies activate

http://tracker.ceph.com/issues/10922 Fixes: #10922

Signed-off-by: Loic Dachary <ldachary@redhat.com>

librbd: Add a paramter:purge_on_error in ImageCtx::invalidate_cache().

If bh_write met error, it will try again. For closing image, if met this
issue, it will trigger a assert:
>>2015-02-03 15:22:49.198292 7ff62d537800 -1 osdc/ObjectCacher.cc: In
function 'ObjectCacher::~ObjectCacher()' thread 7ff62d537800 time
>>2015-02-03 15:22:49.195927osdc/ObjectCacher.cc: 551: FAILED
>>assert(i->empty())

Now add purge_on_error, when shutdown_cache it set true.
In ImageCtx::invalidate_cache, if met error and purge_on_error is true,
purge the dirty bh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

librbd: Remvoe unused func ImageCtx::read_from_cache.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: clean up code in ObjectCacher::Object::map_write

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: Don't pass mutex into ObjectCacher::_wait_for_write.

Because the mutex is the same as ObjectCacher::lock.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: After write try merge bh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: Make last missing bh to wake up the reader.

Avoid wakeup early and wait again.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: For trust_enoent is true, there is only one extent.

Now the judgement only in conditon which will return -ENOENT.
But o->exists don't depend on the extent size. It only depend on
trust_enoent. So move this judgement at the first of _readx().
Make this bug ASAP occur.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: In realease(Object *) make the error bh as clean and later remove.

Avoid met this assert error:
  osdc/ObjectCacher.cc: 551: FAILED assert(i->empty())

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osdc: In _readx() only no error can tidy read result.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #3874 from ceph/wip-10828

objecter: fix linger cancel link bug 10828

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3900 from dachary/wip-recursive-submodules

autogen.sh: do not submodule --force if git does not support it

autogen.sh: do not submodule --force if git does not support it

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3895 from dachary/wip-recursive-submodules

doc,tests: force checkout of submodules

TestAsyncDriver: Fix typo in ceph_test_async_driver

Fix: #11058
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

ReplicatedPG: block write on degraded object if there are waiters

Suppose we have min_size of 2 and size of 3, foo exists only on the
primary.
- block op 1 on foo due to < min_size
- start recovery on foo for replicas 1 and 2
- complete push to replica 1 (2 copies now)
- allow op 2 on foo through since we have 2 copies
- complete recovery on foo, requeue op 1

Fixes: 11057
Signed-off-by: Samuel Just <sjust@redhat.com>

crushtool: test compilation of a rule with straw2 buckets

Signed-off-by: Sage Weil <sage@redhat.com>

crush: parse alg names that include a digit

The str_p("straw2") won't parse straw2, it seems because the digit is throwing
it off.  Use the existing name rule instead which is more robust.  Note that
not constraining the alg value here is better anyway because instead of a
'cannot parse' error that is hard to debug we instead get a 'unknown alg foo'
error when doing the semantic pass.

Fixes: #11015
Signed-off-by: Sage Weil <sage@redhat.com>

tests: add Debian jessie dockerfile

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: DEBIAN_FRONTEND=noninteractive apt-get install

Go into non interactive mode when installing the compilation
dependencies, in case a package has a different default mode.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tests: jq is not available on Ubuntu precise

It should be installed from precise-backports but since it is not
currently used, just remove it.

Signed-off-by: Loic Dachary <ldachary@redhat.com>

install-deps.sh: strip | in the list of packages

Alternatives were introduced lately and the | needs to be stripped from
the list of packages to install otherwise apt-get will try to install
all packages.

Signed-off-by: Yann Dupont <yann@objoo.org>

rgw: flush watch after unregistering

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3835 from ceph/wip-mon-datahealth-fix.hammer

[hammer] mon: initialize data store stats structs to avoid weird behaviors

Merge pull request #3615 from ceph/wip-perf-watch

Performance watching CLI

Merge pull request #3877 from ceph/hammer-backports-hadoop

Hammer backports hadoop

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Client: do not require successful remount when unmounting

Fixes: #10982
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 71f2686252ffe29d0234cc8be7304be1450d090e)

doc,tests: force checkout of submodules

When updating submodules, always checkout even if the HEAD is the
desired commit hash (update --force) to avoid the following:

* a directory gmock exists in hammer
* a submodule gmock replaces the directory gmock in master
* checkout master + submodule update : gmock/.git is created
* checkout hammer : the gmock directory still contains the .git from
  master because it did not exist at the time and checkout won't
  remove untracked directories
* checkout master + submodule update : git rev-parse HEAD is
  at the desired commit although the content of the gmock directory
  is from hammer

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3892 from cernceph/cephdisk

ceph-disk: activate-[all|journal] should suppress

Backport: hammer, giant, firefly
Reviewed-by: Sage Weil <sage@redhat.com>

tests: sync must also be recursive

In case a submodule URL to a nested submodule changed

Signed-off-by: Loic Dachary <ldachary@redhat.com>