Merge pull request #4045 from NTmatter/docs-raid4-removal

doc-rados-operations-crush-map: Swap Raid4 for Erasure

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/hammer'

Conflicts:
src/gmock
src/msg/xio/XioPortal.h

Merge pull request #3910 from athanatos/wip-11027

Wip 11027

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3918 from tchaikov/hammer-fix-leaks-in-C_TwoContexts

osdc: fix a memory leak in C_TwoContexts

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3972 from tchaikov/hammer-fix-msg-leak

osd: fix a msg leak

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3994 from athanatos/wip-11069

PGLog: split divergent priors as well

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

doc-rados-operations-crush-map: Swap Raid4 for Erasure

Signed-off-by: Thomas Johnson <NTmatter@gmail.com>

Merge pull request #4001 from rzarzynski/wip-10650

rgw: send Last-Modified header in response for PUT on Swift's object.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3925 from rzarzynski/wip-11088

rgw: fix handling empty metadata items on Swift container.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #4024 from ceph/wip-unused-var

mds: remove double-define on iterator

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mds: remove double-define on iterator

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #3976 from wonzhq/hit-set-check

osd/ReplicatePG: correctly checking if an object is in hit set

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4017 from ceph/wip-11125

rgw: update keystone cache with token info

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4019 from NTmatter/doc-typo-monap

doc/rados/operations/add-or-rm-mons: correcting minor typo

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc/rados/operations/add-or-rm-mons: correcting minor typo

Signed-off-by: Thomas Johnson <NTmatter@gmail.com>

PGLog: split divergent priors as well

Signed-off-by: Samuel Just <sjust@redhat.com>

rgw: update keystone cache with token info

Fixes: #11125
Backport: hammer, firefly

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3997 from ceph/wip-debian-mv-var-lib-ceph-mds

debian: move /var/lib/ceph/mds to ceph-mds package

Reviewed-by: James Page <james.page@ubuntu.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3945 from tchaikov/hammer-fix-xio-requeue

XIO: Handle requeue case of XIO messages

Merge pull request #3977 from tchaikov/fix-xio-loop-conn-cleanup

Fix XioLoopbackConnection Lifecycle.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3872 from dachary/wip-7334-partx-test

ceph-disk: test activate on the second journal

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #4014 from ceph/wip-mon-doc

doc/rados/operations/add-or-rm-mons: revise doc a bit to be less confusing

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3922 from ceph/wip-11053

mds: fix assertion caused by system clock backwards

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #3752 from ceph/wip-10744

mds: give up replicas of a stopping mds's stuff

Reviewed-by: Yan, Zheng <zyan@redhat.com>

mds: give up replicas of a stopping mds's stuff

In order for an MDS to make it through stopping when
it had some strays, the other ranks that serviced
the migrate_stray renames must ensure that they
give up any cache objects that belonged to
the stopping MDS, so that the stopping MDS
can finish emptying its cache.

Fixes: #10744
Signed-off-by: John Spray <john.spray@redhat.com>

doc/rados/operations/add-or-rm-mons: revise doc a bit to be less confusing

Signed-off-by: Sage Weil <sage@redhat.com>

debian: move /var/lib/ceph/mds to ceph-mds package

On Debian, ceph-mds was split out into its own package in
9d6013e0db47b258cbcde4c692554a2764812099, but the /var/lib/ceph/mds
directory was not moved along with the rest of the mds pieces.

The /var/lib/ceph/mds directory is only necessary if a user has
installed ceph-mds. Move it to the ceph-mds subpackage.

http://tracker.ceph.com/issues/10587 Refs: #10587

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #4002 from zhicheng/master

fix doc/dev/network-protocol.rst typo

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4007 from dachary/wip-warnings

fix a few minor compilation warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>

fix a few minor compilation warnings

Signed-off-by: Loic Dachary <ldachary@redhat.com>

fix doc/dev/network-protocol.rst typo

Signed-off-by: Zhicheng Wei <zhicheng@opensourceforge.net>

Merge pull request #3989 from javacruft/wip/fixup-cycles-crossarch

Add support for PPC architecture, provide fallback

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4006 from dachary/wip-documentation

doc: mark the ascii schema as a literal block

Reviewed-by: Yann Dupont <yann@objoo.org>

doc: mark the ascii schema as a literal block

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4005 from dachary/wip-releases

doc: release timeline

Reviewed-by: Yann Dupont <yann@objoo.org>

Merge pull request #3990 from zaitcev/first_contrib

Specify the actual struct used in protocol handshake

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3992 from ceph/wip-package-perf-objectstore

packaging: include ceph_perf_objectstore

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>

doc: release timeline

Signed-off-by: Loic Dachary <ldachary@redhat.com>

rgw: send Last-Modified header in response for PUT on Swift object.

Fixes: #10650
Backport: hammer
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #3982 from dzafman/wip-10678-2

Update ceph.8 man page to match ceph -h output

Reviewed-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Fix ceph command manpage to match ceph -h (hammer)

Include rebuilt man/ceph.8

Fixes: #10678
Signed-off-by: David Zafman <dzafman@redhat.com>

doc: Fix ceph command manpage to match ceph -h (firefly)

Improve synopsis section
Fixes: #10676
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9ac488c1eb0e30511079ba05aaf11c79615b3940)

Conflicts:

doc/man/8/ceph.rst
man/ceph.8

PGBackend: do not rewrite ec object oi checksums

Deep scrub does not actually give us the whole-object checksum for an ec
object, only the checksum for the first shard.  We ignore it in scrub
for ec pools anyway in be_select_auth_object.

Signed-off-by: Samuel Just <sjust@redhat.com>

PGBackend: add debug option to rewrite digest even if present

Related: #11027
Signed-off-by: Samuel Just <sjust@redhat.com>

ReplicatedPG: finish_ctx: do not assume that the oi is for the head object

If we are setting the oi on the clone object, we want to update the
mod_desc.

Fixes: 11027
Signed-off-by: Samuel Just <sjust@redhat.com>

packaging: include ceph_perf_objectstore

The /usr/bin/ceph_perf_objectstore file is installed by default. Prior
to this commit it was missing from the packaging. This caused the RPM to
fail to build in mock.

Add ceph_perf_objectstore to the "ceph-test" RPM and Debian package.

If we end up developing further ceph_perf_* utilities, it would make
sense to glob them all with a wildcard, similar to what we are doing
with all the ceph_test_* utilities in ceph-test.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Specify the actual struct used in protocol handshake

Maybe it was sockaddr_storage in an older version, but it is
definitely entity_addr_t in "ceph v027".

Signed-off-by: Pete Zaitcev <zaitcev@redhat.com>

Add support for PPC architecture, provide fallback

Add high precision cpu cycles support for powerpc and powerpc64.

Provide a fallback for other architectures and warn during
compilation.

Signed-off-by: James Page <james.page@ubuntu.com>

Merge pull request #3957 from xinxinsh/wip-cleanup

os/{Level,Rocks}DBStore: cleanup in_prefix function

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #3985 from fmarsylle/master

doc: s/that will called every time/that will be called every time/

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #3986 from Al3xis-N/master

doc: How to generate an object corpus : should be a subsection

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: How to generate an object corpus : should be a subsection

Fixes: #11099
Signed-off-by: Alexis Normand <n.al3xis@gmail.com>

Merge pull request #3987 from RobinDehu/master

doc: extra \ in CEPH\_AUTH\_UNKNOWN

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: s/that will called every time/that will be called every time/

fixes: #11098

Signed-off-by: florian marsylle <florian.marsylle@hotmail.fr>

doc: extra \ in CEPH\_AUTH\_UNKNOWN

Suppress all \

fixes: #11097

Signed-off-by: DEHU Robin <robindehu@gmail.com>

rbd: regenerate rbd(8) man page

merge-diff and status commands, --image-features and --image-shared
options, other stuff.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 356a749f63181d401d16371446bb8dc4f196c2a6)

Merge pull request #3979 from ceph/wip-rbd-map-options

rbd: default map options, new options, misc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: Fix typo in ceph pg dump_stuck usage in man page

Cause by 9db596974c82207dc0769ca6bc9810a9d6bcb5cc

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #3981 from ceph/wip-librbd-invalidate-object-map

librbd: invalid snapshot object maps should only affect the snapshot

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3978 from tchaikov/hammer-sca-fixes

sca fixes ported from master

Reviewed-by: Loic Dachary <ldachary@redhat.com>

librbd: invalidating snapshot object maps should set snapshot flag

Instead of setting the HEAD image flag to indicate an invalid
object map, set the flag for the specific snapshot that is
invalid.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test_cls_rbd: verify set_flags updates snapshots

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd_client: add snap_id param to set_flags

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: set_flags can now update snapshots

It's possible for an object map to be invalid only for
a snapshot, so allow snapshot flags to be updated. This
will also be required when rebuilding the object map and
clearing the invalid flag.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: fix rw option skipping logic

We slip in an extraneous comma if { "rw": "rw" } happens to be the
first map_options pair:

    # ./rbd map -o rw,share foo
    /dev/rbd0

    -> 127.0.0.1:6789 name=admin,key=client.admin,,share rbd foo -

The kernel mount options parser can handle it, but fix it nonetheless.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: recognize cephx_require_signatures and tcp_nodelay options

Support for these went into 3.19 and 4.0 kernels respectively.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: add rbd default map options config option

Allow supplying rbd map -o / --options through 'rbd default map
options' config option.  Options specified on the command line take
precedence on a per-option basis, i.e. default and cli options are
merged with a preference for the latter.  For example:

    # grep 'rbd default map options' ceph.conf
            rbd default map options = "ro,nocrc"
    # ./rbd map -o noshare,rw foo
    /dev/rbd0

results in a rw mapping with nocrc,noshare.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: drop redefining map option warning

For example:

    # ./rbd map -o share,noshare foo
    rbd: warning: redefining map option share: 'share' -> 'noshare'
    /dev/rbd0

mount(8) doesn't do this and it's a bit silly to begin with.  Drop this
in preparation for adding rbd default map options config option.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: regenerate rbd(8) man page

merge-diff and status commands, --image-features and --image-shared
options, other stuff.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #3924 from ktdreyer/wip-doc-rm-old-ceph-deploy-reference

rm old ceph-deploy reference

Reviewed-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_main.cc: fix null-deref in case get_storage() fails

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 921c08fce503e171199f8f8b9208dcdf75ac41c4)

rgw/rgw_op.cc: fix potential null-deref in strcmp()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit de7bcfd8d0e578207cbd9237bc166937fdadcd62)

Fix XioLoopbackConnection Lifecycle.

Since XioLoopbackConnection is a RefCountedObject, it can't be
an expanded member of XioMessenger.

Fixes cleanup/shutdown errors.

Signed-off-by: Matt Benjamin <matt@cohortfs.com>
(cherry picked from commit f5735b28d154612b339cf5d3b558e4dc5eaa2884)

Merge pull request #3965 from linuxbox2/xio-fixes

xio: Xio fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/ReplicatePG: correctly checking if an object is in hit set

For the write case when the oid.snap is CEPH_NOSNAP, the obc is created
in find_object_context, and missing_oid is setting to a NULL object.
Checking if the missing_oid is in the hit set is not the correct way
since it's NULL. Instead, we should check the obc->obs.oi.soid.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>

osd: refactor RepScrubWQ::_process()

use Mutex::Locker instead of releasing the lock manually.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: fix a msg leak when OSD is stopping

Signed-off-by: Kefu Chai <kchai@redhat.com>

Fix XioLoopbackConnection Lifecycle.

Since XioLoopbackConnection is a RefCountedObject, it can't be
an expanded member of XioMessenger.

Fixes cleanup/shutdown errors.

Signed-off-by: Matt Benjamin <matt@cohortfs.com>

Fix CMake build w/o RADOSGW.

Just fixes linkage of dencoder in test.  Fix the false assumption
that cls is only used if WITH_RADOSGW.

Signed-off-by: Matt Benjamin <matt@cohortfs.com>

Merge pull request #3625 from majianpeng/objectcacher

Bugs fix about ObjectCacher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #3881 from ceph/wip-da-SCA-20150304

SCA fixes

Reviewed-by: Kefu Chai <kchai@redhat.com>

ceph-disk: more robust parted output parser

In some cases, depending on the implementation or the operating system,

   parted --machine -- /dev/sdh print

may contain empty lines. The current parsing code is fragile and highly
depends on output details. Replace it with code that basically does the
same sanity checks (output not empty, existence of units, existence of
the dev entry) but handles the entire output instead of checking line by
line.

http://tracker.ceph.com/issues/10983 Fixes: #10983

Signed-off-by: Loic Dachary <ldachary@redhat.com>

tools/rados/rados.cc: init some vars in constructor

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados: don't assign value to parameter

Fix for cppcheck issue:
Assignment of function parameter has no effect outside the function.

Remove assignment to need_to_wait before exit.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

src/msg/xio/*: reduce scope of some vars

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_quota.cc: init variables in constructor

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec_thread.cc: reduce scope of var

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ObjectStoreTransactionBenchmark.cc: reduce scope of var

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

objectstore/store_test.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados.cc: reduce scope of variable

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_rados.cc: remove unused string vars

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/internal.cc: reduce scope of some variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

librbd/ObjectMap.cc: reduce scope of variable, prevent redef

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

ErasureCodeShec::minimum_to_decode: reduce scope of variables

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw/rgw_op.cc: fix potential null-deref in strcmp()

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw_main.cc: fix null-deref in case get_storage() fails

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

mds/MDS.cc: add missing asserts to check return values

Change existing asserts in set_up_admin_socket() to
check (VAR == 0).

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

TestErasureCodeShec_all.cc: fix sprintf specifier

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

rgw: use static_cast instead of c-style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

osd/ReplicatedPG.cc: use static_cast instead of c-style cast

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>