Declare libradosstriper library dependencies

When omitted, trying to dynamically load libradosstriper results in "undefined symbol" errors.

Signed-off-by: Javier Guerra <javier@guerrag.com>

rados.py: fix Object.write() method

the file position should advance by the number of bytes written,
but the ioctx.write() function returns 0 on success

Signed-off-by: Javier Guerra <javier@guerrag.com>

Merge pull request #4067 from ceph/wip-rbd-warning

tests: remove unused variable

Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

tests: remove unused variable

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3904 from yuyuyu101/wip-librbd-metadata

Wip librbd metadata

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4013 from ceph/wip-5488

librbd: fix deadlock on image refresh

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Librbd: Add "start" and "max" arguments to metadata_list

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #3953 from wuxingyi/patch-2

rgw/logrotate.conf: Rename service name

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

rgw/logrotate.conf: Rename service name

The service name for ceph rados gateway was changed to "ceph-radosgw",
the previous version of service name "radosgw" would cause a failed reload,
and finally make it impossible to write any log data to the log file.

Signed-off-by: wuxingyi <wuxingyi2015@outlook.com>

Merge pull request #4055 from vuhuong/wip-xio

XIO fixes

Reviewed-by: Sage Weil <sage@redhat.com>

doc/release-notes: v0.67.11 draft notes

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #4045 from NTmatter/docs-raid4-removal

doc-rados-operations-crush-map: Swap Raid4 for Erasure

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge remote-tracking branch 'gh/hammer'

Conflicts:
src/gmock
src/msg/xio/XioPortal.h

Merge pull request #3910 from athanatos/wip-11027

Wip 11027

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3918 from tchaikov/hammer-fix-leaks-in-C_TwoContexts

osdc: fix a memory leak in C_TwoContexts

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #3972 from tchaikov/hammer-fix-msg-leak

osd: fix a msg leak

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3994 from athanatos/wip-11069

PGLog: split divergent priors as well

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

doc-rados-operations-crush-map: Swap Raid4 for Erasure

Signed-off-by: Thomas Johnson <NTmatter@gmail.com>

Merge pull request #4001 from rzarzynski/wip-10650

rgw: send Last-Modified header in response for PUT on Swift's object.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3925 from rzarzynski/wip-11088

rgw: fix handling empty metadata items on Swift container.

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

xio: Add xio_transport_type

Add xio_transport_type to control and run xio messenger over
rdma or tcp

Signed-off-by: Vu Pham <vu@mellanox.com>

xio: Update README.xio with dependencies

Signed-off-by: Vu Pham <vu@mellanox.com>

Merge pull request #4024 from ceph/wip-unused-var

mds: remove double-define on iterator

Reviewed-by: Loic Dachary <ldachary@redhat.com>

mds: remove double-define on iterator

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #3976 from wonzhq/hit-set-check

osd/ReplicatePG: correctly checking if an object is in hit set

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #4017 from ceph/wip-11125

rgw: update keystone cache with token info

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #4019 from NTmatter/doc-typo-monap

doc/rados/operations/add-or-rm-mons: correcting minor typo

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc/rados/operations/add-or-rm-mons: correcting minor typo

Signed-off-by: Thomas Johnson <NTmatter@gmail.com>

PGLog: split divergent priors as well

Signed-off-by: Samuel Just <sjust@redhat.com>

rgw: update keystone cache with token info

Fixes: #11125
Backport: hammer, firefly

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #3997 from ceph/wip-debian-mv-var-lib-ceph-mds

debian: move /var/lib/ceph/mds to ceph-mds package

Reviewed-by: James Page <james.page@ubuntu.com>
Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3945 from tchaikov/hammer-fix-xio-requeue

XIO: Handle requeue case of XIO messages

Merge pull request #3977 from tchaikov/fix-xio-loop-conn-cleanup

Fix XioLoopbackConnection Lifecycle.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #3872 from dachary/wip-7334-partx-test

ceph-disk: test activate on the second journal

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #4014 from ceph/wip-mon-doc

doc/rados/operations/add-or-rm-mons: revise doc a bit to be less confusing

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3922 from ceph/wip-11053

mds: fix assertion caused by system clock backwards

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #3752 from ceph/wip-10744

mds: give up replicas of a stopping mds's stuff

Reviewed-by: Yan, Zheng <zyan@redhat.com>

mds: give up replicas of a stopping mds's stuff

In order for an MDS to make it through stopping when
it had some strays, the other ranks that serviced
the migrate_stray renames must ensure that they
give up any cache objects that belonged to
the stopping MDS, so that the stopping MDS
can finish emptying its cache.

Fixes: #10744
Signed-off-by: John Spray <john.spray@redhat.com>

doc/rados/operations/add-or-rm-mons: revise doc a bit to be less confusing

Signed-off-by: Sage Weil <sage@redhat.com>

debian: move /var/lib/ceph/mds to ceph-mds package

On Debian, ceph-mds was split out into its own package in
9d6013e0db47b258cbcde4c692554a2764812099, but the /var/lib/ceph/mds
directory was not moved along with the rest of the mds pieces.

The /var/lib/ceph/mds directory is only necessary if a user has
installed ceph-mds. Move it to the ceph-mds subpackage.

http://tracker.ceph.com/issues/10587 Refs: #10587

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

librbd: acquire cache_lock before refreshing parent

cache_lock needs to be acquired before snap_lock to avoid
the potential for deadlock.

Fixes: #5488
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librados_test_stub: AIO operation callbacks should be via Finisher

librados will execute all AIO callbacks via a single finisher to
prevent blocking the Objecter.  Reproduce this behavior to avoid
deadlocks that only exist when using the test stub.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #4002 from zhicheng/master

fix doc/dev/network-protocol.rst typo

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4007 from dachary/wip-warnings

fix a few minor compilation warnings

Reviewed-by: Kefu Chai <kchai@redhat.com>

fix a few minor compilation warnings

Signed-off-by: Loic Dachary <ldachary@redhat.com>

fix doc/dev/network-protocol.rst typo

Signed-off-by: Zhicheng Wei <zhicheng@opensourceforge.net>

Merge pull request #3989 from javacruft/wip/fixup-cycles-crossarch

Add support for PPC architecture, provide fallback

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #4006 from dachary/wip-documentation

doc: mark the ascii schema as a literal block

Reviewed-by: Yann Dupont <yann@objoo.org>

doc: mark the ascii schema as a literal block

Signed-off-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #4005 from dachary/wip-releases

doc: release timeline

Reviewed-by: Yann Dupont <yann@objoo.org>

Merge pull request #3990 from zaitcev/first_contrib

Specify the actual struct used in protocol handshake

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #3992 from ceph/wip-package-perf-objectstore

packaging: include ceph_perf_objectstore

Reviewed-by: Loic Dachary <ldachary@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>

doc: release timeline

Signed-off-by: Loic Dachary <ldachary@redhat.com>

rgw: send Last-Modified header in response for PUT on Swift object.

Fixes: #10650
Backport: hammer
Signed-off-by: Radoslaw Zarzynski <rzarzynski@mirantis.com>

Merge pull request #3982 from dzafman/wip-10678-2

Update ceph.8 man page to match ceph -h output

Reviewed-by: Nilamdyuti Goswami <ngoswami@redhat.com>

doc: Fix ceph command manpage to match ceph -h (hammer)

Include rebuilt man/ceph.8

Fixes: #10678
Signed-off-by: David Zafman <dzafman@redhat.com>

doc: Fix ceph command manpage to match ceph -h (firefly)

Improve synopsis section
Fixes: #10676
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 9ac488c1eb0e30511079ba05aaf11c79615b3940)

Conflicts:

doc/man/8/ceph.rst
man/ceph.8

PGBackend: do not rewrite ec object oi checksums

Deep scrub does not actually give us the whole-object checksum for an ec
object, only the checksum for the first shard.  We ignore it in scrub
for ec pools anyway in be_select_auth_object.

Signed-off-by: Samuel Just <sjust@redhat.com>

PGBackend: add debug option to rewrite digest even if present

Related: #11027
Signed-off-by: Samuel Just <sjust@redhat.com>

ReplicatedPG: finish_ctx: do not assume that the oi is for the head object

If we are setting the oi on the clone object, we want to update the
mod_desc.

Fixes: 11027
Signed-off-by: Samuel Just <sjust@redhat.com>

packaging: include ceph_perf_objectstore

The /usr/bin/ceph_perf_objectstore file is installed by default. Prior
to this commit it was missing from the packaging. This caused the RPM to
fail to build in mock.

Add ceph_perf_objectstore to the "ceph-test" RPM and Debian package.

If we end up developing further ceph_perf_* utilities, it would make
sense to glob them all with a wildcard, similar to what we are doing
with all the ceph_test_* utilities in ceph-test.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>

Specify the actual struct used in protocol handshake

Maybe it was sockaddr_storage in an older version, but it is
definitely entity_addr_t in "ceph v027".

Signed-off-by: Pete Zaitcev <zaitcev@redhat.com>

Add support for PPC architecture, provide fallback

Add high precision cpu cycles support for powerpc and powerpc64.

Provide a fallback for other architectures and warn during
compilation.

Signed-off-by: James Page <james.page@ubuntu.com>

Merge pull request #3957 from xinxinsh/wip-cleanup

os/{Level,Rocks}DBStore: cleanup in_prefix function

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #3985 from fmarsylle/master

doc: s/that will called every time/that will be called every time/

Reviewed-by: Kefu Chai <kchai@redhat.com>

librbd: Make metadata_set support multi kv pairs

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Fix typo and prefix ++

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Make image metadata copied during image copy

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

cls_rbd: Fix incorrect assert for prefix metadata key

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

test_cli: update rbd client help description

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Add tests for metadata get operation

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Add metadata_get method to support single metadata key retrieve

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Make metadata support clone operation

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

rbd.cc: add metadata cli operations

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

test_librbd: Add librbd interface tests for metadata

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

test_cls_rbd: add metadata cls tests

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd: Add librbd interface impl and tracing for metadata

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

librbd/internal: Add internal implementation for metatdata ops

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

include/librbd: Add metadata interface to librbd

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

cls_rbd: Add cls rbd implementation for librbd metadata

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

cls_rbd_client: Add cls rbd client interface to support rbd metadata

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>

Merge pull request #3986 from Al3xis-N/master

doc: How to generate an object corpus : should be a subsection

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: How to generate an object corpus : should be a subsection

Fixes: #11099
Signed-off-by: Alexis Normand <n.al3xis@gmail.com>

Merge pull request #3987 from RobinDehu/master

doc: extra \ in CEPH\_AUTH\_UNKNOWN

Reviewed-by: Loic Dachary <ldachary@redhat.com>

doc: s/that will called every time/that will be called every time/

fixes: #11098

Signed-off-by: florian marsylle <florian.marsylle@hotmail.fr>

doc: extra \ in CEPH\_AUTH\_UNKNOWN

Suppress all \

fixes: #11097

Signed-off-by: DEHU Robin <robindehu@gmail.com>

rbd: regenerate rbd(8) man page

merge-diff and status commands, --image-features and --image-shared
options, other stuff.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 356a749f63181d401d16371446bb8dc4f196c2a6)

Merge pull request #3979 from ceph/wip-rbd-map-options

rbd: default map options, new options, misc

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: Fix typo in ceph pg dump_stuck usage in man page

Cause by 9db596974c82207dc0769ca6bc9810a9d6bcb5cc

Signed-off-by: David Zafman <dzafman@redhat.com>

Merge pull request #3981 from ceph/wip-librbd-invalidate-object-map

librbd: invalid snapshot object maps should only affect the snapshot

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #3978 from tchaikov/hammer-sca-fixes

sca fixes ported from master

Reviewed-by: Loic Dachary <ldachary@redhat.com>

librbd: invalidating snapshot object maps should set snapshot flag

Instead of setting the HEAD image flag to indicate an invalid
object map, set the flag for the specific snapshot that is
invalid.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test_cls_rbd: verify set_flags updates snapshots

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd_client: add snap_id param to set_flags

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

cls_rbd: set_flags can now update snapshots

It's possible for an object map to be invalid only for
a snapshot, so allow snapshot flags to be updated. This
will also be required when rebuilding the object map and
clearing the invalid flag.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rbd: fix rw option skipping logic

We slip in an extraneous comma if { "rw": "rw" } happens to be the
first map_options pair:

    # ./rbd map -o rw,share foo
    /dev/rbd0

    -> 127.0.0.1:6789 name=admin,key=client.admin,,share rbd foo -

The kernel mount options parser can handle it, but fix it nonetheless.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: recognize cephx_require_signatures and tcp_nodelay options

Support for these went into 3.19 and 4.0 kernels respectively.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: add rbd default map options config option

Allow supplying rbd map -o / --options through 'rbd default map
options' config option.  Options specified on the command line take
precedence on a per-option basis, i.e. default and cli options are
merged with a preference for the latter.  For example:

    # grep 'rbd default map options' ceph.conf
            rbd default map options = "ro,nocrc"
    # ./rbd map -o noshare,rw foo
    /dev/rbd0

results in a rw mapping with nocrc,noshare.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: drop redefining map option warning

For example:

    # ./rbd map -o share,noshare foo
    rbd: warning: redefining map option share: 'share' -> 'noshare'
    /dev/rbd0

mount(8) doesn't do this and it's a bit silly to begin with.  Drop this
in preparation for adding rbd default map options config option.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd: regenerate rbd(8) man page

merge-diff and status commands, --image-features and --image-shared
options, other stuff.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>