msg/async/rdma: Change port num into int to be printed properly

Issue: 995322
Change-Id: If3095f2cc257cfe110652adca4a6175644c0230e
Signed-off-by: Amir Vadai <amir@vadai.me>

Merge pull request #13986 from dillaman/wip-19081

librbd: image create validates that pool supports overwrites

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13905 from yuyuyu101/wip-fix-assert

msg/async/rdma: fix outstanding queuepair when destruct RDMAStack

Reviewed-by: Adir lev <adirl@mellanox.com>

Merge pull request #13983 from gaurav36/bug-19273-fix-typo

rbd: fix typo in Kernel.cc

Reviewed-by: Nathan Cutler <ncutler@suse.com>

librbd: image create validates that pool supports overwrites

Fixes: http://tracker.ceph.com/issues/19081
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: clean up debug messages within image create state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13628 from jcsp/wip-19075

mds: extend 'p' auth cap to cover all vxattr stuff

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13807 from jcsp/wip-19118

mds: try to avoid false positive heartbeat timeouts

librbd: removed legacy state machine callbacks from CreateRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13913 from wangzhengyong/release

os/bluestore/Allocator: drop unused return value in release function

Reviewed-by: Sage Weil <sage@redhat.com>

rbd: fix typo in Kernel.cc

Fixes: http://tracker.ceph.com/issues/19273
Reported-by: Shinobu Kinjo <shinobu@redhat.com>
Signed-off-by: Gaurav Kumar Garg <garg.gaurav52@gmail.com>

Merge pull request #13928 from oritwas/wip-rgw-19195

rgw: use rgw_zone_root_pool for region_map like is done in hammer

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13977 from runsisi/wip-fix-rbd-py

pybind: fix docstring for librbd Python binding

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13976 from tchaikov/wip-fix-docs-links

doc: fix the links to http://ceph.com/docs

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: fix the links to http://ceph.com/docs

they should point to http://docs.ceph.com/docs/master/.. instead

Fixes: http://tracker.ceph.com/issues/19090
Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind: fix docstring for librbd Python binding

Signed-off-by: runsisi <runsisi@zte.com.cn>

Merge pull request #13817 from ceph/wip-vstart-rgw-help

vstart: add --help documentation for rgw_num

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13935 from chuang-he/fix_mutex_locker_of_compressor

compressor: fix Mutex::Locker used is not correct

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13945 from ceph/wip-selinux-psched

selinux: Allow ceph daemons to read net stats

Reviewed-by: Kefu Chai <kchai@redhat.com>

vstart: --help usage for rgw_num, mgr_num, mon, mds, osd

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #13949 from cbodley/wip-15618

rgw: clear old zone short ids on period update

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13948 from cbodley/wip-rgw-data-sync-versioned

rgw: data sync includes instance in rgw_obj_index_key

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13586 from Liuchang0812/wip-add-override-in-rgw

rgw: add override in header files mostly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12793 from zhangsw/fix-rgw-abortmultipart

rgw: AbortMultipart request returns NoSuchUpload error if the meta obj doesn't exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13900 from oritwas/wip-rgw-19231

fix upgrate from hammer when zone doesn't have zoneparams

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13354 from Liuchang0812/wip-18828

rgw: don't return skew time error in pre-signed url

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13798 from tchaikov/wip-brag-count-mds

brag: count the number of mds in fsmap not in mdsmap

Reviewed-by: John Spray <john.spray@redhat.com>

Merge branch 'wip-omap-getkeys-mds'

qa/suites/fs/basic: vary omap osd-side limit

Ensure that the omap looping paths are exercised.

Signed-off-by: Sage Weil <sage@redhat.com>

mds/CDir: set sane max keys per op

Set default to 16384, which is > than the max frag size of 10000.  We will
loop if there are more.

Signed-off-by: Sage Weil <sage@redhat.com>

mds/CDir: fetch dir omap in batches as needed

Signed-off-by: Sage Weil <sage@redhat.com>

mds/SessionMap: use pmore for loading sessions

Keep the result size check for a while since older OSDs won't populate
the pmore field for us.  Eventually we can remove it.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13812 from idryomov/wip-krbd-feature-disable-msg

rbd: spell out image features unsupported by the kernel

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13924 from tchaikov/wip-silence-gcc-warnings

os/bluestore,test/ceph_test_objectstore: silence gcc warnings

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

rgw/cleanup: add override in headers

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #13700 from yuyuyu101/wip-msgr-lossy

msg/async: avoid lossy connection sending ack message

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13805 from tchaikov/wip-qa-upgrade-tiering

qa/suites/upgrade: add tiering test to hammer-jewel-x

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #13727 from jcsp/wip-send-command

osdc: clean up osd_command/start_mon_command interfaces

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13825 from wjwithagen/wip-wjw-stack-compat

msg/async/Stack.cc: use of pthread_setname_np() needs compat.h

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #10272 from yonghengdexin735/wip-zzz-delete=

msg/simple/Pipe:the returned value for do_recv unequal to zero

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12578 from kylinstorage/wip-filestore-ref

os/filestore: avoid unnecessary copy in filestore::_do_transaction

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13894 from majianpeng/do_op_duplicated_code

osd/PrimaryLogPG: remove duplicated code.

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

rgw: clear old zone short ids on period update

the short ids of old, removed zones were being kept in the period to
guard against hash collisions with new zones

but for a hash collision to cause a wrong object to sync, that object
would have to be uploaded simultaneously to two different zones that had
the same short id

to avoid this, we just have to prevent the period from containing two
colliding zones at the same time - we don't have to remember old zone
short ids forever

Fixes: http://tracker.ceph.com/issues/15618
Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: data sync includes instance in rgw_obj_index_key

this was breaking the versioned object test in test_multi.py, because we
were trying to create delete markers instead of unlinking a specific
object instance

Signed-off-by: Casey Bodley <cbodley@redhat.com>

selinux: Allow ceph daemons to read net stats

Fixes: http://tracker.ceph.com/issues/19254
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #13919 from dillaman/wip-19256

rbd: prevent adding multiple mirror peers to a single pool

Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12264 from cbodley/wip-rgw-print-is-admin

rgw: print is_admin as int instead of __u8

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13722 from dachary/wip-crush-doc

crush: document tunables and rule step set_

Reviewed-by: Sage Weil <sage@redhat.com>

crush: document tunables and rule step set_

Signed-off-by: Loic Dachary <loic@dachary.org>

compressor: fix Mutex::Locker used is not correct

Signed-off-by: hechuang <hechuang@xsky.com>

Merge pull request #13781 from SUSE/wip-19184

build/ops: make package groups comply with openSUSE guidelines

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

rbd: prevent adding multiple mirror peers to a single pool

The rbd-mirror daemon does not currently support replication
from multiple peers. Until that is supported, add a temporary
restriction to prevent confusion.

Fixes: http://tracker.ceph.com/issues/19256
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13926 from trociny/wip-19260

librbd: fix clang compilation error

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: use rgw_zone_root_pool for region_map like is done in hammer

Fixes: http://tracker.ceph.com/issues/19195
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

librbd: make build with clang

Fixes: http://tracker.ceph.com/issues/19260
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13694 from liupan1111/wip-fix-no-error-map

rbd-nbd: warn when kernel parameters are ignored

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd-nbd: only try to load nbd module once when open nbd device, during do_map operation.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

rbd-nbd: warn when kernel parameters are ignored

Fixes: http://tracker.ceph.com/issues/19108
Signed-off-by: Pan Liu <liupan1111@gmail.com>

os/bluestore: silence gcc warning

silences following warning:

src/os/bluestore/BitAllocator.h:410:8: warning: ‘virtual bool
BitMapAreaIN::child_check_n_lock(BitMapArea*, int64_t, bool)’ was hidden
[-Wover\
loaded-virtual]
   bool child_check_n_lock(BitMapArea *child, int64_t required, bool
lock) {
        ^~~~~~~~~~~~~~~~~~
/var/ceph/ceph/src/os/bluestore/BitAllocator.h:489:8: warning:   by
‘BitMapAreaLeaf::child_check_n_lock’ [-Woverloaded-virtual]
   bool child_check_n_lock(BitMapZone* child, int64_t required, bool
lock);
        ^~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/ceph_test_objectstore: silence gcc warning

silence following warning:

store_test.cc: In member function ‘virtual void
StoreTest_ManyObjectTest_Test::TestBody()’:
/var/ceph/ceph/src/test/objectstore/store_test.cc:3153:6: warning:
suggest explicit braces to avoid ambiguous ‘else’ [-Wdangling-else]
   if (listed2.size())
      ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13801 from xiexingguo/wip-fix-func-name

osd: fix func name in log produced by handle_pg_peering_evt()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13856 from zhangsw/cleanup-radosgw-admin-tier_type

rgw: remove useless --tier_type in radosgw-admin.

Reviewed-by: Jiaying Ren <mikulely@gmail.com>

rbd: spell out image features unsupported by the kernel

/sys/bus/rbd/supported_features is available on kernels starting with
4.11.  Example output:

    $ sudo rbd map x
    rbd: sysfs write failed
    RBD image feature set mismatch. This image cannot be mapped because the following immutable features are unsupported by the kernel: data-pool.
    In some cases useful info is found in syslog - try "dmesg | tail".
    rbd: map failed: (6) No such device or address

    $ sudo rbd map -p foo y
    rbd: sysfs write failed
    RBD image feature set mismatch. You can disable features unsupported by the kernel with "rbd feature disable foo/y object-map fast-diff deep-flatten".
    In some cases useful info is found in syslog - try "dmesg | tail".
    rbd: map failed: (6) No such device or address

or, if /sys/bus/rbd/supported_features isn't there:

    $ sudo rbd map z
    rbd: sysfs write failed
    RBD image feature set mismatch. Try disabling features unsupported by the kernel with "rbd feature disable".
    In some cases useful info is found in syslog - try "dmesg | tail".
    rbd: map failed: (6) No such device or address

Fixes: http://tracker.ceph.com/issues/19095
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #13920 from cbodley/wip-rgw-fix-cleanup-meta-sync

rgw: fix transition from full to incremental meta sync

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #11586 from cbodley/wip-rgw-fetchallmeta-yield-break

rgw: fix break inside of yield in RGWFetchAllMetaCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #10407 from yonghengdexin735/wip-zzz-perf-counters

common/perf_counters.: Remove unnecessary judgment

Reviewed-by: Sage Weil <sage@redhat.com>

rgw: fix transition from full to incremental meta sync

when transitioning to incremental sync, we need to write temp_marker
instead of sync_marker, because temp_marker has replaced the full sync
marker (which is a metadata key, rather than an log marker) with
next_step_marker

this was breaking the metadata checkpoint in test_multi.py, because it
was comparing metadata keys (i.e. 'user:zone.user') against mdlog
markers

Signed-off-by: Casey Bodley <cbodley@redhat.com>

build/ops: extend rpm spec file copyright period to 2017

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: make package groups comply with openSUSE guidelines

. . . and put all Group: lines in SUSE conditional blocks.

Fixes: http://tracker.ceph.com/issues/19184
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13737 from dillaman/wip-19104

test: support blacklisting within librados_test_stub

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: correct race conditions within leader watcher

It was possible for multiple, concurrent acquire attempts to be
running for a single instance due to notification messages. Now
only a single acquire state machine can be running.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

os/bluestore/Allocator: drop unused return value in release function

Signed-off-by: wangzhengyong@cmss.chinamobile.com

librbd: improve debug log messages for managed lock

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test: potential shutdown race in TestMockImageReplayer.DecodeError

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: avoid duplicating librados IoCtx objects if not needed

This introduces the potential for shutdown race conditions within
the unit tests.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librados_test_stub: support blacklisting of connections

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librados_test_stub: watch/notify should be connection-aware

Utilize the AIO completion thread of the local connection to best
simulate how librados actually performs when connected a cluster.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

test/librados_test_stub: separate mock cluster from connection

The plan to support independent connections sharing the same
view of the mocked cluster.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13857 from liupan1111/wip-rename-onack-to-oncommit

librados: replace the var name from onack to complete

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #11485 from yehudasa/wip-bucket-cleanup

rgw structures rework

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13758 from dillaman/wip-18748

rbd: demote/promote all mirrored images in a pool

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rgw: fix crash when listing objects via swift

Fixes: http://tracker.ceph.com/issues/19249
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

qa/tasks/radosgw_admin: adjust test to new bucket structure

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: more fixes following rebase

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: new rest api to retrieve object layout

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: get_obj_state() checks for empty oids

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: metadata put of bucket instance sets bucket_id

Need to parse the bucket id off the entry and then set it on the
bucket struct.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw_admin: remove broken check

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

mrgw.sh: fix script

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix async cr operation

Fix crash due to code cleanup. Changes scope of obj ref.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: modify default pools to use namespaces

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: handle pools namespace

Use rgw_pool all around, and replace librados::create_ioctx() with
helper that also sets the namespace.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: add namespace to rgw_pool

add a namespace field to the rgw_pool struct

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: compilation and other fixes following rebase

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: move placement rule out of rgw_bucket

Bucket's placement rule is in the bucket instance's info. Object's
placement rule is in the manifest

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

cls/version: add more useful logging

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: fix ECANCELED handling in rgw_get_system_obj()

Fixes: http://tracker.ceph.com/issues/17996
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: keep rgw_obj key info in rgw_obj_key field

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: add rgw_obj_index_key, kill RGWObjEnt

Use rgw_obj_index_key to represent entries in bucket index (typedef of
cls_rgw_obj_key). Get rid of RGWObjEnt, it was duplicate of rgw_bucket_dir_entry
anyway.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>

rgw: clean rgw_obj

Instead of storing the oid and the name, just store the name
and calculate it when needed (same goes to locator). Also give more
coherent names to the various fields.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>