Merge pull request #14019 from singler/wip-cleanup-create_new_fs

mon/MDSMonitor: remove create_new_fs from header

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #13684 from shashalu/rgw-admin-init-bucket-for-sync

rgw: fix init_bucket_for_sync retcode

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mon/MDSMonitor: remove create_new_fs from header

create_new_fs was refactored and moved earlier. This removes a left
over header entry.

Signed-off-by: Henrik Korkuc <henrik@kirneh.eu>

Merge pull request #14012 from optimistyzy/0317_2

bluestore,NVMEDevice: minor error for get slave core

Reviewed-by: Haomai Wang <haomai@xsky.com>

bluestore,NVMEDevice: minor error for get slave core

The second parameter should set to 1 to skip master core.

Signed-off-by: Ziye Yang <optimistyzy@gmail.com>

Merge pull request #13646 from mslovy/wip-fix-test-pidfile

test: fix test_pidfile

Reviewed-by: Loic Dachary <ldachary@redhat.com>

rgw: fix init_bucket_for_sync retcode

init_bucket_for_sync error retcode should be negative

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>

Merge pull request #13953 from linuxbox2/wip-rmdir-check

rgw_file: implement reliable has-children check (unlink dir)

Merge pull request #13951 from smithfarm/wip-drop-obsolete-bt

tests: drop obsolete Perl scripts

Merge pull request #13944 from liupan1111/wip-fix-nbd-issues

rbd-nbd: don't ignore --read-only option in BLKROSET ioctl

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13893 from hrchu/rgw-admin-api-doc

doc: Update adminops.rst

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13688 from theanalyst/fix/rgw/http-client-dout-msg

rgw: http_client clarify the debug msg function call

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13985 from ebozag/master

doc: Fixes a typo.

Reviewed-by: John Wilkins <jowilkin@redhat.com>

Merge pull request #13761 from zhangsw/cleanup-rgw-unlinkobj

rgw: correct the debug info when unlink instance failed.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

rgw_file:  remove unused rgw_key variable

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file:  rgw_readdir:  return dot-dirs only when *offset is 0

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw_file: implement reliable has-children check (unlink dir)

Bug report and discussion provided by
Gui Hecheng <guihecheng@cmss.chinamobile.com> in nfs-ganesha upstream
github.  Briefly, while a reliable check is potentially costly,
it is necessary.

Fixes: http://tracker.ceph.com/issues/19270
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13408 from zhangsw/fix-rgw-loglevel

rgw: change loglevel to 5 in user's quota sync.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14001 from Adirl/rdma-cm-1

msg/async/rdma: Introduce Device.{cc,h}

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #13930 from drunkard/master

doc: fix a typo

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13958 from wangzhengyong/have_output

mon: drop useless assignment statements

Reviewed-by: Joao Eduardo Luis <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

msg/async/rdma: Introduce Device.{cc,h}

Later on, multidevice support will be added and Device class will be the
main RDMA class (instead of Infiniband as it is today).

Issue: 995322
Change-Id: I060d849de21c8f847dd11ecd3edf1ddfb79b0820
Signed-off-by: Amir Vadai <amir@vadai.me>

msg/async/rdma: Change port num into int to be printed properly

Issue: 995322
Change-Id: If3095f2cc257cfe110652adca4a6175644c0230e
Signed-off-by: Amir Vadai <amir@vadai.me>

Merge pull request #13986 from dillaman/wip-19081

librbd: image create validates that pool supports overwrites

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13905 from yuyuyu101/wip-fix-assert

msg/async/rdma: fix outstanding queuepair when destruct RDMAStack

Reviewed-by: Adir lev <adirl@mellanox.com>

doc: typo fix in rados/operations/crush-map

Signed-off-by: Drunkard Zhang <gongfan193@gmail.com>

Merge pull request #13983 from gaurav36/bug-19273-fix-typo

rbd: fix typo in Kernel.cc

Reviewed-by: Nathan Cutler <ncutler@suse.com>

librbd: image create validates that pool supports overwrites

Fixes: http://tracker.ceph.com/issues/19081
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: clean up debug messages within image create state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13628 from jcsp/wip-19075

mds: extend 'p' auth cap to cover all vxattr stuff

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #13807 from jcsp/wip-19118

mds: try to avoid false positive heartbeat timeouts

doc: Fixes a typo.

Signed-off-by: Edwin F. Boza <eboza@espol.edu.ec>

librbd: removed legacy state machine callbacks from CreateRequest

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

doc: Fixes a typo.

Signed-off-by: Edwin F. Boza <eboza@espol.edu.ec>

Merge pull request #13913 from wangzhengyong/release

os/bluestore/Allocator: drop unused return value in release function

Reviewed-by: Sage Weil <sage@redhat.com>

rbd: fix typo in Kernel.cc

Fixes: http://tracker.ceph.com/issues/19273
Reported-by: Shinobu Kinjo <shinobu@redhat.com>
Signed-off-by: Gaurav Kumar Garg <garg.gaurav52@gmail.com>

Merge pull request #13928 from oritwas/wip-rgw-19195

rgw: use rgw_zone_root_pool for region_map like is done in hammer

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13977 from runsisi/wip-fix-rbd-py

pybind: fix docstring for librbd Python binding

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13976 from tchaikov/wip-fix-docs-links

doc: fix the links to http://ceph.com/docs

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: fix the links to http://ceph.com/docs

they should point to http://docs.ceph.com/docs/master/.. instead

Fixes: http://tracker.ceph.com/issues/19090
Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind: fix docstring for librbd Python binding

Signed-off-by: runsisi <runsisi@zte.com.cn>

Merge pull request #13817 from ceph/wip-vstart-rgw-help

vstart: add --help documentation for rgw_num

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13935 from chuang-he/fix_mutex_locker_of_compressor

compressor: fix Mutex::Locker used is not correct

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13945 from ceph/wip-selinux-psched

selinux: Allow ceph daemons to read net stats

Reviewed-by: Kefu Chai <kchai@redhat.com>

vstart: --help usage for rgw_num, mgr_num, mon, mds, osd

Signed-off-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #13949 from cbodley/wip-15618

rgw: clear old zone short ids on period update

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13948 from cbodley/wip-rgw-data-sync-versioned

rgw: data sync includes instance in rgw_obj_index_key

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13586 from Liuchang0812/wip-add-override-in-rgw

rgw: add override in header files mostly

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #12793 from zhangsw/fix-rgw-abortmultipart

rgw: AbortMultipart request returns NoSuchUpload error if the meta obj doesn't exist

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13900 from oritwas/wip-rgw-19231

fix upgrate from hammer when zone doesn't have zoneparams

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #13354 from Liuchang0812/wip-18828

rgw: don't return skew time error in pre-signed url

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #13798 from tchaikov/wip-brag-count-mds

brag: count the number of mds in fsmap not in mdsmap

Reviewed-by: John Spray <john.spray@redhat.com>

rbd-nbd: should set read_only when --readonly is set.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

rbd-nbd: fixed typo and format in help info.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

Merge branch 'wip-omap-getkeys-mds'

qa/suites/fs/basic: vary omap osd-side limit

Ensure that the omap looping paths are exercised.

Signed-off-by: Sage Weil <sage@redhat.com>

mds/CDir: set sane max keys per op

Set default to 16384, which is > than the max frag size of 10000.  We will
loop if there are more.

Signed-off-by: Sage Weil <sage@redhat.com>

mds/CDir: fetch dir omap in batches as needed

Signed-off-by: Sage Weil <sage@redhat.com>

mds/SessionMap: use pmore for loading sessions

Keep the result size check for a while since older OSDs won't populate
the pmore field for us.  Eventually we can remove it.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #13812 from idryomov/wip-krbd-feature-disable-msg

rbd: spell out image features unsupported by the kernel

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mon: drop useless assignment statements

Signed-off-by: wangzhengyong@cmss.chinamobile.com

Merge pull request #13924 from tchaikov/wip-silence-gcc-warnings

os/bluestore,test/ceph_test_objectstore: silence gcc warnings

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

rgw/cleanup: add override in headers

Signed-off-by: liuchang0812 <liuchang0812@gmail.com>

Merge pull request #13700 from yuyuyu101/wip-msgr-lossy

msg/async: avoid lossy connection sending ack message

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13805 from tchaikov/wip-qa-upgrade-tiering

qa/suites/upgrade: add tiering test to hammer-jewel-x

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

tests: drop obsolete Perl scripts

None of these appear to be used by any tests currently in "master".

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13727 from jcsp/wip-send-command

osdc: clean up osd_command/start_mon_command interfaces

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13825 from wjwithagen/wip-wjw-stack-compat

msg/async/Stack.cc: use of pthread_setname_np() needs compat.h

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #10272 from yonghengdexin735/wip-zzz-delete=

msg/simple/Pipe:the returned value for do_recv unequal to zero

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #12578 from kylinstorage/wip-filestore-ref

os/filestore: avoid unnecessary copy in filestore::_do_transaction

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13894 from majianpeng/do_op_duplicated_code

osd/PrimaryLogPG: remove duplicated code.

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

rgw: clear old zone short ids on period update

the short ids of old, removed zones were being kept in the period to
guard against hash collisions with new zones

but for a hash collision to cause a wrong object to sync, that object
would have to be uploaded simultaneously to two different zones that had
the same short id

to avoid this, we just have to prevent the period from containing two
colliding zones at the same time - we don't have to remember old zone
short ids forever

Fixes: http://tracker.ceph.com/issues/15618
Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: data sync includes instance in rgw_obj_index_key

this was breaking the versioned object test in test_multi.py, because we
were trying to create delete markers instead of unlinking a specific
object instance

Signed-off-by: Casey Bodley <cbodley@redhat.com>

selinux: Allow ceph daemons to read net stats

Fixes: http://tracker.ceph.com/issues/19254
Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #13919 from dillaman/wip-19256

rbd: prevent adding multiple mirror peers to a single pool

Reviewed-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #12264 from cbodley/wip-rgw-print-is-admin

rgw: print is_admin as int instead of __u8

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #13722 from dachary/wip-crush-doc

crush: document tunables and rule step set_

Reviewed-by: Sage Weil <sage@redhat.com>

crush: document tunables and rule step set_

Signed-off-by: Loic Dachary <loic@dachary.org>

compressor: fix Mutex::Locker used is not correct

Signed-off-by: hechuang <hechuang@xsky.com>

Merge pull request #13781 from SUSE/wip-19184

build/ops: make package groups comply with openSUSE guidelines

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

rbd: prevent adding multiple mirror peers to a single pool

The rbd-mirror daemon does not currently support replication
from multiple peers. Until that is supported, add a temporary
restriction to prevent confusion.

Fixes: http://tracker.ceph.com/issues/19256
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #13926 from trociny/wip-19260

librbd: fix clang compilation error

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

rgw: use rgw_zone_root_pool for region_map like is done in hammer

Fixes: http://tracker.ceph.com/issues/19195
Signed-off-by: Orit Wasserman <owasserm@redhat.com>

librbd: make build with clang

Fixes: http://tracker.ceph.com/issues/19260
Signed-off-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #13694 from liupan1111/wip-fix-no-error-map

rbd-nbd: warn when kernel parameters are ignored

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd-nbd: only try to load nbd module once when open nbd device, during do_map operation.

Signed-off-by: Pan Liu <liupan1111@gmail.com>

rbd-nbd: warn when kernel parameters are ignored

Fixes: http://tracker.ceph.com/issues/19108
Signed-off-by: Pan Liu <liupan1111@gmail.com>

os/bluestore: silence gcc warning

silences following warning:

src/os/bluestore/BitAllocator.h:410:8: warning: ‘virtual bool
BitMapAreaIN::child_check_n_lock(BitMapArea*, int64_t, bool)’ was hidden
[-Wover\
loaded-virtual]
   bool child_check_n_lock(BitMapArea *child, int64_t required, bool
lock) {
        ^~~~~~~~~~~~~~~~~~
/var/ceph/ceph/src/os/bluestore/BitAllocator.h:489:8: warning:   by
‘BitMapAreaLeaf::child_check_n_lock’ [-Woverloaded-virtual]
   bool child_check_n_lock(BitMapZone* child, int64_t required, bool
lock);
        ^~~~~~~~~~~~~~~~~~

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/ceph_test_objectstore: silence gcc warning

silence following warning:

store_test.cc: In member function ‘virtual void
StoreTest_ManyObjectTest_Test::TestBody()’:
/var/ceph/ceph/src/test/objectstore/store_test.cc:3153:6: warning:
suggest explicit braces to avoid ambiguous ‘else’ [-Wdangling-else]
   if (listed2.size())
      ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13801 from xiexingguo/wip-fix-func-name

osd: fix func name in log produced by handle_pg_peering_evt()

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #13856 from zhangsw/cleanup-radosgw-admin-tier_type

rgw: remove useless --tier_type in radosgw-admin.

Reviewed-by: Jiaying Ren <mikulely@gmail.com>

rbd: spell out image features unsupported by the kernel

/sys/bus/rbd/supported_features is available on kernels starting with
4.11.  Example output:

    $ sudo rbd map x
    rbd: sysfs write failed
    RBD image feature set mismatch. This image cannot be mapped because the following immutable features are unsupported by the kernel: data-pool.
    In some cases useful info is found in syslog - try "dmesg | tail".
    rbd: map failed: (6) No such device or address

    $ sudo rbd map -p foo y
    rbd: sysfs write failed
    RBD image feature set mismatch. You can disable features unsupported by the kernel with "rbd feature disable foo/y object-map fast-diff deep-flatten".
    In some cases useful info is found in syslog - try "dmesg | tail".
    rbd: map failed: (6) No such device or address

or, if /sys/bus/rbd/supported_features isn't there:

    $ sudo rbd map z
    rbd: sysfs write failed
    RBD image feature set mismatch. Try disabling features unsupported by the kernel with "rbd feature disable".
    In some cases useful info is found in syslog - try "dmesg | tail".
    rbd: map failed: (6) No such device or address

Fixes: http://tracker.ceph.com/issues/19095
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #13920 from cbodley/wip-rgw-fix-cleanup-meta-sync

rgw: fix transition from full to incremental meta sync

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #11586 from cbodley/wip-rgw-fetchallmeta-yield-break

rgw: fix break inside of yield in RGWFetchAllMetaCR

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #10407 from yonghengdexin735/wip-zzz-perf-counters

common/perf_counters.: Remove unnecessary judgment

Reviewed-by: Sage Weil <sage@redhat.com>

rgw: fix transition from full to incremental meta sync

when transitioning to incremental sync, we need to write temp_marker
instead of sync_marker, because temp_marker has replaced the full sync
marker (which is a metadata key, rather than an log marker) with
next_step_marker

this was breaking the metadata checkpoint in test_multi.py, because it
was comparing metadata keys (i.e. 'user:zone.user') against mdlog
markers

Signed-off-by: Casey Bodley <cbodley@redhat.com>

build/ops: extend rpm spec file copyright period to 2017

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: make package groups comply with openSUSE guidelines

. . . and put all Group: lines in SUSE conditional blocks.

Fixes: http://tracker.ceph.com/issues/19184
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #13737 from dillaman/wip-19104

test: support blacklisting within librados_test_stub

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

rbd-mirror: correct race conditions within leader watcher

It was possible for multiple, concurrent acquire attempts to be
running for a single instance due to notification messages. Now
only a single acquire state machine can be running.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>