msg/async/rdma: remove assert from ibv_dealloc_pd in ProtectionDomain
destructor

issue: 1044307

Change-Id: I2fe2099c5f4403f14877e0fbe5d4868dd34d4f9b
Signed-off-by: DanielBar-On <danielbo@mellanox.com>

Merge pull request #15317 from liupan1111/wip-fix-list

rbd-nbd: display pool/image/snap information in list output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15803 from mikulely/cleanup-rgw_rados_doc

rgw: update function doc in rgw_rados.h  and rgw_rados.cc

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15261 from mychoxin/yuanxin_use_func

os/filestore: when print log, use __func__ instead of hard code function name

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #15354 from badone/wip-rados-ls-auth-fix

osd: Reverse order of op_has_sufficient_caps and do_pg_op

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15649 from liewegas/wip-20274

osd/PrimaryLogPG: record prior_version for DELETE events

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15601 from liewegas/wip-no-restful-cert

mgr: remove default cert; disable both restful and dashboard by default

Reviewed-by: Boris Ranto <branto@redhat.com>

rgw: update rgw_rados.h function doc

+ update system_obj_get_attr func comment
+ drop deprecated func comment for get_obj
+ update func comment for pool_iterate_begin

Signed-off-by: Jiaying Ren <jiaying.ren@umcloud.com>

Merge pull request #15755 from Adirl/wip-HUGEPAGES

msg/async/rdma: automatically set RDMAV_HUGEPAGES_SAFE according to conf

Reviewed-by: Haomai Wang <haomai@xsky.com>

qa/workunits/rest/test_mgr_rest_api: remove ruleset

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/rados/rest/mgr-restful: configure cert

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr/restful: add 'restful restart' command

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15757 from runsisi/wip-image-ts

librbd: add create timestamp metadata for image

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15444 from wjwithagen/wip-wjw-freebsd-termwidth

test: Make screencandy optional for FreeBSD

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/PGLog: ignore ERROR entires in _merge_object_divergent_entries

Sometimes ERROR log entries do not have prior_version.  Also,
they aren't actually updates and don't affect the object.

Fixes: http://tracker.ceph.com/issues/20274
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15783 from liewegas/wip-20358

os/bluestore: move sharedblob to new collection in same shard

Merge pull request #15785 from liewegas/wip-bluestore-fsck

qa/objectstore/bluestore*: fsck on mount

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15734 from dillaman/wip-librbd-image-remove

librbd: fix issues with image removal state machine

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #15727 from ovh/bp-runtime-cfg-of-hb-inflator

osd, messages/MOSDPing: bunch of fixes related to ping inflation

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15723 from badone/wip-ceph-mon-startup-error-logging

mon: Log errors at startup

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15784 from liewegas/wip-thrash-cleanup

qa/tasks/ceph_manager: get osds all in after thrashing

Merge pull request #15787 from liewegas/wip-19943-fix

ceph-objectstore-tool: do not populate snapmapper with missing clones

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #15813 from tchaikov/wip-cmake

cmake: do not add dependencies to INTERFACE library on cmake < 3.3

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

pybind/mgr/restful: remove default path names for the cert

I don't think these are useful.

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr/restful: do not start if no certificate is configured

This removes the default filename, by the way.  We also work around a
problem with make_server where it sets up the socket to listen before
checking for the cert, thereby making it problematic to rebind to the
port shortly thereafter when we do have a socket.  (SO_REUSEADDR would
be appropriate but there doesn't seem to be an easy way to make
make_server use it.)

Signed-off-by: Sage Weil <sage@redhat.com>

restful: Introduce create-self-signed-cert command

This implements a create-self-signed-cert command for the ceph-mgr
restful interface.

It is designed so that it will try to restart the module once the cert
is created.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #15060 from shashalu/fix-slo-dlo-range

rgw: fix SLO/DLO range requests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15791 from liewegas/wip-20360

Revert "qa: do not restrict valgrind runs to centos"

cmake: do not add dependencies to INTERFACE library on cmake < 3.3

otherwise we will have

add_dependencies Cannot add target-level dependencies to INTERFACE
library

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #13683 from liewegas/wip-crush-rulesets

crush: detect and (usually) fix ruleset != rule id

Reviewed-by: Loic Dachary <ldachary@redhat.com>

Merge pull request #15789 from linuxbox2/wip-nfs-doc

RGW NFS: add nfs.rst to doc/radosgw
(override because make check fail [missing selinux policy help file] unrelated)

RGW NFS: add nfs.rst to doc/radosgw

This document contains sections by John Wilkins <jowilkin@redhat.com>.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #15255 from david-z/wip-mgr-fsstatus

pybind/mgr/fsstatus: use mds_mem.dn as dentry counter

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #15621 from mingshuaiwang/master

mds: mds perf item 'l_mdl_expos' always behind journaler

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #15662 from renhwztetecs/renhw-wip-client-trim_cache

client: call the lru_remove() twice,when trim cache

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #15715 from jcsp/wip-jt-rmapply

tools/cephfs: remove `apply` mode of cephfs-journal-tool

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #15779 from renhwztetecs/renhw-wip-client-dentry

client: fix Dentry::dump

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: fix SLO/DLO range requests

Sending any SLO/DLO range requests will retrun InvalidRange error. SLO/DLO should call range_to_ofs after got object size instead of read_op.prepare() to deal with negative offsets.

Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>

Merge pull request #15728 from majianpeng/bluefs-remove-memcopy-for-continue-memory

os/bluestore/BlueFS: Rebuild memcopy for bufferlist::page_aligned_app…

Reviewed-by: Sage Weil <sage@redhat.com>

unittest_bufferlist: test page_aligned_appender

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore/BlueFS: replace claim_append with claim_append_piecewise to reduce memory copy.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common/buffer: add function  bufferlist::claim_append_piecewise(list&  bl).

This only useful for bl is bufferlist::page_aligned_appender. Using
this function can remove memcopy for continue ptrs.
Because page_aligned_appender::flush will split a ptr into two or
more ptrs. For this case, rebuild_aligned_size_and_memory can't handle,
it  will rebuild.

For example
a=bl.get_page_aligned_appender(1);
a.append(3K)
a.flush();
t.claim_append(bl);
a.append(1K);
a.flush();
t.claim_append(bl);

dst.claim_append(t);
//3K and 1K ptr are continue in memory. But they are two ptrs..
dst.is_aligned_size_and_memory(4096,4096) is false.

We add new function claim_append_piecewise() to specially
handle this case.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

osd: replicated_ruleset -> replicated_rule

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr/restful: remove default of 127.0.0.1

i.e., disabled until explicitly enabled by an admin

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr/dashboard: remove default of 127.0.0.1

i.e., disabled until explicitly enabled by an admin.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph.spec,debian: remove auto-generated self-signed mgr restful cert

Signed-off-by: Sage Weil <sage@redhat.com>

Revert "qa: do not restrict valgrind runs to centos"

This reverts commit 5923961465e1efd3cc6bc072fef293dda3d975d0.

See http://tracker.ceph.com/issues/20360

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15260 from shashalu/fix-secondary-zonegroup-cors

rgw: update bucket cors in secondary zonegroup should forward to master

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15709 from jcsp/wip-dashboard-loglast

mgr/dashboard: load log lines on startup, split out audit log

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15750 from xiexingguo/wip-bluestore-20170619

os/bluestore: refactor BlueStore::_do_write;  kill dead ExtentMap::find_lextent() method

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #15766 from liewegas/wip-20227-fix

os/bluestore: fix unsharing blob dirty_range args

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15014 from zhangsw/fix-rgw-versionIdMarker-bug

rgw: VersionIdMarker and NextVersionIdMarker should be returned when listing object versions

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #14500 from zhangsw/fix-rgw-multipart-index-bug

rgw: fix the bug that part's index can't be removed after completing multipart upload

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

os/bluestore: debug OmapIteratorImpl

Signed-off-by: Sage Weil <sage@redhat.com>

osd/SnapMapper: debug keys inserted, removed, and iterated

Hunting bug http://tracker.ceph.com/issues/19943

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-objectstore-tool: do not populate snapmapper with missing clones

Cache pools may have clones represented in SnapSet that are not stored
locally.  This is fine as long as ceph-objectstore-tool doesn't go and
add them when it shouldn't.

Fixes: http://tracker.ceph.com/issues/19943
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15771 from fangyuxiangGL/cleanup_rgw_admin

rgw: cleanup unused codes in rgw_admin.cc

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15768 from ukernel/wip-objecter-inflight-ops

osdc/Objecter: fix inflight_ops update

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15737 from wjwithagen/wip-wjw-test_rgw_string

rgw/rgw_string.h: FreeBSD would like errno.h included

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #15782 from cbodley/wip-rgw-vstart-s3tests

vstart: add rgw configuration needed to pass all s3tests

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

vstart: add rgw configuration needed to pass all s3tests

Signed-off-by: Casey Bodley <cbodley@redhat.com>

qa/objectstore/bluestore*: fsck on mount

This will catch more bugs!

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #15751 from xiexingguo/wip-add-state-name

src/common/ceph_string: stringify new osd states

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/tasks/ceph_manager: get osds all in after thrashing

Otherwise we might end up with some PGs remapped, which means they won't
get scrubbed.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: move sharedblob to new collection in same shard

We have to move the SharedBlob to the new collection even if
the new collection is part of the same cache shard.  It's only the buffers
that need to be conditioned on the shard.

Fixes: http://tracker.ceph.com/issues/20358
Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #14703 from zhangsw/feature-lifecycle-deletemarker

rgw: add support for delete marker expiration in s3 lifecycle.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #15540 from joscollin/wip-cleanup-test-commented-code-2

test/osd: Removed Commented Code - 2

Merge pull request #15770 from tchaikov/wip-test-test_mon_crush

qa/workunits/cephtool/test.sh: only include last line for epoch

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #15733 from wjwithagen/wip-wjw-freebsd-ceph-disk-service

ceph-disk: command invocation needs all fields separate

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/osd: Removed Commented Code - 2

Removed commented code found in test/osd/.

Fixes: http://tracker.ceph.com/issues/20207 (Partially)
Signed-off-by: Jos Collin <jcollin@redhat.com>

ceph-disk: command invocation needs all fields separate

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

librbd: removed synchronous watcher check from remove image state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: clean up remove request state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: remove state machine should ignore group not supported errors

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #15592 from david-z/wip-fix-dump-cache

mds: explicitly output error msg for dump cache asok command

Reviewed-by: John Spray <john.spray@redhat.com>

msg/async/rdma: automatically set RDMAV_HUGEPAGES_SAFE according to conf

Signed-off-by: DanielBar-On <danielbo@mellanox.com>

qa/workunits/cephtool/test.sh: only include last line for epoch

/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/qa/workunits/cephtool/test.sh:1606: test_mon_crush:  epoch='2017-06-20 04:44:52.862459 ffffad4d0200 -1 asok(0xffffa8000b10) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: The UNIX domain socket path /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/src/test/td/t-7202/out/client.admin.48876.asok is too long! The maximum length on this system is 107
12'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/qa/workunits/cephtool/test.sh:1607: test_mon_crush:  '[' '2017-06-20 04:44:52.862459 ffffad4d0200 -1 asok(0xffffa8000b10) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: The UNIX domain socket path /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/src/test/td/t-7202/out/client.admin.48876.asok is too long! The maximum length on this system is 107
12' -gt 1 ']'
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/qa/workunits/cephtool/test.sh: line 1607: [: 2017-06-20 04:44:52.862459 ffffad4d0200 -1 asok(0xffffa8000b10) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: The UNIX domain socket path /home/jenkins-build/build/workspace/ceph-pull-requests-arm64/build/src/test/td/t-7202/out/client.admin.48876.asok is too long! The maximum length on this system is 107
12: integer expression expected

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: cleanup unused codes in rgw_admin.cc

Signed-off-by: fang yuxiang <fang.yuxiang@eisoo.com>

Merge pull request #15742 from wjwithagen/wip-wjw-freebsd-host-errno-3

common/freebsd_errno.cc: fixed again a stupid typo

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/cli-integration/rbd: verify formatting for image create timestamp

Signed-off-by: runsisi <runsisi@zte.com.cn>

test/unittest_rbd: add unit tests for image create timestamp

Signed-off-by: runsisi <runsisi@zte.com.cn>

pybind/rbd: add support for image create timestamp

Signed-off-by: runsisi <runsisi@zte.com.cn>

rbd: include create timestamp for info command

Signed-off-by: runsisi <runsisi@zte.com.cn>

librbd: add create timestamp metadata for image

Signed-off-by: runsisi <runsisi@zte.com.cn>

osdc/Objecter: fix inflight_ops update

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge pull request #15651 from zealoussnow/wip-0613

config_opt: use bool instead of int for the default value of filestore_debug_omap_check

Merge pull request #15676 from Linbing/wip-mon-delete-unused-config

mon: delete unused config opts of mon_sync_fs_threshold

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #15697 from gregsfortytwo/wip-mgr-eaccss-20296

mgr: print a more helpful error message for when users lack mgr ceph caps

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #15744 from badone/wip-CID-1412579

ceph-dencoder: Silence coverity CID 1412579

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #15637 from liewegas/wip-point-to-point

qa/upgrade/jewel-x/point-to-point: add a mgr during final upgrade

os/bluestore: fix unsharing blob dirty_range args

Fixes: http://tracker.ceph.com/issues/20227
Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: add_simple_ruleset[_at] -> add_simple_rule[_at]

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: some cosmetic cleanup (ruleset -> rule)

Signed-off-by: Sage Weil <sage@redhat.com>

crush: simplify osd_pool_default_crush_rule config

Make an incompat change here with a release note since
this only affects pool creation, a rare event, and folks
who have customized their configs (also rare).

Keep it simple: a config sets the default rule, or else we pick
the first TYPE_REPLICATED pool in the crush map.

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: rename functions and variables, ruleset -> rule

Signed-off-by: Sage Weil <sage@redhat.com>

mon: scrub 'ruleset' from user-visible output

Signed-off-by: Sage Weil <sage@redhat.com>

mon: remove 'crush_ruleset' property

The 'crush_rule' name has been present since before hammer; users should
use that instead

Signed-off-by: Sage Weil <sage@redhat.com>

osd/osd_types: pg_pool_t: rename crush_ruleset -> crush_rule

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: rename 'osd pool create' argument field name

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: optimize find_rule when rules are uniform

On finalize, set a flag indicating whether our rules are uniform.  If so,
find_rule() is trivial--it just verifies the rule exists.

Signed-off-by: Sage Weil <sage@redhat.com>