Merge pull request #19777 from tchaikov/wip-rpm-fix-upgrade

rpm: adjust ceph-{osdomap,kvstore,monstore}-tool feature move

Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-By: Nathan Cutler <ncutler@suse.com>

Merge pull request #18827 from TaewoongKim/anticipation_timeout

osd: Make dmclock's anticipation timeout be configurable

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: myoungwon oh <omwmw@sk.com>

Merge pull request #19704 from trociny/wip-nbd-format

rbd-nbd: output format support for list-mapped command

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19722 from Songweibin/wip-rbd-import

rbd: replace positional_path parameter with arg_index in get_path()

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #17010 from jgibson/bugfix-rgw-s3-policy-ip-address-condition

rgw: S3 Bucket Policy Conditions IpAddress and NotIpAddress do not work

Reviewed-By: Adam C. Emerson <aemerson@redhat.com>
Reviewed-By: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #19354 from shinobu-x/osd_atomic

osd: Use specializations, typedefs instead

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #19549 from bi-shun/brotli-compressor

compressor: Add Brotli Compressor

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19564 from kungf/list_bucket_osd

mon/OSDMonitor: list osd tree in named bucket

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19816 from tchaikov/wip-22596

qa/standalone: remove osd-map-max-advance related tests

Reviewed-by: Sage Weil <sage@redhat.com>

rpm: adjust ceph-{osdomap,kvstore,monstore}-tool feature move

this is the rpm's counterpart of debian/control changes related to the
ceph-{osdomap,kvstore,monstore}-tool feature move. see #19328 and #19356.
the commit introducing this move is 6dba25e. and

$ git describe 6dba25e
v12.2.2-8-g6dba25e39d

so the first release that have this change is 12.2.2-8. in other words,
ceph-{base,osd,mon} >= 12.2.2.8 cannot co-exist with ceph-test < 12.2.2-8
in the same system. so we let ceph-test Requires ceph-common with the
same version. and since ceph-{osd,mon} Requires ceph-base with the same
version, and ceph-base Requires ceph-common with the same version, so by
tiering ceph-test with ceph-common with the same version, we enforce
this restriction.

Fixes: http://tracker.ceph.com/issues/22558
Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/standalone: remove osd-map-max-advance related tests

this setting was removed in 8967b73

Fixes: http://tracker.ceph.com/issues/22596
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19808 from cbodley/wip-boost-1.66

cmake: update minimum boost version to 1.66

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19358 from taoCH/wip-librgw-multitenancy

librgw: export multitenancy support

cmake: add WITH_BOOST_CONTEXT option

adds a more specific option for this boost::context dependency, which was
previously only used by the radosgw beast frontend. see
http://tracker.ceph.com/issues/20048 for more background

Signed-off-by: Casey Bodley <cbodley@redhat.com>

cmake: remove Beast submodule and include path

the beast library is included in boost 1.66

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: update beast frontend for boost 1.66

Fixes: http://tracker.ceph.com/issues/22600
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19806 from cbodley/wip-common-escape

common: add streaming interfaces for json/xml escaping

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

cmake: update minimum boost version to 1.66

Signed-off-by: Casey Bodley <cbodley@redhat.com>

common: add streaming interfaces for json/xml escaping

adds stream output operators that escape json/xml strings without having
to allocate a separate output buffer

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test: EscapeJson.Utf8 uses json instead of xml

the xml version does not escape the \n, while json does

Signed-off-by: Casey Bodley <cbodley@redhat.com>

test: use hex string literals

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19279 from taoCH/wip-fix-librgw-shutdown-err

librgw: fix shutdown err with resources uncleaned

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #19394 from zhangsw/cleanup-rgw-datasync-lease_cr

rgw: lease_cr->go_down is called twice, remove the needless one.

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19725 from shinobu-x/osd_max_object_size

osd,os/bluestore: Display current size of osd_max_object_size

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19718 from ifed01/wip-ifed-freelist-cleanup

os/bluestore: refactor FreeListManager to get clearer view on the number

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #18324 from Jeegn-Chen/wip-bucket-quota-api

rgw: Admin API Support for bucket quota change

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19719 from shinobu-x/bluestore_extentmap_dup

bluestore: BlueStore::ExtentMap::dup impl

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19698 from Liuchang0812/cleanup-ceph-tool

ceph: add the right bracket to watch-channel argument in the help message

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19660 from cfanz/tmp

include/encoding: fix compat version error message

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19654 from yaozongyou/test-fix-memory-leak

test: fix unittest memory leak to silence valgrind

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19598 from jcsp/wip-22457

packaging: explicit jinja2 dependency for dashboard

Reviewed-by: Wido den Hollander <wido@42on.com>

Merge pull request #19792 from joscollin/wip-doc-cephfs

doc: add mount.fuse.ceph to index

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

doc: fix typo

Signed-off-by: Jos Collin <jcollin@redhat.com>

doc: add mount.fuse.ceph to index

Added mount.fuse.ceph to index.rst

Fixes: http://tracker.ceph.com/issues/22595
Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #19559 from liu-chunmei/fix_dpdk_bug_base_spdk

osd: fix dpdk runtime issue based on spdk/dpdk libarary

Reviewed-by: Haomai Wang <haomai@xsky.com>

doc: Add usage doc for individual bucket quota Admin API

Add usage doc for a new Admin API for individual bucket quota management

Fixes: http://tracker.ceph.com/issues/21811
Signed-off-by: Jeegn Chen <jeegnchen@gmail.com>

osd: fix dpdk runtime issue based on spdk/dpdk libarary

1.misunderstand mbuf_overhead before, so correct it.
2.spdk/dpdk library will set refcnt, so needn't set it here.
3.avoid dpdk library modification. when create memory pool set elt_size as
mbuf_overhead + mbuf_data_size to avoid dpdk library check size assert.
4. call rte_pktmbuf_prefree_seg to set mbuf->next = null avoid dpdk/lib
modification .
5. use memzon to allocate mbuf data part, so these data buf can be
processes the same way as mbuf allocated by mempool create.

Signed-off-by: chunmei Liu <chunmei.liu@intel.com>

Merge PR #19766 into master

* refs/pull/19766/head:
mds: fix typo

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19756 from shinobu-x/objecter_unique_lock

osdc: Do not use lock_guard as unique_lock

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Revert "Merge PR #19369 into master"

This reverts commit 3189ba19a779e4a54265aa831a57d686a7911187, reversing
changes made to b7620de0200bdbac6a1a7abbdfbafd8b43d7bf47.

Despite the change in json format being positive, the unfortunate side-effect
is that it broke upgrade testing (because the QA framework must handle the
transition of mdsmap["info"] to a list from object) and the ceph-mgr.

Fixes: http://tracker.ceph.com/issues/22527
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #19763 from tchaikov/wip-cmake

cmake: pass static linkflags to the linker who links libcommon

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #19780 from tchaikov/wip-22266

tools/ceph_monstore_tool: include mgrmap in initial paxos epoch

Reviewed-by: Sage Weil <sage@redhat.com>

tools/ceph_monstore_tool: include mgrmap in initial paxos epoch

before this change, the "mgr" map is not included paxos, thus why the
peon mon fails to have the mgr map after it is sync'ed with its leader.
and hence there is chance that ceph-mgr is unable to get a mgr map from
the monitor it is connected to.

Fixes: http://tracker.ceph.com/issues/22266
Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: pass static linkflags to the linker who links libcommon

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: Add URL format support for bucket quota modification

Allow admin to change the quota of a individual bucket through Admin API
with params in URL

Fixes: http://tracker.ceph.com/issues/21811
Signed-off-by: Jeegn Chen <jeegnchen@gmail.com>

rgw: Admin API Support for bucket quota change

Allow admin to change the quota of a individual bucket through Admin API

Fixes: http://tracker.ceph.com/issues/21811
Signed-off-by: Jeegn Chen <jeegnchen@gmail.com>

rbd: replace positional_path parameter with arg_index in get_path()

Signed-off-by: songweibin <song.weibin@zte.com.cn>

mds: fix typo

Signed-off-by: dongdong tao <tdd21151186@gmail.com>

Merge pull request #19768 from cbodley/wip-rgw-cache-dedeque

rgw: change ObjectCache::lru from deque back to list

rgw: add invalidate_lru() call back to ObjectCache::put()

the removal of this line was causing s3test failures independently of
the change from list-deque

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: change ObjectCache::lru from deque back to list

ObjectCacheEntry stores an lru_iter into this lru, and depends on that
iterator remaining valid as other entries are inserted/erased

Fixes: http://tracker.ceph.com/issues/22560
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge tag 'v13.0.1'

v13.0.1

Signed-off-by: Sage Weil <sage@redhat.com>

rbd-nbd: 'id' instead of 'pid' as column name in list-mapped output

(to make output look similar to krbd)

Signed-off-by: Mykola Golub <mgolub@suse.com>

rbd-nbd: output format support for list-mapped command

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #19724 from dillaman/wip-18480

rbd: support cloning an image from a non-primary snapshot

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #19750 from badone/wip-dump_loads-formatter-null-check

mds: Remove redundant null pointer check

osdc: Do not use lock_guard as unique_lock

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

Merge pull request #19752 from Liuchang0812/cleanup-dev-doc

doc: fix CFLAGS in doc/dev/cpu-profiler.rst

Reviewed-by: Kefu Chai <kchai@redhat.com>

doc: fix CFLAGS in doc/dev/cpu-profiler.rst

Signed-off-by: Chang Liu <liuchang0812@gmail.com>

mds: Remove redundant null pointer check

Resolves Coverity CID 1426746

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #19666 from trociny/wip-nbd-unmap-by-image

rbd-nbd: allow to unmap by image or snap spec

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19745 from ceph/wip-use-installer0-master2

qa/tests - Added options to use both cases: mon.a and installer.0

qa/tests - Added options to use both cases: mon.a and installer.0

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #19512 from Liuchang0812/wip-dashboard-rgw

mgr/dashboard: RGW page

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19645 from tchaikov/wip-install-deps

install-deps: use DTS-7 on aarch64 and only download mirrored package indexes

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #19375 from qrGitHub/wip-rgw-redundant-parenthesis

rgw: remove redundant parenthesis in logs

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #19650 from yaozongyou/fix-erasure-code-memory-leak

erasure-code: use jerasure_free_schedule to properly free a schedule

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19649 from yaozongyou/fix-dns-resolve-memory-leak

common/dns_resolve: fix memory leak

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19659 from Songweibin/wip-doc-rbd

 doc/rbd: add missing several commands in rbd CLI man page

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #19634 from jcsp/wip-22514

qa: configure zabbix properly before selftest

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #19599 from yaozongyou/config-cleanup

common/config: cleanup remove some unused macros

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

Merge pull request #19581 from adamemerson/wip-cache-on-the-barrelhead

Cache on the barrelhead

Merge pull request #19546 from ZVampirEM77/wip-em-pg-cleanup3

osd/PG: some cleanups && add should_gather filter for loop logging

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #19524 from fungusakafungus/fix-22313

pybind/mgr/prometheus: fix metric type undef -> untyped

Reviewed-by: John Spray <jspray@redhat.com>

Merge pull request #19519 from ifed01/wip-ifed-fix-col-split

os/bluestore: fix potential assert when splitting collection.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #19452 from wido/mgr-zabbix-runtimerror

mgr/Zabbix: Various fixes to Zabbix module

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #19735 from shinobu-x/osd_oprequest_xlist

osd: Remove unnecessary headers

Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge pull request #19738 from lixiaoy1/minor_fix

msg/async: use device before checking

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/rbd: new unit test cases for clone request state machine

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: permit creation of clones against non-primary images

Fixes: http://tracker.ceph.com/issues/18480
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: switching to pool mode shouldn't affect existing mirrored images

Previously, non-primary images within a pool would prevent the switch from
image-mode to pool-mode mirroring.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

msg/async: use device before checking

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

Merge pull request #19737 from tchaikov/wip-dedup-rgw-admin-opt

doc: remove duplicated --max-buckets option desc

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: remove duplicated --max-buckets option desc

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: Remove unnecessary headers

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

osd, bluestore: Display current size of osd_max_object_size

It is better to show current value than showing same value twice.

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

install-deps.sh: check if have access to stdout

$- is "hB" if launched via command line, and it is "himBH" only when
we are actually *in* an interactive shell. so checking "test -t 1" is
what we want.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph.spec.in: use DTS-7 on aarch64 also

see https://bugs.centos.org/view.php?id=14078

Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps: bump DTS version on aarch64 to 7

see https://bugs.centos.org/view.php?id=14078

Signed-off-by: Kefu Chai <kchai@redhat.com>

install-deps.sh: only download mirrored package index

E: Failed to fetch http://mirror.cs.uchicago.edu/ubuntu-toolchain-r/dists/xenial/main/binary-i386/Packages  404  Not Found
E: Failed to fetch http://mirror.yandex.ru/mirrors/launchpad/ubuntu-toolchain-r/dists/xenial/main/i18n/Translation-en  404  Not Found [IP: 213.180.204.183 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.

On Ubuntu amd64 installations, i386 is enabled as an additional
architecture by default. apt & co. expect repositories to provide all
configured architectures, which causes the error above.

Signed-off-by: Songbo Wang <wangsongbo@cloudin.cn>

install-deps.sh: do not download i18n apt index

and do not fail the `apt-get update` command, if any mirror is not
reachable.

Signed-off-by: Kefu Chai <kchai@redhat.com>

os/bluestore: add asserts for fsck's used_blocks bitmap access as
boost doesn't always handle out-of-range access properly.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: refactor FreeListManager to get clearer view on the number
of alloc units it tracks.
This also fixes out-of-range access for fsck's used_blocks bitmap that
might happen when checking stores created prior to v12.2.2
Fixes http://tracker.ceph.com/issues/22535

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

bluestore: BlueStore::ExtentMap::dup impl

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>

Merge pull request #19010 from shinobu-x/src_cls_func

cls_acl/_crypto: Add modeline

Merge pull request #15855 from linuxbox2/wip-nfs-docsync

RGW NFS:  mount cmdline example missing -osync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #16932 from amitkumar50/cov-cls-rgw-1

cls/rgw: Initialization of uninitialized members

Merge pull request #17089 from amitkumar50/test-cov-journal-1

test/journal: Initialize member variable m_work_queue

Merge pull request #17090 from amitkumar50/test-cov-messages-8

messages: Initializing variable ceph_mds_reply_head