Merge pull request #22264 from liewegas/wip-mimic

ceph_release: s/rc/stable/

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #22277 from tspmelo/fix-lint-error

mgr/dashboard: fix linting problem

Reviewed-by: Kanika Murarka <murarkakanika@gmail.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22184 from tchaikov/mimic-23464

mimic: common/crc/aarch64: Added cpu feature pmull and make aarch64 specific…

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: fix linting problem

Problems were introduced in 50fbc04bf45b8cbf4db57c492652615625cd72c6,
when cherry picking some changes from master.

Signed-off-by: Tiago Melo <tmelo@suse.com>

ceph_release: s/rc/stable/

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22260 from liewegas/wip-rbd-cli-tests-mimic

mimic: test: fix CLI unit formatting tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test/cli-integration/rbd: fixed unit formatting

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4aea23cf1b488c0fe57b3cbe910ddcc4cb5360eb)

qa/workunits/rbd: fix unit formatting tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit d5b7d672b949767dc65359f26f5af3cad887e444)

librbd: clones should not inherit implicit feature bits from parent

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit bb357062853cc225fd7489eba9a70aa7ec7226e3)

Merge PR #22225 into mimic

* refs/pull/22225/head:
msg/async/Event: clear time_events on shutdown

Merge PR #22226 into mimic

* refs/pull/22226/head:
tests/crypto: print compile warning when NSS is unavailable.
tests/crypto: add tests for the no-bl encrypt/decrypt, part 2.
tests/crypto: add tests for the no-bl encrypt/decrypt.
auth: use OpenSSL for CryptoAESKeyHandler's no-bl encrypt/decrypt.
auth: extend CryptoKey with no-bl encrypt/decrypt.
auth: CryptoAESKeyHandler switches from NSS to OpenSSL.
auth: the outbuf of AES should be multiple of block size
auth: cache the PK11Context for CryptoAESKeyHandler

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge PR #22227 into mimic

* refs/pull/22227/head:
mgr/telemetry: Check if boolean is False or not present

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22242 into mimic

* refs/pull/22242/head:
doc: rgw: fix tagging support status

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #22245 into mimic

* refs/pull/22245/head:
mgr/dashboard: Add unit tests for all frontend pipes

Merge PR #22247 into mimic

* refs/pull/22247/head:
qa: use $TESTDIR for testing mkfs

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #22254 into mimic

* refs/pull/22254/head:
osd/ReplicatedBackend: fix use-after-free on InProgressOp

Reviewed-by: Kefu Chai <kchai@redhat.com>

osd/ReplicatedBackend: fix use-after-free on InProgressOp

- op in flight to disk...
- on_change() clears the InProgressOp
- C_OSD_OnOpCommit calls op_commit() w/ bare pointer
- crash!

Fix by refcounting InProgressOp and clearing on_commit when it is
canceled.

Fixes: http://tracker.ceph.com/issues/24219
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e84c2d097440aea5980fba2a2ef065769dbf1271)

qa: use $TESTDIR for testing mkfs

as tmpfs does not support O_DIRECT, but bluefs use this flag for
accessing the underlying file.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 1692f49b7b51ca599bf2e180d7a69778a5a5ba68)

Merge PR #22235 into wip-sage-testing-20180525.132253

* refs/pull/22235/head:
mgr/iostat: enable module by default on new and vstart clusters
doc: add documentation for iostat

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

common/arch/arm: Improve CPU feature check.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit d998580c7dbeff5ced5e9eb1e11d160a2a8d7852)

common/crc/aarch64: Added cpu feature pmull and make aarch64 specific crc function to depend on it.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit 90b1c1f1c4a57a59f598cd93dc9ce8a1532db238)

Merge pull request #22169 from tchaikov/mimic-23386

mimic: crush: fix device_class_clone for unpopulated/empty weight-sets

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22199 from dzafman/wip-minor-mimic

mimic: test: wait_for_pg_stats() should do another check after last 13 secon…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22170 from tchaikov/mimic-24211

mimic: os/bluestore: simplify and fix SharedBlob::put()

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Add unit tests for all frontend pipes

Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit b6305630c417bcb92c55a6b461c63ca0911a8b03)

doc: rgw: fix tagging support status

As of mimic, policy and LC conditionals for tagging are merged, so let's drop
the unsupported status.

Fixes: http://tracker.ceph.com/issues/24164
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 17dab9b3f24c1454a12f42184fd262119eb97e3e)

mgr/iostat: enable module by default on new and vstart clusters

Signed-off-by: Mohamad Gebai <mgebai@suse.com>
(cherry picked from commit a776755b704065bb0b3dc648b3c5e1efd887e003)

doc: add documentation for iostat

Conflicts:
        doc/releases/mimic.rst: remove file
        doc/mgr/index.rst: trivial resolution

Signed-off-by: Mohamad Gebai <mgebai@suse.com>
(cherry picked from commit 1e2bf1ec8c0dbe18b816ff9eb3243b73bb40d237)

Merge PR #22136 into mimic

* refs/pull/22136/head:
client: avoid freeing inode when it contains TX buffer heads

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22137 into mimic

* refs/pull/22137/head:
mds: properly setup client_need_snapflush for snap inode
Revert "mds: properly setup need_snapflush for snapped inode"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22138 into mimic

* refs/pull/22138/head:
mds: reply session reject for open request from blacklisted client
qa/tasks/cephfs: add timeout parameter to kclient umount_wait

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22139 into mimic

* refs/pull/22139/head:
client: delay dentry trimming until after cap traversal

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22140 into mimic

* refs/pull/22140/head:
client: fix issue of revoking non-auth caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22141 into mimic

* refs/pull/22141/head:
mds: broadcast quota to relevant clients when quota is explicitly set

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22153 into mimic

* refs/pull/22153/head:
mon/MonClient: set configs via finisher

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22154 into mimic

* refs/pull/22154/head:
client: increase verbosity level for log messages in helper methods

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/telemetry: Check if boolean is False or not present

Now we disable the module if the boolean is True

Signed-off-by: Wido den Hollander <wido@42on.com>
(cherry picked from commit 20f5696a735094cbaa0a698f41094cd5a5ac96f4)

tests/crypto: print compile warning when NSS is unavailable.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit ff56a6b7e1e502170779802fb121b4f8f0cdc1b6)

tests/crypto: add tests for the no-bl encrypt/decrypt, part 2.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 2535d11713aa837015e5028923ac97a271f41081)

tests/crypto: add tests for the no-bl encrypt/decrypt.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit e70ef0fb483c34e4944a996eb84ea09f7d1b30ca)

auth: use OpenSSL for CryptoAESKeyHandler's no-bl encrypt/decrypt.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 24e0532f81671fcea31a1fd2fb17e8b41fe5e9d9)

auth: extend CryptoKey with no-bl encrypt/decrypt.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit b5acef206c94142464f7ff314bb11c843c5420e8)

auth: CryptoAESKeyHandler switches from NSS to OpenSSL.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 4860bb70e1f47377ff69e1dc44e9b11bc69a7c2a)

auth: the outbuf of AES should be multiple of block size

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 3cfcd02af6381022081d51367f7338bde1fcb1d2)
(cherry picked from commit 7635485d34c99671030a10e1da67945c19b4fc88)

auth: cache the PK11Context for CryptoAESKeyHandler

in the flame graph, 0.50% of total time is used by
CephxSessionHandler::check_message_signature(), in which 0.27% is used
by PK11_CreateContextBySymKey(). so we should cache the PK11Context.

Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit f8c2036f7663be54030e13f1f9c1511cb40070f6)
(cherry picked from commit 7f6751bb5522c00b78d33af402470a659a07d4a3)

msg/async/Event: clear time_events on shutdown

Works around crash from http://tracker.ceph.com/issues/24162

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 923bd4649aacfb02a783fab9a0a71d1395d6e474)

Merge pull request #22135 from tchaikov/mimic-24175

mimic: mgr/status: output to stdout, not stderr

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #22160 from liewegas/wip-24037-mimic

common: OpTracker doesn't visit TrackedOp when nref == 0.

Merge pull request #22183 from tchaikov/mimic-24025

mimic: cmake/modules/BuildRocksDB.cmake: enable compressions for rocksdb

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

test: wait_for_pg_stats() should do another check after last 13 second sleep

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 151de1797b9163918b95a5996f422688e0964126)

cmake/modules/BuildRocksDB.cmake: enable compressions for rocksdb

we should enable them if they are found.

currently, we don't have bzip2 compressor plugin, so it's not detected
in the cmake script. we can always enable it for rocksdb in future.

Fixes: http://tracker.ceph.com/issues/24025
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 71462ddf05afa59bdaed8c471ee0617ee08dc71a)

Merge pull request #22167 from dzafman/wip-test-fixes-mimic

test: Whitelist corrections

Reviewed-by: Kefu Chai <kchai@redhat.com>

os/bluestore: simplify and fix SharedBlob::put()

There is a narrow race possible:

A: lookup foo
A: put on foo
A:   foo --nref == 0
B: lookup foo
B: put foo
B:   foo --nref == 0
B: try_remove() succeeds, removes
A: try_remove() tries to remove foo again, probably crashes

We could fix this by flagging the object in some way to indicate it was
removed (maybe clearing parent?), but then we need to be careful about
dereferencing foo to get parent from put().

Fix this by moving to a simpler model: make lookup fail if nref == 0.
This eliminates the races around put() entirely because once nref reaches
0 it never goes up again.

Fixes: http://tracker.ceph.com/issues/24211
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8c8944b2c45ca9dc5b8fd4db1590e1d24206c0b3)

crush: fix device_class_clone for unpopulated/empty weight-sets

If the bucket is empty or does not have weight-set weights yet, avoid
crashing when populating the parent bucket.

Fixes: http://tracker.ceph.com/issues/23386
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 452b5217406fc44cfd89003032ff1af9f6498675)

test: Whitelist corrections

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ee4acb6e1ff7458ceaefdb288cbcb158c6a3bed3)

common: OpTracker doesn't visit TrackedOp when nref == 0.

The patch fixes a race condition that happens between
`unregister_inflight_op` and `visit_ops_in_flight` of
`OpTracker`. When a callable passed to the former one
turns the plain reference it gets into `TrackedOpRef`,
an almost-to-terminate `TrackedOp` (with `nref == 0`)
can be resurrected (`nref++`). This will be reflected
in extra call to `unregister_inflight_op` for same op
leading to e.g. use-after-free. For more details see:
https://tracker.ceph.com/issues/24037#note-5.

The fix deals with the problem by ensuring there will
be no call to the visitor for ops with zeroized `nref`.

Fixes: http://tracker.ceph.com/issues/24037
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit 574aeb8130926fab5632d9f7a6ac0b132c6a60b8)

Merge pull request #22152 from tspmelo/fix-rbd-task-metadata-mimic

mimic: mgr/dashboard: Fix RBD task metadata

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge PR #21860 into mimic

* refs/pull/21860/head:
qa/rgw: tests target ceph-mimic branches
qa: add ragweed to luminous-x upgrade suite
qa/rgw: remove rgw/upgrade suite

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge PR #21907 into mimic

* refs/pull/21907/head:
radosgw-admin: add pretty 'bucket sync status' command
rgw: expose struct bucket_index_marker_info in header
rgw: rgw_bucket_sync_status takes bucket info
radosgw-admin: rename 'bucket sync status' to 'bucket sync markers'

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

githubmap: update contributors

Signed-off-by: Sage Weil <sage@redhat.com>

client: increase verbosity level for log messages in helper methods

Fixes: http://tracker.ceph.com/issues/21014
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit e276fdaaa8b737ea845e19388796796ca30d2162)

mon/MonClient: set configs via finisher

The config observers may want to take locks that are ordered relative
to monc_lock.

We could simply drop monc_lock for this call, but that would implicitly
rely on a single-threaded dispatch to avoid having two incoming MConfig
messages get reordered.  Explicitly putting it on a finisher is safer.
Note that we adjust the get_monmap_and_config() to start, drain, and stop
the finisher to ensure we have incoming config processed and applied
before returning.

Fixes: http://tracker.ceph.com/issues/24118
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 02a92360171240be9a66d1939c019b3427ed825d)

mgr/dashboard: Fix RBD task metadata

Error message template for RBD copy was trying to read
an unexistent property of the returned metada.

Metadata for RBD edit was missing the new image name.
The new name should be displayed, instead of the old one,
when the user tries to use an existent image name.

Fixes: https://tracker.ceph.com/issues/24171
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 27d4d3d5694b475fb59d285ab48b9e3507a42cef)

mds: broadcast quota to relevant clients when quota is explicitly set

Try to broadcast quota to relevant clients proactively if quota is
explicitly set by someone, in case that client won't get quota update
for a long time.

Fixes: http://tracker.ceph.com/issues/24133
Signed-off-by: Zhi Zhang <zhangz.david@outlook.com>
(cherry picked from commit b2a7643b102dbbb8221dcb8a785db5e4276ac284)

mds: properly setup client_need_snapflush for snap inode

MDCache::cow_inode() checks "cap->issued() & CEPH_CAP_ANY_WR" to decide
if it needs to setup client_need_snapflush for the new snap inode. If
cap message flushes dirty caps and releases the same caps, cap->issued()
may have no WR caps when MDCache::cow_inode() gets called. The solution
is temporarily setting NEEDSNAPFLUSH on Capability::state.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 9fb61cd65895dbdb92cb46d24be8bf79ce57409d)

Revert "mds: properly setup need_snapflush for snapped inode"

commit de3f3d88b3e make Locker::_do_cap_update() get called before
adjusting wanted caps. This is wrong because Locker::_do_cap_update()
need uptodate wanted caps to calculate max size.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: https://tracker.ceph.com/issues/24111
(cherry picked from commit 4b13d3ffe260fa4539e2d341604c34d2e8df7c94)

mds: reply session reject for open request from blacklisted client

Kernel client and old version libcephfs do not check if themselves
are blacklisted. They can be stuck at opening session after getting
blacklisted. The session reject message can avoid this.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: https://tracker.ceph.com/issues/24054
(cherry picked from commit b7c6cd8a54f094acb58603b8c6bae9e570a73e27)

qa/tasks/cephfs: add timeout parameter to kclient umount_wait

Just make caller happy. there is no easy way to support timeout.

Signed-off-by: Yan, Zheng <zyan@redhat.com>
Fixes: https://tracker.ceph.com/issues/24053
(cherry picked from commit e7d0b41deae7ec99ddf0a1f5f30ea82683b7b474)

client: delay dentry trimming until after cap traversal

Fixes: http://tracker.ceph.com/issues/24137
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 9199179799b41d9534b27bcceefc1bc9f2763bac)

client: fix issue of revoking non-auth caps

when non-auth mds revokes caps, Fcb caps can still be issued by auth
auth mds. It's wrong to flush buffer or invalidate cache when non-auth
mds revokes other caps. This bug can cause client to not respond the
revoke.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
Fixes: https://tracker.ceph.com/issues/24172
(cherry picked from commit 341a9114e0726e1a7cbb7e6f22adb54c2024c506)

client: avoid freeing inode when it contains TX buffer heads

ObjectCacher::discard_set() prematurely delete TX buffer heads. But
the pending writebacks still pin parent objects of these buffer heads.
Assertion "oset.objects.empty()" gets triggered if inode with pending
writebacks get freed.

Fixes: http://tracker.ceph.com/issues/23837
Signed-off-by: Guan yunfei <yunfei.guan@xtaotech.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 8a03757ca0ab493c6c2ea4fa4307e053e8ebc944)

Merge pull request #22081 from smithfarm/wip-gcc-8-silence

mimic: silence various warnings to enable GCC-8 build

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/status: output to stdout, not stderr

I think this was just me forgetting the order
of the arguments when writing the return statements.

Fixes: http://tracker.ceph.com/issues/24175
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit c18db575ca87906faa201c4d3bde75b99148d18f)

Merge pull request #22120 from liewegas/wip-24167-mimic

mimic: crush: update choose_args on bucket removal

Merge pull request #22125 from liewegas/wip-old-clients-mimic

qa/suites/rados/thrash-old-clients: only centos and 16.04

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #22075 from dzafman/wip-parens-mimic

mimic: test: Need to escape parens in log-whitelist for grep

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge branch 'mimic' of github.com:ceph/ceph into mimic

Merge pull request #22113 from tchaikov/mimic-24174

mimic: osd/PrimaryLogPG: fix try_flush_mark_clean write contention case

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22117 from tchaikov/mimic-qa-test_envlibrados_for_rocksdb

mimic: qa/workunits/rados/test_envlibrados_for_rocksdb: install g++ not g++-4.7

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22107 from liewegas/wip-unit-spaces-mimic

mimic: include/types: space between number and units

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test: Add CACHE_POOL_NO_HIT_SET to whitelist for mon/pool_ops.sh

Ignore
  cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET)

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 4fad800043d44024a496f78869e9bb02a16af063)

test: Need to escape parens in log-whitelist for grep

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 918921ab2f939c8b52c43a10bf0266347a2bdcf3)

qa/suites/rados/thrash-old-clients: only centos and 16.04

We don't have old client builds for the newer distros.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit dd46ae1006e580b4f25d190732e7b4b400501955)

13.1.1

crush: update choose_args on bucket removal

The specific bug I see is that a bucket no longer exists but its
choose_args still does.  However, I'm also taking the opportunity to
verify that the choose_args agrees with the bucket sizes and position
counts everywhere else, too.  Check for

- ids or weight_sets for buckets that don't exist or aren't straw2
- weight_set_positions that don't match the choose_args
  - don't fix this, just warn.  i'm not sure how it would happen.  :/
- weight_set sizes that don't match the bucket size

Fixes: http://tracker.ceph.com/issues/24167
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit a75ffcd17cc171a383315ec9738865de6c455086)

crush: update choose_args on bucket removal, resize, or position mismatch

The specific bug I see is that a bucket no longer exists but its
choose_args still does.  However, I'm also taking the opportunity to
verify that the choose_args agrees with the bucket sizes and position
counts everywhere else, too.  Check for

- ids or weight_sets for buckets that don't exist or aren't straw2
- weight_set_positions that don't match the choose_args
- weight_set sizes that don't match the bucket size

Fixes: http://tracker.ceph.com/issues/24167
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 564ef28a4014cb5d9959b1925154bbf7863fd0d4)

crush: create weight-set on demand when doing a choose-args reweight

If we try to reweight an item in a choose-args set for a given item
but the bucket doesn't have a weight_set yet, create it on the fly, and
populate it with the canonical bucket weights.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit d8d08b8cbf5b0ad3646dd1e177a1418e96273244)

test/cli/crushtool: use straw2 buckets for choose-args test

Choose-args only works for straw2 so it is misleading to test using a
map with straw buckets.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 7bfebfacfaf830fa9cb420b9d3e1d008e9891b3c)

crush: weight_set_size -> weight_set_positions

This naming was confusing!  This is the number of positions we
have weight_sets for.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c173a275dff3f5fc050832cb10f75bda51e653c5)

Merge pull request #22105 from dillaman/wip-rbd-mirror-policy-mimic

mimic: rbd-mirror: optionally support active/active replication

Reviewed-by: Mykola Golub <mgolub@suse.com>

qa/workunits/rados/test_envlibrados_for_rocksdb: use cmake not make

* so we just rely on a single build system instead of two of them, the
  other place we use cmake is cmake/modules/BuildRockDB.cmake.
* disable gflags when building rocksdb, it's optional and does not help
  in the sense of testing librados support.
* disable prompts when installing on debian, to silence warnings like:
  debconf: unable to initialize frontend: Dialog
* drop --force-yes option, as it is deprecated, and is replaced with
  --allow-downgrades, --allow-remove-essential,
  --allow-change-held-packages,  but none of them apply in our case.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit f8d47223e2e67871d5155406af4c4cda1451f521)

qa/workunits/rados/test_envlibrados_for_rocksdb: install g++ not g++-4.7

since fog now deploys ubuntu 18.04 where g++-4.7 is not available
anymore, so g++ is good enough.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 63a35625a3416ee4c210530fd38d38585b6eca95)

osd/PrimaryLogPG: fix try_flush_mark_clean write contention case

Release ctx *after* we dereference it to queue up the other flush waiters.
This was a dumb mistake in the recent fix from 41609ba1b24e5384494c7959494e19302b5a2429

Fixes: http://tracker.ceph.com/issues/24174
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 6d6cf388b245e21cc78be00d838e8b214b1e9497)

Merge pull request #22099 from liewegas/wip-pgmap-digest-decode-mimic

mon/PGMap: fix PGMapDigest decode

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22104 from tchaikov/mimic-24180

mimic: mon/LogMonitor: call no_reply() on ignored log message

Reviewed-by: Sage Weil <sage@redhat.com>

qa/workunits/cephtool/test.sh: update unit formatting

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0b956ef37db1f4bad1cc75cc2a78852832ab7b65)

qa/workunits/rbd/import_export: update unit formatting

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c6a747d1d801e8d1b3a9af0bc91770cf8544e85e)

qa/workunits/rbd/cli_generic: update unit formatting

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ddd3095bc0ab324b37ee0ed8cfb40dfe7c1c0a3f)

test/cli-integration/rbd: update unit formatting

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 35ae9acc322a272c9d02d5610822c7455721e730)

# Conflicts:
# src/test/cli-integration/rbd/formatted-output.t

[adjacent snapshot lines aren't in mimic]

include/types: space between number and units

not

    client:   504B/s rd, 10.6MiB/s wr, 0op/s rd, 30op/s wr

but

    client:   504 B/s rd, 10.6 MiB/s wr, 0 op/s rd, 30 op/s wr

This was changed as part of d3cecebacdcebcf475808a6204de22dfa94d729d.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 9cbd283bc60c35f84458a343421f513508ae0f04)