os/bluestore: fix races on SharedBlob::coll in ~SharedBlob.

Example scenario:
  ```
  A: BlueStore::Collection::split_cache(src, dest)
  A: std::lock(src->cache->lock, dest->cache->lock)

  B: SharedBlob::~SharedBlob
  B: waits coll->cache->lock // coll == src

  A: sb->coll := dest
  A: unlocks both src and dest's cache mutexes

  C: // any locked operation on dest's cache
  C: acquires dest->cache->lock
  C: begins the op

  B: // with the src's cache mutex acquired
  B: BufferSpace::_clear(coll->cache) // coll == dest
  B: // oops, B operates on cache already locked by C
  ```

Fixes: http://tracker.ceph.com/issues/24859
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
(cherry picked from commit d1b16749c7d32eead6e326c492ef9edf0d03a2b8)

Merge pull request #23034 from tchaikov/mimic-clang-4

include: fix build with older clang

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

include: fix build with older clang

to fix the FTBFS on clang shipped with xcode 9.1 (based on Clang 4.0).

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22997 from liewegas/wip-24597-mimic

mimic: osd: do not blindly roll forward to log.head

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22842 from pdvian/wip-24547-mimic

mimic: rgw: have a configurable authentication order

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22888 from smithfarm/wip-24617-mimic

mimic: ValueError: too many values to unpack due to lack of subdir

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #22986 from joscollin/wip-24843-mimic

mimic: rgw: require --yes-i-really-mean-it to run radosgw-admin orphans find

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22291 from pdvian/wip-24204-mimic

mimic: ceph_test_rados_api_misc: fix LibRadosMiscPool.PoolCreationRace

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22412 from tchaikov/mimic-cmake-boost-1.67

mimic: cmake: bump up the required boost version to 1.67

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22492 from tchaikov/mimic-24321

mimic: qa: wait longer for osd to flush pg stats

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22777 from croit/mimic-22977

mimic:  common/hobject: improved hash calculation for hobject_t etc

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22805 from smithfarm/wip-24708-mimic

mimic: ceph osd safe-to-destroy crashes the mgr

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #22813 from smithfarm/wip-24747-mimic

mimic: change default filestore_merge_threshold to -10

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22887 from smithfarm/wip-24473-mimic

mimic: cosbench stuck at booting cosbench driver

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22889 from smithfarm/wip-24618-mimic

mimic: osd: choose_acting loop

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22910 from ifed01/wip-ifed-fix-23540-mimic

mimic: os/bluestore: fix incomplete faulty range marking when doing compression

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22960 from pdvian/wip-24805-mimic

mimic: osd/PrimaryLogPG: rebuild attrs from clients

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22550 from smithfarm/wip-24296-mimic

mimic: repeated eviction of idle client until some IO happens

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #22810 from smithfarm/wip-24703-mimic

mimic: PurgeQueue sometimes ignores Journaler errors

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22816 from pdvian/wip-24537-mimic

mimic: common/DecayCounter: set last_decay to current time when decoding dec…

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22884 from smithfarm/wip-24704-mimic

mimic: mds: low wrlock efficiency due to dirfrags traversal

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/standalone/osd/ec-error-rollforward: reproduce bug 24597

This reproduces http://tracker.ceph.com/issues/24597

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4108ebc0abae83777ebc69bda610167f39e22109)

qa/standalone/osd/repro_long_log.sh: fix test

The log trimming case wasn't quite right.  Before HEAD^ we were
rolling forward too aggressively and miscalculating the can_rollforward_to,
which affected the trim_to calculation.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 4f9fdd98e2dde3979ec88a3f761d76f7b297415c)

osd/PG: do not blindly roll forward to log.head

If we are told we can roll forward by the primary, we should only roll
forward as far as the primary says we can.

This probably came out of the similar case in append_log(), but notably
that roll_forward() only happens if !transaction_applied (i.e., backfill
target), and that condition is not checked here.

Fixes: http://tracker.ceph.com/issues/24597
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 858ff647ff7bd1f6277823b5185ead5ffebc05b0)

Merge pull request #22886 from smithfarm/wip-24707-mimic

mimic: mgr/dashboard: Float numbers incorrectly formatted

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22265 from liewegas/wip-24308-mimic

mimic: mon/MDSMonitor: do not send redundant MDS health messages to cluster log

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22707 from pdvian/wip-24475-mimic

mimic: librados: fix buffer overflow for aio_exec python binding

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #22826 from smithfarm/wip-24738-mimic

mimic: Bring back diff -y for non-FreeBSD

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #22851 from smithfarm/wip-24766-mimic

mimic: build/ops: Mimic build fails with -DWITH_RADOSGW=0

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Bassam Tabbara <bassam.tabbara@quantum.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>

Merge pull request #22859 from smithfarm/wip-24769-mimic

mimic: set correctly shard for existed Collection

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22861 from smithfarm/wip-24771-mimic

mimic: osd: may get empty info at recovery

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22917 from smithfarm/wip-24803-mimic

mimic: Python bindings use iteritems method which is not Python 3 compatible

Reviewed-by: John Spray <john.spray@redhat.com>

rgw: require --yes-i-really-mean-it to run radosgw-admin orphans find

Incorrect use of orphans find can lead to data loss. Warn users to be
extra cautious.

Fixes: http://tracker.ceph.com/issues/24146
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
(cherry picked from commit 3ff47c7f3eb5964464c8cd49144546ce532ed7f7)

Merge pull request #22549 from smithfarm/wip-24391-mimic

mimic: [rbd-mirror] entries_behind_master will not be zero after mirror over

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #22768 from pdvian/wip-24499-mimic

mimic: librbd: deep_copy: resize head object map if needed

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22770 from jcsp/wip-24689

mimic: mon: exception for dashboard in config-key warning

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #22268 from xiaoxichen/backport_22243_mimic

mimic: mon/OSDMonitor: Respect paxos_propose_interval

Reviewed-by: Sage Weil <sage@redhat.com>

osd/PrimaryLogPG: rebuild attrs from clients

Ensure that buffers coming in via client ops are rebuilt before populating
the PGTransaction.  This ensures that we don't pin the raw buffers for
the entire incoming message in memory.

In the past we've addressed this issue at the ObjectStore layer, but we
did not consider the attr_cache in ObjectContext.  Rebuilding the buffers
at this point will sanitize any incoming attribute before it reaches
either PGBackend or ObjectContext object_cache or the ObjectStore
implementation.

Fixes: http://tracker.ceph.com/issues/24768
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 0a0f8d5e6fa8276f06a9b7a7b18ea6f15fa8cb86)

Merge pull request #22700 from yuriw/wip-yuriw-22532-backport

mimic: qa/suites: Add supported-random-distro$ links.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by:  Warren Usui <wusui@redhat.com>

Merge pull request #22791 from pdvian/wip-24534-mimic

mimic: client: fix bug #24491 _ll_drop_pins may access invalid iterator

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22819 from pdvian/wip-24539-mimic

mimic: client:  update inode fields according to issued caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22841 from pdvian/wip-24541-mimic

mimic: mds: add magic to header of open file table

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #22882 from smithfarm/wip-24310-mimic

mimic: pjd: cd: too many arguments

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22900 from pdvian/wip-24585-mimic

mimic: ceph-volume lvm.common use destroy-new, doesn't need admin keyring

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #22849 from smithfarm/wip-24764-mimic

mimic: mgr/dashboard: Missing breadcrumb on monitor performance counters page

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge remote-tracking branch 'private/wip-mon-snap-caps-mimic' into mimic

Merge remote-tracking branch 'private/wip-cx-mimic' into mimic

Merge pull request #22297 from tchaikov/mimic-24322

mimic: mon/OSDMonitor: no_reply on MOSDFailure messages

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22333 from pdvian/wip-24246-mimic

mimic: mon/MgrMonitor: change 'unresponsive' message to info level

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22361 from tchaikov/mimic-24361

mimic: mon: enable level_compaction_dynamic_level_bytes for rocksdb

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22382 from pdvian/wip-24261-mimic

mimic: bluestore: flush_commit is racy

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22394 from tchaikov/mimic-24406

mimic: osd: retry to read object attrs at EC recovery

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22406 from pdvian/wip-24292-mimic

mimic: common: fix typo in rados bench write JSON output

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #22525 from tchaikov/mimic-pr-22336

mimic: os/bluestore: firstly delete db then delete bluefs if open db met error.

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #22552 from smithfarm/wip-24468-mimic

mimic: tell ... config rm <foo> not idempotent

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22559 from pdvian/wip-24332-mimic

mimic: PG: add custom_reaction Backfilled and release reservations after bac…

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #22651 from liewegas/wip-24423-mimic

mimic: osd: fix getting osd maps on initial osd startup

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22722 from liewegas/wip-hobject-hash-mimic

mimic: common/hobject: improved hash calculation

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #22730 from pdvian/wip-24494-mimic

mimic: osd/Session: fix invalid iterator dereference in Sessoin::have_backoff()

Reviewed-by: Neha Ojha <nojha@redhat.com>

mgr: replace iteritems with six.iteritems (mgr_module.py)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 093562f6de706cb810fef5b0a297fb2422c65c78)

mgr: replace iteritems with six.iteritems (telegraf)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit ab001e134238f609823d0aad25f5e758107d9369)

Conflicts:
src/pybind/mgr/telegraf/module.py (trivial)

mgr: replace iteritems with six.iteritems (status)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 5866377ec39a3f0954afc8b0a3a8a544f188f307)

mgr: replace iteritems with six.iteritems (restful)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit c02c6601755efbe5fb06021d314d5b2d8aa6c79e)

mgr: replace iteritems with six.iteritems (influx)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit a62ab8ae8c91b91bb6c5367a6af197e5d85fe10a)

mgr: replace iteritems with six.iteritems (balancer)

Fixes: http://tracker.ceph.com/issues/24779
Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 962f0c416c90c2b319d0a9c403fe235086446080)

mgr/dashboard: Fix formatter service unit test

Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 5058ff960733f19937be1be4c401e89bf744138b)

os/bluestore: fix incomplete faulty range marking when doing compression
GC.

Under some scenarios GC might process an extent range where some inner extents are left untouched by GC (as there is no need for that). Hence GC doesn't invaliate these inner extents with fault_range call. If untouched extents are mapped to unloaded shards it results in subsequent assertion on o->extent_map.dirty_range() call.
The solution is to invalidate the whole extent range when doing GC.

Fixes: https://tracker.ceph.com/issues/23540
Fixes: https://tracker.ceph.com/issues/24799
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 0f04d4484c8663767bdb60f743a8835897013b5a)

mon: exception for dashboard in config-key warning

This warning went in with the expectation that nobody
would be using config-key commands for modules any more,
but the dashboard does use these in order to get the
"-i" functionality on the CLI for loading certs/keys.

In Nautilus they can switch to using "-i" on real module
commands, but for Mimic let's silence the warning for
the dashboard module.

Fixes: https://tracker.ceph.com/issues/24689
Signed-off-by: John Spray <john.spray@redhat.com>

ceph-volume lvm.common use destroy-new, doesn't need admin keyring

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 57fb124ce74fe4fb091091be6b4fa63e36d567a2)

osd/PG: restrict async_recovery_targets to up osds

When an osd that is part of the acting set and not the up set, gets chosen
as an async_recovery_target, it gets removed from the acting set. Since this
osd is no longer in the up or acting set, it is classified as a stray in
the next peering cycle. This results in choose_acting() looping between two
proposed acting sets.

To avoid this, we will only choose up osds as async_recovery_targets.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 7f1b6adec31e71ef4b3ec9b3c5cba8b1a4b7746f)

PG: do not choose stray osds as async_recovery_targets

Without this change, we might accept stray osds as async_recovery_targets,
and need to ensure that they get a chance to become part of the acting set
after recovery is over.

However, when choose_acting() is called in the Recovered state, we set
restrict_to_up_acting=true, which does not allow them to get back to the
acting set.

Therefore, similar to backfill, do not allow stray osds to become
async_recovery_targets.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit d96732d2448173f70f6514610d1c3780a7c8fab1)

qa/suites/rados/objectstore: move yaml files under a subdir

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 7cfc95b367d8b80d7e42b92f3c0f18147f6aae9c)

qa: whitelist SMALLER_PGP_NUM

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit dca3275f0c8f3ce96a73a4143c5658d4a3d73b59)

qa/tasks/cbt.py: changes to run on bionic

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit bf62e1fadc6a49a511e3e569cc5c6fa1a0cf78f0)

mgr/dashboard: Format small numbers correctly

The issue was triggered by numbers that a lower than 1.
Doing a logarithm with a number lower than 1 leads to
negative value that is not handled anywhere in The formatter service as
a result the final value will be quirky.
The negative number will also be used as index in the units array, where
it will return "undefined".

Fixes #24081

Signed-off-by: Stephan Müller <smueller@suse.com>
(cherry picked from commit b8ecac3603c01764d83e2b8862f3b6f847321e85)

mds: avoid traversing all dirfrags when trying to get wrlocks

Fixes: http://tracker.ceph.com/issues/24467
Signed-off-by: Xuehan Xu <xuxuehan@360.cn>
(cherry picked from commit da02fbcb0afd1b72d6c030f06613d8b18028a13c)

qa/workunits/suites/pjd.sh: use correct dir name

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 8fbab63436467aeb72b43f9db8c6b182e48d993c)

Merge pull request #22854 from jcsp/wip-mimic-cephfs-doc

[mimic] doc/cephfs: remove lingering "experimental" note about multimds

Reviewed-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: Improve max-line-length tslint rule

Prettier is not breaking strings if they are bigger
then the defined max line length.

With this rule we can make sure lines are not exceeding it,
with the exception of import/export lines.

Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit fc67af05b67cb2bbb76fcbea35cf10b1284d25de)

Conflicts:
src/pybind/mgr/dashboard/frontend/tslint.json

osd/PG: do not send notify to empty peer

This is mostly paranoia to avoid doing something clearly silly if the
add_source_info() implementation incorrectly decided we found something
new (as it did in http://tracker.ceph.com/issues/24588).

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3407b39bf36cb5fa5b5bd91c25f571134b7b929b)

osd/PG: do not assume delete event means found_missing

This condition was introduced in 3a9d056d843bcafd26d78950b84e2844f8a3a9a1
as part of the missing deletes series, without a clear motivation.  The
best guess is that it was either compensating for some other unfound bug
or simply being a bit overaggressive.

The problem is that it triggers a notify being sent to the sender, or
restarts recovery, both of which are overreactions.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 81d918d449f0211c8eac262e700930473f6e98a3)

os/bluestore: set correctly shard for existed Collection.

For existed Collection, the constructor of Collection will be called in _open_collections.
But m_finisher_num can't setup when enable bluestore_shard_finishers.

So move m_finisher_num setup before _open_collections && _kv_start.

Fixes: http://tracker.ceph.com/issues/24761
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 42cd25b794d2a2c04e96a24abea7f773bb7a3c2e)

doc/cephfs: remove lingering "experimental" note about multimds

Signed-off-by: John Spray <john.spray@redhat.com>

Merge pull request #22803 from joscollin/wip-24746-mimic

mimic: doc: update experimental features - snapshots

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Remove embedded 'cephd' code

Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit 39b2626c7a36b3fef8d97cd276e23dc6f1f197f3)

Conflicts:
src/compressor/CMakeLists.txt
src/libcephd/libcephd.cc

doc: update experimental features - snapshots

* drop the experimental warning of multi-MDS
* update snapshotting blocked off version
* update Multiple filesystems availability in Jewel

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 9ed1158e1de3addf0e768fdb489fc128d09335eb)

doc: highlight the flags

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 1586580d33ff3f1ca4b168d9e3bd1182643e031f)

mgr/dashboard: Add performance counters breadcrumb

Fixes: https://tracker.ceph.com/issues/23806
Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit e89d5bef7fb768d508db6600174be6388a314b15)

mgr/dashboard: Remove useless observable unsubscriptions

Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit ae75f15dedf6e9f19744dd4ba568ceb9b2c768e4)

Conflicts:
src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-user-form/rgw-user-form.component.ts
        (retain mimic "Create an observable to add the S3 key when the form is submitted" logic
        but add whitespace so it blends in with the rest of the refactor)

mgr/dashboard: Add support for queryParams on routerLinkTpl

Signed-off-by: Ricardo Marques <rimarques@suse.com>
(cherry picked from commit 9e710e1ad785c86ec32c5c46a6c63349f8615709)

Merge pull request #22807 from smithfarm/wip-24699-mimic

mimic: mgr/dashboard: Replace Pool with Pools

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

rgw: have a configurable authentication order

This implements a configurable authentication order, currently used only for s3
authentication and only supporting external & local authentication, though there
is potential for more finegrained control by allowing for a map of various
engines and the control strategy (required vs sufficient vs fallback)

The current implementation just focuses on setting control fallback if the
engine is the last in the order (and hence the stack) and just sets sufficient to
every other element, so that errors from the last sufficient engine is returned.

The configuration option is rgw_s3_auth_order which takes a comma/space seperated
list of authentication engines where currently we support the keywords `external`
and `local`.

Fixes: http://tracker.ceph.com/issues/23089
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 9c7fc682ca23259037115db3437c2bc9dd91fa22)

mds: add magic to header of open file table

Fixes: http://tracker.ceph.com/issues/24240
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit c60d55d19276c5c65c18bebbfd5939bb8c53f57c)

Conflicts:
src/mds/OpenFileTable.cc: Resolved in _load_finish

Merge pull request #22832 from ceph/backport-mimic-22828

mimic: ceph-volume: provide a nice errror message when missing ceph.conf

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: always ignore a missing ceph conf in main.py

Now that we have a nice error message when a ceph.conf is missing
and we try to use values from it, maintaining a list of commands that
don't need ceph.conf isn't as helpful. We had actually missed 'simple
trigger' when we first implemented this causing all our luminous tests
for simple to fail when we backported.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 0487a9ac60c5ba289b7dd7c730b014a298c5998f)

ceph-volume: show a nice error message when ceph.conf is not loaded

If the ceph configuration file is not loaded correctly and then values
from it are used then an undescript error message is shown, e.g.

AttributeError: 'property' object has no attribute 'get'

With this change that same error condition shows the message:

RuntimeError: No valid ceph configuration file was loaded.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit e42ecb831914b07c7c8559d4556a8aabd2accd0c)

Merge pull request #22747 from ceph/backport-mimic-22724

mimic: ceph-volume: error on commands that need ceph.conf to operate

Reviewed-by: Alfredo Deza <adeza@redhat.com>

test: run-standalone move aside cores found before testing starts

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit ea75e03d2222f890033dee1b2f1f9eba71f9a142)

test: Fail immediately if some operations fail

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit fe09fc5e9da3fd2681e7c78d1df19142dc9e0eb7)

test: Fix standalone main usage

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 33538aca359546667a9b336901379ec2bde07fdb)