Merge pull request #24056 from smithfarm/wip-24985-luminous

luminous: multisite: object metadata operations are skipped by sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24069 from pdvian/wip-35709-luminous

luminous: rgw: fix deadlock on RGWIndexCompletionManager::stop

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23884 from smithfarm/wip-32084-luminous

luminous: mds: MDBalancer::try_rebalance() may stop prematurely

mds: fix premature stop of MDBalancer::try_rebalance()

When mds wants to export loads to multiple targets. The amount of single
export can easily be less than (0.2 * target_loads). For example:

6 active mds, mds.0's load is 11, other ranks' loads are 5. target_load
is 6. Export amount for each target is 1.

Fixes: http://tracker.ceph.com/issues/26973
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit e68cbec6a647f0252d848620ce6e51b4f3eea1e3)

mds: don't queue straydir for split

straydir can not get splitted (for now at least)

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
(cherry picked from commit 959e63c1f88b6ce6b2492cfb98e5875d928558e4)

Conflicts:
src/mds/MDBalancer.cc
- finesse split/merge conditional expression

Merge pull request #23789 from ukernel/luminous-mds-export-size

luminous: mds: optimize the way how max export size is enforced

Merge branch 'luminous' into luminous-mds-export-size

Merge pull request #23594 from dillaman/wip-23604-luminous

luminous: librbd: discard should wait for in-flight cache writeback to complete

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #24100 from pdvian/wip-35711-luminous

luminous: librbd: journaling unable request can not be sent to remote lock owner

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23793 from pdvian/wip-26983-luminous

luminous : client: fix choose_target_mds for requests that do name lookup

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23881 from smithfarm/wip-26977-luminous

luminous: cephfs-data-scan: print the max used ino

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #23946 from ukernel/luminous-35721

luminous: mds: prevent MDSRank::evict_client from blocking finisher thread

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23990 from batrick/i35852

luminous: MDSMonitor: lookup of gid in prepare_beacon that has been removed will cause exception

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #24033 from smithfarm/wip-24934-luminous

luminous: cephfs-journal-tool: wrong layout info used

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24134 from batrick/qa-.qa-symlink

luminous: qa: add .qa helper link

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23408 from ukernel/luminous-24820

luminous: mds: avoid using g_conf->get_val<...>(...) in hot path

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23505 from ukernel/luminous-26834

luminous: mds: calculate load by checking self CPU usage

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23507 from ukernel/luminous-26858

luminous: mds: reset heartbeat map at potential time-consuming places

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23554 from batrick/i26906

luminous: MDSMonitor: consider raising priority of MMDSBeacons from MDS so they are processed before other client messages

Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23556 from batrick/i26924

luminous: mds: mds got laggy because of MDSBeacon stuck in mqueue

Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #23617 from pdvian/wip-26915-luminous

luminous: client: check for unmounted condition before printing debug output

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23677 from ukernel/luminous-26894

luminous: mds: crash when dumping ops in flight

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23678 from ukernel/luminous-24840

luminous: mds: explain delayed client_request due to subtree migration

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23693 from tserlin/tserlin-require-fuse-luminous

luminous: debian/control: require fuse for ceph-fuse

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23726 from joscollin/wip-25205-luminous

luminous: CephVolumeClient: delay required after adding data pool to MDSMap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23758 from dillaman/wip-27987

luminous: librbd: fix refuse to release lock when cookie is the same at rewatch

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #23892 from pdvian/wip-27061-luminous

luminous: librbd: potential race on image create request complete

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #24003 from pdvian/wip-35704-luminous

luminous: rbd: fix error import when the input is a pipe

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

librbd: discard should wait for in-flight cache writeback to complete

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(derived from commit 0e04de4bb9c08755bd0915776bd538ce8f7c2b5b)

qa: add .qa helper link

Barebones backport of #22719.

Command used:

    $ find qa/suites -type d -execdir ln -sfn ../.qa/ {}/.qa \;
    $ ln -sf ../ qa/suites/.qa

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #23680 from k0ste/luminous_backports

luminous: osd: backport 'bench' and stdout changes

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #23740 from k0ste/luminous_backports2

luminous: mgr/balancer: deepcopy best plan - otherwise we get latest

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23791 from tchaikov/wip-23370-luminous

luminous: mgr/MgrClient: guard send_pgstats() with lock

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23862 from smithfarm/wip-25177-luminous

luminous: osd,mon: increase mon_max_pg_per_osd to 250

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #23864 from smithfarm/wip-25203-luminous

luminous: rados python bindings use prval from stack

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23871 from smithfarm/wip-32106-luminous

luminous: object errors found in be_select_auth_object() aren't logged the same

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #23901 from pdvian/wip-32078-luminous

luminous: test: Use pids instead of jobspecs which were wrong

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #23943 from batrick/i35716

luminous: msg: "challenging authorizer" messages appear at debug_ms=0

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #23969 from zmedico/backport-lumious-17121-17239-17410-17820

luminous: PGPool::update optimizations

Reviewed-by: Robin H. Johnson <robbat2@gentoo.org>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23499 from pdvian/wip-26844-luminous

luminous: rgw_file: deep stat handling

Reviewed-by: Matt Benjamin <mbenjami@redhat.com>

Merge pull request #23522 from pdvian/wip-26846-luminous

luminous: rgw: Limit the number of lifecycle rules on one bucket.

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #23545 from pdvian/wip-26848-luminous

luminous: rgw: The delete markers generated by object expiration should have owner

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #23665 from pdvian/wip-26922-luminous

luminous: rgw: Fix log level of gc_iterate_entries

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #23974 from pdvian/wip-35069-luminous

cls/rgw: ready rgw_usage_log_entry for extraction via ceph-dencoder

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24034 from smithfarm/wip-25087-luminous

luminous: change default rgw_thread_pool_size to 512

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24046 from pdvian/wip-35707-luminous

luminous: rgw: raise default rgw_curl_low_speed_time to 300 seconds

Reviewed-by: Casey Bodley <cbodley@redhat.com>

librbd: journaling unable request can not be sent to remote lock owner

Fixes: http://tracker.ceph.com/issues/26939
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 985535bae74106aec1938813bdb18661967f468f)

Merge pull request #24082 from alfredodeza/luminous-rm35970

luminous: ceph-volume fix zap not working with LVs

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24080 from alfredodeza/luminous-rm35913

luminous: ceph-volume batch: allow --osds-per-device, default it to 1

Reviewed-by: Andrew Schoen aschoen@redhat.com

ceph-volume tests.functional comment why not use --destroy when zapping lvs in filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 11858c74a1e704d63185f0ef4b6cd6aa02f17e52)

ceph-volume tests.functional use --destroy when zapping lvs in bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 87af815691a7f43d14067c24e308d60f39d58051)

ceph-volume lvm.zap use remove_pv only for physical devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 036683fde799ab1cfad7d6192981515d6b3419cc)

ceph-volume tests.lvm.strategies initial filestore validation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d1d68eade56ba3f898cd3bfd29aa1ee3bf5460ec)

ceph-volume lvm.tests conftest fixtures for custom vgs

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9f14db432317c15b29b187d1e1a4bd3a729a7043)

ceph-volume tests.devices.lvm add bluestore strategy cases

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 2ced05a2bc6ba01cf8946f43d5839890a28cab20)

ceph-volume tests verify strategy validators for minimum size

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9167f76c2a7107c82b781cbe5f51e24ee72e4db4)

ceph-volume lvm.batch.filestore allow osds-per-device

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 66947537b542da82185ea4d79bfaedaae3117918)

ceph-volume lvm.batch.validators account for osds per device, include collocation

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 04a40eda0ac089c07654d9ab07958126f0fc8f84)

ceph-volume lvm.batch.bluestore allow osds-per-device with MixedType

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 028ef60df768c24a9573a044a451164dcb037677)

ceph-volume lvm.batch.bluestore allow osds-per-device with SingleType

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8c0e707110e284626fe4161097a3121170a3f09d)

ceph-volume lvm.batch add --osds-per-device flag, defaulting to 1

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fe7c5e0129172a4b2a216e7c49e67d8ead570b39)

Merge pull request #24075 from alfredodeza/luminous-rm34535

luminous: ceph-volume batch carve out lvs for bluestore

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests.util verify Disk objects don't change state with divisions

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 9ca526336221ea2ad32e7daf3e5e46b07291a1d8)

ceph-volume util.disk fix an issue where Disk objects would mutate on div operations

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 7d1a3c74240e67621e993a304eaf566d4e2789d1)

ceph-volume util.prepare add a helper to get block.db sizes from ceph.conf

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 860068ec5daca6d7087cf1ce0594a2fe36e91193)

ceph-volume lvm.batch.bluestore add TODOs for custom fast/slow devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 0eb0bdfc55d8ceb27cd12bc3a7fb807a7e65c788)

-f  ceph-volume lvm.batch.bluestore validation and reporting with VG reuse

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d117d66903fa740108c9948b499150b02523fdf4)

ceph-volume lvm.batch fix error reporting, Device objects aren't strings

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 267a386cc270081b2cd284bf78bf94b52be113dc)

ceph-volume lvm.batch.bluestore validation and reporting with VG reuse

Reworks the bluestore validation and reporting to account for reusable
VGs from fast devices, and adds validation calls to ensure the new way
to calculate this process will work.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 27f47565a0d43ecee03dea4f6addc739e3e78655)

ceph-volume lvm.batch.filestore capture SizeAllocationErrors

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 71fcd35c3d44795f92b17e9aedb7ca868b3750d3)

ceph-volume lvm.batch make sure data devices don't have existing LVs on bluestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d2ea49a4e64820fc3cd95c2a6ca8869309638e00)

rgw: fix deadlock on RGWIndexCompletionManager::stop

The lock has been acquired in complete_op_data::stop
Fixes: http://tracker.ceph.com/issues/26949
Signed-off-by: Yao Zongyou <yaozongyou@vip.qq.com>
(cherry picked from commit 519cea1bf58461a65bab1e4aecff1fd3ea3191b0)

Merge pull request #23216 from b-ranto/wip-mgr-prom-bp-luminous

luminous: Sync the prometheus module

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #23461 from pdvian/wip-26838-luminous

luminous: mgrc: enable disabling stats via mgr_stats_threshold

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #23566 from pdvian/wip-26908-luminous

luminous: kv/KeyValueDB: return const char* from MergeOperator::name()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #23579 from VictorDenisov/backport_26934

luminous: osd/OSDMap: fix apply upmap segfault

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #24039 from ceph/wip-s3a-fix-mirror

luminous: qa/tasks:  s3a fix mirror

rgw: bucket sync only allows one olh op at a time

if olh operations on different instances of an object race to sync,
their olh epochs could be applied in the wrong order and result in one
operation being rejected by the olh log. serialize olh operations on the
same object name to prevent this from happening

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d120ca1734d247a1b8c78e1a8bfd7441f4fef8b4)

Conflicts:
    src/rgw/rgw_data_sync.cc
- replaced tn->log with ldout

rgw: bucket sync updates high marker for squashed entries

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 918fc3813c8d8c96c7c071645b5ae5c1090a7c15)

Conflicts:
src/rgw/rgw_data_sync.cc
- trivial resolution

rgw: CompleteMultipart applies its olh_epoch

RGWCompleteMultipart was using get_system_versioning_params() to read an
olh_epoch, but was never applying it to the bucket index operation

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e4d1cc1204dd588a6cdbb70b0b48403a97302412)

rgw: bucket sync only provides an epoch for olh operations

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2de295c2cf88212ecdf536ee491ac0ecb6de9f65)

Conflicts:
src/rgw/rgw_data_sync.cc
- luminous uses ldout instead of tn->log
- use boost::optional instead of std::optional

rgw: SyncModule::sync_object() takes optional olh epoch

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c0bc17b48cce7395528bd58f9572040895529fdd)

Conflicts:
src/rgw/rgw_sync_module_aws.cc
- file does not exist in luminous
src/rgw/rgw_data_sync.cc
src/rgw/rgw_sync_module.h
src/rgw/rgw_sync_module_es.cc
 src/rgw/rgw_sync_module_log.cc
- use boost::optional instead of std::optional

Merge pull request #23211 from neha-ojha/wip-24988-luminous

luminous: osd: Limit pg log length during recovery/backfill so that we don't run out of memory

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

rgw: fetch_remote_obj() applies olh even if object is current

even if we already have the latest version of an object, it may have
synced without a valid olh epoch. so we still need to apply its olh
modifications with this epoch. this change also has to be added to the
data changes log

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b5ecc4eaaaaa4f9950ad934a015255171abeb01e)

rgw: fetch_remote_obj() takes optional olh epoch

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit ff0280b30180b73a53f08fc919febc03410e4c1e)

Conflicts:
    src/rgw/rgw_cr_rados.h
    src/rgw/rgw_rados.cc
    src/rgw/rgw_rados.h
- use boost::optional instead of std::optional

rgw: Object::Write::_do_write_meta() takes optional olh epoch

take an optional olh_epoch, and use its empty state to skip set_olh()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 519f1b407db91695a49de7bda04b02e86fd35c1b)

Conflicts:
    src/rgw/rgw_rados.h
- use boost::optional instead of std::optional

rgw: bucket sync allows OP_ADD on versioned objects

operations like PutACL that only mutate object metadata do not
generate a link_olh entry, so they cannot be skipped. their bilog
entries don't have a versioned_epoch, so we skip the call to
set_olh() by passing an empty value to fetch_remote_obj()

Fixes: http://tracker.ceph.com/issues/24367
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7fb034ee5e98e7e991595c768a3a1da6c56f119c)

Conflicts:
src/rgw/rgw_data_sync.cc
- trivial resolution

rgw: bucket sync doesn't squash over olh entries

all bucket index operations that mutate the olh_epoch must be applied
by sync to keep it consistent between zones. avoid squashing later
operations on the same object version

Fixes: http://tracker.ceph.com/issues/24367
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 08cd49ab5cdb50565c554adb586a2230cd058d42)

test/rgw: test incremental sync of acls on versioned object

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b300e1424127b9bba94f394ccadc25b5279d9c7b)

rgw: raise default rgw_curl_low_speed_time to 300 seconds

this timeout is mainly to avoid waiting forever when a sync connection
goes away without seeing a close. a 30-second timeout is much more
likely to catch other delays like loaded osds - and timeout/retry will
increase that load

Fixes: http://tracker.ceph.com/issues/27989
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 2fe357657158f6454865b9c37314fa8cfdfc4771)

Merge pull request #23761 from k0ste/luminous_backports3

luminous: docs: radosgw: ldap-auth: fixed option name 'rgw_ldap_searchfilter'

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

qa/tasks: update mirror link for maven, the original mirror no longer exists

Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
(cherry picked from commit 13e100259e3658fcfd2123474cb8ea8498ac3fb6)

Merge pull request #23621 from joscollin/wip-26917-luminous

luminous: doc: Fix broken urls

Reviewed-by: Nathan Cutler <ncutler@suse.com>

qa/rgw: override valgrind --max-threads for radosgw

radosgw now uses 512 frontend threads by default, and valgrind won't
start with its default --max-threads=500

Fixes: http://tracker.ceph.com/issues/25214
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit e617990efe4c39f592cff1541ea55fbcf9f2d6f2)

Conflicts:
qa/suites/rgw/tempest/tasks/rgw_tempest.yaml
qa/suites/rgw/verify/tasks/0-install.yaml

rgw: change default rgw_thread_pool_size to 512

This value is commonly tuned up and demonstrates better performance on
large systems.

Fixes: http://tracker.ceph.com/issues/24544
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
(cherry picked from commit 28b0c1054fbc0d390d12fc32540c66b1e8c0bae4)

cephfs-journal-tool: get and check layout info during dump/undump

when cephfs-journal-tool import journal, it uses default layout
to get object_size, this is wrong. Because default objetct_size
can be changed by Config option change mds_log_segment_size when
cluster initialized.

The fix is save the layout info to header during journal export,
When improt journal, first try to get layout from cluster, if failed,
then get layout info from dumped journal file, if still failed, set
layout to default.

Fixes: http://tracker.ceph.com/issues/24644
Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>
Signed-off-by: yupeng chen chenyupeng-it@360.cn
(cherry picked from commit 989ecf234a24c53884cf202de588692a17ccec2a)

cephfs-journal-tool: check fsid when import purge queue
import purge queue will triger file delete in cluster.
we occasionally imported a purge queue from one production
cluster to a test cluster and found the the fs in test cluster
was damaged because some objects are really delted as its inode
recorded in purge queue.

add fsid when exporting purge queue, check if the fsid matched when
importing purge queue. This could make cephfs-journal-tool safer to
handle purge queue issues.

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>
Signed-off-by: yupeng chen chenyupeng-it@360.cn
(cherry picked from commit e0eef2aab10747de0efc128133c567204b4d0322)

Merge pull request #23967 from alfredodeza/luminous-rm35535

luminous: ceph-volume: batch tests for mixed-type of devices

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

rbd: fix error import when the input is a pipe

Fixes: http://tracker.ceph.com/issues/34536
Signed-off-by: songweibin <song.weibin@zte.com.cn>
(cherry picked from commit 3168d21629eb4d33f95dbd9399225e3e3817dc4f)

mon: test if gid exists in pending for prepare_beacon

If it does not, send a null map. Bug introduced by
624efc64323f99b2e843f376879c1080276e036f which made preprocess_beacon only look
at the current fsmap (correctly). prepare_beacon relied on preprocess_beacon
doing that check on pending.

Running:

    while sleep 0.5; do bin/ceph mds fail 0; done

is sufficient to reproduce this bug. You will see:

    2018-09-07 15:33:30.350 7fffe36a8700  5 mon.a@0(leader).mds e69 preprocess_beacon mdsbeacon(24412/a up:reconnect seq 2 v69) v7 from mds.0 127.0.0.1:6813/2891525302 compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2}
    2018-09-07 15:33:30.350 7fffe36a8700 10 mon.a@0(leader).mds e69 preprocess_beacon: GID exists in map: 24412
    2018-09-07 15:33:30.350 7fffe36a8700  5 mon.a@0(leader).mds e69 _note_beacon mdsbeacon(24412/a up:reconnect seq 2 v69) v7 noting time
    2018-09-07 15:33:30.350 7fffe36a8700  7 mon.a@0(leader).mds e69 prepare_update mdsbeacon(24412/a up:reconnect seq 2 v69) v7
    2018-09-07 15:33:30.350 7fffe36a8700 12 mon.a@0(leader).mds e69 prepare_beacon mdsbeacon(24412/a up:reconnect seq 2 v69) v7 from mds.0 127.0.0.1:6813/2891525302
    2018-09-07 15:33:30.350 7fffe36a8700 15 mon.a@0(leader).mds e69 prepare_beacon got health from gid 24412 with 0 metrics.
    2018-09-07 15:33:30.350 7fffe36a8700  5 mon.a@0(leader).mds e69 mds_beacon mdsbeacon(24412/a up:reconnect seq 2 v69) v7 is not in fsmap (state up:reconnect)

in the mon leader log. The last line indicates the problem was safely handled.

Fixes: http://tracker.ceph.com/issues/35848
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit f26752a8a8b5041189a4e6d638946826b5403feb)

Conflicts:
    src/mon/MDSMonitor.cc: needs to use old message creation mechanism