qa/standalone/ceph-helpers: fix test_wait_for_clean

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 30fc7f5e972bd41537c92eb135c9763b46dd09d2)

Conflicts:
qa/standalone/ceph-helpers.sh
- mimic does not have separate WAIT_FOR_CLEAN_TIMEOUT

qa/standalone: make sure an osd is running before create_rbd_pool

'rbd pool init' now does IO.  Drop the pool, or change the pool size to 1.

Fixes: http://tracker.ceph.com/issues/38585
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit cba0483b090a6370390e8ed88d3f9d7767d56be5)

qa: specify filestore for misc tests

Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 8a694fc2f9e75864e064fe591feda9fab943c15e)

qa/standalone/ceph-helpers: resurrect all OSD before waiting for health

address the regression introduced by e62cfceb
in e62cfceb, we wanted to test the newly introduced TOO_FEW_OSDS
warning, so we increased the number of OSD to the size of pool, so if
the number of OSD is less than pool size, monitor will send a warning
message.

but we need to bring all OSDs back if we are expecting a healthy
cluster. in this change, all OSDs are resurrect before
`wait_for_health_ok`.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cdba0f14201a327e824b6ec48e014233281dee2d)

qa/standalone/ceph-helpers: fix health_ok test

Stopping the osd daemon won't reliably get you HEALTH_WARN or ERR; you have
to make sure it is also marked down.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit dcdca44aa45e7dc5aab56da26caec644e32024a6)

mon/PGMap: enable/disable TOO_FEW_OSDS warning with an option

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 3b74fbc93d32d6cc367049d3bcbe83d3837abba5)

Conflicts
        src/mon/PGMap.cc
- cct->_conf->get_val in luminous
src/common/options.cc
- no mon_max_snap_prune_per_epoch in luminous
- in luminous, osd_pool_default_size is INT_T (int64_t). In master, it is UINT_T
  (uint64_t). So I changed the type to int64_t.

qa/standalone: remove osd_pool_default_size in test_wait_for_health_ok

Signed-off-by: zjh <jhzeng93@foxmail.com>
(cherry picked from commit e62cfceb9549e806f774ef3da549efd9f4d90171)

osd: Better error message when OSD count is less than osd_pool_default_size

Fixes: http://tracker.ceph.com/issues/38617
Signed-off-by: zjh <jhzeng93@foxmail.com>
(cherry picked from commit 94237d36932b27961afffbe65d433af26c3fe513)

Conflicts:
doc/rados/operations/health-checks.rst
- trivial
        src/mon/PGMap.cc
- cct->_conf->get_val in luminous

Merge pull request #29713 from cbodley/wip-41322

luminous: rgw: datalog/mdlog trim commands loop until done

Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #29269 from smithfarm/wip-40135-luminous

luminous: rgw: the Multi-Object Delete operation of S3 API wrongly handles the Code response element

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #30345 from dillaman/wip-41772-luminous

luminous: common/config: hold lock while accessing mutable container

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #27974 from iain-buclaw-sociomantic/luminous-rgw-cls-bi-list-log-level

luminous: rgw: cls/rgw: raise debug level of bi_log_iterate_entries output

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27992 from smithfarm/wip-39016-luminous

luminous: rgw: unable to cancel reshard operations for buckets with tenants

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29153 from davebiffuk/s3_policy_tracker_39272

luminous: rgw: policy fix for nonexistent objects

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29170 from cbodley/wip-qa-rgw-tempest-luminous

luminous: qa/rgw: remove incomplete rgw/tempest suite from luminous

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #29271 from smithfarm/wip-40143-luminous

luminous: rgw: multisite: 'radosgw-admin bucket sync status' should call syncs_from(source.name) instead of id

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29295 from dillaman/wip-40233-luminous

luminous: rbd/action: fix error getting positional argument

Reviewed-by: Mykola Golub <mgolub@mirantis.com>

Merge pull request #29826 from linuxbox2/wip-luminous-apiname

luminous: rgw: fix issue for CreateBucket with BucketLocation param

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29874 from trociny/wip-41285-luminous

luminous: rbd-mirror: don't overwrite status error returned by replay

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #30375 from smithfarm/wip-41480-luminous

luminous: rgw: make dns hostnames matching case insensitive

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #30666 from smithfarm/wip-41104-luminous

luminous: rgw: mitigate bucket list with max-entries excessively high

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #30677 from sobelek/wip-42049-luminous

luminous: ceph-volume: fix warnings raised by pytest

Merge pull request #28991 from smithfarm/wip-40249

luminous: tests: qa/rados/upgrade/jewel-x-singleton: set RBD_FEATURES

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

qa/rados/upgrade/jewel-x-singleton: do not run on Bionic

The test was failing because it did not explicitly set "os_type" and
"os_version", and teuthology defaulted to Ubuntu Bionic.

Jewel is not supported on Ubuntu Bionic, so we should not try to run tests on
that Ubuntu version.

NOTE: This commit is not cherry-picked from master because the fix is
luminous-specific (there is no "rados/upgrade/jewel-x-singleton" test after
luminous).

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/rados/upgrade/jewel-x-singleton: set RBD_FEATURES

This commit cannot be cherry-picked from master because luminous is the only
non-EOL stable branch that still includes the
qa/rados/upgrade/jewel-x-singleton suite.

Fixes: https://tracker.ceph.com/issues/40249
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #30665 from smithfarm/wip-42106-luminous

luminous: build/ops: admin/build-doc: use python3

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24169 from tchaikov/wip-17516-luminous

luminous: cmake: pass -march to detect compiler support of arm64 crc/crypto

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #26247 from liewegas/wip-bluestore-disable-csum-luminous

luminous: bluestore: add bluestore_ignore_data_csum option

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #29859 from athanatos/sjust/wip-luminous-20924

luminous:  mds,osd,mon,msg: use intrusive_ptr for holding Connection::priv

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #27557 from smithfarm/wip-39277-luminous

luminous:  platform.linux_distribution() is deprecated; stop using it

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #27912 from ifed01/wip-ifed-fix-39565

luminous: os/bluestore_tool: bluefs-bdev-expand: indicate bypassed for main dev

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29025 from ifed01/wip-ifed-fix-stupid-len0-lum

luminous: os/bluestore: avoid length overflow in extents returned by Stupid Alloc

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29248 from smithfarm/wip-38436-luminous

luminous: crc cache should be invalidated when posting preallocated rx buffers

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #29366 from dzafman/wip-37748

luminous: tools: Add clear-data-digest command to objectstore tool.

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #28812 from trociny/wip-40592-luminous

luminous: rbd-mirror: handle duplicates in image sync throttler queue

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

ceph-volume: avoid backported version of configparser

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 88a8d5d967c9060e525e91f48b623beebee00c6d)

ceph-volume: fix warnings raised by pytest

Warnings are related to configparser. Update tests too.

Fixes: https://tracker.ceph.com/issues/41907
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 31a0c7e77bf5d0193e90e6693b424ca5305b47cc)

rgw: fix minimum of unordered bucket listing

A recent PR made sure that a bucket listing could not request too many
entries at once. It also did a minimum computation for number of
entries. For ordered listing the minimum was 0, as required to pass
all unit tests. However the minimum for unordered listing was left at
1. In order to make ordered and unordered listing behave the same --
with the exception of ordering -- the minimum for unordered listing is
modified to 0.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 38e02a7157d4c534cc9fd8ccfe20fdd93a9b07a8)

rgw: mitigate bucket list with max-entries excessively high

When listing a bucket with radosgw-admin, the user can specify the
maximum number of entries. That number can be unreasonably large, and
can affect the performance and memory availability. For example:

    radosgw-admin bucket list --bucket mybucket1 --max-entries=10000000

This has the potential for creating large data structures at multiple
levels in the the call stack of the radosgw(-admin) process,
potentially causing the process to run out of memory. This change
limits the maximum number of entries requested in all but the high
level code to help mitigate this issue.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
(cherry picked from commit 300429c9e98a27e17c2a20ade82c6c63ac276c20)

Conflicts:
src/rgw/rgw_rados.cc
- in luminous, RGWRados::Bucket::List::list_objects_ordered does not take
  optional_yield parameter
- no std::string_view in C++11

common/config: hold lock while accessing mutable container

The 'call_gate_leave' method was accessing the 'obs_call_gate' map
without holding the required lock. This data structure could be
manipulated by another thread underneath the observer callback
thread context.

Fixes: https://tracker.ceph.com/issues/41354
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0cc2dd7bd1021dd8b5e22850495ce97da7d98716)

Conflicts:
src/common/config_proxy.h: logic incorporated into config.cc

admin/build-doc: use python3

to address https://github.com/sphinx-doc/sphinx/issues/3620, we need to
use sphinx with its fix at
https://github.com/sphinx-doc/sphinx/commit/e049f86b2de1cfdf8a74c88dc9593d047c85d5cb
in other words, we need to use sphinx v2.0.0 and up. but sphinx 2.0
requires python >= 3.5, so we have to use python3 for building the
documents.

in this change:

* doc-requirements.txt: install python3 packages on debian derivatives
* build-doc: install python3.6 packages from EPEL7, and use python3
  venv for using sphinx2
* doc-requirements.txt: bump up all python packages to latest
  stable.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit ace8cb15a600daf5261c0ab349b43010a238a1d9)

Conflicts:
admin/doc-requirements.txt
admin/build-doc

admin/build-doc: support renamed python2 RPMs

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit f309180b961094e1e3ab88b76eb113578b07d752)

Merge pull request #27639 from wumingqiao/luminous

luminous: mgr: drop GIL in StandbyPyModule::get_config

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29736 from smithfarm/wip-41334-luminous

luminous: build/ops: ceph-test RPM not built for SUSE

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.de>

Merge pull request #30556 from alfredodeza/luminous-guits-41392

luminous ceph-volume: do not fail when trying to remove crypt mapper

ceph-volume: do not fail when trying to remove crypt mapper

In a containerized context, at some point, need to run `simple scan` on a device
from a separate container (not the existing and running corresponding container
to that device), but this can't work because when it tries to remove the
mapper which is still in use by the corresponding running osd container,
it fails.
This can be a bit more permissive and simply throw a warning.

Closes: https://tracker.ceph.com/issues/41392
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 80d3e4ca054b4d1334c2fb99fede46ab54ca4305)

rgw: fix dns name comparison for virtual hosting

Fixes: https://tracker.ceph.com/issues/41692
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 60b2dbe67af4a919df0d1b802c1fe21909c48d4c)

rgw: make dns hostnames matching case insensitive

Currently when parsing host in requests, we try to case match against supplied
hostnames set, which violates dns hostnames being case insensitive. Do a case
insensitive comparision instead

Fixes: http://tracker.ceph.com/issues/40995
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 9bc7d7bd0cf42663ebf1131d9cac6c96e6d9f141)

Merge pull request #29038 from dcasier/wip-40100-luminous

luminous: ceph-volume: lvm.activate: Return an error if WAL/DB devices absent

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: lvm.activate: Return an error if WAL/DB devices absent

Fixes: https://tracker.ceph.com/issues/40100
Signed-off-by: David Casier <david.casier@aevoo.fr>
(cherry picked from commit de439a0a236e4752e64a0cd63b7ce1ad3278eff3)

Merge pull request #29771 from alfredodeza/luminous-bz-1738379

luminous ceph-volume: use the OSD identifier when reporting success

Merge pull request #30296 from alfredodeza/luminous-rm41378-2

luminous ceph-volume: pre-install python-apt and its variants before test runs

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #30305 from alfredodeza/luminous-rm40664

luminous ceph-volume  zap always skips block.db, leaves them around

Merge pull request #30299 from alfredodeza/luminous-rm41660

luminous ceph-volume: fix stderr failure to decode/encode when redirected

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph-volume tests create a test file for checking unicode output

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 79196354aca3b3d30c7de1e2ac8b5a3c06ee8330)

ceph-volume tests create a shell test for functional unicode

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 595e492783283359b613d199a75f3e7495bfb9ed)

ceph-volume tests verify new logging fallback and encodings in terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit ee18ebc078b2d380b88f3726737184f34345e2f3)

ceph-volume terminal remove unicode stream handler

This caused problems in environments where stderr was redirected, since
stderr sets the encoding to None. Getting it back again allows
everything to work correctly, and keeps all the current unit tests
passing

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit fe66e01f6a4b19bb15695537713a629a951b28ce)

ceph-volume: instantiate the new terminal logger in main()

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit bfb8422e262dcce49c50610a3977bfe1edc2a435)

ceph-volume create a logger for the terminal

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 95b16b516c0736839fe50f12ecbeb7cd62c67335)

ceph-volume create a new tox.ini for shell-based tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 4612ab3ae62a3e8e1cc9e51489feab0c938296d3)

Merge pull request #30302 from alfredodeza/luminous-ceph-volume-zap-fix

luminous ceph-volume lvm.zap fix cleanup for db partitions

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

ceph-volume lvm.zap fix cleanup for db partitions

this uses the correct type 'db' for db type partitions, else
a block.db parition does not get cleaned up by ceph-volume zap

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
(cherry picked from commit 140af8a7eee8d230ac9d94ba100320a20c137a12)

Merge pull request #30331 from jan--f/wip-9e2175c-luminous

luminous: ceph-volume: set a lvm_size property on the fakedevice fixture

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: set a lvm_size property on the fakedevice fixture

This is so tests can continue to set sys_api['size'] and the code
can retrieve that as Device.lvm_size

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 9e2175c4001434f7b0ed8ce6f723e063cf001440)

ceph-volume tests: verify that wipefs tries several times

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 845e62cb4ba2636325304db2c2d65a185ace25cf)

ceph-volume lvm.zap: retry wipefs several times to prevent race condition failures

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit b7b8b106fa485d3b8345b583fbc7f74e30cbc03e)

ceph-volume tests pre-instrall python-apt to prevent auto-install failing later

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 40cac7b9510f7904c2f9dfb1294d809e04bdcf7e)

Merge pull request #30129 from smithfarm/wip-41644-luminous

luminous: install-deps.sh: Remove CR repo

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #29910 from vumrao/wip-vumrao-bluefs-log-level

luminous: os/bluestore: apply shared_alloc_size to shared device with log level change

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #29190 from badone/wip-luminous-lazy-omap-stats-backport-tracker-38551

luminous: core: lazy omap stat collection

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #29175 from neha-ojha/wip-40583-luminous

luminous: common/options.cc: Lower the default value of osd_deep_scrub_large_omap_object_key_threshold

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #30149 from neha-ojha/wip-40769-luminous

luminous: common/options: Set concurrent bluestore rocksdb compactions to 2

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

common/options: Set concurrent bluestore rocksdb compactions to 2

Signed-off-by: Mark Nelson <mnelson@redhat.com>
(cherry picked from commit 26dc5189e73a1f5a22f89ee67eef5430ef660b5d)

install-deps.sh: Remove CR repo

Remove the continuous release repos for CentOS and Virtuozzo 7 as they
should no longer be needed since http://tracker.ceph.com/issues/13997 is
no longer relevant and the newer versions of selinux packages pulled in
by the build system are causing problems for systems without CR repos
enabled.

Fixes: https://tracker.ceph.com/issues/37335
Signed-off-by: Alfredo Deza <adeza@redhat.com>
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit f39d9b7d1f4f2b63233802211a9ed0e641446ca1)

Conflicts:
install-deps.sh

Merge pull request #30094 from jan--f/wip-41613-luminous

luminous: ceph-volume: lvm list is O(n^2)

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: test number of times LVs list was created

Add a test that cofirm that the LVs list was created only once when
calling devices.lvm.listing.list()

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 20d6264538dab444c0ef13f40c6c21ec3d2057f8)

ceph-volume: reuse list of LVs

Avoid creating of list of LVs in different methods of the same module
and resue them.

Fixes: https://tracker.ceph.com/issues/37490
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit ab3a58e78124a984d162b2f63992d01c5e59e2a7)

ceph-volume devices.lvm zap use the identifier to report success

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit d0d5750c1ba508152c1d5111f2003a039a27145f)

ceph-volume tests check success message when zapping

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 88c7429a60fdab6222d14132bea93b9bf773c720)

osdc/Objecter: invalidate crcs on preallocated rx buffers

Both simple and async messengers use c_str() when copying the data from
the socket into the receive buffer, going behind bufferlist's back.  If
the receive buffer is preallocated, we need to invalidate its crc cache
by hand to avoid possible data crc mismatches on the client side.

Fixes: https://tracker.ceph.com/issues/38416
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 3eeab975b604d9fc4290747f85a78d59a2452c2e)

Conflicts:
src/test/librados/io_cxx.cc
- In master, 3730d10623650ce8569be96b28cbba599a9a0db6 renamed this file from
  src/test/librados/io.cc but that commit is not being backported to luminous.
  Manually cherry-picked the test mods into src/test/librados/io.cc.

Merge pull request #29901 from alfredodeza/luminous-rm41378

luminous ceph-volume tests set the noninteractive flag for Debian

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

os/bluestore/BlueFS: Move bluefs alloc size initialization log message to log level 1
Fixes: https://tracker.ceph.com/issues/41399
Signed-off-by: Vikhyat Umrao <vumrao@redhat.com>
(cherry picked from commit 96d48af3252f0ec2c1864383ad5eee993ecfeb4f)

Merge pull request #29791 from jan--f/wip-41373-luminous

luminous: batch functional idempotency test fails since message is now on stderr

Reviewed-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests set the noninteractive flag for Debian, to avoid prompts in apt

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 89231c9a60a46ba0691f787efbfe5ab35f05c679)

rbd-mirror: don't overwrite status error returned by replay

Fixes: https://tracker.ceph.com/issues/39980
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit c06ebf0c075e0533149572fde6c203ab22495cf6)

os/bluestore: record gifts < expected size in superblock

Returning 0 from _balance_bluefs_freespace() skips recording
allocations in the superblock, so we fail the consistency check on
startup. The elseif branch handles this case already, so just remove
it from the first branch. This is luminous/mimic specific, since
bluefs extents are not recorded in the superblock in later releases.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

qa: fix for s3tests.functional.test_s3.test_bucket_get_location failed

Fixes: https://tracker.ceph.com/issues/41401
Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
(cherry picked from commit ad6edd4dd2d6e57682ff82ad0ad0bfcde5942d66)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

rgw: fix issue for CreateBucket with BucketLocation param under default zonegroup

Because there is not a default realm, the has_zonegroup_api always returns false
when create bucket with the BucketLocation parameter under default zonegroup,
which always leads to the CreateBucket with BucketLocation parameter request failed.

Fixes: https://tracker.ceph.com/issues/41401
Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
(cherry picked from commit 18115915fa861a4b32e8d442266e90e9589215f7)

rgw: set "api_name" to default_zonegroup_name for default zonegroup

Fixes: https://tracker.ceph.com/issues/41401
Signed-off-by: Enming Zhang <enming.zhang@umcloud.com>
(cherry picked from commit 1f8ee65c1c6515dd8006b9035a993a386ce2b7e4)

src/osd/: Move session variables out of if for backport

That construction isn't allowed in luminous.

Signed-off-by: Samuel Just <sjust@redhat.com>

mds,osd,mon,msg: use intrusive_ptr for holding Connection::priv

See-also: http://tracker.ceph.com/issues/20924
Signed-off-by: Kefu Chai <kchai@redhat.com>
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 72883956c26fdc4345324e9b27b45c3dfac17fa2)

 Conflicts:
src/osd/OSD.cc
src/osdc/Objecter.cc

Merge pull request #27551 from smithfarm/wip-osd-markdown-luminous

luminous: qa/standalone/osd/osd-markdown: fix dup command disabling

osdc: cleanup code.

Remvoe useless conditon.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 964068ca8f60a9409abbfdd5836eb89a29553229)

osdc/Objecter: using session::put() rather than put_session().

If using connection::get_priv(), it use session::put rather than
put_session() to reduce reduanant check.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit d485b1bf34e9e6bc7bc189a97f1b86718c5f0da5)

Merge pull request #27471 from dzafman/wip-38567

luminous: doc: osd_recovery_priority is not documented (but osd_recovery_op_priority is)

mgr/DaemonServer: [Cleanup] Remove redundant code

Signed-off-by: Shinobu Kinjo <shinobu@redhat.com>
(cherry picked from commit e1b4c9de677a9041ba9dfba37d1b37999842789a)

ceph-volume: fix batch functional tests, idempotent test must check stderr

Fixes: https://tracker.ceph.com/issues/41295
Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 88807110f3014e7a2a39c37ae5c64bb3ab107d80)

 Conflicts:
src/ceph-volume/ceph_volume/tests/functional/batch/playbooks/test_explicit.yml
        removed

os/bluestore: try to gift large extents, then fall back to small extents

First try to gift using the larger alloc_size (normally bluefs_alloc_size,
but here max(bluefs_alloc_size,bluefs_shared_alloc_size) just in case the
settings are weird.  If that fails, then try the shared_alloc_size.  If
that fails, fail and complain as before, with an more accurate error
message.

Signed-off-by: Sage Weil <sage@redhat.com>

os/bluestore: gift freespace to bluefs based on shared_alloc_size

This does not appear in master because this code was removed with the
allocator fallback behavior, which was not backported.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 98c04c48c318ea15fdcbb331f879d9a962243559)