rgw: ES sync: wrap all the decode bls in try block

since decode can throw, wrap all the try block

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 12b12ccea23871688cc4101c72f00b0575f1c01a)

Conflicts:
src/rgw/rgw_sync_module_es.cc
- mimic uses val.begin() where master has val.cbegin()

Merge pull request #27451 from badone/wip-mimic-spdk-rte_table_hash_cuckoo_create-build-error

mimic: spdk: update to latest spdk-18.05 branch

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26495 from pdvian/wip-38337-mimic

mimic: test: run-standalone.sh set local library location so mgr can find li…

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #26777 from pdvian/wip-38532-mimic

mimic: mgr/PyModule: put mgr_module_path first in sys.path

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #26811 from ashishkumsingh/wip-38565-mimic

mimic: run-standalone.sh: Need double-quotes to handle | in core_pattern on all distributions

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #27860 from ashishkumsingh/wip-39464-mimic

mimic: msg: output peer address when detecting bad CRCs

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #26709 from pdvian/wip-38507-mimic

mimic: osd/PrimaryLogPG: handle object !exists in handle_watch_timeout

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #26760 from pdvian/wip-38511-mimic

mimic: CLI: ability to change file ownership

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26810 from ashishkumsingh/wip-38587-mimic

mimic: common/str_map: fix trim() on empty string

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26833 from pdvian/wip-38561-mimic

mimic: mgr/BaseMgrModule: drop GIL for ceph_send_command

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26862 from ashishkumsingh/wip-38646-mimic

mimic: osd: fixup OpTracker destruct assert, waiting_for_osdmap take ref      with OpRequest

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27078 from yuriw/wip-yuriw-clients-mimic_2

qa/tests: initial checkin for `client-upgrade-mimic` suite

qa/tests: removed conflicting tests rbd_api_tests and rbd_import_export
and added devstack-tempest-gate.yaml instead

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #27447 from ifed01/wip-ifed-fix-39144

mimic: os/bluestore_tool: bluefs-bdev-expand: indicate bypassed for main dev

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #27558 from tspmelo/wip-26986-mimic

mimic: mgr/dashboard: Use human readable units on the OSD I/O graphs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #27587 from pdvian/wip-38972-mimic

mimic: osd: process_copy_chunk remove obc ref before pg unlock

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #27366 from ifed01/wip-ifed-fix-alloc-overflow-mimic

mimic: os/bluestore: fix length overflow.

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27570 from ifed01/wip-ifed-fix-21312-mimic

mimic: os/bluestore: call fault_range properly prior to looking for blob to …

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27629 from pdvian/wip-39044-mimic

mimic: osd/PGLog: preserve original_crt to check rollbackability

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #27635 from smithfarm/wip-39342-mimic

mimic: ceph-objectstore-tool: rename dump-import to dump-export

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #27647 from smithfarm/wip-39353-mimic

mimic: mon/OSDMonitor: further improve prepare_command_pool_set E2BIG error message

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #27738 from ifed01/wip-ifed-fix-39445

mimic: os/bluestore: fix out-of-bound access in bmap allocator.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27745 from ashishkumsingh/wip-39433-mimic

mimic: osd/PG: discover missing objects when an OSD peers and PG is degraded

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27747 from s0nea/wip-39347-mimic

mimic: mgr/dashboard: Manager should complain about wrong dashboard certificate

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27845 from mykaul/patch-1

doc: Update bluestore config docs - fix typo (as -> has)

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/rados: Update bluestore config - fix typo

as -> has

Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

Merge pull request #26643 from pdvian/wip-38448-mimic

mimic: MDSMonitor: do not assign standby-replay when degraded

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26804 from pdvian/wip-38542-mimic

mimic: qa: unmount clients before deleting fs

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26807 from pdvian/wip-38544-mimic

mimic: qa: remove libcephfs-java tests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #26963 from ashishkumsingh/wip-38734-mimic

mimic: qa: extend MDS heartbeat grace for valgrind

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #27023 from pdvian/wip-38670-mimic

mimic: mds: wait for client to release shared cap when re-acquiring xlock

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #27058 from pdvian/wip-38689-mimic

mimic: mds: inode filtering on 'dump cache' asok

Reviewed-by: Venky Shankar <vshankar@redhat.com>

msg: output peer address when detecting bad CRCs

Fixes: http://tracker.ceph.com/issues/39367
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
(cherry picked from commit 19679231fc32daa405cc060847a02ac1cfd51300)

cmake: update BuildSPDK for spdk-18.05

in spdk v18.05, libuuid is linked by libspdk_util.a, in which,
it is used by lib/util/uuid.c. and libspdk_vol.a uses the wrapper
function exposed by libspdk_util.a, so update the CMakefile script to
reflect the change.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c2026b7473173e021c1f5ee072828538ea5c5d61)

mgr/dashboard: .pylintrc: ignore too-many-statements check

Signed-off-by: Ricardo Dias <rdias@suse.com>
(cherry picked from commit c3465e03ad11d8c39b61a221aaee78762bc9465c)

osd/PG: discover missing objects when an OSD peers and PG is degraded

When a PG is remapped from OSD `a` to OSD `b`, the objects are
backfilled. When OSD `a` is restarted, objects become degraded
as `a` is no longer queried or considered as a backfill source.

As the PG is degraded, `PG::discover_all_missing` is not called
when a candidate OSD peers with the primary: The PG is already
active, thus `PG::activate` (and in turn missing object discovery)
is not called. Discovery is also not initiated from
`PG::RecoveryState::Active::react(const MNotifyRec& notevt)`
as there are no unfound objects.

This patch adds a call to `discover_all_missing` when
when an OSD sends its `MNotifyRec` message and the PG is degraded.

Fixes: https://tracker.ceph.com/issues/37439
Signed-off-by: Jonas Jelten <jj@stusta.net>
(cherry picked from commit e152d092f7b7839bb27ac7a5cf1c95f4d3752b32)

mgr/dashboard: Manager should complain about wrong dashboard certificate

Fixes: https://tracker.ceph.com/issues/24453
Signed-off-by: Volker Theile <vtheile@suse.com>
(cherry picked from commit a9ba5ba152cd54c3b4c94f5627cc85239906fe67)

os/bluestore: fix out-of-bound access in bmap allocator.

Fixes: https://tracker.ceph.com/issues/39334
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit b0d2411c9c48b49fff827841c5ec6b66533d9c58)

 Conflicts:
src/os/bluestore/fastbmap_allocator_impl.h

os/bluestore_tool: bluefs-bdev-expand: indicate bypassed for main dev

Pre-nautilis releases are unable to expand main device but they do not
 report about that hence user thinks expansion succeeds. Which is confusing.

Fixes: https://tracker.ceph.com/issues/39144
Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #26829 from pdvian/wip-38546-mimic

mimic: doc: updated reference link for log based PG

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #27606 from ifed01/wip-ifed-fix-alloc-ut

mimic: os/bluestore: fix improper backport for p2 macros for bmap allocator

Reviewed-by: Xie Xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

os/bluestore: fix improper backport for p2 macros for bmap alocator

Fixes: 3244c871f36ee22ee826a96f30e82a8d05c9170c
This caused TestAllocatorLevel01.test_4G_alloc_bugN to fail in mimic.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #27228 from tchaikov/mimic-36737

mimic: ceph-disk/tests: use random unused port for CEPH_MON

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #27259 from pdvian/wip-38872-mimic

mimic: pybind/rados: fixed Python3 string conversion issue on get_fsid

Reviewed-by: Kefu Chai <kchai@redhat.com>

mon/OSDMonitor: further improve prepare_command_pool_set E2BIG error message

d2c0fe9b5319a4404965c40ec92e291802ef30f6 improved this error message,
but it can be improved further by suggesting that the pg_num be increased in
smaller increments.

This commit is not cherry-picked from master because 4d3407ba93b2c15dd86aa7f97bd9f29c3508c60e
(merged for Nautilus) removed the error message in question.

Fixes: http://tracker.ceph.com/issues/39353
Signed-off-by: Nathan Cutler <ncutler@suse.com>

ceph-objectstore-tool: Rename dump-import to dump-export

If user specifies dump-import it will still work, but isn't
in the usage that way.

Fixes: http://tracker.ceph.com/issues/39284
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 96861a8116242bdef487087348c24c97723dfafc)

Conflicts:
qa/standalone/special/ceph_objectstore_tool.py
src/tools/ceph_objectstore_tool.cc
- trivial resolution

osd/PGLog: preserve original_crt in merge_log()

Fixes: https://tracker.ceph.com/issues/21174
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 47215a5c0e1544c9080fda1eee8f04c9890eea42)

osd/PGLog: preserve original_crt in proc_replica_log()

This is to prevent rolling back further than deleted object version.

Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit cd25c1d79d189b4e0389927e7c20d75afd2661d9)

osd/PGLog: should not rollback further than deleted object version

When a deleted object becomes a divergent entry in the pg log,
we should not be able to rollback to a version of the deleted
object that doesn't exist.

To avoid this, we need to preserve the original crt of the pg log,
before we update it in rewind_from_head() and use that to decide whether
we can rollback or not in _merge_object_divergent_entries().

Fixes: http://tracker.ceph.com/issues/36739
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit de18c592259816ad013ac82983cdec71bbfa51a0)

Conflicts:
src/osd/PGLog.h : Resolved in struct PGLog

mgr/dashboard: Use human readable units on the sparkline graphs

Fixes: http://tracker.ceph.com/issues/25075
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 6a8cdcef779a84a4610bbf78a87ec9e58f18465c)

osd: process_copy_chunk remove obc ref before pg unlock

drop the obc ref when pg locked is more safer, if not, on_change
may failed to clear it, and on_flushed may find leak.

Fixes: http://tracker.ceph.com/issues/38842
Signed-off-by: Kefu Chai kchai@redhat.com
Signed-off-by: Zengran Zhang <zhangzengran@sangfor.com.cn>
(cherry picked from commit 70874399da47411e1e07824ef50dd82c4ebb6922)

os/bluestore: call fault_range properly prior to looking for blob to reuse

Fixes: https://tracker.ceph.com/issues/21312
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit e0a395ee2240d7565953a9bb0a1e5c336efe6336)

Merge pull request #27284 from pdvian/wip-38904-mimic

mimic: osd/PGLog.h: print olog_can_rollback_to before deciding to rollback

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #27298 from ifed01/wip-ifed-alloc-issues-mimic

mimic: os/bluestore: bitmap allocator might fail to return contiguous chunk despite having enough space

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27318 from ifed01/wip-ifed-fix-p2-mimic

mimic: include/intarith: enforce the same type for p2*() arguments

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #27188 from smithfarm/wip-38901-mimic

mimic: doc: Minor rados related documentation fixes

Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #27476 from tchaikov/mimic-39164

mimic: qa: install python36-{cephfs,rados} instead of python34-*

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>

install-deps.sh: install '*rpm-macros'

so `yum-builddep` can have access to the latest macros for preparing the
build dependencies

Fixes: http://tracker.ceph.com/issues/39164
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 087ea813a061a1d6858aeae57950e90965f4ae15)

qa/tasks/ceph_deploy: install python3.6 instead of python3.4 for py3 tests

EPEL7 has switched over to python3.6 as the main python3. and we started
packaging python bindings for python3.6 since
https://github.com/ceph/ceph-build/pull/1283

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 42167b87909c20c9bb4d32bac02b1e01ec94af13)

PendingReleaseNotes: note on python3.6 changes

Fixes: http://tracker.ceph.com/issues/39164
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 77c5ee06311d610f9accd94edf14becdf5820542)

Conflicts: resolve conflict as the PendingReleaseNotes in master is
quite different than that of mimic

qa: install python3-{cephfs,rados} instead of python34-*

we install the latest python-rpm-macros on all builders since
https://github.com/ceph/ceph-build/pull/1283 . now that we started
building python36-* after that change, for testing the python3 packages on
CentOS/RHEL 7, we need to install python36-* instead of python34-*.

and after the change of 8ae1947, python36-* now "Provides" python3-*, we
can just install python3-* for fulfill the requirement for testing
python3 cephfs bindings.

Fixes: http://tracker.ceph.com/issues/39164
Signed-off-by: Kefu Chai <kchai@redhat.com>
Conflicts: this change is not cherry-picked from master, because,
  in master, we don't install python3 packages after 7e5c85b604.

rpm: add "Provides: python3-*" for python packages

so user can install python3-rados, instead of python36-rados, without
specifying the minor version of python. also, we should not break our
teuthology tests with this naming scheme change. for instance, our
cephfs qa suite installs `python3-cephfs` for testing the `cephfs-shell`

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 8ae1947728837bf4a4bcc8e9325aea69ff14b706)

rpm: always use 3 as the default python3_{pkgversion,version}

as we have the rpm macros installed in
https://github.com/ceph/ceph-build/pull/1283 .

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 11f0b8d5a8951cb50437f5e87055f3b9405f4556)

rpm: use python 3.6 as the default python3

some of our centos7 jenkins builders are failing to build ceph master and
nautilus branches. because EPEL7 recently switched from python3.4 to
python3.6 as the native python3. see
https://lists.fedoraproject.org/archives/list/epel-announce@lists.fedoraproject.org/message/EGUMKAIMPK2UD5VSHXM53BH2MBDGDWMO/

and one of our BuildRequires, cmake3,
was offered by EPEL7. it also followed the python3.6 switch-over to
rebuild against python3.6. as a result, the cmake3-data-3.13.4-2.el7
started to depend on /usr/bin/python3.6, which is in turn offered by
python36 package. after installing python36 as a dependency of the
updated cmake3. but in cmake, we originally checks for the latest
python3 interpreter if WITH_PYTHON3 is enabled, that's why these
builders which happen to install these updated packages started to fail
when detecting the existence of python3.6 related build dependencies.

as a fix, in d1e83082,
python%{python3_pkgversion}-{devel,setuptools,Cython} are listed as
BuildRequires to reflect this change in EPEL7. before d1e83082, we
hardwired them to python34-*.

but as following analysis puts, there are cases where `yum-builddep`
is inconsistent with `rpmbuild`. as `yum-builddep` changes the how
`python3_pkgversion` and `python3_version` macros are expanded:

- none of the packages installed by `yum-builddep` installs the python3
  related rpm macros, so the system stays with whatever python3 it was
  using. in this case, `rpmbuild` won't complain, as the
  `python3_pkgversion` and `python_version` are consistent before and
  after `yum-builddep`.
- system has python3.4 installed before `yum-builddep`. but
  `yum-builddep` installed python3.6 and also the updated
  `python-rpm-macros` packages, which points `python3_version` and
  `python3_pkgversion` to 3.6 and 36 respectively. in this case,
  `rpmbuild` will complain, because when we run `yum-builddep`,
  `python3_version` was still "3.4".
- system does not have python3 installed before `yum-builddep`. so
  it was using python34 for preparing the "BuildRequires". but some
  of the packages installed by `yum-builddep` installs python36, and
  also the updated `python-rpm-macros` packages, which points
  `python3_version` and `python3_pkgversion` to 3.6 and 36 respectively.
  in this case, `rpmbuild` will complain, because the python36 related
  dependencies are missing. what the system has is python34
  dependencies.
- system does not have python3 installed before `yum-builddep`. so
  it was using python34 for preparing the "BuildRequires". but some
  of the packages installed by `yum-builddep` installs python34, and
  also the updated `python-rpm-macros` packages, which points
  `python3_version` and `python3_pkgversion` to 3.4 and 34 respectively.
  in this case, `rpmbuild` won't complain, as the
  `python3_pkgversion` and `python_version` are also consistent before and
  after `yum-builddep`.

as we cannot tell if the system has python3 or what the python3 version
the system has before `yum-builddep`, so what we can do is to ensure
`rpmbuild` has what it needs to build Ceph. so let's just stick with
python3.6.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 658ca2b0d79022edbc6cb1e488b243844e87bfbe)

rpm: use python3.4 on RHEL7 by default

python3.4 is the native python3 before 7.6

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cea9d18ced929a330f2299681ef93768449a05ed)

rpm: install python%{python3_pkgversion}-devel on el7

as RHEL 7.7 support a native python3.6, we cannot hardwire to python3.4
on RHEL7/CentOS7 anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d1e83082b42f784d83b2342bb953d3931a940173)

rpm: pass %{python3_version} to cmake

to force cmake to use the python3 and python3 modules for building
python3 bindings

on the debian side, it's okay to continue using "-DWITH_PYTHON3=ON", as

- cmake does normalize "ON" to 3
- debian's cmake extension lives on /usr/lib/python3/dist-packages/
  not in a specific /usr/lib/python3.x/dist-packages directory

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7681a38ce51743a19ec5679d49b362f04e271ddb)

cmake: should PYTHON3_VERSION_STRING of libpython3

and make sure the version matches exactly the requested one

in future, we should use FindPython.cmake

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 7b15b682b1201929e0ba9efc332ebb8652be40a8)

cmake: check for MAJOR.MINOR version of python3

We can only check for MAJOR.MINOR version of python3 since
FindPython3Libs does not support checking for MAJOR.MINOR.PATCH version
of python3. We also need to make sure we use the PYTHON3 versions of
these variables.

This should fix a regression introduced by c961e00.

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 8b3f0d5bcce8a5c8ddff0a362fbccb05d18335f6)

cmake: check for libpython of the same version of interpreter

actually cython or python3-cython's dependencies should take care of it.
but we should at least get it right on our side if we check it.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c961e002d22ab7ceec20f1019f8c558ec9fcce6c)

cmake: use specified python3 version if any

use might have multiple python3 installed, some of them has/have all
dependencies installed and is good enough for building Ceph. we should
not always use the latest python installed in the system and complain that
there is missing dependencies, even if user has installed all the
python3 dependencies for the older python3.

put in other words, if user only installs cython module for python3.4, but
she has both python3.6 and python3.4 in her system. we should not force
her to uninstall python3.6 for installing Ceph.

this change also aligns with MGR_PYTHON_VERSION. i am not applying the
same change to WITH_PYTHON2, because python2 is already stablized. and distros
are not likely to release new python2 releases.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d8c13e86048b7aac458f3eda101f9778e3d45dcf)

Merge pull request #26901 from pdvian/wip-38566-mimic

mimic: osd_recovery_priority is not documented (but osd_recovery_op_priority is)

Reviewed-by: David Zafman <dzafman@redhat.com>

spdk: update to latest spdk-18.05 branch

also bump dpdk to to spdk-18.05 branch. this should fix the build
failure on Fedora 28

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 8098e549643cd98f527ee923d6e3ab804cfe5b65)

Conflicts:
src/spdk

Merge pull request #24639 from dmick/wip-crashdump-mimic-backport

mimic: mgr: crashdump feature backport

Merge pull request #26497 from k0ste/mimic_backports

mimic: mgr/balancer: blame if upmap won't actually work

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #26537 from k0ste/mimic_backports3

mimic: mgr/prometheus: add interface and objectstore to osd metadata

Reviewed-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #26750 from noahdesu/cherrypy-ipv6-mimic

mimic: mgr/dashboard: fix for using '::' on hosts without ipv6

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #27082 from pdvian/wip-38720-mimic

mimic: crush/CrushWrapper: ensure crush_choose_arg_map.size == max_buckets

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #27257 from pdvian/wip-38858-mimic

mimic: crush: add root_bucket to identify underfull buckets

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: fix length overflow.

In fact, length of 'struct interval_t' and 'struct bluestore_pextent_t'
is uint32_t. But len of AllocatorLevel02::_mark_allocated is uint64_t.
So it may cause data overflow which cause bug.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
(cherry picked from commit 40d13406b85ad1630e61296b0b4f2a85f228d2a4)

Merge pull request #27351 from liewegas/wip-fix-osd-check-mimic

mimic: mon/OSDMonitor: fix osd boot check

Merge pull request #27039 from pdvian/wip-38684-mimic

mimic: librbd: avoid aggregate-initializing IsWriteOpVisitor

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #27041 from pdvian/wip-38685-mimic

mimic: librbd: avoid aggregate-initializing any static_visitor

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #26529 from pdvian/wip-38353-mimic

mimic: rgw: check for non-existent bucket in RGWGetACLs

Reviewed-by: Adam Emerson <aemerson@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>

Merge pull request #26641 from pdvian/wip-38447-mimic

mimic: rgw: only update last_trim marker on ENODATA

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27029 from pdvian/wip-38672-mimic

mimic: rgw: sync module: avoid printing attrs of objects in log

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27174 from pdvian/wip-38756-mimic

mimic: rgw: ldap: fix LDAPAuthEngine::init() when uri !empty()

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27179 from pdvian/wip-38772-mimic

mimic: rgw: nfs: skip empty (non-POSIX) path segments

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #27297 from theanalyst/mimic-sse-c-fixes

Mimic sse c fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mon/OSDMonitor: relax osd check to allow octopus osd with mimic mon

This mirrors commit 1a689766a50d346bef9f74d45a5086f54f41cca5 in master.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #27321 from liewegas/wip-valgrind-mimic

mimic: qa/valgrind: backport of latest valgrind suppressions

qa/valgrind: backport of latest valgrind suppressions

in the correct location

Signed-off-by: Sage Weil <sage@redhat.com>

include/intarith: enforce the same type for p2*() arguments

x and align must be of the same width.  Otherwise the lack of
sign-extension can bite, e.g.:

  uint64_t x = 128000;
  uint32_t align = 32768;
  uint64_t res = p2roundup(x, align);
  // res = 18446744069414715392

While the templates added in commit c06b97b3d7e3 ("include: Add
templates along side macros in intarith") are technically correct,
P2*() macros weren't meant to be used with different types.  The
Solaris kernel (which is where they originated) has P2*_TYPED()
variants for that.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit d0293dd2d8c0327c9d4d6b826a966dd7b18ea318)

 Conflicts:
src/librbd/io/ImageRequest.cc
src/msg/async/frames_v2.h
src/os/bluestore/StupidAllocator.cc

os/bluestore: unconditionally cap chunks returned by allocator to 2^31

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 1337443e37ffbfcaf494bdb2bb80db4cf41b97f9)

os/bluestore: start using 64-bit intervals for bitmap allocator

This helps to avoid 4Gb+ values wrapping which resulted in
allocation failures.

Fixes: https://tracker.ceph.com/issues/38761
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit d1466963a60ed2345a6fc159071ec09ce422b656)

os/bluestore: make bluestore interval base template.

We might need intervals at BlueStore with both 32-bit and 64-bit
lengths. Having base template eliminates the need for copy/paste

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit f775d4227fb9432840c795115669766565c5e718)

tests/fastbmap_alloc: UT to reproduce 4G allocation bug

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit ed8a93ba387458961444913a8695f538595efacf)

os/bluestore: os/bluestore: implement dump for bitmap allocator

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit 80310d98a7456afdb2002eaf2891853ea21eaf68)

rgw: fixup_range() clamps offsets to valid part range

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4b3bac20d550cae274c07a1d555751483dde463b)

test_rgw_crypto: add invalid ranges beyond obj boundary

This is just to assert we're not doing an invalid memory access

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit b34a00de0f6ecea6d97fd80f65d2a2dca0876e6e)