mgr/dashboard: remove pyOpenSSL version pinning

Tests require a newer version of pyOpenSSL.

Fixes: https://tracker.ceph.com/issues/48506
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #38497 from tchaikov/wip-seastar

seastar: update seastar submodule

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #38491 from agayev/fix-hmsmr-compilation

os/bluestore: Fix HMSMRDevice.cc compilation.

Reviewed-by: Kefu Chai <kchai@redhat.com>

seastar: update seastar submodule

to pick up recent fixes in file io

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38437 from ifed01/wip-ifed-alloc-replay-dump

os/bluestore: add more means to troubleshoot allocator(s)

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #38454 from tchaikov/wip-compressor-cleanup

compressor: put Compressor into TOPNSPC namespace and use string_view

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38458 from tchaikov/wip-librados-cct

librados: move definition of cct_deleter to .cc

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38418 from anthonyeleven/anthonyeleven/clarify-op-priorities

doc: clarify osd recovery op priority and fix a couple of typos

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

os/bluestore: Fix HMSMRDevice.cc compilation.

Signed-off-by: Abutalib Aghayev <agayev@psu.edu>

Merge PR #38419 into master

* refs/pull/38419/head:
vstart_runner: print result line unless opt_rotate_log is set

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #37956 from dorindabassey/example-notific

examples/boto3/README: examples notification

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #38374 from rzarzynski/wip-crimson-backfill-remove

crimson: add support for backfill, part 2 -- MOSDPGBackfillRemove

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38392 from rzarzynski/wip-crimson-backfill-virtual_pgfacade

crimson: use explicit, abstract interfaces for backfill's facades.

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38208 from sebastian-philipp/mgr-cephadm-type-annotations-mandatory

mgr/cephadm: make type annotations mandatory for some modules

Reviewed-by: Michael Fritch <mfritch@suse.com>

crimson: use explicit, abstract interfaces for backfill's facades.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #37130 from pcuzner/cephadm-exporter

cephadm:Add a daemon mode for cephadm to provide a metadata endpoint

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #38483 from tchaikov/wip-crimson-cleanup

crimson: cleanups

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

mgr/cephadm: make annotations for schedule.py mandatory

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: make annotations for inventory.py mandatory

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: make annotations for serve.py mandatory

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: make annotations for upgrade.py mandatory

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #38106 from varshar16/wip-fix-rook-orch-ps-img-id-name

mgr/rook: Fix container id and image id in 'orch ps'

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

Merge pull request #38346 from varshar16/wip-remove-container-id-field-orch-ps

mgr/orchestrator: Remove 'CONTAINER ID' col from orch ps if unknown

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #37952 from mgfritch/cephadm-json-decode

mgr/cephadm: handle JSONDecodeError

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #38032 from mgfritch/cephadm-last-local-image-name

cephadm: reference the last local image by digest

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>
Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #38064 from mgfritch/cephadm-rm-host-key-err

mgr/cephadm: verify the `host` key exists during refresh

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

crimson/test: make the backfill testing quieter.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/test: make use of maybe_flush() in the backfill testing.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: add support for MOSDPGBackfillRemove at primary.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: add support for MOSDPGBackfillRemove at replicas.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #38130 from sebastian-philipp/cephadm-upgrade-fail-daemon

mgr/cephadm: upgrade: fail gracefully, if daemon redeploy fails

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #38317 from sebastian-philipp/cephadm-reduce-log-dg-noise

mgr/cephadm: OSD service: reduce log noise

Reviewed-by: Michael Fritch <mfritch@suse.com>

crimson/osd: return something in PG::do_delete_work()

this silences the warning of

crimson/osd/pg.cc:316:1: warning: no return statement in function returning non-void [-Wreturn-type]
  316 | }
      | ^

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/os: use a constexpr function for sanity test

since match_stage_t is an alias of uint_8. it is an unsigned type, so an
instance of this type should be always greater or equal to 0.

this silences warning like:

stage_types.h:37:50: warning: comparison is always true due to limited range of data type [-Wtype-limits]
   37 |     static_assert(STAGE >= STAGE_BOTTOM && STAGE <= STAGE_TOP);
      |                                            ~~~~~~^~~~~~~~~~~~

also, GCC complains if it is able to assure that the compare always
returns true. so a function helps to silence it.

this warning is meaningless per-se, as we are using static_assert() to
perform sanity test at *compile-time*.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38460 from ronen-fr/wip-ronenf-lua-move

rgw: removing a pessimizing move

doc: clarify osd recovery op priority and fix a couple of typos

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #36986 from cyx1231st/wip-seastar-onode-tree-upstream

crimson/seastore: add initial onode staged-fltree

Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/test: fix test/tool make target names

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/staged-onode-tree: use LT, EQ, GT for comparison results

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/onode-staged-tree: add and fix comments

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #38356 from icepic/patch-2

doc/radosgw: Fix misspelled word in index.rst

Reviewed-by: Zac Dover <zac.dover@gmail.com>

rgw: removing a pessimizing move

"warning: moving a temporary object prevents copy elision"

Co-authored-by: Kefu Chai <kefu@redhat.com>
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #38324 from tchaikov/wip-mon-cleanups

mon: drop QuorumService and cleanups.

Reviewed-by: Joao Eduardo Luis <joao@suse.com>

mon/ConfigKeyService: drop ConfigKeyService::dispatch()

this method is basically an alias of
ConfigKeyService::service_dispatch(), so get rid of it.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/ConfigKeyService: drop ConfigKeyService::get_name()

the only caller of this function is prefix(), so hardcode the service
name there instead.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/ConfigKeyService: remove unused bits

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/ConfigKeyService: store mon and paxos using reference

before this change, we always have a valid pointer as member variable.
so, a reference serve this purpose better.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon: drop QuorumService

so far we only implemented ConfigKeyService, so move QuorumService into
the its only child class -- ConfigKeyService.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon/ConfigKeyService: move private member variable declaration to the end of class

to follow https://google.github.io/styleguide/cppguide.html

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon: store a reference as member variable

before this change, we always have a valid pointer as member variable.
so, a reference serve this purpose better.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mon: use unique_ptr<> to manage lifecycle of Monitor::paxos

it is less error-prone, and has better readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>

librados: move definition of cct_deleter to .cc

so cct->get() and cct->put() are visually closer. this improves the
readability.

Signed-off-by: Kefu Chai <kchai@redhat.com>

compressor: pass string_view instead of string

this allows us to find by string without creating a std::string
instance, even if this string is not nul terminated.

Signed-off-by: Kefu Chai <kchai@redhat.com>

compressor: put Compressor into TOPNSPC namespace

we want to add the support for on-the-wire compression to msgr v2, and
this feature will be shared by classic osd and crimson. but
Compressor.cc is also used by bluestore which is linked against by
crimson also. Compressor depends on CephContext which has two different
implementation in two namespaces for classic osd and crison.

so to avoid violating ODR, we should also put Compressor into two
different namespaces so it can be shared by alienstore and crimson in
the same executable.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38442 from badone/wip-bump-fmt-package-min-version

cmake: require fmt 6.0.0 and up

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38389 from trociny/wip-48412

librbd: fix regression in object map diff request

Reviewed-by: Jason Dillaman <dillaman@hotmail.com>

Merge pull request #35821 from Aran85/fix-ec-read

osd/ECBackend: optimize remaining read as readop contain multiple objects

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: David Zafman <dzafman@redhat.com>

Merge pull request #37543 from ianwatsonrh/master

docs: With the addition of STS x-amz-security-token is now supported

Merge pull request #38401 from a16bitsysop/alpine-tests

test/lazy-omap-stats: Fix compilation on alpine linux

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38314 from badone/wip-monitor-reset-abort-if-is_shutdown

mon: paxos: Delete logger in destructor

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38410 from badone/wip-monitor-logger-deelete-in-destructor

mon: Delete logger in destructor

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #37314 from ifed01/wip-ifed-faster-rm-p1

osd: optimize PG removal (part1)

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #38438 from ifed01/wip-ifed-fix-avl

os/bluestore: fix inappropriate ENOSPC from avl/hybrid allocator

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38436 from ktdreyer/prometheus-err

mgr/prometheus: don't store exception as e

Reviewed-by: Boris Ranto <branto@redhat.com>

cmake: require fmt 6.0.0 and up

The cmake search for a local package has lagged in regard to our usage
of it.

Fixes: https://tracker.ceph.com/issues/48453
Signed-off-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #38400 from tchaikov/wip-pybind-use-non-deprecated-functions

pybind/rados: use rados_set_pool_full_try instead of the deprecated one

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #36384 from linuxbox2/wip-doc-ippolicy

rgw:doc: document support for (Not)IpAddress in S3 policy

os/bluestore: bring back probing available space for main dev allocator.

This feature was mistakenly removed while making a single allocator
for main device (https://github.com/ceph/ceph/pull/30838).

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #37546 from myoungwon/wip-revamp-tier-flush

osd: refactoring tier_flush()

Reviewed-by: Samuel Just <sjust@redhat.com>

doc/osd-config-ref: adjust default value for osd_delete_sleep_hybrid

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

osd: adjust osd_delete_sleep_ssd[_hybrid]

1) Do not differentiate all-flash and hybrid(ssd+hdd) deployments since
they both bound to SSD drive backing DB volume.
2) Drop osd_delete_sleep_hybrid from 2s to 1s
3) Raise osd_delete_sleep_ssd to 1s since 0 is a way too low - having
it that low tend to cause a significant negative imact on OSD's regular
operation.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: introduce removal perf counter

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: track kv utilization in kv_sync_thread

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: perf counters for omap_get_keys[_get_values]

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

osd/pg: use next when calling collection_list for pg removal

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #38449 from trociny/wip-rbd-flush-finisher

librbd: flush task finisher queue on unregistering watcher

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38399 from es-gyt/master

rbd: current pc value should be greater than last_pc

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #38403 from zdover23/wip-doc-CLT-update-02-Dec-2020-1

doc/governance.rst - updating CLT list

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #38440 from bk201/wip-48454

mgr/dashboard: fix trusted_ip_list in iSCSI service form

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #38398 from tchaikov/wip-48210

pybind/mgr: unpin urllib3

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #38298 from bk201/wip-43165

mgr/dashboard: refactor /api/orchestrator/* endpoints

Reviewed-by: Courtney Caldwell <ccaldwel@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #38432 from rhcs-dashboard/orch-not-available-incorrect-status

mgr/dashboard: Fix for misleading "Orchestrator is not available" error

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

test/librbd: fix race in TestLibRBD.ConcurentOperations

It was possible that before image1 was closed, both quiesce2 and
quiesce3 requests were received and the test got stuck on
create_snap1.join() waiting for the image1 to be closed, while
the close was waiting for quiesce3 to be completed.

Signed-off-by: Mykola Golub <mgolub@suse.com>

rbd: current pc value should be greater than last_pc

If rbd image enabled striping feature, stderr will
print repeat rate of progress when execu rbd export-diff.

Fixes: https://tracker.ceph.com/issues/48431
Signed-off-by: Yite Gu <yite.gu@easystack.cn>

Merge pull request #38379 from dillaman/wip-librbd-migration-5

librbd/migration: add thick-provisioned snapshot support to the raw format

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #38285 from jmolmo/include_hw_info_in_HostCache

mgr/cephadm: Get host facts data in host cache

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

doc/governance.rst - updating CLT list

This PR removes the following people from the
CLT. They've left the CLT:
* Lenz Grimmer
* Alfredo Deza
* Zack Cerza

This commit also adds:
* Ernesto Puerta

Fixes: https://tracker.ceph.com/issues/48436
Signed-off-by: Zac Dover <zac.dover@gmail.com>

mgr/cephadm: Get host facts data in host cache

Include cephadm gather-facts information in host cache

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

os/bluestore: print allocator params on init and failures.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

test/objectstore: bmap_allocator_replay_test: rename .cc file

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

test/allocator_replay_test: implements allocator's dump replay.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: write out general allocator parameters on allocator dump.

This includes allocator type, name, size, allocation unit which enables
allocator dump replay implementation.

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: fix inappropriate ENOSPC from avl/hybrid allocator

Under high fragmentation circumstances (in best-fit mode) these allocators might fail to
return continuous(longer than a single alloc unit) extents. Occurs when
requested size is longer than the maximum available extents and these
extents are unaligned with alloc unit. The latter results in the actual lengths of these extens are shorter
than the requested length and they do not match to the lookup. Which
finally causes no allocation at all.

Fixes: https://tracker.ceph.com/issues/47883
Signed-off-by: Igor Fedotov <ifedotov@suse.com>

librbd: flush task finisher queue on unregistering watcher

To make sure all inflight requests like scheduled
cancel_async_requests are complete on shutdown.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #37631 from zjcmszh/aio_setxattr

pybind/rados: add Ioctx::aio_setxattr() and remove_omap_range2()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38168 from agayev/cleaner-plumbing

os/bluestore: Introduce plumbing necessary for garbage collecting zones.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #38323 from wjwithagen/wjw-fix-nasm-wrapper

nasm-wrapper: Make nasm-wrapper OS independent

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #38325 from aclamk/bs-onode-put

os/bluestore: Only pass that decremented nref to 0 deletes object

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #38387 from hualongfeng/vstart_add_path

vstart.sh: add PATH in vstart_environment

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

doc/radosgw: fix a typo in index.rst

miniscule spelling nit.

Signed-off-by: Janne Johansson <icepic.dz@gmail.com>