qa/vstart_runner: fix args to os.path.join()

Result of os.path.join() before "./bin/ceph-mds" and after
"./bin/./ceph-mds".

Before -

2022-05-05 19:36:11,100.100 DEBUG:__main__:> ./bin/./ceph-mds -i a

After -

2022-05-05 19:38:48,179.179 DEBUG:__main__:> ./bin/ceph-mds -i a

Signed-off-by: Rishabh Dave <ridave@redhat.com>

qa/vstart_runner: minor bug fix and minor adjustments

The message regarding deletion of helper tools is printed for every
command. This message should be printed only when applicable.

Besides -

* Move XXX comments to _do_run() since it increases visibility of
  these messages.
* Move omission of arguments stuff to new method clear up the clutter.
* And remove shell as a parameter from _perform_checks_and_adjustments
  since it's redundant.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #37667 from rishabh-d-dave/vr-inherit-Remote

vstart_runner: let LocalRemote inherit Remote

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #46194 from athanatos/sjust/wip-55565

crimson: fix op tracking related bugs

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/common/operation: trim AggregateBlockingEvent::events entries

PglogBasedRecovery and BackfillRecovery reuse the same Operation
until their respective operations are complete.  Each recovery
operation adds an entry to AggregateBlockingEvent::events.  This
way, we only retain entries that are currently blocking.

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #46127 from Matan-B/wip-matanb-cyan-clone

crimson/os: Add OP_CLONE to cyanstore

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #46205 from rzarzynski/wip-crimson-cls_rgw-objapi

crimson/osd, rgw: implement the new Objclass API methods for cls_rgw

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #44418 from AmnonHanuhov/wip-fix_futurized_store

crimson/os: Don't limit the amount of returned keys per omap get call

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/osd, rgw: implement the new Objclass API methods for cls_rgw

They were introduced by 3877c1e37f2fa4e1574b57f05132288f210835a7.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge pull request #45732 from mchangir/mgr-snap_schedule-validate-fs-for-all-commands

mgr/snap_schedule: validate fs name argument

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #46064 from liu-chunmei/crimson-pool-quota

Crimson pool quota

Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/os: Add OP_CLONE to cyanstore

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/osd: check full

need check if pool exceeds its quota

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

crimson/errorator: change errorator template to int

need define EDQUOT which is not included in std::errc.

Signed-off-by: chunmei-liu <chunmei.liu@intel.com>

crimson/osd/pg_recovery: remove new_started, fix parallel_for_each call

Presumably typo from b7e2b11595c6e2cd2c638135ef6f044c40efaa87.

Fixes: https://tracker.ceph.com/issues/55565
Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/common/operation: use std::list for AggregateBlockingEvent::events

std::vector's references aren't stable after emplace_back.

Fixes: https://tracker.ceph.com/issues/55565
Signed-off-by: Samuel Just <sjust@redhat.com>

crimson/osd/pg_recovery: avoid using create_part_trigger() directly

AggregateBlockingEvent::TriggerI::maybe_record_blocking() should handle
invoking create_part_trigger().

Signed-off-by: Samuel Just <sjust@redhat.com>

crimson: retain reference to WaitForObjectRecovery until blocker resolves

PGRecovery::on_global_recover destroys the map entry without waiting for
the future returned from

    seastar::future<> wait_for_recovered(BlockingEvent::TriggerI& trigger) {

This commit changes WaitForObjectRecovery to be refcounted and retains a
reference until the future resolves.

Fixes: https://tracker.ceph.com/issues/55565
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #46170 from elacunza/patch-1

doc/rados/operations: Add missing required parameters to stretch-mode CRUSH rule example

Merge pull request #46108 from rhcs-dashboard/fix-ingress-ssl-cert-validation

mgr/dashboard: fix ssl cert validation for ingress service creation

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: sunilangadi2 <NOT@FOUND>

doc/rados/operations: Add missing required parameters to stretch-mode CRUSH rule example

Rule example is missing min_size and max_size, both are required.

Signed-off-by: Eneko Lacunza <elacunza@binovo.es>

crimson: remove unused PGRecovery method declarations

Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #45935 from dparmar18/cephfs-shell-doc-update

doc/man/cephfs-shell.rst: add use of exclamation mark with cmds

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #46097 from tchaikov/wip-crimson-cleanups

crimson/osd/main: s/values/config_values/

Reviewed-by: Chunmei Liu <chunmei.liu@intel.com>

Merge pull request #45788 from rzarzynski/wip-crimson-op-tracking-2

crimson: bring the core of op tracking

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #46006 from phlogistonjohn/jjm-nfs-cleanups-pt4

mgr/nfs: minor organizational cleanups

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #46035 from rkachach/fix_issue_52906

mgr/cephadm: adding logic to close ports when removing a daemon

Reviewed-by: Adam King <adking@redhat.com>

crimson/osd: drop legacy base classes for pipeline stages

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: drop the old blocking infrastructure

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: strip blokcing_future from OperationT

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate OperationThrottler to new tracking infra.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: crimson/osd: drop blocking_future from PGActivationBlocker

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: crimson/osd: drop blocking_future from PGMap

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: drop blocking_future from OSDMapGate

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson: bring the WaitBarrier tracking

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

vstart_runner: make LocalRemote inherit RemoteShell

And therefore get rid of methods duplicated in LocalRemote and add a
call to empty constructor of RemoteShell in LocalRemote.__init__().

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #46033 from iqbalredkhan/quota2

rgw: Refactoring Quota Management

crimson: move PipelineStageIT::enter() dispatch to compile-time

The optimization isn't the goal (but rather a side effect). The
reason behind this change is to let pipeline stages define exit
barriers incorporating `OpT`-dependant `Triggers` to track blocking
that happens on the barriers.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/os: add junction between OSD's ops and Seastore's OrderingHandle.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate AggregateBlocker-related ops to new tracking infra

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: bring AggregateBlockingEvent.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate PeeringEvent to new tracking infra, part 2

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: add BlockingEvent-aware variant of OSD::get_or_create_pg

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate InternalClientRequest to new tracking infra.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate RecoverySubRequest to new tracking infra.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate RepRequest to new tracking infra

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate PGAdvanceMap to new tracking infra.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: reduce unnecessary indentation level in PGAdvanceMap::start()

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate PeeringEvent to new tracking infra.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate BackgroundRecovery to new tracking infra.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate BackfillRecoveryPipeline to new tracking infra.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: CompoundPeeringRequest tracks its start and completion too

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: migrate CompoundPeeringRequest into new tracking infra

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: move SubOpBlocker from .cc to header

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: PGActivationBlocker does support new tracking infra

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: dissect PG::WaitForActiveBlocker into PGActivationBlocker

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: PGMap blocker does support new tracking infra

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: make PhaseOperationT able to deal with interruptors

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: pg_map.h doesn't pull pg.h anymore

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: convert entire ClientRequest to the new infra

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: slightly optimize OperationRegistryT::do_register()

There is no need to call the virtual op::get_type() twice.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: fix an assertion failure in Formatter when dumping ops

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: implement dump_ops_in_flight

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: track start and completion of ClientRequest

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: bring boilerplate for LTTng-based external tracking backend

At the moment it's mostly for verification & demonstration purposes.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: drop forward declaration of OSDMap osd.h

Having it doesn't make sense as we already include the header.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: add registry of external handlers of tracking events

This commit intentionally fails the build. It does that to verify
the static assertion on per-op-type lookup of event registry.

The concrete assertion is the one about `ClientRequest` as we
already moved two of its blocker into the new op tracking infra.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: switch one OSDMapGate of ClientRequest to new blocking infra

This time the demo of `BlockerT::BlockingEvent::TriggerI` which
is supposed to handle situations where:

1) `BlockerT` is separated from `TrackableOperationT` by many
intermediaries while
2) we don't want to templatize them all.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: switch a stage of ClientRequest to new blocking infra

This is a debut of the new blocking & tracking infrastructure
using the type-aware `BlockerT::BlockingEvent::Trigger<T>`.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: introduce PhasedOperationT to deduplicate PipelineHandles

This new class in the hierarchy will be handy when it will come to
introduce helpers for deailing with entering a pipeline stage.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: switch all OSD operations to TrackableOperationT

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: get rid of unnecessary private in ClientRequest

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: switch all ClientRequest-related pipeline stages to the new infra

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: pipeline stage classes can be bases for CRTP now

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: pipeline stage classes derive from BlockerT now

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: document OperationThrottler's behavior towards op's blocker list.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: simplify OperationThrottler::with_throttle()

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: move with_blocking_future from Operation to OperationT.

Buildability of this commit proves all users of `with_blocking_future()`
have the concrete operation type which is necessary on the the way
to the compile-time defined op's blocker registry.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/os/seastore: migrate OrderingHandle away from blocking_future

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: OperationThrottler derives from BlockerT instead of Blocker.

One another step closer. See the previous' commit description.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: make OperationThrottler aware about exact type of operation

This is a step towards making op's blocker registry static and
compile-time checked.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: make the OperationThrottler::with_throttle private

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/osd: add types PG and OSD-hosted OSDMapGate instances.

`ClientRequest` is going to hold all tracking events as a `std::tuple`
instance and access them through the type-related variant of `std::get`.
As two instances of `OSDMapGate` are used there, a need to differentiate
at the type level arises.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/common: add basic infrastructure for op tracking

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

rgw: user_quota and bucket_quota are defined under RGWQuota

struct RGWQuota {
      RGWQuotaInfo user_quota;
      RGWQuotaInfo bucket_quota;
};

Signed-off-by: Iqbal Khan <iqkhan@redhat.com>

Merge pull request #46080 from cbodley/wip-55420

cmake/rgw: add missing dependency on Arrow::Arrow

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
Reviewed-by: Gal Salomon <gsalomon@redhat.com>

Merge pull request #45341 from idryomov/wip-use-system-libpmem-on-c9s

rpm: use system libpmem on Centos 9 Stream

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #46073 from idryomov/wip-rbd-mirror-exit-code

rbd-mirror,immutable-object-cache: exit code is inverted

Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #46103 from tchaikov/cmake-alloc-cleanup

cmake: re-enable TCMalloc and allocator related cleanups

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #45845 from SUSE/wip-fix-55237

ceph.spec.in: openSUSE: require gcc11-c++, disable parquet

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

crimson/common: abstract from staticness of BlockerT::type_name.

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge pull request #37655 from rishabh-d-dave/vr-run-multiple-cmds

vstart_runner: accepts multiple commands as str too

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #45473 from neesingh-rh/fix_50470

cephfs-top: Multiple filesystem support

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #45776 from neesingh-rh/fix_48863

cephfs-shell: chmod should be allowed for changing all mode bits.

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #45974 from rishabh-d-dave/shell-unrecognized-cmds

cephfs-shell: exit with non-zero on getting unrecognized cmd

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #46135 from athanatos/sjust/wip-vstart-asok

vstart.sh: default CEPH_ASOK_DIR to out/

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #46137 from athanatos/sjust/wip-crimson-typos

crimson: misc typos

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #46134 from athanatos/sjust/wip-55326

crimson/common/format: remove ceph::coarse_real_clock format speciali…

Reviewed-by: Kefu Chai <tchaikov@gmail.com>

crimson/common/format: remove ceph::coarse_real_clock format specialization

...
..., value=..., loc=...) at /usr/include/fmt/ostream.h:114
...

Introduced: e9784125241064709d25e9c64d16e1c72b4d7bed
Fixes: https://tracker.ceph.com/issues/55326
Signed-off-by: Samuel Just <sjust@redhat.com>

Merge pull request #46128 from vumrao/wip-vumrao-quincy-release-note-update

docs/releases: update quincy release notes tracker 55383 known issue