qa/rgw: remove 1999-shard tests from rgw/verify

causing test failures because the host runs out of disk space

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 01516061ece6ebe4d265a994e5f1f8d255334fcd)

Merge pull request #34828 from rhcs-dashboard/wip-45328-octopus

octopus: monitoring: fix decimal precision in Grafana %percentages

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #34924 from smithfarm/wip-45401-octopus

octopus: mon/OSDMonitor: allow trimming maps even if osds are down

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34917 from sseshasa/wip-45392-octopus

octopus: mon/OSDMonitor: Always tune priority cache manager memory on all mons

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34449 from rhcs-dashboard/wip-44979-octopus

octopus: monitoring: Fix pool capacity incorrect

Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #34912 from batrick/i45389

octopus: qa: install task runs twice with double unwind causing fatal errors

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34766 from smithfarm/wip-45214-octopus

octopus: client: reset requested_max_size if file write is not wanted

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Yan, Zheng <zyan@redhat.com>

Merge pull request #34273 from ukernel/octopus-44800

octopus: mds: fix 'if there is lock cache on dir' check

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #34528 from smithfarm/wip-45039-octopus

octopus: mon: calculate min_size on osd pool set size

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Deepika Upadhyay <dupadhya@redhat.com>

Merge pull request #34416 from smithfarm/wip-44836-octopus

octopus: mgr/DaemonServer: fetch metadata for new daemons (e.g., mons)

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34769 from smithfarm/wip-45211-octopus

octopus: ceph-fuse: link to libfuse3 and pass `-o big_writes` to libfuse if libfuse < 3.0.0

Reviewed-by: Yan, Zheng <zyan@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #34775 from ajarr/octopus-fix-fs-commands

octopus: mon/FSCommands: Fix 'add_data_pool' command and 'fs new' command

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #34803 from smithfarm/wip-45220-octopus

octopus: tools/cephfs: add accounted_rstat/rstat when building file dentry

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34802 from smithfarm/wip-45219-octopus

octopus: qa/cephfs: update TestClusterFull:test_barrier

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #34801 from smithfarm/wip-45216-octopus

octopus: qa/tasks/cephfs: umount the mountpoints when tearDown

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34816 from vumrao/wip-vumrao-45317

octopus: radosgw-admin: add support for --bucket-id in bucket stats command

Merge pull request #34800 from smithfarm/wip-45049-octopus

octopus: mgr: force purge normal ceph entities from service map

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34799 from smithfarm/wip-44844-octopus

octopus: qa/test_config_session_timeout: fix options conflicting

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34798 from smithfarm/wip-44843-octopus

octopus: test/libcephfs: retry and wait for caps when setting the RD deleg

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34804 from smithfarm/wip-45222-octopus

octopus: cephfs-journal-tool: correctly parse --dry_run argument

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34805 from smithfarm/wip-45227-octopus

octopus: test_cephfs_shell: fix UnicodeDecodeError

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34681 from kotreshhr/octopus-subvolume-info

octopus: mgr/volumes: Add interface to get subvolume metadata

Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #34477 from jan--f/wip-44999-octopus

octopus: ceph-volume/batch: return success when all devices are filtered

Merge pull request #34480 from jan--f/wip-45003-octopus

octopus: ceph-volume/batch: check lvs list before access

Merge pull request #34886 from jan--f/wip-34341-no-tracker-octopus

octopus: ceph-volume: update functional testing deploy.yml playbook

Merge pull request #34446 from jan--f/wip-44975-octopus

octopus: devices/simple/scan: Fix string in log statement

ceph-volume: update functional testing deploy.yml playbook

This commit adds a call to `ceph-facts` role in the first play of this
playbook. This is needed so `ceph-validate` won't fail because of
following error:

```
fatal: [osd0]: FAILED! => {}

MSG:

'osd_pool_default_size' is undefined
```

`osd_pool_default_size` is set in ceph-facts.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5bf7cc87ec91e128bd106e101b43328cacef1c7b)

mon/OSDMonitor: ensure lec only accounts for up osds

If we also consider down osds, we may very well be in a healthy state
but keeping maps as far back as the last epoch when a given osd went
down. If said osd stays down for eons, we will be keeping bajillions of
maps that we shouldn't.

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit e62269c8929e414284ad0773c4a3c82e43735e4e)

qa/standalone: exercise osdmon's last epoch clean

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit 3d682c21f6246276fa30d179766a49f0eb78d73a)

mon/OSDMonitor: dump last epoch clean info on report

Signed-off-by: Joao Eduardo Luis <joao@suse.de>
(cherry picked from commit bd2e5c62757c17d00f30620c043ed06abb8a0bca)

mon/OSDMonitor: Always tune priority cache manager memory on all mons

Always call into priority cache manager (pcm) to tune the memory on the
leader and on all the followers (peons) as part of each tick(). This
ensures that the pcm on all the mons continuously tunes the tcmalloc
memory thereby ensuring that the peons don't run out of memory eventually.

Fixes: https://tracker.ceph.com/issues/45266
Signed-off-by: Sridhar Seshasayee <sseshasa@redhat.com>
(cherry picked from commit 5083e21192c3cfdec274c8bafdc4d73e7c843af9)

qa: move fsx deps into common cephfs install yaml

The double install task causes errors during unwind.

Fixes: fe3961fc36e4d0af896fe140ba2450e1556ab563
Fixes: https://tracker.ceph.com/issues/45387
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d791daa732e122c4b81ec5693eeb3971a87199cb)

Merge pull request #34797 from smithfarm/wip-45207-octopus

octopus: ceph-object-corpus: update to octopus

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34790 from smithfarm/wip-drop-is_opensuse-octopus

octopus: rpm: drop "is_opensuse" conditional in SUSE-specific bcond block

Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #34610 from ifed01/wip-ifed-better-bluefs-expand-octopus

octopus: os/bluestore: open DB in read-only when expanding DB/WAL

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #34830 from smithfarm/wip-45314-octopus

octopus: qa/standalone/scrub/osd-scrub-repair.sh: fix race in TEST_auto_repair_bluestore_failed

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34809 from smithfarm/wip-45041-octopus

octopus: osd/PrimaryLogPG: fix SPARSE_READ stat

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34807 from smithfarm/wip-44842-octopus

octopus: osd/PeeringState: do not trim pg log past last_update_ondisk

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #34806 from smithfarm/wip-44685-octopus

octopus: qa/standalone/osd/osd-backfill-stats.sh: get_latest_osdmap to propagate map change

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34353 from neha-ojha/wip-44757-octopus

octopus: common/options: Disable bluefs_buffered_io by default again.

Reviewed-by: Vikhyat Umrao <vikhyat@redhat.com>

Merge pull request #34795 from smithfarm/wip-45044-octopus

octopus: os/bluestore: Don't pollute old journal when add new device

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #34739 from votdev/wip-45262-octopus

octopus: mgr/dashboard: RGW auto refresh is not working

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #34742 from sebastian-philipp/octopus-backport-34691-34600-34620-34674-34678-34673-34675-34693-34619

octopus: cephadm: Batch backport April (3)

Reviewed-By: Michael Fritch <mfritch@suse.com>

rgw/rgw_bucket: clear tenant string for non tenanted buckets
in rgw_bucket_parse_bucket_key function.

Fixes: https://tracker.ceph.com/issues/45355
Signed-off-by: Vikhyat Umrao <vikhyat@redhat.com>
(cherry picked from commit 6c4594a6aa6db0aee05287a82eaa937f715dc8b0)

radosgw-admin: add support for --bucket-id in bucket stats command
Fixes: https://tracker.ceph.com/issues/41061
Signed-off-by: Vikhyat Umrao <vikhyat@redhat.com>
(cherry picked from commit 4cd16e13ca0c8709091737ad2cb2b37a3b19840d)

Merge pull request #34793 from smithfarm/wip-45063-octopus

octopus: os/bluestore: fix 'unused' calculation

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #34734 from smithfarm/wip-45259-octopus

octopus: rgw: reshard: skip stale bucket id entries from reshard queue

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34825 from smithfarm/wip-44895-octopus

octopus: rgw: pubsub sync module ignores ERR_USER_EXIST

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34826 from smithfarm/wip-45052-octopus

octopus: rgw: Disable prefetch of entire head object when GET request with range header

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34819 from idryomov/wip-krbd-unmap-spacing-octopus

octopus: tests: update unmap.t for table spacing changes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34810 from smithfarm/wip-45047-octopus

octopus: rbd-mirror: improved replication statistics

Reviewed-by: Mykola Golub <mgolub@mirantis.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #34425 from smithfarm/wip-44839-octopus

octopus: pybind/rbd: ensure image is open before permitting operations

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@mirantis.com>

os/bluestore: invoke _prepare_ondisk_format_super as the last op

Fixes: https://tracker.ceph.com/issues/45133
Signed-off-by: Igor Fedotov <ifedotov@suse.com>
(cherry picked from commit e792e19331d7cde91b0e8601fd33c41765a10232)

Merge pull request #34626 from shyukri/wip-45042-octopus

octopus: mgr/PyModule: fix missing tracebacks in handle_pyerror()

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #34629 from shyukri/wip-45069-octopus

octopus: mgr/telegraf: catch FileNotFoundError exception

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #34731 from smithfarm/wip-45215-octopus

octopus: rgw: use DEFER_DROP_PRIVILEGES flag unconditionally

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #34758 from callithea/wip-45272-octopus

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #34759 from callithea/wip-45280-octopus

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

qa/*/osd-scrub-repair.sh: don't fail if PG is in active+clean+wait

a0b453ad335671bd92f165115d6ee984d2412448 added the wait state, which can
make PGs stay in active+clean+wait for a while instead of going into
active+clean directly. As far as TEST_auto_repair_bluestore_failed is
concerned, we only care about the repair state being cleared.

Fixes: https://tracker.ceph.com/issues/45075
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 7c8b627eaa478a97b5694704170336ed4fc40666)

Conflicts:
qa/standalone/scrub/osd-scrub-repair.sh

qa/standalone/scrub/osd-scrub-repair.sh: fix race in TEST_auto_repair_bluestore_failed

We need to flush_pg_stats before checking for active+clean.

Fixed: https://tracker.ceph.com/issues/45075
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit 4f82ebf41b0150a77083c665e7cac8678a31fb09)

Conflicts:
        qa/standalone/scrub/osd-scrub-repair.sh

rbd-mirror: skip snapshot image sync if mirror snapshot is marked clean

This is currently only utilized for the case where a newly created image
has mirroring enabled at time of creation, but it could be expanded in the
future if we track writes.

Fixes: https://tracker.ceph.com/issues/44596
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit fbede28d1a7713248765cf91136bb19b3b3fdac2)

librbd: EnableRequest now accepts a boolean to indicate a clean image

If the image is clean, it's treated as if it was newly created and
therefore clean since snapshot id 0. The CreateRequest and
CloneRequest state machines pass true for this bool if mirroring
is being enabled during creation.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit ea64ba5a6f2a73e3440244c0b358b948647670c5)

librbd: pass clean_since_snap_id to CreatePrimaryRequest

This will be stored in the primary MirrorSnapshotNamespace for use by
the rbd-mirror snapshot replayer.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 959d5e60d8a7eec96c71875cf69d1d4a7d80816d)

cls/rbd: added clean_since_snap_id to MirrorSnapshotNamespace

It is only valid for primary snapshots so we can re-use the space
previously used by the primary_snap_id without the need to bump the
version id.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 50702eece0b1d7535d387bdf923b88cd4bbfee37)

rbd: add '--mirror-image-mode' optional to image creation actions

This allows the user to enable mirroring while creating, cloning,
copying, importing, or migrating an image.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6da56d798248369f15191d8d5ec3b093f8dca831)

librbd: save requested mirror image mode during migration prepare

Once the migration is committed, the requested mode will be applied.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 25c034f0a0a627b54a1e4b948292890385cb7b6d)

librbd: permit mirroring to be enabled upon image cloning

The librbd clone API's ImageOptions parameter now supports passing
the mirror image mode.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 407f2e5c172cf5bef49a28b3156099487e7c2bf6)

librbd: add non-primary global image id to mirror enable

The constructor that took an ImageCtx did not support passing the
non-primary global image id and instead defaulted to an empty string.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a90042b2546bdaa056610bd10d7dae0f5e5da18e)

librbd: permit mirroring to be enabled upon image creation

The librbd create API's ImageOptions parameter now supports passing
the mirror image mode.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 37f6fa8fe95729af845100e28df8b8e0668c2270)

librbd: mirror enable state machine might need to open image

If attempting to create a snapshot-based mirroring primary snapshot,
the image needs to first be opened. If we weren't supplied an image,
open the image, create the snapshot, and close the image again.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 931c9f0fd9332136466a8d81c9708b86f597f4be)

librbd: pass bit-flags to image::CreateRequest

The current boolean for skip enable mirror will be able to be changed to
a tri-state to force enable mirror (in addition to the current auto-enable
if in pool-mode).

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit b0c3ca63c2dff11c823d0d4456a2a07f1594fb9f)

monitoring: fix decimal precision in Grafana %

Set decimal precision to 2 positions for charts using percentunits.

Fixes: https://tracker.ceph.com/issues/45183
Signed-off-by: Ernesto Puerta <epuertat@redhat.com>
(cherry picked from commit 3fd804f10b2335ebcee6b885e2f870e86ba14388)

rgw: Disable prefetch of entire head object when GET request with range header

Disable prefetch of entire head object when GET request with range header.
The current behavior for the RGW is getting the whole object although the client asked only for a small bytes offset.
For example: If the client asked for bytes=0-1, The RGW will anyway fetch 0-4194304

Fixes: https://tracker.ceph.com/issues/44508
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit 2be5af0006169cb54547034aa98b7eacb8751d59)

rgw: pubsub sync module ignores ERR_USER_EXIST

otherwise, this error gets returned by RGWPSDataSyncModule::start_sync()
and data sync fails to start

Fixes: https://tracker.ceph.com/issues/44857
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 768c9f3839c682cb3f92745396773faa96b18a9f)

qa/workunits/rbd: fix peer_add helper

We need to temporary disable "exit on error" mode so it does not
abort when `rbd mirror pool peer add` returns "already exists"
error code.

Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 1ad816ae0c2d4f0db3d26e0ae993947154bab6c8)

qa/workunits/rbd: retry the addition of a mirror pool peer

fb4311f5 has fixed this for setup, but "remove mirroring pool"
test needs fixing too.

Fixes: https://tracker.ceph.com/issues/44938
Signed-off-by: Mykola Golub <mgolub@suse.com>
(cherry picked from commit 7eced158a9a3c47cc408b35219b4428e97e018fb)

tests: update unmap.t for table spacing changes

Introduced in 4d42b4c5a0ed ("common/TextTable: default to 2 spaces
separating columns") and 41f003518a07 ("common/TextTable: only pad
between columns").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 314b14aebeef4ef1dfb450d191e2f53eaf063e67)

rbd-mirror: track in-flight start/stop/restart in instance replayer

The shut down waits for in-flight ops to complete but the
start/stop/restart operations were previously not tracked. This
could cause a potential race and crash between an image replayer
operation and the instance replayer shutting down.

Fixes: https://tracker.ceph.com/issues/45072
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 31140a940ea1909c4b5d68ef4593cb582a527354)

rbd-mirror: switch to common C_TrackedOp context class

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 64f8d9c30c53077f48107e83ba836897aa99e609)

common: add helper C_TrackerOp context class

This wraps the functionality of starting and finishing a tracked op
into the standard context interface.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 4bd9d1501f3832206ef12155cec3f008e3160822)

rbd-mirror: utilize structured binding in 'for' loop

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 847f178c4bfb50ab1c1b21222fcb47759119aa43)

rbd-mirror: unlink from remote snapshot if required

If a previous remote snapshot was synced but the unlink failed,
ensure we retry the unlink so that the remote can cleanup the unused
snapshot.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit cb7b91dc02b64cb15f8d21e830a698bd4173b35a)

rbd-mirror: prune unnecessary non-primary mirror snapshots

Once a non-primary snapshot is no longer required for syncing, delete it
from the image.

Fixes: https://tracker.ceph.com/issues/44105
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 281af0de86b17244a2c64af80db0bf84de6d9819)

rbd-mirror: propagate full snap-seq mapping in non-primary snapshots

Previously only newly created user snapshots were included in the
non-primary snapshot snap-seq mapping table. However, we need to
retain a full history of the mapping table if we want to be able to
prune non-primary snapshots.

Failovers are a special case since we won't have a valid snap seq mapping
so it will need to be rebuilt. Luckily, both sides should be read-only
in the previous state so we can use the snapshot names to find matches.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit cb8187c0dd22ed9098ab51e47777aee5aec471a7)

rbd-mirror: ignore non-primary read-only state for remote images

snapshot-based mirroring needs to be able to potentially delete a
demotion snapshot during the unlink process. Previously, these
snapshots have been left while the read-only error was ignored.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6bf2132cf041f6ab16c86e46b5acd4cf2ce12060)

rbd: fix missing space when listing non-primary mirror snapshots

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0ca7817eceafd7756eee967e7d740afeedd517e7)

librbd: fixed race condition on demotion of snapshot-based mirrored image

A pending refresh could occur after setting the non-primary feature flag but
before the creation of the demotion snapshot. This would prevent the snapshot
from being created and would leave the image in a half-primary state.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit a3acdbd069b5c5bd62e528cbf2fbc33fe6f23d92)

librbd: store mirror peer uuids in non-primary demoted snapshots

This will allow a remote rbd-mirror process to have a snapshot to use for
delta sync operations during failover.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0102ce88701dd52208520b8c0ab2e505548ed3e1)

librbd: additional debug logs for mirror snapshot unlink peer

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit eed00eb17969791107ddb8df0e909790c17bd573)

test/rbd-mirror: fix gmock warnings during snapshot-based replayer tests

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 6a342bb5e01fea205088592128f62b55261408bf)

rbd-mirror: fixed race condition with snapshot sync and shutdown

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 8a18a7fc71944c55e7a5fa47e55572546b4088db)

rbd-mirror: add snapshot-based replay performance metrics

The mirror image status for replaying snapshot-based images now includes
bytes per second and per snapshot, in addition to an estimated number of
seconds until the image is fully synced.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 03aa081435abf12429fbe98b7ff1095a4b37d830)

rbd-mirror: switch to json_spirit formatter for snapshot image status

This will make it cleaner and easier to add additional data fields
to the existing JSON replaying status.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit de445810e97b345f647ffc5f5792dfc2ad589f06)

rbd-mirror: add journal-based replay performance metrics

The mirror image status for replaying journal-based images now includes
bytes and entries per second in addition to an estimated number of seconds
until the image is fully synced.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c0dc96bf137dac6942e14800e173bfb9bda233eb)

rbd-mirror: switch to json_spirit formatter for journal image status

The free-form journal replay status description is now JSON-encoded. The
"master"/"mirror" designators have been changed to "primary"/"non_primary"
to better align with RBD terminology.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit e04422f2c60a9ce891da496b71a568264e39233e)

rbd-mirror: periodically poll image replayer status

When metrics are incorporated, there might not be a forced status update
if no new data is available to replicate. However, we will want the metrics
to decrease over time.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 133cce8af50653d147d7147fc4bc7269ad10b5e1)

librbd: pass bytes-read statistics from deep-copy through handler

These simple stats will be utilized by rbd-mirror to compute throughput
metrics for snapshot-based mirroring.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit 0b6cbdeb9deec8184a43f1b7533c8316b22b9f5d)

librbd: pass specialized handler interface to deep-copy

The initial version mimics the existing ProgressContext callback
interface. Later commits will add additional deep-copy unique methods.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
(cherry picked from commit c7d6e762d19284a05b88daba6c27cbbdbdb210e9)

osd/PrimaryLogPG: fix SPARSE_READ stat

22960192 use readv to reimplement SPARSE_READ, however it is
still using total_read to accumulate total bytes it reads from
bluestore, which is always zero in code.

Fix by dropping the redundant local total_read counter.
By definition objects_readv_sync should return the correct bytes
it has read. Use that instead.

Signed-off-by: Yan Jun <yan.jun8@zte.com.cn>
(cherry picked from commit 528fb95326e76518e93fe43274f7a8e86049851d)

suites/rados/thrash: add rados-high-concurrency

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit ceb8e457e7c3bd1ce1c14833c091f517a024d6d5)