rbd-mirror: add peer_uuids for non-primary demoted group snapshot

GroupReplayer should add peer uuid for group snapshot if it is a
non-primary demoted snapshot, other wise this snapshot will be
unconditionally unlinked further, as doesn't have a peer uuid leading to
split-brain scenarios.

Credits to N Balachandran <nithya.balachandran@ibm.com> for the find.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

librbd: fix remove_interim_snapshots()

Fixed remove_interim_snapshots() to remove the image
snaps.

Signed-off-by: N Balachandran <nithya.balachandran@ibm.com>

rbd: rbd group info now displays mirroring info

The "rbd group info" will now display the mirroring mode, global-id
and whether primary for mirror enabled groups.

Signed-off-by: N Balachandran <nithya.balachandran@ibm.com>

qa/workunits/rbd: updates to mirror group bash scripts

- support cli parameters to specify the test to run
- support cli parameter to specify the number of times to repeat the test
- new tests
- added RBD_MIRROR_NEW_IMPLICIT_BEHAVIOUR env variable in preparation for
  changes to group snapshot behaviour

Signed-off-by: John Agombar <agombar@uk.ibm.com>

rbd-mirror: improvements to unlink group snapshot

1. If group snapshot is syncing do not remove mirror peer uuid of last complete
   snapshot [ i.e. currently incomlpete - 1] (on remote)
2. If group snapshot is synced do not remove mirror peer uuid of its
   respective on remote yet.

both will lead to split brain issues.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: synchronize image replayer and group replayer

Here on, image replayer waits for group_replayer to set remote image snap
end id inorder for image replayer to proceed with the image snapshot copy.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

mgr/rbd_support: create mirror gp snap asynchronously

... in the mirror group snapshot scheduler.

Signed-off-by: Ramana Raja <rraja@redhat.com>

librbd: add C and Python bindings for async mirror group snapshot create

Signed-off-by: Ramana Raja <rraja@redhat.com>

librbd/group/UnlinkPeerGroupRequest: convert sync to async calls

Signed-off-by: Ramana Raja <rraja@redhat.com>

librbd/mirror/snapshot: add async create mirror group snapshot

Signed-off-by: Ramana Raja <rraja@redhat.com>

librbd/mirror/snapshot: add async group prepare images

... state machine class.

Signed-off-by: Ramana Raja <rraja@redhat.com>

librbd: add C and python bindings for async mirror gp snap info

Signed-off-by: Ramana Raja <rraja@redhat.com>

librbd: add async state machine for "mirror_group_get_info"

Signed-off-by: Ramana Raja <rraja@redhat.com>

mgr/rbd_support: disallow scheduling mirror snapshots

... of images that are part of a group.

Creating a global, pool or namespace level mirror snapshot
schedule shouldn't schedule mirror snapshots of images that
are part of a group and reside in the pool or namespace.

Also disallow directly scheduling mirror image snapshots on
images that are part of a group.

Signed-off-by: Ramana Raja <rraja@redhat.com>

qa/workunits/rbd: add basic tests for mirror group snapshot scheduler

Add tests to check the basic functionality of the
mirror_group_snapshot_schedule module. Check that
- `rbd mirror group snapshot schedule add/rm/status/ls` commands work
- the module can recover from blocklisting of its client and continue
  to process requests

Signed-off-by: Ramana Raja <rraja@redhat.com>

tools/rbd: add CLI to call into mirror group snapshot scheduler

Signed-off-by: Ramana Raja <rraja@redhat.com>

pybind/mgr/rbd_support: add scheduler for mirror group snapshots

Signed-off-by: Ramana Raja <rraja@redhat.com>

librbd: add API to fetch group name using group ID

Signed-off-by: Ramana Raja <rraja@redhat.com>

librbd: add python and C bindings for group_list2

... that lists IDs and names of groups in a pool.

Signed-off-by: Ramana Raja <rraja@redhat.com>

pybind/rbd: add interface to create mirror group snapshot

... synchronously.

Signed-off-by: Ramana Raja <rraja@redhat.com>

pybind/rbd: add interfaces to fetch and list mirror group info

Signed-off-by: Ramana Raja <rraja@redhat.com>

librbd/api/Mirror: set the boolean field 'primary' in group_info and group_list

outputs to identify whether a mirror group is primary or not.

Signed-off-by: Ramana Raja <rraja@redhat.com>

cleanup: revert some of the unrequired changes

With this changes will be able to build and run librbd unit tests

Note: This changes will be later merged into the subsequent commits.

Signed-off-by: Vinay Bhaskar Varada <vvarada@redhat.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: more improvements in the group replayer

- Arresting the interruptions after shutdown
- Restart GroupReplayer if bootstrap returns remote group id as empty
- Fix a test failure in "regular group snapshots test"

Issue#23

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: create group snapshots only if remote group snapshot is complete

Also do not create a regular group snapshot until the next mirror group
snapshot is created. Otherwise we simply have an incomplete regular group
snapshot until the next mirror group snapshot is created.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

cls/rbd: changes needed to align rest with the proposed ones

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

cls/rbd: proposed changes

Changes proposed by 'N Balachandran'

Signed-off-by: N Balachandran <nibalach@redhat.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: group replayer work in-progress changes

This commit is a WIP targeting stability of tests failures, it might be diluted
into various existing commits

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

qa/workunits/rbd: mirror group tests improvements

Signed-off-by: N Balachandran <nibalach@redhat.com>
Signed-off-by: John Agombar <agombar@uk.ibm.com>
Signed-off-by: Adam Lyon-Jones <adamlyon@uk.ibm.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: move the rename detection logic to snapshot GroupReplayer

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: use group_header object for resync flagging

Also move the resync checks to snapshot GroupReplayer

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: rollback to last good snapshot just while force promote

fix the group membership to match the rollback snapshot

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: fix the below bugs

* fix braces in the imageMap update_images_added & update_images_removed
* do not allow image add from non-primary
* `down+unknown` status shown on querying individual images which are part
  of group enabled for mirroring
* `mirror pool status` shows down+unknown status
* fix imageMap being overwritten when multiple images are enabled for mirroring
* fix misleading error msg when getting status of a non-mirror enabled group

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: prune snapshots added to the list on image_replayer shutdown

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: discover primary demote snapshot in group_replayer

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: remove group snaps on primary at snapshot creation

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: Independent GroupReplayer

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

qa/workunits/rbd: mirror group functional tests

Also sets the RBD_MIRROR_INSTANCES to 1 to avoid any deviations.

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: implement group resync functionality

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: support mirroring regular non-mirror group snapshots

This commit also enable deep copying `.group` snaps.

$ rbd --cluster site-a snap ls pool1/test_image1 --all --debug-rbd=0
SNAPID  NAME                                                                                       SIZE     PROTECTED  TIMESTAMP                 NAMESPACE
     4  .group.2_10416b8b4567_104d6b8b4567                                                         128 MiB             Mon Apr 22 17:07:57 2024  group (pool1/test_group@group_snap1)
     7  .group.2_10416b8b4567_104f6b8b4567                                                         128 MiB             Mon Apr 22 17:07:59 2024  group (pool1/test_group@group_snap2)
     8  .mirror.primary.72855be4-1ffb-4094-8426-fb1d5f082c21.71b62dad-f515-43cf-91b2-bf1225c5a0fc  128 MiB             Mon Apr 22 17:08:03 2024  mirror (primary peer_uuids:[5f9ea7aa-fa5b-4d2e-a098-3afe006361aa])

$ rbd --cluster site-b snap ls pool1/test_image1 --all --debug-rbd=0
SNAPID  NAME                                                                                           SIZE     PROTECTED  TIMESTAMP                 NAMESPACE
     5  .group.2_10416b8b4567_104d6b8b4567                                                             128 MiB             Mon Apr 22 17:08:07 2024  group
     7  .group.2_10416b8b4567_104f6b8b4567                                                             128 MiB             Mon Apr 22 17:08:08 2024  group
     8  .mirror.non_primary.72855be4-1ffb-4094-8426-fb1d5f082c21.64488da7-3350-4d04-8e4d-90047182e004  128 MiB             Mon Apr 22 17:08:09 2024  mirror (non-primary peer_uuids:[] 6d963873-6679-483b-b05a-8bf2536f4fdf:8 copied)

$ rbd --cluster site-a group snap ls pool1/test_group --debug-rbd=0
NAME                                 STATUS
group_snap1                              ok
group_snap2                              ok
.mirror.2_10416b8b4567_10536b8b4567      ok

$ rbd --cluster site-b group snap ls pool1/test_group --debug-rbd=0
NAME                                 STATUS
.mirror.2_10376b8b4567_1037327b23c6      ok
group_snap1                              ok
group_snap2                              ok
.mirror.2_10416b8b4567_10536b8b4567      ok

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: support rename with group mirroring

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: add undo code, exclusive locking and quiescing

* add essential logic to undo partially succeeded API's like, group promote,
  group demote, group enable, group disable, group image add and
  group image remove
* add exclusive locking and quiescing with-in all the required group API's
* adress code duplication and optimization with in the group API's

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: address partial group snapshots case

Make sure group snapshots doesn't get copied to secondary if the group snapshot
is incomplete on primary. On creation time of a group snapshots on primary,
make sure to delete the previous snapshot in case it is incomplete.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: request group snapshot creation when creating group image snapshot

It makes the group image replayers to synchronize and to have the
group snapshot created.

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: unlink group snapshot when pruning non-primary snapshot

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: initial group replayer implementation

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: hook GroupReplayer

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: make pool watcher also refresh groups

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: introduce generalized mirror entity (type, global_id, size)

where type may be image or group. Make pool watcher and image
mapper use it currently for images. The group support is coming.

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd-mirror: remove dead code

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

rbd: add 'mirror group' commands

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

librbd: link group snapshot when creating non-primary snapshot

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

librbd: unlink group snapshot when removing mirror snapshot

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

librbd: allow to add image to group on creation

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

librbd: add mirror group API

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

librbd: don't send 'image updated' notifications

when enabling/disabling group mirroring

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

librbd: introduce 'group updated' mirroring watcher notification

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

librbd: introduce 'group add/remove image' async requests

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

librbd/api: assume user namespace for group snapshot

when the snapshot is specified by name only

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

cls/rbd: add method to unlink image snapshot from group snapshot

When no image snapshots are left the group snapshot will be
automatically removed.

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

cls/rbd: make possible mirror_image_list filter out group images

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

cls/rbd: add async versions of some group functions

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

cls/rbd: add group_spec and group_snap_id to mirror snapshot

which are going to be used when creating a group mirror snapshot.

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

cls/rbd: add mirror group types and methods

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

cls/rbd: add group snapshot namespace

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

cls/rbd: rename GroupSnapshotNamespace to GroupImageSnapshotNamespace

Signed-off-by: Mykola Golub <mgolub@suse.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

Merge pull request #63945 from ArbitCode/wip-raja-get-account-summary

rgw/iam: getAccountSummary API

Merge pull request #64592 from ArbitCode/wip-raja-get-caller-identity-docs

doc/radosgw: get caller identity docs

Merge pull request #64289 from sseshasa/wip-mclock-new-benchmark-mechanism

src/ceph_osd, osd: Implement running benchmark during OSD creation - Phase 1

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #64593 from chanyoung/fix-omap-iterate

crimson/.../omap_btree_node_impl: fix inner node iterate

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #64585 from myoungwon/wip-parallel-write-ool

crimson/os/seastore/rbm: improve large sequential write by issuing writes in parallel

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #64498 from xxhdx1985126/wip-seastore-io-block-time

crimson/os/seastore/async_cleaner: measure io block time

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #64418 from ifed01/wip-ifed-discard-mkfs

os/bluestore: make full device discard on mkfs a default behavior

Merge pull request #64142 from phlogistonjohn/jjm-smb-ip-bind

smb: support custom ip address binds

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>

Merge pull request #64690 from zdover23/wip-doc-2025-07-25-cephfs-ceph-dokan

doc/cephfs: edit ceph-dokan.rst (1 of x)

doc/cephfs: edit ceph-dokan.rst (1 of x)

Edit the first 62 lines of doc/cephfs/ceph-dokan. This is part of a
larger project whose goal is the improvement of the CephFS
documentation.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge PR #64523 into main

* refs/pull/64523/head:

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge pull request #64409 from bolzo/mds_disallow_root_fragmentation

MDS: disallow root directory fragmentation

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #64308 into main

* refs/pull/64308/head:

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #64297 into main

* refs/pull/64297/head:

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #63524 from batrick/i71462

 mds: wrong snap check for directory with parent snaps

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #62554 into main

* refs/pull/62554/head:

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #64465 from tchaikov/wip-crimson-formatter-sans-fmt-local

crimson/common: replace deprecated fmt::localtime() with localtime_r()

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #63408 from aainscow/ec_recovery_zero_detect

OSD: EC recovery zero detect

Merge pull request #64545 from ajarr/cleanup-librbd-mirror-enable-req

librbd/mirror: cleanup EnableRequest::handle_get_mirror_image()

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #64193 from tchaikov/wip-bluestore-test-shutdown

os/bluestore: fix memory leak in HybridAllocator destructor

Merge pull request #64661 from afreen23/revert-stylus

Revert "mgr/dashboard: Fix stylus issue"

Reviewed-by: Naman Munet <nmunet@redhat.com>

Merge pull request #64352 from aclamk/aclamk-bs-remove-cache_blob

os/bluestore: Get rid of unused CACHE_BLOB_BL

Merge pull request #64219 from kamoltat/wip-ksirivad-fix-qa-rados-test

qa/workunits/rados/test.sh: add timeout mechanism and more info to workloads with parallel tests

Merge pull request #63642 from ifed01/wip-ifed-dynamic-vsel-params

os/bluestore: make vselector reserved* parameters applicable in run-time

Merge pull request #64667 from afreen23/fix-localization-load

mgr/dashboard: Fix issue with loading localization module

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #62790 from ceph/rm-ceph-common-postun

ceph.spec.in: don't rm ceph conf and logs in ceph-common postun

Merge pull request #64535 from yuvalif/wip-yuval-71979

rgw/test/notification: add more info when retry test fail

librbd/mirror: cleanup EnableRequest::handle_get_mirror_image()

In the EnableRequest state machine, clean up the handling of the async
request to fetch the mirror image, particularly when a non-primary image
is being created by the rbd-mirror daemon.

Signed-off-by: Ramana Raja <rraja@redhat.com>

Merge pull request #64402 from sam0044/sam0044-bug_72034

mgr/cephadm: updating maintenance health status in the serve loop

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #63825 from leomylonas/main

mgr/cephadm: handle possibly undefined template variable in haproxy.cfg.j2

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #63391 from Kushal-deb/fix-rgw-bootstrap

mgr/rgw: Improve error handling and add --resume flag to 'ceph rgw realm bootstrap'

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #64185 from kshtsk/wip-add-cephadm-file-path

qa/tasks/cephadm: allow to select from 'cephadm' and 'cephadm.py'

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #64635 from abitdrag/tracker_72134

krbd: "rbd device map" command should use msgr2 by default

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #62886 from Kushal-deb/fix-issue-2302464-ceph_orch_commands_to_report_the_exit_code

mgr/cephadm: Provide appropriate exit codes for orch operations

Reviewed-by: John Mulligan <jmulligan@redhat.com>