doc/rados: edit control.rst (1 of x)

Edit doc/rados/operations/control.rst.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #52109 from idryomov/wip-61616

librbd: avoid object map corruption in snapshots taken under I/O

Reviewed-by: Mykola Golub <mgolub@suse.com>

qa/workunits/rbd: make continuous export-diff test actually work

The current version is pretty useless:

- "rbd bench" writes the same byte (0xff) over and over again, so
  almost all checksumming is in vain
- snapshots are taken in a steady state (i.e. not under I/O), so no
  race conditions can get exposed
- even with these caveats, it's not wired up into the suite

Redo this workunit to be a reliable reproducer for the issue fixed
in the previous commit and wire it up for both krbd and rbd-nbd.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #51827 from liu-chunmei/get_assert

crimson:  fix RefCountedObject::_get() assert

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #52104 from anthonyeleven/anthonyeleven/fixfft

doc/rados/operations: Improve formatting in crush-map.rst

doc/rados/operations: Improve formatting in crush-map.rst

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge PR #51500 into main

* refs/pull/51500/head:
test/libcephfs: add test case for revoking caps
client: issue a cap release immediately if no cap exists
mds: add the revoking caps back to _revokes list
mds: move confirm_receipt() to Capability.cc

Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #51655 into main

* refs/pull/51655/head:
mds: display sane hex value (0x0) for empty feature bit

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #51681 from cbodley/wip-61349

librados: aio operate functions can set mtimes

Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge pull request #51737 from cbodley/wip-rocksdb-44

bump rocksdb submodule to latest ceph-reef-v7.9.2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #51669 from ronen-fr/wip-rf-max-scrub-3

osd/scrub: increasing max_osd_scrubs to 3

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #48209 from kamoltat/wip-ksirivad-fix-tracker-57570

osd/OSDMap: Check for uneven weights & != 2 buckets post stretch mode

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

mds: display sane hex value (0x0) for empty feature bit

Print a valid hex (0x0) during empty feature bit, so that the clients could recognize it.
When the _vec size becomes 0, print() function creates an invalid hex (0x) and 'perf stats'
crashes with the below error:
"
File "/opt/ceph/src/pybind/mgr/stats/fs/perf_stats.py", line 177, in notify_cmd
metric_features = int(metadata[CLIENT_METADATA_KEY]["metric_spec"]["metric_flags"]["feature_bits"], 16)
ValueError: invalid literal for int() with base 16: '0x'
"
This patch creates  a valid hex (0x0), when _vec size is 0.

Fixes: https://tracker.ceph.com/issues/59551
Signed-off-by: Jos Collin <jcollin@redhat.com>

librbd: stop passing IOContext to image dispatch write methods

This is a major footgun since any value passed e.g. at the API layer
may be stale by the time we get to object dispatch.  All callers are
passing the IOContext returned by get_data_io_context() for their
ImageCtx anyway, highlighting that the parameter is fictitious.

Only the read method can meaningfully take IOContext.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

librbd: use an up-to-date snap context when owning the exclusive lock

By effectively moving capturing of the snap context to the API layer,
commit 1d0a3b17f590 ("librbd: pass IOContext to image-extent IO
dispatch methods") introduced a nasty regression.  The snap context can
be captured only after exclusive lock is safely held for the duration
of dealing with the image request and even then must be refreshed if
a snapshot creation request is accepted from a peer.  This is needed to
ensure correctness of the object map in general and fast-diff states in
particular (OBJECT_EXISTS vs OBJECT_EXISTS_CLEAN) and object deltas
computed based off of them.  Otherwise the object map that is forked
for the snapshot isn't guaranteed to accurately reflect the contents of
the snapshot when the snapshot is taken under I/O (as in disabling the
object map may lead to different results being returned for reads).

The regression affects mainly differential backup and snapshot-based
mirroring use cases with object-map and/or fast-diff enabled: since
some object deltas may be incomplete, the destination image may get
corrupted.

This commit represents a reasonable minimal fix: IOContext passed
through to ImageDispatch is effected only for reads and just gets
ignored for writes.  The next commit cleans up further by undoing the
passing of IOContext through the image dispatch layers for writes.

Fixes: https://tracker.ceph.com/issues/61616
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #52105 from gsantella/patch-update-typo-on-dashboard

doc/mgr/dashboard: fix a typo

doc/mgr/dashboard: fix a typo

Signed-off-by: Guido Santella <gsantella@southhills.edu>

Merge pull request #52100 from zdover23/wip-doc-2023-06-17-rados-operations-crush-map-firefly-tunables

doc/rados: edit firefly tunables section

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>

doc/rados: edit firefly tunables section

Edit the firefly tunables section in
/doc/rados/operations/crush-map.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #52092 from zdover23/wip-doc-2023-06-16-rados-operations-crush-map-4-of-x

doc/rados: edit crush-map.rst (4 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: edit crush-map.rst (4 of x)

Edit doc/rados/operations/crush-map.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #50874 from rishabh-d-dave/mds-log-variable

mds: use variable g_ceph_context directly in MDSAuthCaps

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #52095 from rhcs-dashboard/jsonschema-fix

mgr/dashboard: fix the jsonschema issue in install-deps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #51549 from vedanshbhartia/coverity_1517658

rgw: modify RGWRados::cls_bucket_list_ordered to store dirent key before move

Merge pull request #51719 from vedanshbhartia/coverity_1510555

rgw: Use a unique_ptr when creating a D3nCacheAioWriteRequest instance

Merge pull request #52076 from rhcs-dashboard/cephfs-no-mds-clients

mgr/dashboard: empty list of fs clients with no mds

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #52089 from samarahu/d4n-cmake-fix

RGW: Place D4N files under cmake conditional

mds: use variable g_ceph_context directly in MDSAuthCaps

Variable g_ceph_context is global, therefore use it directly instead of
passing it as a parameter to method.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #52010 from pkalever/fix-ImageMap

rbd-mirror: don't refer to (remote) mirror_uuid as peer_uuid

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #51132 from lxbsz/wip-59349

qa: wait for 100 seconds to make sure the quota to be enforced

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

mgr/dashboard: fix the jsonschema issue in install-deps

Fixes: https://tracker.ceph.com/issues/61690
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #52067 from zdover23/wip-doc-2023-06-15-rados-operations-crush-map-3-of-x

doc/rados: edit crush-map.rst (3 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: edit crush-map.rst (3 of x)

Edit doc/rados/operations/crush-map.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

mgr/dashboard: empty list of fs clients with no mds

Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>
Fixes: https://tracker.ceph.com/issues/61708

rbd-mirror: don't refer to (remote) mirror_uuid as peer_uuid

peer_uuid and mirror_uuid are different identifiers, we should not use
peer_uuid naming for remote mirror_uuid.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>

RGW: Place D4N files under cmake conditional

Signed-off-by: Samarah <samarah.uriarte@ibm.com>

Merge pull request #52064 from ajarr/wip-61688

mgr/rbd_support: add user-friendly stderr message when module is not ready

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #51962 from vedanshbhartia/coverity_1511072

rgw: pass boost intrusive_ptr to DataLogBackends flat_map

Merge pull request #51676 from vedanshbhartia/coverity_move

rgw: Do not forward functor objects in rgw_iam_policy.h

Merge pull request #51965 from vedanshbhartia/coverity_1510687

rgw: Fix use after move in cls_fifo_legacy.cc

Merge pull request #50330 from mohamedawnallah/fix-RGWEnv.get()-nullptr-default-value-to-the-default-returned-value

src/rgw/rgw_env.cc: Fix potential null pointer dereference in RGWEnv::get()

mgr/rbd_support: log number of images

... that have one snapshot request pending when the
mirror_snapshot_schedule handler is shutting down.

Signed-off-by: Ramana Raja <rraja@redhat.com>

Merge pull request #52058 from guits/tracker_61673

ceph-volume: set lvm membership for mpath type devices

Merge PR #52029 into main

* refs/pull/52029/head:
debian: install cephfs-mirror systemd unit files and man page

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #52039 from zdover23/wip-doc-2023-06-14-operations-crush-map-2-of-x

doc/rados: edit crush-map.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

debian: install cephfs-mirror systemd unit files and man page

Fixes: https://tracker.ceph.com/issues/59682
Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge PR #50862 into main

* refs/pull/50862/head:
mon/MDSMonitor: batch last_metadata update with pending
mon/MDSMonitor: add debugging for update_metadata

Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc/rados: edit crush-map.rst (2 of x)

Edit doc/rados/operations/crush-map.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #52040 from zdover23/wip-doc-2023-06-14-operations-crush-map-ordered-list-fix

doc/rados: fix list in crush-map.rst

mgr/rbd_support: add user-friendly stderr message

... when the rbd_support module is not ready.

Fixes: https://tracker.ceph.com/issues/61688
Signed-off-by: Ramana Raja <rraja@redhat.com>

Merge pull request #51946 from phlogistonjohn/jjm-cephadm-write-new

cephadm: add write_new function for robust file writes

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #51924 from adk3798/limit-policy-log-level

python-common/drive_selection: lower log level of limit policy message

Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #51861 from adk3798/spec-no-spec-validate

python-common/drive_group: handle fields outside of 'spec' even when 'spec' is provided

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #51267 from phlogistonjohn/jjm-cephadm-versioning

Support cephadm "binary" versioning

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #51832 from rkachach/fix_issue_61499

cephadm: adding support for ipv6 when deploying ceph-exporter

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #51817 from rkachach/fix_issue_61494

mgr/cephadm: validate host label before removing

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #51070 from adk3798/reconfig-firewall-ports

cephadm: allow ports to be opened in firewall during adoption, reconfig, redeploy

Reviewed-by: Michael Fritch <mfritch@suse.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

ceph-volume: set lvm membership for mpath type devices

If any lvs is present, self.lvs should be set accordingly for mpath
type devices as well.
Otherwise, when re-running a command like following:

```
ceph-volume lvm batch --bluestore --no-auto  /dev/mapper/mpatha /dev/mapper/mpathb
```

fails with error : "Device /dev/mapper/mpatha has a filesystem."

where the expected behavior is:

```
--> passed data devices: 2 physical, 0 LVM
--> relative data size: 1.0
--> All data devices are unavailable

Total OSDs: 0

  Type            Path                                                    LV Size         % of device
```

Fixes: https://tracker.ceph.com/issues/61673
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>

Merge pull request #52044 from vedanshbhartia/coverity_fix_tags

rgw: user correct tags for coverity annotations and add the SUPPRESS keyword

rgw: user correct tags for coverity annotations and add the SUPPRESS
keyword

Signed-off-by: Vedansh Bhartia <vedanshbhartia@gmail.com>

Merge pull request #52012 from cbodley/wip-61428

qa: suppress SyscallParam error during startup on jammy

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ali Maredia <amaredia@redhat.com>

doc/rados: fix list in crush-map.rst

Correct the numbering and formatting of an ordered list in
doc/rados/operations/crush-map.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #52014 from cbodley/wip-test-cls-refcount-rename

test/cls: rename cls_refcount tests

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #52009 from zdover23/wip-doc-2023-06-12-rados-operations-placement-groups-math-markup

doc/rados: add math markup to placement-groups.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #52011 from samarahu/d4n-warning-cleanup

RGW: Cleanup D4N unit test warning

Merge pull request #52028 from zdover23/wip-doc-2023-06-13-dev-deploying-dev-cluster-link

doc/dev: fix "deploying dev cluster" link

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #51989 from samarahu/remove-trace-comment

RGW: Replace trace with d4n in yaml long desc

Merge pull request #52018 from zdover23/wip-doc-2023-06-13-operations-crush-map-1-of-x

doc/rados: edit crush-map.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/dev: fix "deploying dev cluster" link

Fix a link to "Deploying a Development Cluster" that had been broken
since the transition from master to main.

This is the first part of an initiative to improve the vstart
documentation.

Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/rados: edit crush-map.rst (1 of x)

Edit doc/rados/operations/crush-map.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #51966 from rhcs-dashboard/grafana-url-deprecated

mgr/dashboard: change deprecated grafana URL in daemon logs

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #51953 from rhcs-dashboard/dashboard-status-popover

mgr/dashboard: add popover to cluster status card

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge pull request #51945 from rzarzynski/wip-bug-61504

crimson/osd: fix Notify life-time mismanagement in Watch::notify_ack

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #51973 from Matan-B/wip-matan-lsan-supp

qa/lsan.supp: update heap_profiler suppression and ASAN_OPTIONS

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #51703 from lxbsz/wip-59683

xfstests_dev: install extra packages from powertools repo for xfsprogs

Merge pull request #50728 from lxbsz/wip-59195

qa: switch to use the merge fragment for fscrypt

qa: suppress SyscallParam error during startup on jammy

Fixes: https://tracker.ceph.com/issues/61428
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #51972 from vedanshbhartia/coverity_resource_leak

rgw: Add coverity annotations for resource leak false positives

test/cls: rename cls_refcount tests

test cases were copy/pasted from test_cls_rgw.cc without renaming

Signed-off-by: Casey Bodley <cbodley@redhat.com>

RGW: Cleanup D4N unit test warning

Signed-off-by: Samarah <samarah.uriarte@ibm.com>

doc/rados: add math markup to placement-groups.rst

Add some math markup to some of the mathematical expressions in
doc/rados/operations/placement-groups.rst.

This change has advantages as well as disadvantages. On the one hand,
the mathematical expressions look nicer and are easier to read.  On the
other hand, the rendered mathematical expressions are not searchable as
a part of the text, which makes CTRL+F searches useless in finding them.

This PR is offered as an RFC on the matter of whether mathematical
expressions should be rendered with the :math: extension.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #51864 from yuvalif/wip-yuval-fix-61540

rgw/kafka: make sure that destroy is called after connection is removed

xfstests_dev: install extra packages from powertools repo for xfsprogs

Centos Stream 8 has removed the 'device-mapper-devel', 'libedit-devel'
and 'userspace-rcu-devel' packages from the mirrors and we need to
install it from powertools repo.

Fixes: https://tracker.ceph.com/issues/59683
Signed-off-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #51994 from zdover23/wip-doc-2023-06-11-rados-operations-placement-groups-3-of-x

doc/rados: edit placement-groups.rst (3 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: edit placement-groups.rst (3 of x)

Edit doc/rados/operations/placement-groups.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #51366 from SMIL-Infra/fix-test-build

rgw: fix tests build w/o AMQP

Merge pull request #51468 from vedanshbhartia/coverity_time

rgw: Add coverity annotations to ignore warnings about 32 bit time

Merge pull request #51992 from zdover23/wip-doc-2023-06-10-rados-operations-change-mon-elections

doc/rados: edit change-mon-elections

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #51991 from zdover23/wip-doc-2023-06-09-rados-operations-placement-groups-2-of-x

doc/rados: edit placement-groups.rst (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: edit placement-groups.rst (2 of x)

Edit doc/rados/operations/placement-groups.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/rados: edit change-mon-elections

Edit doc/rados/operations/change-mon-elections.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #51988 from ljflores/wip-release-checklist

doc/dev: update release checklist

crimson/osd: suppress the assertion in Watch::notify_ack

Replaced the assert with an error log entry but ultimately
this commit should be reverted.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: fix Notify life-time mismanagement in Watch::notify_ack

Fixes: https://tracker.ceph.com/issues/61504
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #49316 from BryceCao/wip-add-initialization-for-RGWGC-process-call

rgw : fix add initialization for RGWGC::process()

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

Merge pull request #51715 from awojno-bloomberg/replication-race-condition-fix

RGW: Pass and check destination zone trace when fetching object

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>

RGW: Replace trace with d4n in yaml long desc

Signed-off-by: Samarah <samarah.uriarte@ibm.com>

doc/dev: update release checklist

Signed-off-by: Laura Flores <lflores@redhat.com>

Merge pull request #51975 from zdover23/wip-doc-2023-06-09-rados-operations-placement-groups-1-of-x

doc/rados: edit placement-groups.rst (1 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: edit placement-groups.rst (1 of x)

Edit doc/rados/operations/placement-groups.rst.

https://tracker.ceph.com/issues/58485

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Co-authored-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>