mgr/dashboard: Object gateway sync status cards keeps loading when multisite is not configured

Fixes: https://tracker.ceph.com/issues/62665
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge PR #52409 into main

* refs/pull/52409/head:
mds: rework FSMap to avoid shared_ptrs

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #53100 from rhcs-dashboard/fix-rbd-mirror-label

mgr/dashboard: fix image columns naming

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53182 from rhcs-dashboard/fix-subvolume-rm-snapshots

mgr/dashboard: subvolume rm with snapshots

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53107 from ceph/wip-yuriw-release-16.2.14-main

doc: 16.2.14 Release Notes

Merge pull request #53211 from rhcs-dashboard/kill-rbd-mirror-leftovers

mgr/dashboard: cleanup rbd-mirror process in dashboard e2e

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Zack Cerza <zack@redhat.com>

Merge pull request #53208 from rhcs-dashboard/fs-volume-validator-fix

mgr/dashboard: fix cephfs create form validator

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: fix image columns naming

Fixes: https://tracker.ceph.com/issues/62551
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: subvolume rm with snapshots

Fixes: https://tracker.ceph.com/issues/62452
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: cleanup rbd-mirror process in dashboard e2e

Fixes: https://tracker.ceph.com/issues/61703
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53213 from zdover23/wip-doc-2023-08-30-cephadm-install-adding-hosts-2

doc/cephadm: edit "Adding Hosts" in install.rst

doc/cephadm: edit "Adding Hosts" in install.rst

Edit the English in the "Adding Hosts" section of
doc/cephadm/install.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #52655 from rzarzynski/wip-crimson-ertr-sassert-on-wrong-valuefunc

crimson/ertr: assert on invocability of func provided to safe_then()

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #53136 from petrutlucian94/rbd_tests

rbd-wnbd: wait for disks to become available

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #53133 from rhcs-dashboard/rgw-addr-fix

mgr/dashboard: fix rgw page issues when hostname not resolvable

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

doc: 16.2.14 Release Notes

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge PR #52686 into main

* refs/pull/52686/head:
PendingReleaseNotes: note about mandatory fs argument
doc/cephfs: add note about mandatory --fs argument to snap-schedule
qa: add test for mandatory fs argument to snap-schedule commands
mgr/snap-schedule: tweaks to keep mypy happy
mgr/snap_schedule: validate fs before execution

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #53193 from rhcs-dashboard/progress-bar-color-fix

mgr/dashboard: fix progress bar color visibility

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard: fix cephfs create form validator

dashboard was allowing to create a filesystem with / in its name but the
cli throws out error in doing so. And the created volume in dashboard
just gets ended up as a stale volume

Fixes: https://tracker.ceph.com/issues/62628
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix rgw page issues when hostname not resolvable

Fixes: https://tracker.ceph.com/issues/62396
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #51964 from idryomov/wip-doc-drop-journal-max-fetch-bytes

doc: drop mention of rbd_mirror_journal_max_fetch_bytes option

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

doc: drop mention of rbd_mirror_journal_max_fetch_bytes option

It was removed in commit 1ef12ea0d29f ("rbd-mirror: remove
rbd_mirror_journal_max_fetch_bytes option") in 2019.  Commit
32375cb789d7 ("doc: misc clarity and capitalization") added a "tip"
mentioning it in 2020 in an attempt to capture advice [1].

[1] https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/IHMGNFLWCD5E5R4W5S2BSSKEB5X3N4S4/

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

rbd-wnbd: wait for disks to become available

After a WNBD mapping is created, the driver informs Storport that
the bus changed, expecting it to rescan the bus and expose the disk.
This is an asynchronous process and it usually takes a matter of
milliseconds, up to a few seconds under significant load.

The fsx librbd test ocasionally fails to open the disk as it isn't
ready yet. Unlike the Python rbd-wnbd test, it doesn't perform
any polling.

For convenience, WNBD now includes a helper function called
WnbdPollDiskNumber. We're going to use it to wait for the new
disk attachments to become available.

Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

Merge pull request #53181 from joscollin/wip-stop-cephfs-mirror

stop.sh: kill cephfs-mirror, rbd-mirror

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #53192 from roysahar/nvmeof-bump-version-0.0.3

Update default nvmeof gw version to 0.0.3

Merge pull request #53171 from rhcs-dashboard/unauthorized-resp-headers

mgr/dashboard: set CORS header for unauthorized access

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

stop.sh: kill cephfs-mirror, rbd-mirror

kill cephfs-mirror/rbd-mirror daemon in vstart cluster stop,
instead of having multiple mirror daemons on restart.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #53198 from zdover23/wip-doc-2023-08-29-cephadm-install

doc/cephadm: add ssh note to install.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc: test

Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/cephadm: add ssh note to install.rst

Add a note instructing the reader to have ssh running before the
"cephadm bootstrap" command is run.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #52204 from xxhdx1985126/wip-61825

crimson/osd/object_context: consider clones found as long as they're in SnapSet::clones

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com

Merge pull request #53032 from xxhdx1985126/wip-crimson-rebuild-missing

osd/PGLog: return a ready future when found missing objects

Reviewed-by: Samuel Just <sjust@redhat.com>

mgr/dashboard: fix progress bar color visibility

Side effect of a recent change https://github.com/ceph/ceph/pull/52915/files#diff-747a9554e053df3b4d3fcc144bfb5c002654dc0fd5aa59b0ec18db56d85d5b28R29

Fixes: https://tracker.ceph.com/issues/62614
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #52633 from kamoltat/wip-ksirivad-autoscale-performance-improve

pybind/mgr/pg_autoscaler: Cut back osdmap.get_pools calls

Reviewed-by: Nitzan Mordechai <nmordech@redhat.com>

mgr/dashboard: allow CORS for unauthorized access

Fixes: https://tracker.ceph.com/issues/62612
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge PR #53144 into main

* refs/pull/53144/head:
doc: reformat so note is not part of code

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>

Merge PR #50502 into main

* refs/pull/50502/head:
mon/OSDMonitor: do not propose on error in prepare_update

Reviewed-by: Laura Flores <lflores@redhat.com>

Merge PR #52292 into main

* refs/pull/52292/head:
mgr: register OSDs in ms_handle_accept
msg: indicate ms_handle_authentication is fast

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #51979 from rhcs-dashboard/angular-14

mgr/dashboard: update to angular 14

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Update default nvmeof gw version to 0.0.3

Signed-off-by: Roy Sahar <royswi@gmail.com>

Merge pull request #53096 from zdover23/wip-doc-2023-08-24-start-os-recommendations-platforms

doc/start: edit os-recommendations.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #48052 from rzarzynski/wip-crimson-qa-backfill

qa/suites/crimson-rados: bring backfill testing

Reviewed-by: Samuel Just <sjust@redhat.com>

doc/start: edit os-recommendations.rst

Improve the grammar in one sentence of the "Platforms" section of
doc/start/os-recommendations.rst. Improving that grammar involved
splitting the sentence into two sentences, but that's life. Update:
Anthony substantially rewrote this, so credit for this should rightly
go to him.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge PR #52921 into main

* refs/pull/52921/head:
pybind/mgr/volumes: log mutex locks to help debug deadlocks

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #52940 into main

* refs/pull/52940/head:
qa: Wait for purge to complete in test_volume_info_pending_subvol_deletions

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #52005 from rhcs-dashboard/entries

mgr/dashboard: n/a entries behind primary snapshot mode

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53015 from ceph/dependabot-npm_and_yarn-src-pybind-mgr-dashboard-frontend-protobufjs-6.11.4

mgr/dashboard: bump protobufjs from 6.11.3 to 6.11.4 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53066 from rhcs-dashboard/fix-protect-option-rbd

mgr/dashboard: disable protect if layering is not enabled on the image

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53082 from Matan-B/wip-matanb-crimson-asock-cleanup

crimson/admin/admin_socket: remove path file if it exists

Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>

crimson/admin/admin_socket: remove path file if it exists

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

qa: bring backfill testing to crimson-rados

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #53031 from xxhdx1985126/wip-crimson-backfill-state

crimson/osd/pg: check if backfill_state exists when judging objects'

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #53163 from zdover23/wip-doc-2023-08-27-cephadm-services-mgr

doc/cephadm: edit sentence in mgr.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/cephadm: edit sentence in mgr.rst

Improve the grammar in the first sentence of
doc/cephadm/services/mgr.rst. This is a small change, but I could not
resist.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #52663 from ronen-fr/wip-rf-map-update

osd/scrub: reduce calls to on_scrub_schedule_input_change()

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #53161 from Svelar/doc_cephadm

doc/cephadm: update cephadm reef version

Reviewed-by: Zac Dover <zac.dover@proton.me>

doc/cephadm: update cephadm reef version

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

Merge pull request #52532 from guits/fix-tracker-58946

python-common: drive_selection: fix KeyError when osdspec_affinity is not set

PendingReleaseNotes: note about mandatory fs argument

Signed-off-by: Milind Changire <mchangir@redhat.com>

doc/cephfs: add note about mandatory --fs argument to snap-schedule

Signed-off-by: Milind Changire <mchangir@redhat.com>

qa: add test for mandatory fs argument to snap-schedule commands

Signed-off-by: Milind Changire <mchangir@redhat.com>

mgr/snap-schedule: tweaks to keep mypy happy

Signed-off-by: Milind Changire <mchangir@redhat.com>

mgr/snap_schedule: validate fs before execution

Stop command execution if there are more than one filesystem and --fs
argument is missing in command-line, i.e. do not use the first fs in the
fsmap if there are more than one filesystem.
This is to ensure that user doesn't mistakenly run the command against
the first fs by missing to specify the desired fs.

Fixes: https://tracker.ceph.com/issues/62218
Signed-off-by: Milind Changire <mchangir@redhat.com>

Merge pull request #53141 from rhcs-dashboard/fix-host-e2e

mgr/dashboard: remove unnecessary failing hosts e2e

Merge PR #52944 into main

* refs/pull/52944/head:
PendingReleaseNotes: add a note for `mds_session_metadata_threshold` mds config
test: add test to verify that a buggy client is blocklisted
mds: add perf counter to track number of sessions evicted due to metadata threshold being exceeded
mds: blocklist clients with "bloated" session metadata

Reviewed-by: Robin H. Johnson <robbat2@orbis-terrarum.net>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #53131 from rhcs-dashboard/subvolume-e2e-test

mgr/dashboard: cephfs subvolume management e2e tests

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

mgr/dashboard: n/a entries behind primary snapshot mode

Fixes: https://tracker.ceph.com/issues/62576
Signed-off-by: Pere Diaz Bou <pere-altea@hotmail.com>

mgr/dashboard: adapt jest unit tests to angular 14

Important change would be the introduction of `TypedFormControl` which
is now more stricter in typing the FormControl and FormGroups as well.
Right now the dashboard has many number of untypedforms which by default
is migrated to an `UntypedFormControl` class when I ran the angular
upgrade script

Fixes: https://tracker.ceph.com/issues/61641
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #52723 from lxbsz/wip-62245

libcephfs/deleg: Wait for the delegation to be timedout

Teuthology run: http://pulpito.front.sepia.ceph.com/mchangir-2023-08-09_06:59:15-fs-wip-mchangir-testing-20230808.041738-testing-default-smithi/

Reviewed-by: Milind Changire <mchangir@redhat.com>

Merge pull request #53074 from cbodley/wip-62504

qa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist

Reviewed-by: Ali Maredia <amaredia@redhat.com>

Merge pull request #53125 from batrick/volume-spec

doc: expand and consolidate mds placement

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc: reformat so note is not part of code

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: remove unnecessary failing hosts e2e

These host e2e test were failing, since we are already checking this on the Dashboard Cephadm e2e tests we can get rid of these ones.

Fixes: https://tracker.ceph.com/issues/62491
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

qa/rgw: add new POOL_APP_NOT_ENABLED failures to log-ignorelist

Fixes: https://tracker.ceph.com/issues/62504
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #53039 from AliMasarweh/wip-alimasa-persistant-q-enhance-fix

RGW | Bucket Notification: Fix the issue of miscounting the entries to remove

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

mgr/dashboard: cephfs subvolume management e2e tests

includes subvolume and subvolume groups e2es

Also taking care of renaming of Volume to File Systems in the remaining
actions like Edit and Remove

Fixes: https://tracker.ceph.com/issues/62564
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #53091 from rhcs-dashboard/fix-62453-main

mgr/dashboard: Create realm sets to default

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

doc: expand and consolidate mds placement

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #52575 into main

* refs/pull/52575/head:
ceph_fs.h: add separate owner_{u,g}id fields

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #52676 into main

* refs/pull/52676/head:
mds/Server: mark a cap acquisition throttle event in the request

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

mgr/dashboard: disable protect if layering is not enabled on the image

Fixes: https://tracker.ceph.com/issues/62498
Signed-off-by: avanthakkar <avanjohn@gmail.com>

mgr/dashboard: cleanup behave logics

Fixes: https://tracker.ceph.com/issues/62564
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #52829 from rhcs-dashboard/cephfs-e2e-test

mgr/dashboard: add e2e tests for cephfs management

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

mgr/dashboard: Create realm sets to default

In Multisite page, When we create a realm the realm sets to default even if some other realm is already default and default checkbox in unchecked as well while creating.

Fixes: https://tracker.ceph.com/issues/62453
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #52895 from cloudbehl/minor-improvements

mgr/dashboard: Added some minor usability improvements

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #52896 from cyx1231st/wip-seastar-multi-core-msgr-UT

crimson/net: consolidate messenger implementations and enable multi-shard UTs

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #53083 from Matan-B/wip-matanb-crimson-centos9-supp

crimson/osd/lsan_suppressions: add MallocExtension::Register

Reviewed-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge pull request #53117 from batrick/doc-name-fix

doc: correct option name

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #53104 from ceph/revert-49428-maintain_prefix_itr

Revert "osd/SnapMapper: Maintain the prefix_itr between calls to avoid search…"

Reviewed-by: Gabriel BenHanokh <gbenhano@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

doc: correct option name

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Revert "osd/SnapMapper: Maintain the prefix_itr between calls to avoid search…"

Signed-off-by: Radosław Zarzyński <rzarzyns@redhat.com>

Merge PR #53053 into main

* refs/pull/53053/head:
libcephsqlite: fill 0s in unread portion of buffer

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #49428 from benhanokh/maintain_prefix_itr

osd/SnapMapper: Maintain the prefix_itr between calls to avoid search…

osd/SnapMapper:
Maintain the prefix_itr between calls to SnapMapper::get_next_objects_to_trim() to prevent searching depleted prefixes.
We got 8 distinct hash prefixes used for searching objects owned by a given PG.
On each call to SnapMapper::get_next_objects_to_trim() we start from the first prefix even after all objects mapped to it were depleted.
This means that we will be searching for 1 non-existing prefix after the first prefix was depleted, 2 after the first two prefixes were depleted... and so on until we will search 7 non-existing prefixes after the first 7 prefixes were depleted.

This is a performance improvement PR only!
It maintains the existing behavior and does not try to fix/change any of the TRIM logic.
I added an extra step after the last object is trimmed doing a full scan of the DB and only if no object was found it will return ENOENT.
This should make the new code no-worse than existing code which returns ENOENT after a full scan found no object.
It should not impact performance in real life snaps as it should only happen once per-snap.

added snap-mapper tests to rados-test-suite
disabled osd_debug_trim_objects when running (SnapMapperTest, prefix_itr) to prevent asserts(as this code does illegal inserts into DELETED snaps)
Code beautifing

Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>

Merge pull request #53076 from zdover23/wip-doc-2023-08-22-start-os-recommendations-chart

doc/start: refactor ABC test chart

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge PR #53077 into main

* refs/pull/53077/head:
qa: add `POOL_APP_NOT_ENABLED` to ignorelist for cephfs tests

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #53051 from xxhdx1985126/wip-crimson-omap-list

crimson/os/seastore/omap_manager: fix the entry leak issue in BtreeOMapManager::omap_list()

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson: increase smp settings for test_socket and test_messenger

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson/test_messenger: add delay to suite markdown

markdown() needs delay to be propagated to the primary core.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson/test_messenger: enable multi-core messenger in FailoverSuite

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>