doc: elide superuser discussion from mount prereq

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc: clarify superuser note for ceph-fuse

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #57775 from rhcs-dashboard/add-urls

mgr/dashboard: adds links to the services

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #58143 from rhcs-dashboard/add-ec-profile-fields

mgr/dashboard: add fields to EC profile form

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #58581 from rkhudov/src-test-rgw-test_multi-update-documentation

src/test/rgw/test_multlti.md:update documentation for multi tests run

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #49974 from neesingh-rh/wip-58619

mds: fix session/client evict command.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #56193 from joscollin/wip-B64927-test_cephfs_mirror_blocklist-fail

cephfs_mirror, qa: fix mirror daemon doesn't restart when blocklisted or failed

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #57859 from vshankar/wip-debug-reclaim-noaddrs

mds: derr log and dump sessionmap when reclaim session uuid isn't found

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #58216 from hit1943/56288_fix

client: retry readdir if dir is invalidated

Reviewed-by: Venky Shankar <vshankar@redhat.com>

mgr/dashboard: add fields to EC profile form

Fixes: https://tracker.ceph.com/issues/66571
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

mgr/dashboard: add url links to services

Fixes: https://tracker.ceph.com/issues/66736
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #58167 from rhcs-dashboard/multi-cluster-cleanup

mgr/dashboard: Multi-cluster cleanup

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #58241 from afreen23/wip-nvmeof-subsystems

mgr/dashboard: Configure subsystems from dashboard

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #58357 from xxhdx1985126/wip-seastore-btree-interruptible-bug-fix

crimson/os/seastore/btree: interrupt transactions immediately when getting children

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #58467 from cyx1231st/wip-seastore-track-outstanding

crimson/os/seastore: track transactions/conflicts/outstanding periodically

Reviewed-by: Myoungwon Oh <myoungwon.oh@samsung.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #58250 from cyx1231st/wip-seastore-optimize-ool-padding

crimson/os/seastore: write ool extents without padding

Reviewed-by: Samuel Just <sjust@redhat.com>

src/test/rgw/test_multlti.md:update documentation for multi tests run

Signed-off-by: rkhudov <rhudov@gmail.com>

Merge pull request #58574 from zdover23/wip-doc-2024-07-13-cephfs-mount-using-kernel-driver

doc/cephfs: edit "is mount helper present"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #58451 from zdover23/wip-doc-2024-07-07-dev-encoding

doc/dev: edit "Principles for format change"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/dev: edit "Principles for format change"

Improve "Principles for format change" in doc/dev/encoding.rst. This
commit started as a response to Anthony D'Atri's suggestion here: https://github.com/ceph/ceph/pull/58299/files#r1656985564

Review of this section suggested to me that certain minor English usage
improvements would be of benefit. The numbered lists in this section
could still be made a bit clearer.

Co-authored-by: Anthony D'Atri <anthony.datri@gmail.com>
Signed-off-by: Zac Dover <zac.dover@proton.me>

doc/cephfs: edit "is mount helper present"

Edit the section called "Is mount helper present?", the title of which
prior to this commit was "Is mount helper is present?". Other small
disambiguating improvements have been made to the text in the section.
An unselectable prompt has been added before a command.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #58324 from kamoltat/wip-ksirivad-fix-63784

qa/standalone/mon: Fix mkfs  test & TEST_LOG failures

Merge pull request #58554 from ceph/wip-yuriw-release-process-main

docs: removed centos 8 and added squid to the build matrix

Reviewed-by: Casey Bodley <cbodley@redhat.com>

docs: removed centos 8 and added squid to the build matrix

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #58416 from cbodley/wip-cmake-zfs

cmake/os: remove WITH_ZFS option

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>

Merge pull request #58311 from rishabh-d-dave/fs-auth-nonalpha-test

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #58386 from YiteGu/dashboard-rename-api

mgr/dashboard: add cephfs rename REST API

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #58531 from rhcs-dashboard/fix-66901-main

mgr/dashboard: fix rgw configuration e2e test

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #57989 from rhcs-dashboard/multicluster-e2e

mgr/dashboard: multicluster management e2e tests

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #57712 from rhcs-dashboard/dashboard-nvmf-mtls-conf

mgr/dashboard: use secure_channel for grpc requests

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: baum <NOT@FOUND>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@redhat.com>

Merge pull request #58101 from rishabh-d-dave/async-clone-race

mgr/vol: handle case where clone index entry goes missing

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>

crimson/os/seastore: report outstanding ios for each shard

Detailed outstanding ios might reflect some issues, remove if it is no
longer a problem.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastar: report transaction iops/conflicts/outstanding

Also report details in shard 0.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: track shard io stats below transaction manager

Including background transactions.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

qa/cephfs: fix test_single_path_authorize_on_nonalphanumeric_fsname

This test deletes the CephFS already present on the cluster at the very
beginning and unmounts the first client beforehand. But it leaves the
second client mounted on this deleted CephFS that doesn't exist for the
rest of the test. And then at the very end of this test it attempts to
remount the second client (during tearDown()) which hangs and causes
test runner to crash.

Unmount the second client beforehand to prevent the bug and delete
mount_b object to avoid confusion for the readers in future about
whether or not 2nd mountpoint exists.

Fixes: https://tracker.ceph.com/issues/66077
Signed-off-by: Rishabh Dave <ridave@redhat.com>

mgr/vol: handle case where clone index entry goes missing

In `async_cloner.py`, clone index entry is fetched to get next clone job
that needs to be executed. It might happen that the clone job was
cancelled just when it was going to be picked for execution (IOW, when
it was about to move from pending state to in-progress state).

Currently, MGR hangs in such a case because exception `ObjectNotFound`
from CephFS Python bindings is raised and is left uncaught. To prevent
this issue catch the exception, log it and return None to tell
`get_job()` of `async_job.py` to look for next job in the queue.

Increase the scope of try-except in method `get_oldest_clone_entry()` of
`async_cloner.py` so that when exception `cephfs.Error` or any exception
under it is thrown by `self.fs.lstat()` is not left uncaught.

FS object is also passed to the method `list_one_entry_at_a_time()`, so
increasing scope of try-except is useful as it will not allow exceptions
raised in other calls to CephFS Python binding methods to be left
uncaught.

Fixes: https://tracker.ceph.com/issues/66560
Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #58491 from cbodley/wip-66880

rgw/datalog: fix LazyFIFO race fix

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #58530 from zdover23/wip-doc-2024-07-11-radosgw-s3-authentication

doc/radosgw: s/http/HTTP/

Merge pull request #58507 from idryomov/wip-66886

qa: account for rbd_trash object in krbd_data_pool.sh + related ceph{,adm} task fixes

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: N Balachandran <nibalach@redhat.com>

mgr/dashboard: Configure subsystems from dashboard

Fixes https://tracker.ceph.com/issues/66659

- adds subsytems tab
- adds subsystem listing view
- adds create subsystem modal
- adds delete subsystem
- adds unit tests

Signed-off-by: Afreen Misbah <afreen23.git@gmail.com>

Merge pull request #57492 from rishabh-d-dave/qa-fs-mds-fail-improve

qa/cephfs: improvements for "mds fail" and "fs fail"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #58225 from guojidan/fix-mem-leak

crimson/osd/osd_operation: fix dump_historic_slow_ops command works

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #58396 from NitzanMordhai/wip-nitzan-crimson-stat-pybind-test

suite/crimson: enable stats tests

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

mgr/dashboard: fix rgw configuration e2e test

Fixes: https://tracker.ceph.com/issues/66901
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard: multicluster management e2e tests

auth, edit, reconnect and disconnect another cluster

Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #58224 from Serrini/fix-66490-main

mgr/dashboard: add restful api for creating crush rule with type of 'erasure'

Reviewed-by: Nizamudeen A <nia@redhat.com>

doc/radosgw: s/http/HTTP/

s/http/HTTP/ in doc/radosgw/s3/authentication.rst, as suggested by
Anthony D'Atri in the location linked in the "Fixes:" line below.

Fixes: https://github.com/ceph/ceph/pull/58436#discussion_r1667088045
Signed-off-by: Zac Dover <zac.dover@proton.me>

crimson/os/seastore: track transactional read/write/flush

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: improve store stats

Calculate average values and remove verbose per-shard data.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #58382 from adk3798/cephadm-upgrade-test-start-points

qa/cephadm: change upgrade test start point to reef/squid

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #58440 from jmolmo/tls_4_exporter

exporter: Added https(TLSv13) support

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #58343 from skritik098/rgw-module-period-commit-retry-fix

mgr/rgw: Adding a retry config while calling zone_create()

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #58326 from adk3798/mgr-rgw-realm-token

mgr/rgw: fix setting rgw realm token in secondary site rgw spec

Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #58321 from adk3798/jinja2-cephadm-rhel9

cephadm: make cephadm compatible with jinja2 2.11.3

Reviewed-by: Adam King <adking@redhat.com>

mgr/dashboard: fall back to server_cert

if root_ca is not found

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 0a393ca9bde714524da369caaef1c097472f791c)

Merge pull request #58298 from thegreenbear/cephadm-ingress-fix-ipv6-binding

Ingress service: Fix HAProxy to listen on IPv4 and IPv6

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #57535 from rkachach/fix_issue_66095

Adding support for ceph mgmt-gateway

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #58048 from mohit84/issue_66268

mgr: the mojibake characters in the MGR OP tracker output

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #57808 from rzarzynski/wip-bug-66297

 mon: validate everybody understands MSR on set-require-min-compat-client

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

Merge pull request #57777 from rzarzynski/wip-bug-66285

osd: bump versions of decoders for upmap-primary

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Josh Salomon <josh.salomon@gmail.com>

Merge pull request #57005 from cfsnyder/wip-cfsnyder-63195

mgr/Mgr.cc: clear daemon health metrics instead of removing down/out osd from daemon state

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #58506 from aclamk/wip-aclamk-fix-deferred-tests

test/objectstore: Fix DeferredReplayTest

qa/tasks/cephadm: drop --cluster and sudo in create_rbd_pool()

The cluster (name) is already specified in the arguments passed to
_shell() and this command doesn't need privileges.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa/tasks/cephadm: don't wait for OSDs in create_rbd_pool()

This fails because teuthology.wait_until_osds_up() wants to use
adjust-ulimits wrapper which isn't available in "cephadm shell"
environment.  The whole thing is also redundant because cephadm task
is supposed to wait for OSDs to come up earlier, in ceph_osds().

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa/tasks: initialize rbd pool instead of just tagging it

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa: krbd_data_pool.sh: account for rbd_trash object

... created since commit 0bcb5cfce9a2 ("librbd/image: create rbd_trash
object during RBD pool initialization").

Fixes: https://tracker.ceph.com/issues/66886
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

test/objectstore: Fix DeferredReplayTest

Created ch = CollectionHandle was not reset, causing valgrind & other
problems.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>

Merge pull request #55196 from rzarzynski/wip-osd-ec-partial-reads

osd: EC Partial Stripe Reads (Retry of #23138 and #52746)

Reviewed-by: Samuel Just <sjust@redhat.com>

exporter: Added https(TLSv13) support

Ceph exporter can run now as http or https server

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>

mgr/dashboard: add restful api for creating crush rule with type of 'erasure'
Fixes: https://tracker.ceph.com/issues/66490
Signed-off-by: sunlan <sunlan@asiainfo.com>

rgw/datalog: fix LazyFIFO race fix

initialize `fifo_tmp` to prevent the race on member variable `fifo` that
was identified in 8fa844383f9c22e758f39ecdda74f70de054ad68

Fixes: https://tracker.ceph.com/issues/66880
Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #58484 from mcv21/patch-2

doc: Correct link to "Device management"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc: Correct link to "Device management"

The current link goes to `cephadm/services/rados/operations/devices` which is a 404, it instead needs to be `rados/operations/devices/`.

Signed-off-by: Matthew Vernon <mvernon@wikimedia.org>

mgr/cephadm: introducing nobody/nogroup constants

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding documentation for cephadm mgmt-gateway service

Signed-off-by: Redouane Kachach <rkachach@ibm.com>

mgr/cephadm: adding new cephadm service mgmt-gateway

adding mgmt-gateway, a new cephadm service based on nginx, to act as
the front-end and single entry point to the cluster. This gateway
offers unified access to all Ceph applications, including the
Ceph dashboard and monitoring tools (Prometheus, Grafana, ..),
while enhancing security and simplifying access management
through nginx.

Fixes: https://tracker.ceph.com/issues/66095
Signed-off-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #58466 from Svelar/test_cookie_cleanup

test/librbd: clean up unused TEST_COOKIE variable

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #55906 from pereman2/synthetic-seed

os/bluestore: store_test reproducible tests with seeds

Merge pull request #57890 from aclamk/fix-aclamk-66361

os/bluestore: Fix ExtentDecoderPartial::_consume_new_blob

Merge pull request #57369 from YiteGu/bluestore-offline-trim

tools/bluestore: Add command 'trim' to ceph-bluestore-tool

Merge pull request #54327 from pereman2/buffer-space-onode

os/bluestore: BufferSpace per Onode

mgr/cephadm: create OrchSecretNotFound exception type

This exception type is made to handle the formatting
of errors where we try to find a cert/key in the
cert/key store and can't

Signed-off-by: Adam King <adking@redhat.com>

mgr/dashboard: use secure_channel for grpc requests

Store the certificates to config-key stores and then later on used by
dashboard to set-up the secure_channel for grpc nvmeof requests

By storing the certificates we can ensure that the dashboard nvmeof apis
will be configurable even if the deployments are not cephadm based

Signed-off-by: Nizamudeen A <nia@redhat.com>
(cherry picked from commit 0a393ca9bde714524da369caaef1c097472f791c)

mgr/cephadm: fix nvmeof conf unit test

This needed changes to reflect changes made
to the conf to not have the certs stored at a
relative path and the addition of the root ca
cert

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 91883b9efc396779dfb61494e9aa140251f22b54)

mgr/cephadm: add get cert/key commands

In order to be able to grab certs/keys stored
in the new CertKeyStore class

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 2dd0ce91a53a898b8b3635f8943cc2d0e39b4ec1)

mgr/cephadm: integrate nvmeof spec certs/keys into cert/key store

Now that we're taking actual certs/keys in the spec, they
should go into the cert/key store with the others

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 3b1ec5851902c6b672db2ea81ae7eebcb1002101)

test/ librbd: clean up unused 'TEST_COOKIE'

Clean them up because
```
/home/jenkins-build/build/workspace/ceph-pull-requests-arm64/src/test/librbd/exclusive_lock/test_mock_PreAcquireRequest.cc:47:26: error: unused variable 'TEST_COOKIE' [-Werror,-Wunused-const-variable]
static const std::string TEST_COOKIE("auto 123");
                         ^
1 error generated.
```

Ref: https://jenkins.ceph.com/job/ceph-pull-requests-arm64/59242/consoleFull#-5120798632a811ea2-3e7b-466b-84b4-d13df7e35809

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

Merge pull request #58303 from rhcs-dashboard/nfs-export-subvol

mgr/dashboard: nfs export enhancement for CEPHFS

Reviewed-by: afreen23 <NOT@FOUND>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #58462 from rkhudov/src-test-objecstore-allocism-ops_replayer_fix_import_rados_buffer_fwd

src/test/objectstore/allocism/ops_replayer.cc: fix rados import

Reviewed-by: Svelar <NOT@FOUND>
Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>

Merge PR #53301 into main

* refs/pull/53301/head:
qa: adding test for preventing scrub when mds is inactive
mds: prevent scrub start for standby-replay MDS

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Milind Changire <mchangir@redhat.com>

python-common/service_spec: add root_ca_cert to nvmeof spec

Also improves the error messaging around when spec/key
attributes are missing when enable_auth is set to true

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit 9212914be65fe3adde2108f5a2cfd2587d17c0ff)

mgr/cephadm: allow passing client/server cert/key in nvmeof spec

Before this patch the client/server cert/key fields were
just filepaths that told the nvmeof gw daemon where to look
for the cert/key. There's not much reason why users would
care where in the nvmeof gw container the cert goes. It's more
useful to use these fields as a way to pass the certs/keys
to the daemon and then just hardcode where in the container
we'll place the certs/keys

Signed-off-by: Adam King <adking@redhat.com>
(cherry picked from commit e9fca39092348e6c08022341116875e831c175f0)

src/test/objectstore/allocism/ops_replayer.cc: fix rados import

src/test/objectstore/allocism/ops_replayer.cc: fix rados import

Signed-off-by: rkhudov <rkhudov@gmail.com>

Merge pull request #58454 from spdfnet/docs

doc/cephfs: fix typo

Merge pull request #57353 from Svelar/asan_shec

crush/builder: free 'crush_rule' before return

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

Merge pull request #55917 from Aequitosh/fix-ceph-crash-permissions

debian: recursively adjust permissions of /var/lib/ceph/crash

Reviewed-by: Pere Diaz Bou <pdiazbou@redhat.com>
Reviewed-by: Kefu Chai <tchaikov@gmail.com>

qa: adding test for preventing scrub when mds is inactive

Fixes: https://tracker.ceph.com/issues/62537
Signed-off-by: Neeraj Pratap Singh <neesingh@redhat.com>

Merge pull request #58423 from pereman2/scrapper-replayer-argparse

test/allocsim: ops scrapper arg parsing

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

doc/cephfs: fix typo

blanacer -> balancer

Signed-off-by: spdfnet <32593931+spdfnet@users.noreply.github.com>

Merge pull request #58114 from rhcs-dashboard/add-configuration-page-rgw

mgr/dashboard: add a new configuration page in side nav bar  Object > Configuration

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>