qa/suites/rados/thrash-old-clients: switch to cls/test_cls_rbd.sh workunit

For consistency with other upgrade jobs, don't invoke ceph_test_cls_rbd
directly.  While at it, drop a long outdated filter.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #60514 from rhcs-dashboard/fix-68733-main

mgr/dashboard: fix total objects/Avg object size in RGW Overview Page

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #60904 from cyx1231st/wip-seastore-hex-log

crimson/os/seastore: convert offset/length in logs to hex format

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #60755 from dmick/wip-release-container

Modify container/ software to support release containers and the promotion of prerelease containers

Merge pull request #60018 from smanjara/rgw-script-multiple-zg

rgw/multisite: test-rgw-multisite.sh can create multiple zonegroups

Merge pull request #60919 from zdover23/wip-doc-2024-12-03-rados-ops-health-checks-2

doc/rados: fix sentences in health-checks (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

container/{Containerfile,build.sh}: use a secret to pass secrets

Use podman's build --secret mechanism to pass credentials for
download.ceph.com

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile: allow failure from curl to stop build

curl doesn't actually fail unless -f is used; also, add
some sh -e flags.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile: purge .repo files with secrets before commit

ceph.repo had creds in it for download.ceph.com/prerelease.
Remove the .repo files we construct, since they're not necessary
once the container is built (no one should be dnf'ing anything
in the container).

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile fix boolean IS_RELEASE usage

Existing syntax was just wrong.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #59862 from jamiepryde/isa-xor-handle-misaligned-buffers

erasure-code/isa: Handle case where data and coding buffers are not 3…

Reviewed-by: Jose J Palacios-Perez <perezjos@uk.ibm.com>

Merge pull request #58473 from YiteGu/add-mon-scrub-lat

mon/scrub: add mon scrub latency

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #60131 from Shreyasapale/wip-fix-shreya-68009

osd/scrubber: Clean up unused function add_double

container/make-manifest-list.py: use LOCALMANIFEST throughout

Signed-off-by: Dan Mick <dan.mick@redhat.com>

containers/make-manifest.py: Add --promote

The promote function finds the latest image on the prerelease repo,
finds all its tags, and copies them all to the release repo.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile, build.sh: Add PRERELEASE_USERNAME/PASSWORD

Used for accessing download.ceph.com

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/build.sh:

- change repo path to prerelease
- remove image id arg for release push
- include 'v' in version name for release push

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/make-manifest-list.py

- don't print command failure in worker; let the caller print them
  if desired (allow silent failure)
- allow for empty tags list
- look for CEPH_SHA1.  GIT_COMMIT was the sha1 of the ceph-container.git
  commit
- change default paths to prerelease
- add --dry-run to avoid final push
- rename 'HOST' to 'CONTAINER_HOST'
- Use ARCH_SPECIFIC_HOST instead of CONTAINER_HOST (which is used by podman)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #60172 from smanjara/rgw-multisite-two-zonegroups

qa/multisite: enable two zonegroup yaml

Merge pull request #60591 from smanjara/wip-datalog-autotrim

qa/multisite: add extra checkpoints in datalog_autotrim testcase

Merge pull request #60918 from gbregman/main

python-common/ceph/cephadm: Change NVME image version to 1.4.

Merge pull request #60917 from zdover23/wip-doc-2024-12-03-rados-ops-health-checks

doc/rados: make sentences agree in health-checks.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: fix sentences in health-checks (2 of x)

Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.

This commit edits the second third of
doc/rados/operations/health-checks.rst.

Zac: cf. 000228

Signed-off-by: Zac Dover <zac.dover@proton.me>

python-common/ceph/cephadm: Change NVME image version to 1.4.
Fixes https://tracker.ceph.com/issues/69099

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

Merge pull request #60767 from rhcs-dashboard/smb-cluster

mgr/dashboard: list smb

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #60099 from MaxKellermann/json_spirit_thread

json_spirit: add missing dependency on Boost::thread

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/rados: make sentences agree in health-checks.rst

Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.

This commit edits the first third of
doc/rados/operations/health-checks.rst.

Zac: cf. 000228

Signed-off-by: Zac Dover <zac.dover@proton.me>

crimson/os/seastore: convert offset/length in seastore logs to hex format

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: convert offset/length in segment logs to hex format

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60151 from shreyanshjain7174/wip-skoduri-cloud-restore

rgw/restore: Fixed status codes and response headers
Reviewed-by: Soumya Koduri <skoduri@redhat.com>

Merge pull request #59449 from tth-trang/wip-65772-quincy

rgw/lc: Fix lifecycle not working while bucket versioning is suspended

Reviewed-by: Soumya Koduri <skoduri@redhat.com>

Merge pull request #60910 from clwluvw/putacl-log

rgw: relax RGWPutACLs_ObjStore::get_params read data log

Merge pull request #60254 from clwluvw/zonegroups-createbucket

rgw: initialize createparams zone_placement to avoid garbage value

Merge pull request #60589 from clwluvw/requested-gen-sync

rgw/multisite: revert "rgw/multisite: avoid redundant error repo entry logging

qa/multisite: enable two zonegroup yaml

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #60906 from zdover23/wip-doc-2024-12-02-cephadm-services-mon-capitalize-ceph

doc/cephadm: s/ceph/Ceph/ in mon.rst

rgw: relax RGWPutACLs_ObjStore::get_params read data log

the input can be logged with level 20 than zero.

Fixes: https://tracker.ceph.com/issues/69093
Signed-off-by: Seena Fallah <seenafallah@gmail.com>

Merge pull request #60412 from ShwetaBhosale1/fix_issue_68605_modify_structure_of_default_images

cephadm: Modify the structure of the default container images

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #60903 from nbalacha/wip-nbalacha-rbd-man

doc/rbd: Add mirror snapshot schedule information

Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/rbd: Add mirror snapshot schedule information

Signed-off-by: N Balachandran <nibalach@redhat.com>

mgr/dashboard: list smb
Tracker: https://tracker.ceph.com/issues/69080

Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge pull request #60862 from rhcs-dashboard/fix-multisite-e2e-failures

mgr/dashboard: fix multisite e2e failures

Reviewed-by: Afreen Misbah <afreen@ibm.com>

doc/cephadm: s/ceph/Ceph/ in mon.rst

s/ceph/Ceph/ in doc/cephadm/services/mon.rst where appropriate.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60759 from rhcs-dashboard/add-smb-endpoints

mgr/dashboard: add smb endpoints

Reviewed-by: Afreen Misbah <afreen@ibm.com>

crimson/os/seastore: minor cleanup

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: drop unused file

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60892 from zdover23/wip-doc-2024-11-30-glossary-dashboard-plugin

doc/glossary.rst: add "Dashboard Plugin"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/glossary.rst: add "Dashboard Plugin"

Add an entry below the (Mimic-era and therefore outdated but
nonetheless historically important) Dashboard Plugin key word in the
glosssary, which before now had never been added to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60882 from anthonyeleven/59466-followup

os/bluestore: Improve documentation introduced by #57722

Reviewed-by: Zac Dover <zac.dover@proton.me>

mgr/dashboard: add smb endpoints

Adds following SMB endpoints:
- cluster: list, get, create
- share: list, delete

Fixes: https://tracker.ceph.com/issues/69044
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #60560 from ivoalmeida/carbon-tree-component

mgr/dashboard: carbon tree component

Signed-off-by: nizamial09 <nia@redhat.com>

Merge pull request #60884 from zdover23/wip-doc-2024-11-29-radosgw-s3-common

doc/radosgw: update rgw_dns_name doc

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

mgr/dashboard: fix multisite e2e failures

Fixes: https://tracker.ceph.com/issues/69072
Signed-off-by: Naman Munet <namanmunet@li-ff83bccc-26af-11b2-a85c-a4b04bfb1003.ibm.com>

Merge pull request #60817 from cyx1231st/wip-seastore-improve-get_extents_if_live

crimson/os/seastore/transaction_manager: improve get_extents_if_live()

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

doc/radosgw: update rgw_dns_name doc

Update doc/radosgw/s3/commons.rst with the changes made by Jiffin Tony
Thottan in https://github.com/ceph/ceph/pull/54524 and the suggestions
made in that same PR by Anthony D'Atri.

Explain how to set rgw_dns_name to a domain name in order to configure
access to virtual hosted buckets.

Signed-off-by: Zac Dover <zac.dover@proton.me>

os/bluestore: Improve documentation introduced by #57722

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #60427 from mkogan1/wip-vstart-dev-err

vstart: log faulty physical disk device

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard: carbon tree component

Replaces the deprecated npm package '@circlon/angular-tree-component' by
Carbon Tree component.

Fixes: https://tracker.ceph.com/issues/68249
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>

crimson/os/seastore/transaction_manager: add log in get_extents_if_live()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: fix get_extents_if_live

It should not assume the paddr type from get_mappings is SEGMENT.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: adjust do_with for pin_list in get_extents_if_live

To make pin_list alive during parallel_for_each().

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: verify type in read_pin_by_type

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60654 from cyx1231st/wip-seastore-fine-grained-cache

crimson/os/seastore: support partial read for data extents

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore/transaction_manager: fully read the extent before rewritting it

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: convert transaction related paths with interruptor

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: split out transaction_interruptor.h

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: implement and use maybe_indirect_extent_t::get_bl()

Return bufferlist because the extent may be partially loaded under
indirection.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: delay setting bptr for all types of extent in case of read

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson/seastore: add test case for partial reads

Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore: disable seastore_max_data_allocation_size by default

Supposing that fine-grained-cache should address the read amplification
issue. By-default disable seastore_max_data_allocation_size with
fine-grained-cache since seastore_full_integrity_check is by-default
disabled.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: update lru size based on loaded extent size

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore: implement partial reads from cached_extent to object_data_handler

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore/cached_extent: introduce BufferSpace for partial reads

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore/cached_extent: minor adjustments to print

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: add missing final keywords

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: hide the complexity of extent indirection

Make it easier for TM::read_pin() users to consume extent without
worrying about the indirections.

This basically reverts 9cdcd06caadef6cdca64a974c6525ba300c1e289

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: fix to use direct key/length to get absent extent

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: adjust logs

Mostly convert length to the hex format.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: misc cleanups and asserts

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: clarify zero length is only possible for root

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cached_extent: clarify invarients in the cstors

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: cleanup, use the same method to create extents

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cached_extent: unify extent ptr creation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60857 from zdover23/wip-doc-2024-11-27-glossary-dc

doc/glossary: add "DC" (Data Center) to glossary

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/glossary: add "DC" (Data Center) to glossary

Add the initialism "DC" (Data Center) to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60860 from anthonyeleven/update-60846

doc/radosgw: Improve archive-sync-module.rst more

Merge pull request #60721 from anuradhagadge/fix_issue_68748_add_multiple_lable

cephadm: Add multiple labels in single command to host

Reviewed-by: Adam King <adking@redhat.com>

doc/radosgw: Improve archive-sync-module.rst more

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #60856 from ceph/uadk_version

build/uadk: fix uadk repo's commit id

build/uadk: fix uadk repo's commit id

Signed-off-by: Rongqi Sun <sunrongqi@huawei.com>

Merge pull request #60846 from anthonyeleven/improve-archive-sync

doc/radosgw: Improve archive-sync-module

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60801 from cbodley/wip-qa-rgw-verify-java-last

qa/rgw: the rgw/verify suite runs java tests last

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

doc/radosgw: Improve archive-sync-module.rst

Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #60747 from mohit84/crimson_device_class

crimson: Set device class during spawn of a crimson osd

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #60736 from guits/fix-stale-bot

.github/workflows: Do not exempt PRs with milestone

Merge pull request #59897 from avanthakkar/note-cephfs-earmark

doc: add pendingreleasenotes for cephfs subvolume earmarking feature

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>

Merge pull request #60768 from VallariAg/wip-prometheus-tested-limits

monitoring: Add alert NVMeoFTooManyNamespaces

Reviewed-by: Afreen Misbah <afreen@ibm.com>

mgr/dashboard: fix total objects/Avg object size in RGW Overview Page

Till now we are calculating the total number of objects and the average
object size in the RGW Overview Page using `ceph df` command's output.
As per the discussion with RGW team, this data is not correct as S3
objects in rgw can occupy more than one rados object. This PR tends to
make the overview page's info in sync with the RGW bucket page's info.

Fixes: https://tracker.ceph.com/issues/68733
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #60702 from ceph/wip-yuriw-release-17.2.8-main

doc: 17.2.8 Release Notes

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

doc/releases: fix some formatting for the 17.2.8 release notes

Signed-off-by: Laura Flores <lflores@ibm.com>
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>