mgr/dashboard: Show correct token expiration date in Manage Clusters
page

Currently wrong date is being displayed if we hover of the Token Expires
filed in the Clusters list table in Manage Clusters page. This PR
intends to fix this issue.

Fixes: https://tracker.ceph.com/issues/69172
Signed-off-by: Aashish Sharma <aasharma@redhat.com>

Merge pull request #60929 from dmick/wip-release-docs

doc/dev/release-process.rst: New container build/release process

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #60947 from idryomov/wip-67759

qa/suites/upgrade/*-x: skip TestClsRbd.group_snap_{set,remove}

Reviewed-by: N Balachandran <nibalach@redhat.com>
Reviewed-by: Laura Flores <lflores@ibm.com>

Merge pull request #60966 from gbregman/main

mgr/cephadm/nvmeof: Add encryption key file for nvmeof encryption

Merge pull request #60948 from idryomov/wip-68588

qa/suites/rados/thrash-old-clients: skip TestClsRbd.group_snap_{set,remove}

Reviewed-by: N Balachandran <nibalach@redhat.com>

mgr/cephadm/nvmeof: Add key encryption support to nvmeof configuration

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

Merge pull request #60926 from ktdreyer/container-labeler

labeler: add container labelers

Merge pull request #60946 from nbalacha/wip-nbalacha-69112

cls/rbd: fix mirror_image_map_list object

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #60913 from myoungwon/wip-fix-null-pointer-crimson

crimson/osd/ops_executer: prevent access to null pointers caused by gcc bug

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #60944 from dalgaaf/dalgaaf-fix-doc-ec

doc: fix EC overhead table m-numbering

Merge pull request #60954 from zdover23/wip-doc-2024-12-04-dev-development-workflow-backporter-manual

doc/dev: update link to backporter manual

doc/dev: update link to backporter manual

Update the link to the backporter manual, in
doc/dev/development-workflow.rst. Laura Flores requested this change on
04 Dec 2024.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60924 from ktdreyer/containerfile-date-utc

container/build.sh: use UTC dates for tags

Merge pull request #60930 from zdover23/wip-doc-2024-12-03-rados-ops-health-checks-3

doc/rados: fix sentences in health-checks (3 of x)

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>

qa/suites/upgrade/*-x: skip TestClsRbd.group_snap_{set,remove}

These tests assert on the number of keys on the group header object,
which changed with commits ec6224db3e55 ("cls/rbd: save group snapshot
creation order in a new key") and 3662bc7b3dc3 ("cls/rbd: save max
group snap order").

Fixes: https://tracker.ceph.com/issues/67759
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

cls/rbd: fix mirror_image_map_list object

The rbd-mirror image map is stored in the rbd_mirror_leader object.
Fixed mirror_image_map_list() to operate on the correct object.

Fixes: https://tracker.ceph.com/issues/69112
Signed-off-by: N Balachandran <nibalach@redhat.com>

qa/suites/rados/thrash-old-clients: skip TestClsRbd.group_snap_{set,remove}

These tests assert on the number of keys on the group header object,
which changed with commits ec6224db3e55 ("cls/rbd: save group snapshot
creation order in a new key") and 3662bc7b3dc3 ("cls/rbd: save max
group snap order").

Fixes: https://tracker.ceph.com/issues/68588
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

qa/suites/rados/thrash-old-clients: switch to cls/test_cls_rbd.sh workunit

For consistency with other upgrade jobs, don't invoke ceph_test_cls_rbd
directly.  While at it, drop a long outdated filter.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #60514 from rhcs-dashboard/fix-68733-main

mgr/dashboard: fix total objects/Avg object size in RGW Overview Page

Reviewed-by: Afreen Misbah <afreen@ibm.com>

doc/dev/release-process.rst: New container build/release process

Signed-off-by: Dan Mick <dan.mick@redhat.com>

crimson/osd/ops_executer: prevent access to null pointers caused by gcc bug

This commit assigns the parameter eariler to work around a coroutine related gcc bug
that causes a null pointer access.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

Merge pull request #60904 from cyx1231st/wip-seastore-hex-log

crimson/os/seastore: convert offset/length in logs to hex format

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #60755 from dmick/wip-release-container

Modify container/ software to support release containers and the promotion of prerelease containers

Merge pull request #60018 from smanjara/rgw-script-multiple-zg

rgw/multisite: test-rgw-multisite.sh can create multiple zonegroups

Merge pull request #60919 from zdover23/wip-doc-2024-12-03-rados-ops-health-checks-2

doc/rados: fix sentences in health-checks (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: fix sentences in health-checks (3 of x)

Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.

This commit edits the second third of
doc/rados/operations/health-checks.rst.

Note to (I hope soon) future Zac: There are a a couple of places near
the end of this file where the sentences are ungrammatical. Update these
in a separate PR (in isolation, so that the grammar and technical
accuracy of these sentences can be the primary focus of the reviewers).

Zac: cf. 000228

Signed-off-by: Zac Dover <zac.dover@proton.me>

container/{Containerfile,build.sh}: use a secret to pass secrets

Use podman's build --secret mechanism to pass credentials for
download.ceph.com

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile: allow failure from curl to stop build

curl doesn't actually fail unless -f is used; also, add
some sh -e flags.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile: purge .repo files with secrets before commit

ceph.repo had creds in it for download.ceph.com/prerelease.
Remove the .repo files we construct, since they're not necessary
once the container is built (no one should be dnf'ing anything
in the container).

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile fix boolean IS_RELEASE usage

Existing syntax was just wrong.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #59862 from jamiepryde/isa-xor-handle-misaligned-buffers

erasure-code/isa: Handle case where data and coding buffers are not 3…

Reviewed-by: Jose J Palacios-Perez <perezjos@uk.ibm.com>

Merge pull request #58473 from YiteGu/add-mon-scrub-lat

mon/scrub: add mon scrub latency

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #60131 from Shreyasapale/wip-fix-shreya-68009

osd/scrubber: Clean up unused function add_double

container/make-manifest-list.py: use LOCALMANIFEST throughout

Signed-off-by: Dan Mick <dan.mick@redhat.com>

containers/make-manifest.py: Add --promote

The promote function finds the latest image on the prerelease repo,
finds all its tags, and copies them all to the release repo.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile, build.sh: Add PRERELEASE_USERNAME/PASSWORD

Used for accessing download.ceph.com

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/build.sh:

- change repo path to prerelease
- remove image id arg for release push
- include 'v' in version name for release push

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/make-manifest-list.py

- don't print command failure in worker; let the caller print them
  if desired (allow silent failure)
- allow for empty tags list
- look for CEPH_SHA1.  GIT_COMMIT was the sha1 of the ceph-container.git
  commit
- change default paths to prerelease
- add --dry-run to avoid final push
- rename 'HOST' to 'CONTAINER_HOST'
- Use ARCH_SPECIFIC_HOST instead of CONTAINER_HOST (which is used by podman)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #60172 from smanjara/rgw-multisite-two-zonegroups

qa/multisite: enable two zonegroup yaml

Merge pull request #60591 from smanjara/wip-datalog-autotrim

qa/multisite: add extra checkpoints in datalog_autotrim testcase

labeler: add container labelers

Signed-off-by: Ken Dreyer <kdreyer@ibm.com>

container/build.sh: use UTC dates for tags

Do not rely on the builder host's local timezone setting.

Signed-off-by: Ken Dreyer <kdreyer@ibm.com>

doc: fix EC overhead table m-numbering

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #60918 from gbregman/main

python-common/ceph/cephadm: Change NVME image version to 1.4.

Merge pull request #60917 from zdover23/wip-doc-2024-12-03-rados-ops-health-checks

doc/rados: make sentences agree in health-checks.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: fix sentences in health-checks (2 of x)

Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.

This commit edits the second third of
doc/rados/operations/health-checks.rst.

Zac: cf. 000228

Signed-off-by: Zac Dover <zac.dover@proton.me>

python-common/ceph/cephadm: Change NVME image version to 1.4.
Fixes https://tracker.ceph.com/issues/69099

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

Merge pull request #60767 from rhcs-dashboard/smb-cluster

mgr/dashboard: list smb

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #60099 from MaxKellermann/json_spirit_thread

json_spirit: add missing dependency on Boost::thread

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/rados: make sentences agree in health-checks.rst

Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.

This commit edits the first third of
doc/rados/operations/health-checks.rst.

Zac: cf. 000228

Signed-off-by: Zac Dover <zac.dover@proton.me>

crimson/os/seastore: convert offset/length in seastore logs to hex format

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: convert offset/length in segment logs to hex format

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60151 from shreyanshjain7174/wip-skoduri-cloud-restore

rgw/restore: Fixed status codes and response headers
Reviewed-by: Soumya Koduri <skoduri@redhat.com>

Merge pull request #59449 from tth-trang/wip-65772-quincy

rgw/lc: Fix lifecycle not working while bucket versioning is suspended

Reviewed-by: Soumya Koduri <skoduri@redhat.com>

Merge pull request #60910 from clwluvw/putacl-log

rgw: relax RGWPutACLs_ObjStore::get_params read data log

Merge pull request #60254 from clwluvw/zonegroups-createbucket

rgw: initialize createparams zone_placement to avoid garbage value

Merge pull request #60589 from clwluvw/requested-gen-sync

rgw/multisite: revert "rgw/multisite: avoid redundant error repo entry logging

qa/multisite: enable two zonegroup yaml

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #60906 from zdover23/wip-doc-2024-12-02-cephadm-services-mon-capitalize-ceph

doc/cephadm: s/ceph/Ceph/ in mon.rst

rgw: relax RGWPutACLs_ObjStore::get_params read data log

the input can be logged with level 20 than zero.

Fixes: https://tracker.ceph.com/issues/69093
Signed-off-by: Seena Fallah <seenafallah@gmail.com>

Merge pull request #60412 from ShwetaBhosale1/fix_issue_68605_modify_structure_of_default_images

cephadm: Modify the structure of the default container images

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #60903 from nbalacha/wip-nbalacha-rbd-man

doc/rbd: Add mirror snapshot schedule information

Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/rbd: Add mirror snapshot schedule information

Signed-off-by: N Balachandran <nibalach@redhat.com>

mgr/dashboard: list smb
Tracker: https://tracker.ceph.com/issues/69080

Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge pull request #60862 from rhcs-dashboard/fix-multisite-e2e-failures

mgr/dashboard: fix multisite e2e failures

Reviewed-by: Afreen Misbah <afreen@ibm.com>

doc/cephadm: s/ceph/Ceph/ in mon.rst

s/ceph/Ceph/ in doc/cephadm/services/mon.rst where appropriate.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60759 from rhcs-dashboard/add-smb-endpoints

mgr/dashboard: add smb endpoints

Reviewed-by: Afreen Misbah <afreen@ibm.com>

crimson/os/seastore: minor cleanup

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: drop unused file

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60892 from zdover23/wip-doc-2024-11-30-glossary-dashboard-plugin

doc/glossary.rst: add "Dashboard Plugin"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/glossary.rst: add "Dashboard Plugin"

Add an entry below the (Mimic-era and therefore outdated but
nonetheless historically important) Dashboard Plugin key word in the
glosssary, which before now had never been added to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60882 from anthonyeleven/59466-followup

os/bluestore: Improve documentation introduced by #57722

Reviewed-by: Zac Dover <zac.dover@proton.me>

mgr/dashboard: add smb endpoints

Adds following SMB endpoints:
- cluster: list, get, create
- share: list, delete

Fixes: https://tracker.ceph.com/issues/69044
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #60560 from ivoalmeida/carbon-tree-component

mgr/dashboard: carbon tree component

Signed-off-by: nizamial09 <nia@redhat.com>

Merge pull request #60884 from zdover23/wip-doc-2024-11-29-radosgw-s3-common

doc/radosgw: update rgw_dns_name doc

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

mgr/dashboard: fix multisite e2e failures

Fixes: https://tracker.ceph.com/issues/69072
Signed-off-by: Naman Munet <namanmunet@li-ff83bccc-26af-11b2-a85c-a4b04bfb1003.ibm.com>

Merge pull request #60817 from cyx1231st/wip-seastore-improve-get_extents_if_live

crimson/os/seastore/transaction_manager: improve get_extents_if_live()

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

doc/radosgw: update rgw_dns_name doc

Update doc/radosgw/s3/commons.rst with the changes made by Jiffin Tony
Thottan in https://github.com/ceph/ceph/pull/54524 and the suggestions
made in that same PR by Anthony D'Atri.

Explain how to set rgw_dns_name to a domain name in order to configure
access to virtual hosted buckets.

Signed-off-by: Zac Dover <zac.dover@proton.me>

os/bluestore: Improve documentation introduced by #57722

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #60427 from mkogan1/wip-vstart-dev-err

vstart: log faulty physical disk device

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard: carbon tree component

Replaces the deprecated npm package '@circlon/angular-tree-component' by
Carbon Tree component.

Fixes: https://tracker.ceph.com/issues/68249
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>

crimson/os/seastore/transaction_manager: add log in get_extents_if_live()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: fix get_extents_if_live

It should not assume the paddr type from get_mappings is SEGMENT.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: adjust do_with for pin_list in get_extents_if_live

To make pin_list alive during parallel_for_each().

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: verify type in read_pin_by_type

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60654 from cyx1231st/wip-seastore-fine-grained-cache

crimson/os/seastore: support partial read for data extents

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore/transaction_manager: fully read the extent before rewritting it

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: convert transaction related paths with interruptor

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: split out transaction_interruptor.h

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: implement and use maybe_indirect_extent_t::get_bl()

Return bufferlist because the extent may be partially loaded under
indirection.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: delay setting bptr for all types of extent in case of read

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson/seastore: add test case for partial reads

Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore: disable seastore_max_data_allocation_size by default

Supposing that fine-grained-cache should address the read amplification
issue. By-default disable seastore_max_data_allocation_size with
fine-grained-cache since seastore_full_integrity_check is by-default
disabled.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: update lru size based on loaded extent size

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore: implement partial reads from cached_extent to object_data_handler

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore/cached_extent: introduce BufferSpace for partial reads

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore/cached_extent: minor adjustments to print

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: add missing final keywords

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: hide the complexity of extent indirection

Make it easier for TM::read_pin() users to consume extent without
worrying about the indirections.

This basically reverts 9cdcd06caadef6cdca64a974c6525ba300c1e289

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: fix to use direct key/length to get absent extent

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>