mgr/cephadm/nvmeof: Add key encryption support to nvmeof configuration

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

Merge pull request #60926 from ktdreyer/container-labeler

labeler: add container labelers

Merge pull request #60946 from nbalacha/wip-nbalacha-69112

cls/rbd: fix mirror_image_map_list object

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #60913 from myoungwon/wip-fix-null-pointer-crimson

crimson/osd/ops_executer: prevent access to null pointers caused by gcc bug

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #60944 from dalgaaf/dalgaaf-fix-doc-ec

doc: fix EC overhead table m-numbering

Merge pull request #60954 from zdover23/wip-doc-2024-12-04-dev-development-workflow-backporter-manual

doc/dev: update link to backporter manual

doc/dev: update link to backporter manual

Update the link to the backporter manual, in
doc/dev/development-workflow.rst. Laura Flores requested this change on
04 Dec 2024.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60924 from ktdreyer/containerfile-date-utc

container/build.sh: use UTC dates for tags

Merge pull request #60930 from zdover23/wip-doc-2024-12-03-rados-ops-health-checks-3

doc/rados: fix sentences in health-checks (3 of x)

Reviewed-by: Cole Mitchell <cole.mitchell.ceph@gmail.com>

cls/rbd: fix mirror_image_map_list object

The rbd-mirror image map is stored in the rbd_mirror_leader object.
Fixed mirror_image_map_list() to operate on the correct object.

Fixes: https://tracker.ceph.com/issues/69112
Signed-off-by: N Balachandran <nibalach@redhat.com>

Merge pull request #60514 from rhcs-dashboard/fix-68733-main

mgr/dashboard: fix total objects/Avg object size in RGW Overview Page

Reviewed-by: Afreen Misbah <afreen@ibm.com>

crimson/osd/ops_executer: prevent access to null pointers caused by gcc bug

This commit assigns the parameter eariler to work around a coroutine related gcc bug
that causes a null pointer access.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Myoungwon Oh <myoungwon.oh@samsung.com>

Merge pull request #60904 from cyx1231st/wip-seastore-hex-log

crimson/os/seastore: convert offset/length in logs to hex format

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #60755 from dmick/wip-release-container

Modify container/ software to support release containers and the promotion of prerelease containers

Merge pull request #60018 from smanjara/rgw-script-multiple-zg

rgw/multisite: test-rgw-multisite.sh can create multiple zonegroups

Merge pull request #60919 from zdover23/wip-doc-2024-12-03-rados-ops-health-checks-2

doc/rados: fix sentences in health-checks (2 of x)

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: fix sentences in health-checks (3 of x)

Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.

This commit edits the second third of
doc/rados/operations/health-checks.rst.

Note to (I hope soon) future Zac: There are a a couple of places near
the end of this file where the sentences are ungrammatical. Update these
in a separate PR (in isolation, so that the grammar and technical
accuracy of these sentences can be the primary focus of the reviewers).

Zac: cf. 000228

Signed-off-by: Zac Dover <zac.dover@proton.me>

container/{Containerfile,build.sh}: use a secret to pass secrets

Use podman's build --secret mechanism to pass credentials for
download.ceph.com

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile: allow failure from curl to stop build

curl doesn't actually fail unless -f is used; also, add
some sh -e flags.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile: purge .repo files with secrets before commit

ceph.repo had creds in it for download.ceph.com/prerelease.
Remove the .repo files we construct, since they're not necessary
once the container is built (no one should be dnf'ing anything
in the container).

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile fix boolean IS_RELEASE usage

Existing syntax was just wrong.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #59862 from jamiepryde/isa-xor-handle-misaligned-buffers

erasure-code/isa: Handle case where data and coding buffers are not 3…

Reviewed-by: Jose J Palacios-Perez <perezjos@uk.ibm.com>

Merge pull request #58473 from YiteGu/add-mon-scrub-lat

mon/scrub: add mon scrub latency

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #60131 from Shreyasapale/wip-fix-shreya-68009

osd/scrubber: Clean up unused function add_double

container/make-manifest-list.py: use LOCALMANIFEST throughout

Signed-off-by: Dan Mick <dan.mick@redhat.com>

containers/make-manifest.py: Add --promote

The promote function finds the latest image on the prerelease repo,
finds all its tags, and copies them all to the release repo.

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/Containerfile, build.sh: Add PRERELEASE_USERNAME/PASSWORD

Used for accessing download.ceph.com

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/build.sh:

- change repo path to prerelease
- remove image id arg for release push
- include 'v' in version name for release push

Signed-off-by: Dan Mick <dan.mick@redhat.com>

container/make-manifest-list.py

- don't print command failure in worker; let the caller print them
  if desired (allow silent failure)
- allow for empty tags list
- look for CEPH_SHA1.  GIT_COMMIT was the sha1 of the ceph-container.git
  commit
- change default paths to prerelease
- add --dry-run to avoid final push
- rename 'HOST' to 'CONTAINER_HOST'
- Use ARCH_SPECIFIC_HOST instead of CONTAINER_HOST (which is used by podman)

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #60172 from smanjara/rgw-multisite-two-zonegroups

qa/multisite: enable two zonegroup yaml

Merge pull request #60591 from smanjara/wip-datalog-autotrim

qa/multisite: add extra checkpoints in datalog_autotrim testcase

labeler: add container labelers

Signed-off-by: Ken Dreyer <kdreyer@ibm.com>

container/build.sh: use UTC dates for tags

Do not rely on the builder host's local timezone setting.

Signed-off-by: Ken Dreyer <kdreyer@ibm.com>

doc: fix EC overhead table m-numbering

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>

Merge pull request #60918 from gbregman/main

python-common/ceph/cephadm: Change NVME image version to 1.4.

Merge pull request #60917 from zdover23/wip-doc-2024-12-03-rados-ops-health-checks

doc/rados: make sentences agree in health-checks.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/rados: fix sentences in health-checks (2 of x)

Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.

This commit edits the second third of
doc/rados/operations/health-checks.rst.

Zac: cf. 000228

Signed-off-by: Zac Dover <zac.dover@proton.me>

python-common/ceph/cephadm: Change NVME image version to 1.4.
Fixes https://tracker.ceph.com/issues/69099

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>

Merge pull request #60767 from rhcs-dashboard/smb-cluster

mgr/dashboard: list smb

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #60099 from MaxKellermann/json_spirit_thread

json_spirit: add missing dependency on Boost::thread

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/rados: make sentences agree in health-checks.rst

Make sentences agree at the head of each section in
doc/rados/operations/health-checks.rst. The sentences were sometimes in
the imperative mood and sometimes in the declarative mood.

This commit edits the first third of
doc/rados/operations/health-checks.rst.

Zac: cf. 000228

Signed-off-by: Zac Dover <zac.dover@proton.me>

crimson/os/seastore: convert offset/length in seastore logs to hex format

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: convert offset/length in segment logs to hex format

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60151 from shreyanshjain7174/wip-skoduri-cloud-restore

rgw/restore: Fixed status codes and response headers
Reviewed-by: Soumya Koduri <skoduri@redhat.com>

Merge pull request #59449 from tth-trang/wip-65772-quincy

rgw/lc: Fix lifecycle not working while bucket versioning is suspended

Reviewed-by: Soumya Koduri <skoduri@redhat.com>

Merge pull request #60910 from clwluvw/putacl-log

rgw: relax RGWPutACLs_ObjStore::get_params read data log

Merge pull request #60254 from clwluvw/zonegroups-createbucket

rgw: initialize createparams zone_placement to avoid garbage value

Merge pull request #60589 from clwluvw/requested-gen-sync

rgw/multisite: revert "rgw/multisite: avoid redundant error repo entry logging

qa/multisite: enable two zonegroup yaml

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #60906 from zdover23/wip-doc-2024-12-02-cephadm-services-mon-capitalize-ceph

doc/cephadm: s/ceph/Ceph/ in mon.rst

rgw: relax RGWPutACLs_ObjStore::get_params read data log

the input can be logged with level 20 than zero.

Fixes: https://tracker.ceph.com/issues/69093
Signed-off-by: Seena Fallah <seenafallah@gmail.com>

Merge pull request #60412 from ShwetaBhosale1/fix_issue_68605_modify_structure_of_default_images

cephadm: Modify the structure of the default container images

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Redouane Kachach <rkachach@ibm.com>

Merge pull request #60903 from nbalacha/wip-nbalacha-rbd-man

doc/rbd: Add mirror snapshot schedule information

Reviewed-by: Zac Dover <zac.dover@proton.me>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

doc/rbd: Add mirror snapshot schedule information

Signed-off-by: N Balachandran <nibalach@redhat.com>

mgr/dashboard: list smb
Tracker: https://tracker.ceph.com/issues/69080

Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge pull request #60862 from rhcs-dashboard/fix-multisite-e2e-failures

mgr/dashboard: fix multisite e2e failures

Reviewed-by: Afreen Misbah <afreen@ibm.com>

doc/cephadm: s/ceph/Ceph/ in mon.rst

s/ceph/Ceph/ in doc/cephadm/services/mon.rst where appropriate.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60759 from rhcs-dashboard/add-smb-endpoints

mgr/dashboard: add smb endpoints

Reviewed-by: Afreen Misbah <afreen@ibm.com>

crimson/os/seastore: minor cleanup

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: drop unused file

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60892 from zdover23/wip-doc-2024-11-30-glossary-dashboard-plugin

doc/glossary.rst: add "Dashboard Plugin"

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/glossary.rst: add "Dashboard Plugin"

Add an entry below the (Mimic-era and therefore outdated but
nonetheless historically important) Dashboard Plugin key word in the
glosssary, which before now had never been added to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60882 from anthonyeleven/59466-followup

os/bluestore: Improve documentation introduced by #57722

Reviewed-by: Zac Dover <zac.dover@proton.me>

mgr/dashboard: add smb endpoints

Adds following SMB endpoints:
- cluster: list, get, create
- share: list, delete

Fixes: https://tracker.ceph.com/issues/69044
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #60560 from ivoalmeida/carbon-tree-component

mgr/dashboard: carbon tree component

Signed-off-by: nizamial09 <nia@redhat.com>

Merge pull request #60884 from zdover23/wip-doc-2024-11-29-radosgw-s3-common

doc/radosgw: update rgw_dns_name doc

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

mgr/dashboard: fix multisite e2e failures

Fixes: https://tracker.ceph.com/issues/69072
Signed-off-by: Naman Munet <namanmunet@li-ff83bccc-26af-11b2-a85c-a4b04bfb1003.ibm.com>

Merge pull request #60817 from cyx1231st/wip-seastore-improve-get_extents_if_live

crimson/os/seastore/transaction_manager: improve get_extents_if_live()

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

doc/radosgw: update rgw_dns_name doc

Update doc/radosgw/s3/commons.rst with the changes made by Jiffin Tony
Thottan in https://github.com/ceph/ceph/pull/54524 and the suggestions
made in that same PR by Anthony D'Atri.

Explain how to set rgw_dns_name to a domain name in order to configure
access to virtual hosted buckets.

Signed-off-by: Zac Dover <zac.dover@proton.me>

os/bluestore: Improve documentation introduced by #57722

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #60427 from mkogan1/wip-vstart-dev-err

vstart: log faulty physical disk device

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard: carbon tree component

Replaces the deprecated npm package '@circlon/angular-tree-component' by
Carbon Tree component.

Fixes: https://tracker.ceph.com/issues/68249
Signed-off-by: Ivo Almeida <ialmeida@redhat.com>

crimson/os/seastore/transaction_manager: add log in get_extents_if_live()

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: fix get_extents_if_live

It should not assume the paddr type from get_mappings is SEGMENT.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: adjust do_with for pin_list in get_extents_if_live

To make pin_list alive during parallel_for_each().

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: verify type in read_pin_by_type

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60654 from cyx1231st/wip-seastore-fine-grained-cache

crimson/os/seastore: support partial read for data extents

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore/transaction_manager: fully read the extent before rewritting it

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: convert transaction related paths with interruptor

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: split out transaction_interruptor.h

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: implement and use maybe_indirect_extent_t::get_bl()

Return bufferlist because the extent may be partially loaded under
indirection.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: delay setting bptr for all types of extent in case of read

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

test/crimson/seastore: add test case for partial reads

Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore: disable seastore_max_data_allocation_size by default

Supposing that fine-grained-cache should address the read amplification
issue. By-default disable seastore_max_data_allocation_size with
fine-grained-cache since seastore_full_integrity_check is by-default
disabled.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: update lru size based on loaded extent size

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore: implement partial reads from cached_extent to object_data_handler

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore/cached_extent: introduce BufferSpace for partial reads

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>
Signed-off-by: Jianxin Li <jianxin1.li@intel.com>

crimson/os/seastore/cached_extent: minor adjustments to print

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: add missing final keywords

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: hide the complexity of extent indirection

Make it easier for TM::read_pin() users to consume extent without
worrying about the indirections.

This basically reverts 9cdcd06caadef6cdca64a974c6525ba300c1e289

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/transaction_manager: fix to use direct key/length to get absent extent

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: adjust logs

Mostly convert length to the hex format.

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: misc cleanups and asserts

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore: clarify zero length is only possible for root

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cached_extent: clarify invarients in the cstors

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cache: cleanup, use the same method to create extents

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

crimson/os/seastore/cached_extent: unify extent ptr creation

Signed-off-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #60857 from zdover23/wip-doc-2024-11-27-glossary-dc

doc/glossary: add "DC" (Data Center) to glossary

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

doc/glossary: add "DC" (Data Center) to glossary

Add the initialism "DC" (Data Center) to the glossary.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60860 from anthonyeleven/update-60846

doc/radosgw: Improve archive-sync-module.rst more