test/crimson/seastore/CMakeLists: [arm64] disable omap manager

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson: bump up seastar fixes

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

seastar: bump up seastar submodule

Update branch to wip-matanb-seastar-feb-25

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #61441 from ivancich/wip-dynamic-resharding-min-shards

rgw: allow per-bucket minimum number of shards

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #62010 from ronen-fr/wip-rf-monotime

osd/scrub: tolerate non-monotonic scrub queue cutoff updates

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #61541 from rishabh-d-dave/mgr-vol-comment

mgr/vol: add a helpful comment in async_job.py

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #61993 from ronen-fr/wip-rf-nbq-sign

test/test_not_before_queue: fix Signed-vs-Unsigned warnings

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc/rgw: update dynamic resharding docs to reflect recent changes

The documentation on dynamic resharding is updated to include a) a
description of reducing the number of shards, b) related configuration
options, and c) the radosgw-admin sub-command to set a minimum number
of shards for a specific bucket.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

rgw: add radosgw-admin sub-command to set-min-shards for a bucket

There is now a mechansim to set the minimum number of shards when a
bucket is created, and dynamic resharding adheres to that
setting. This adds the ability to modify that minimum shard count that
exists within the bucket layout of the bucket instance
object. Example:

    radosgw-admin bucket set-min-shards --bucket=<bucket> \
        --num-shards=<value>

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

rgw: allow per-bucket minimum number of shards

Dynamic resharding can now reduce the number of shards. The code
currently has a hard-coded value of 11 as the minimum number of shards
dynamic resharding can reshard to. There may be cases where the user
wants to set an alternate minimum, such as when they have a sense of
how many objects the bucket will eventually hold.

This PR builds off of https://github.com/ceph/ceph/pull/61269 .

That PR allows the user to specify an initial number of shards during
bucket creation. This PR then takes that number to be the minimum and
saves it in the layout field of the bucket instance object
(RGWBucketInfo).

When dynamic resharding is triggered, it will use that stored value as
a minimum number of shards for resharing.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #62021 from dmick/container-rmi-tags

container/build.sh: remove local container images

Merge pull request #61946 from cbodley/wip-70013

rgw: PutObjectLockConfiguration can enable object lock on existing buckets

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #61901 from cbodley/wip-69936

rgw/cksum: GetObject omits checksum headers for Range requests

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #61626 from zdover23/wip-doc-2025-02-03-rados-ops-pgs

doc/rados: improve pg_num/pgp_num info

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #55838 from neesingh-rh/wip-58090

mgr/volumes: fix dangling symlink in clone index

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
Reviewed-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #60394 from joscollin/wip-B68446-B68001-get-metrics-fix

qa: reconfigure only the appropriate client auth caps

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>

Merge pull request #61554 from rishabh-d-dave/check-mds-id

cephfs,mon: don't disallow "mds/fs fail" when other MDS/FS has health warning

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #61706 from rishabh-d-dave/subvol-meta-filename-too-long

mgr/vol: print proper message when subvolume metadata filename is too long

Reviewed-by: Milind Changire <mchangir@redhat.com>

test/test_not_before_queue: fix Signed-vs-Unsigned warnings

Fixes: https://tracker.ceph.com/issues/69525
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

osd/scrub: log scrub-queue advance_time() failures

such failures occur when the "Ceph clock" goes backwards,

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

common/not_before_queue_t: tolerate non-monotonic cut-off values

as the system monotonic clock is used when the container is used
in Scrub implementation, and on some kernels there are rare cases
where the monotonic clock can go backwards, we need to tolerate
such events.

Fixes: https://tracker.ceph.com/issues/70055
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>

Merge pull request #62032 from Matan-B/wip-matanb-crimson-disable-txn-mgr-arm

test/crimson/seastore/CMakeLists: disable unittest-transaction-manager (arm)

Merge pull request #62022 from xxhdx1985126/wip-seastore-tree-pointer-abstract-cleanup

crimson/so/seastore: clean up logical_child_node.cc

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>

Merge pull request #62030 from zdover23/wip-doc-2025-02-27-rbd-nvmeof-requirements

doc/rbd: improve nvmeof-requirements.rst

Merge pull request #59213 from myoungwon/wip-dec-omap-leaf-size

crimson/os/seastore: redirect log operations to 16K-leaf-size omap

Reviewed-by: Yingxin Cheng <yingxin.cheng@intel.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge PR #61682 into main

* refs/pull/61682/head:
mds/snap: include cleanup
mds/MetricsHandler: include cleanup
mds/LogSegment: add missing includes
mds/JournalPointer: include cleanup
mds/cephfs_features: include cleanup
mds/MDSMap: include cleanup
mds/Migrator: include cleanup
mds/MDLog: include cleanup
mds/SnapServer: include cleanup
mds/MDSTableServer: include cleanup
mds/MetricAggregator: include cleanup
mds/SnapClient: include cleanup
mds/MDSPinger: include cleanup
mds/flock: include cleanup
mds/MDSAuthCaps: include cleanup
mds/QuiesceAgent: include cleanup
mds/QuiesceDb: include cleanup
mds/mdstypes: include cleanup
msg/Locker: include cleanup
mds/journal: add missing includes
mds/Server: include cleanup
mds/LogEvent: include cleanup
mds/Capability: include cleanup
mds/DamageTable: include cleanup
mds/StrayManager: include cleanup
mds/Mutation: include cleanup
mds/SnapRealm: include cleanup
mds/InoTable: add missing include
mds/ScatterLock: add missing include
mon/MDSMonitor: add missing include
mon/MDSDaemon: add missing include

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61991 into main

* refs/pull/61991/head:
qa: remove unreacheable/redundant code from test_data_scan.py

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #62019 from ceph/template-updates

Links to Jenkins jobs in PR comment commands / Remove deprecated commands

Merge pull request #61863 from spuiuk/ctdb_ports

cephadm/smb: add ctdb ports to smb daemon endpoint

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61925 from JoshuaGabriel/joshua_70058

mgr/cephadm: Check if a daemon is ok-to-stop before scheduling a restart

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #61897 from cbodley/wip-70077

qa/rgw: configure 'iam root' accounts outside of rgw/verify

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #61920 from adk3798/cephadm-rgw-ingress-test-timeouts

qa/cephadm: add timeouts and debug commands to rgw-ingress test

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61710 from adk3798/mgr-nfs-upgrade-start-reef

qa/cephadm: start mgr-nfs-upgrade test from reef

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #61013 from Kushal-deb/fix_issue-2047153

cephadm: orch upgrade status to return output according to the format specified

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

src/test/crimson/seastore/CMakeLists: disable unittest-transaction-manager

See: https://tracker.ceph.com/issues/70202

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

doc/rbd: improve nvmeof-requirements.rst

Address quibbles raised in the course of assessing
https://github.com/ceph/ceph/pull/61982.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #61504 from kchheda3/wip-fix-healthchecks

rgw:lua: Skip the healthchecks and system requests from going to backend storage

Merge pull request #61509 from kchheda3/wip-config-lua

rgw/lua: Add rgw config to enable/disable lua

Merge pull request #61560 from oshrey16/fix-lua-runtime-background

rgw/lua: Fix Lua background thread execution timing and config updates

Merge pull request #61938 from ceph/retrigger-rtd

doc: Support to retrigger docs/readthedocs.org:ceph PR build

crimson/so/seastore: clean up logical_child_node.cc

logical_child_node.cc is totally duplicated with lba_mapping.cc and
should be removed

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>

container/build.sh: remove local container images

Optionally, for those that want to run build.sh locally and
use the images.  The default is to remove, for Jenkins builders,
which will build, push, and rmi.

Fixes: https://tracker.ceph.com/issues/70196
Signed-off-by: Dan Mick <dan.mick@redhat.com>

doc: PR Template - Remove non-functional trigger phrases

Signed-off-by: David Galloway <david.galloway@ibm.com>

doc: PR Template - Add Jenkins job URLs to commands

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #61822 from rhcs-dashboard/smb-edit-cluster

mgr/dashboard: SMB - Edit Cluster

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #61815 from ljflores/wip-tracker-69456

qa/suites/upgrade: ignore when pg is stuck peering

Merge pull request #61703 from MaxKellermann/kstore_includes

os/kstore: add missing include

Merge pull request #61674 from connorfawcett/wip-ec-default-fixes

erasure-code: Set reed_sol_van to be the default technique for Jerasure if none is specified.

Merge pull request #61015 from nbalacha/wip-nbalacha-rbd-mirror-namespace-2

rbd-mirror: default <-> non-default namespace remapping

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #61269 from cbodley/wip-rgw-createbucket-layout

rgw/s3: CreateBucket extension for layout type and shard count

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #61982 from anthonyeleven/nvme-proxy

doc/rbd: Improve nvmeof-requirements.rst with proxy and other info

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60524 from thotz/multisitie-fix-cloud-restore

rgw/cloudrestore: sync cloud restore objects accoridngly

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Soumya Koduri <skoduri@redhat.com>

Merge pull request #61849 from yuvalif/wip-yuval-69944

rgw: be explicit on moving rados op in APIs

Merge pull request #61376 from athanatos/sjust/wip-crimson-gcc-13

src/CMakeLists: require gcc13 for crimson, clarify error messages

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #61846 from rhcs-dashboard/tox-generate-proto

mgr/dashboard: tox command to generate the nvmeof proto files

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #61903 from ronen-fr/wip-rf-crimson-conf

crimson/osd: replace obsolete get_tracked_conf_keys()

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #62001 from cybozu/fix-stale-delete

doc: fix incorrect radosgw-admin subcommand

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60871 from leonidc/leonidc-epoch-filter

Epoch filtering

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Aviv Caro <Aviv.Caro@ibm.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

doc: fix incorrect radosgw-admin subcommand

Signed-off-by: Toshikuni Fukaya <toshikuni-fukaya@cybozu.co.jp>

Merge PR #61889 into main

* refs/pull/61889/head:
msg/async/Stack: include cleanup
msg/Message: include cleanup
msg/async/crypto_onwire: include cleanup
msg/async/ProtocolV1: add missing includes
msg/async/frames_v2: add missing includes
msg/msg_types: add missing includes

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61888 into main

* refs/pull/61888/head:
os/memstore: add missing includes

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61876 into main

* refs/pull/61876/head:
qa/cephfs: ignore warning that pg is stuck peering for upgrade jobs

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61802 into main

* refs/pull/61802/head:
perfglue: add missing include

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61801 into main

* refs/pull/61801/head:
kv: add missing includes

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61799 into main

* refs/pull/61799/head:
erasure-code: add missing include

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61798 into main

* refs/pull/61798/head:
crush: include cleanup

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #61955 from idryomov/wip-70075-test

qa/workunits/rbd: add a test for force promote with a user snapshot

Reviewed-by: N Balachandran <nibalach@redhat.com>

rgw/cksum: GetObject omits checksum headers for Range requests

if we send response headers containing full-object checksums, clients
will compare them with the returned data and fail with:

> botocore.exceptions.FlexibleChecksumError: Expected checksum 4AAr8A== did not match calculated checksum: WbxXZw==

the only thing aws docs[1] say about Range requests refers to multipart
uploads:

> For completed uploads, you can get an individual part's checksum by using the GetObject or HeadObject operations and specifying a part number or byte range that aligns with a single part.

Fixes: https://tracker.ceph.com/issues/69936
[1] https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html#Part-level-checksums

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: reject PutBucketLifecycleConfiguration on indexless buckets

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: improved error when listing indexless bucket

without this change, ListObjects would attempt to list index shard
objects that don't exist and fail with:

> ERROR: S3 error: 404 (NoSuchKey)

after:

> ERROR: S3 error: 405 (MethodNotAllowed): Indexless buckets cannot be listed

Signed-off-by: Casey Bodley <cbodley@redhat.com>

examples/rgw: register boto extension for BucketIndex on CreateBucket

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/s3: CreateBucket extension for layout type and shard count

extend s3's CreateBucketConfiguration [1] with a custom BucketIndex element
that can override rgw's default bucket index type and shard count

to create an indexless bucket:
  <CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <BucketIndex>
      <Type>Indexless</Type>
    </BucketIndex>
  </CreateBucketConfiguration>

to create a normal pre-sharded bucket:
  <CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <BucketIndex>
      <Type>Normal</Type>
      <NumShards>1023</NumShards>
    </BucketIndex>
  </CreateBucketConfiguration>

[1] https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html#API_CreateBucket_RequestSyntax

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: create_bucket() can override index type and shards

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: indexless buckets skip init_index/clean_index

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/s3: CreateBucket's LocationConstraint is optional

clean up the parsing and make the LocationConstraint optional

Signed-off-by: Casey Bodley <cbodley@redhat.com>

mgr/vol: add a helpful comment in async_job.py

Signed-off-by: Rishabh Dave <ridave@redhat.com>

Merge pull request #61740 from Matan-B/wip-matanb-clang-16

script/run-make: clang 14->19

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #61319 from rhcs-dashboard/gklm-kmip

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #61619 from ronen-fr/wip-rf-more-keys

rgw: replace obsolete get_tracked_conf_keys()
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

mgr/dashboard: SMB - Edit Cluster
Fixes: https://tracker.ceph.com/issues/69964
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

qa: remove unreacheable/redundant code from test_data_scan.py

call to self.fs.journal_tool(["journal", "reset", "--force", "--yes-i-really-really-mean-it"], 0)
is already made below it, the same call under if False is unreachable.

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #61928 from rhcs-dashboard/smb-share-edit

mgr/dashboard: SMB - Edit Share

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #61976 from guits/cv-update-batch-unit-tests

ceph-volume: update lvm batch unit tests

Merge pull request #61985 from zdover23/wip-doc-2025-02-25-releases-squid-order

doc/releases: correct squid release order

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #61788 from VallariAg/wip-nvmeof-teuthology-improve

qa/suite/nvmeof: Add asserts to scalability_test and extra logs in fio_test

Merge pull request #61984 from cbodley/wip-doc-rgw-path-style

doc/rgw: clarify path-style vs virtual-hosted-style access

Reviewed-by: Zac Dover <zac.dover@proton.me>

doc/releases: correct squid release order

Put the releases of Squid in descending order. This change alters the
order of the Squid releases so that it is the same as the order of the
other Ceph releases.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #61875 from rhcs-dashboard/update-bucket-details

mgr/dashboard: Update bucket details section after making bucket lifecycle changes

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard: Add GKLM(KMIP) option to SSE-KMS dropdown Menu in the Dashboard UI

Fixes: https://tracker.ceph.com/issues/69490
Signed-off-by: Prachi Goel <prachi.goel@li-25d455cc-279f-11b2-a85c-b097dffa3f04.ibm.com>
Changes for GKLM dropdown UI:

1.Added option kmip in KMS dropdown in configuration page RGW
2.Made API dynamic for setting form data for Vault, kmip both for (S3 and
KMS)
3.Both Getdata and setdata API for RGW config page is modified

mgr/dashboard: SMB - Edit Share.

Fixes: https://tracker.ceph.com/issues/70094
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Merge PR #61979 into main

* refs/pull/61979/head:
script/ceph-backport: remove upstream remote inferencing

Reviewed-by: Guillaume Abrioux <gabrioux@redhat.com>

doc/rgw: clarify path-style vs virtual-hosted-style access

instead of referring to "vhost-style", copy the "path-style" and
"virtual-hosted-style" language from https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html

expand the FQDN acronym to avoid potential confusion

"The second method is deprecated by AWS" had incorrectly referred to the
vhost-style method - clarify that it refers to path-style access

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge PR #61943 into main

* refs/pull/61943/head:
qa: ignore variant of down fs

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #61792 into main

* refs/pull/61792/head:
libcephsqlite: include cleanup

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61701 into main

* refs/pull/61701/head:
client/Client: add missing include and forward declaration

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61697 into main

* refs/pull/61697/head:
osdc/ObjectCacher: include cleanup
osdc/Objecter: include cleanup
osdc/Journaler: include cleanup
osdc/Striper: add missing includes
osdc/Filer: include cleanup
crush/CrushWrapper: un-inline methods to fix linker errors
client/Client: add missing include and forward declaration

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61872 into main

* refs/pull/61872/head:
crush/CrushWrapper: un-inline methods to fix linker errors

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #61733 from phlogistonjohn/jjm-ctr-cleaner

container: small container image improvements

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Dan Mick <dan.mick@redhat.com>

doc/rbd: Improve nvmeof-requirements.rst with proxy and other info

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #61400 from salieri11/igolikov-wip-fscrypt-68776

client: Use custom fscrypt cli in fs/fscrypt suite

Reviewed-by: Christopher Hoffman <choffman@redhat.com>