qa: add helpful exceptions for attr changes

It's not possible to extract the error message from stderr by the caller.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

qa: ignore libicu leaks

These are likely spurious and not interesting.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

client: add wrappings for charmap manipuluation of dentry names

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

client: add dir_result_t::dentry::print

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

win32: add libicu Windows build

cephfs introduces a new dependency: libicu. We need to update the
Windows build scripts to build and link against libicu.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Signed-off-by: Lucian Petrut <lpetrut@cloudbasesolutions.com>

CMakeLists: add boost::locale dependency for client

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

install-deps: unconditionally install boost libraries

If we add a new Boost component, the install-deps.sh script will not install it
because other Boost libraries are installed with the desired version. Just
unconditionally install. The package manager is smart enough to skip already
installed packages.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

test/libcephfs: update root operation return values

Switch from EBUSY to reasonable and normal errors for operations on the root
directory. Use the Client::path_walk machinery changes from the previous commit
to catch most of them, otherwise return EINVAL (which is normal for other POSIX
fs).

Fixes: 6ed7f2364ae5507bab14c60b582929aa7b0ba400
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

client: refactor all path traversals through path_walk

This is groundwork to ensure that all code paths properly apply dentry name
transformations during traversal, specifically for casefolding.

Note some ancillary changes:

* readlink now correctly does lookup permission checks during path walk (previously a TODO).

* There has been some tweaking of the method signatures to accept an `const
  InodeRef&`. This was convenient when passing walk_dentry_result::diri around
  but the cascade in changes prompted me to leave it partially done to avoid
  blowing up this changeset.

It's worth noting that this change is somewhat "half done". I had made an
effort to have all operations (whether fuse or libcephfs) call path_walk once.
This was done for Client::_mkdir and a few others. See for example:

  - do_mkdirat -> _mkdir
  - mksnap -> _mkdir
  - ll_mkdir -> _mkdir
  - mkdirs -> _mkdir (this had other significant simplification/cleanup);
    however: note that we still have two calls to path_walk for any given
    directory to be created. This is not a problem because:

Client::path_walk is now more tolerant of being called in a openat-style way.
The method now accepts an anchor directory and relative path (which may be a
single dentry name for ll_* calls). It is also tolerant of the directory inode
in fact referencing the target inode with relative path == "". That is a useful
property for many openat-style APIs (although largely unofficially).  It also
means that if we resolve a path then we can pass the resolved Inode (directory)
/ string (Dentry name) pair to another method which may also call path_walk
with minimal replication of path walk work. This is done a few times in this
changeset.

Fixes: https://tracker.ceph.com/issues/66373
Fixes: https://tracker.ceph.com/issues/70100
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

test/libcephfs: test parallel creates

That the cap_shared_gen value does not result in unexpected ENOENT.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

test/libcephfs: add test for lookup failure after readdir

That we do not get unexpected ENOENT following readdirs (particularly thinking
of cap_shared_gen differences).

Test-case-for: https://tracker.ceph.com/issues/70100
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

client: init dentry shared_gen with invalid value

Directories and dentries are initialized with value 0 which makes detecting a
null  (or placeholder) dentry created via Client::get_or_create difficult.  We
already do checks to see if a dentry is invalid when the directory's shared_gen
changes so use an invalid value for these synthetic dentries until they can be
appropriately updated from Client::update_dentry_lease.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

client: add _lookup debugging

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

client: remove redundant check

This condition:

    (!dn->inode || dn->inode->caps_issued_mask(mask, true))

is already checked in the surrounding `if`.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

client: dump InodeStat from mds

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

mds: encode optmetadata in InodeStat sent to clients

Deliberately do not dump the entire struct. Some metadata may not be
appropriate to share to clients.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

mds: check client features for charmap

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

mds: add client feature bit for charmap

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

mds: wire up vxattr for changing charmap

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Fixes: https://tracker.ceph.com/issues/66373

mds: inherit charmap on mkdir

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

mds,include: add charmap optmetadata

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Fixes: https://tracker.ceph.com/issues/66373

mds,include: add inode_t optional metadata

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Fixes: https://tracker.ceph.com/issues/66373

client: hide alternate_name from API

The dentry alternate_name was exposed primarily to facilitate testing. Instead,
put these methods in the TestClient scaffolding to allow reading/manipulating
alternate_name.

Because we will be using alternate_name to handle encrypted names and the
normalized / case folded name, we do not want the application to change the
meaning of the metadata out-of-band.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

client: move alternate_name once

Client::update_dentry_lease also moves alternate_name from the lease
which causes it to zero out alternate_name on the second move.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Fixes: https://tracker.ceph.com/issues/66373

client: optimize alternate_name passing to helper

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Fixes: https://tracker.ceph.com/issues/66373

client: relocate definition

It doesn't need to be public; the API does not expose alternate_name.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Fixes: https://tracker.ceph.com/issues/66373

client: print dentry with alternate_name on dump

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

client: move inode dump to print method

There is no functional change.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

mds: add debugging for encoding lease stat

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

mds: make encode_lease a proper method

So we can print debugging.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

mds: add fscrypt metadata for inode stat size

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

client: use DentryRef for ref counting in MetaRequest

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

client: add DentryRef

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

client: add helper for determining if a perm check is necessary

To be used in subsequent commits.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

client: cache client_permissions config

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

client: add debugging for conf changes

So logs show when they are modified.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

client: sort configs

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

client/UserPerm: add print method

For debug prints.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

client: note mount parameters in debug log

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

client: print stat mode in octal

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

common: add missing op string

This gets converted in messenger douts.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

include/filepath: add empty path check

When passing an empty string to filepath, it would test if the first character
is '/' which is an invalid access through std::string_view.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
Fixes: https://tracker.ceph.com/issues/66373

Merge PR #61991 into main

* refs/pull/61991/head:
qa: remove unreacheable/redundant code from test_data_scan.py

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #62019 from ceph/template-updates

Links to Jenkins jobs in PR comment commands / Remove deprecated commands

Merge pull request #61863 from spuiuk/ctdb_ports

cephadm/smb: add ctdb ports to smb daemon endpoint

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61925 from JoshuaGabriel/joshua_70058

mgr/cephadm: Check if a daemon is ok-to-stop before scheduling a restart

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #61897 from cbodley/wip-70077

qa/rgw: configure 'iam root' accounts outside of rgw/verify

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>

Merge pull request #61920 from adk3798/cephadm-rgw-ingress-test-timeouts

qa/cephadm: add timeouts and debug commands to rgw-ingress test

Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61710 from adk3798/mgr-nfs-upgrade-start-reef

qa/cephadm: start mgr-nfs-upgrade test from reef

Reviewed-by: Avan Thakkar <athakkar@redhat.com>

Merge pull request #61013 from Kushal-deb/fix_issue-2047153

cephadm: orch upgrade status to return output according to the format specified

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>

Merge pull request #61504 from kchheda3/wip-fix-healthchecks

rgw:lua: Skip the healthchecks and system requests from going to backend storage

Merge pull request #61509 from kchheda3/wip-config-lua

rgw/lua: Add rgw config to enable/disable lua

Merge pull request #61560 from oshrey16/fix-lua-runtime-background

rgw/lua: Fix Lua background thread execution timing and config updates

Merge pull request #61938 from ceph/retrigger-rtd

doc: Support to retrigger docs/readthedocs.org:ceph PR build

doc: PR Template - Remove non-functional trigger phrases

Signed-off-by: David Galloway <david.galloway@ibm.com>

doc: PR Template - Add Jenkins job URLs to commands

Signed-off-by: David Galloway <david.galloway@ibm.com>

Merge pull request #61822 from rhcs-dashboard/smb-edit-cluster

mgr/dashboard: SMB - Edit Cluster

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #61815 from ljflores/wip-tracker-69456

qa/suites/upgrade: ignore when pg is stuck peering

Merge pull request #61703 from MaxKellermann/kstore_includes

os/kstore: add missing include

Merge pull request #61674 from connorfawcett/wip-ec-default-fixes

erasure-code: Set reed_sol_van to be the default technique for Jerasure if none is specified.

Merge pull request #61015 from nbalacha/wip-nbalacha-rbd-mirror-namespace-2

rbd-mirror: default <-> non-default namespace remapping

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #61269 from cbodley/wip-rgw-createbucket-layout

rgw/s3: CreateBucket extension for layout type and shard count

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #61982 from anthonyeleven/nvme-proxy

doc/rbd: Improve nvmeof-requirements.rst with proxy and other info

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60524 from thotz/multisitie-fix-cloud-restore

rgw/cloudrestore: sync cloud restore objects accoridngly

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Soumya Koduri <skoduri@redhat.com>

Merge pull request #61849 from yuvalif/wip-yuval-69944

rgw: be explicit on moving rados op in APIs

Merge pull request #61376 from athanatos/sjust/wip-crimson-gcc-13

src/CMakeLists: require gcc13 for crimson, clarify error messages

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #61846 from rhcs-dashboard/tox-generate-proto

mgr/dashboard: tox command to generate the nvmeof proto files

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #61903 from ronen-fr/wip-rf-crimson-conf

crimson/osd: replace obsolete get_tracked_conf_keys()

Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #62001 from cybozu/fix-stale-delete

doc: fix incorrect radosgw-admin subcommand

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #60871 from leonidc/leonidc-epoch-filter

Epoch filtering

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Aviv Caro <Aviv.Caro@ibm.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>

doc: fix incorrect radosgw-admin subcommand

Signed-off-by: Toshikuni Fukaya <toshikuni-fukaya@cybozu.co.jp>

Merge PR #61889 into main

* refs/pull/61889/head:
msg/async/Stack: include cleanup
msg/Message: include cleanup
msg/async/crypto_onwire: include cleanup
msg/async/ProtocolV1: add missing includes
msg/async/frames_v2: add missing includes
msg/msg_types: add missing includes

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61888 into main

* refs/pull/61888/head:
os/memstore: add missing includes

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61876 into main

* refs/pull/61876/head:
qa/cephfs: ignore warning that pg is stuck peering for upgrade jobs

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61802 into main

* refs/pull/61802/head:
perfglue: add missing include

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61801 into main

* refs/pull/61801/head:
kv: add missing includes

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61799 into main

* refs/pull/61799/head:
erasure-code: add missing include

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #61798 into main

* refs/pull/61798/head:
crush: include cleanup

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #61955 from idryomov/wip-70075-test

qa/workunits/rbd: add a test for force promote with a user snapshot

Reviewed-by: N Balachandran <nibalach@redhat.com>

rgw: reject PutBucketLifecycleConfiguration on indexless buckets

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw: improved error when listing indexless bucket

without this change, ListObjects would attempt to list index shard
objects that don't exist and fail with:

> ERROR: S3 error: 404 (NoSuchKey)

after:

> ERROR: S3 error: 405 (MethodNotAllowed): Indexless buckets cannot be listed

Signed-off-by: Casey Bodley <cbodley@redhat.com>

examples/rgw: register boto extension for BucketIndex on CreateBucket

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/s3: CreateBucket extension for layout type and shard count

extend s3's CreateBucketConfiguration [1] with a custom BucketIndex element
that can override rgw's default bucket index type and shard count

to create an indexless bucket:
  <CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <BucketIndex>
      <Type>Indexless</Type>
    </BucketIndex>
  </CreateBucketConfiguration>

to create a normal pre-sharded bucket:
  <CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <BucketIndex>
      <Type>Normal</Type>
      <NumShards>1023</NumShards>
    </BucketIndex>
  </CreateBucketConfiguration>

[1] https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html#API_CreateBucket_RequestSyntax

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: create_bucket() can override index type and shards

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/rados: indexless buckets skip init_index/clean_index

Signed-off-by: Casey Bodley <cbodley@redhat.com>

rgw/s3: CreateBucket's LocationConstraint is optional

clean up the parsing and make the LocationConstraint optional

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #61740 from Matan-B/wip-matanb-clang-16

script/run-make: clang 14->19

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Adam Emerson <aemerson@redhat.com>

Merge pull request #61319 from rhcs-dashboard/gklm-kmip

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #61619 from ronen-fr/wip-rf-more-keys

rgw: replace obsolete get_tracked_conf_keys()
Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>

mgr/dashboard: SMB - Edit Cluster
Fixes: https://tracker.ceph.com/issues/69964
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

qa: remove unreacheable/redundant code from test_data_scan.py

call to self.fs.journal_tool(["journal", "reset", "--force", "--yes-i-really-really-mean-it"], 0)
is already made below it, the same call under if False is unreachable.

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>

Merge pull request #61928 from rhcs-dashboard/smb-share-edit

mgr/dashboard: SMB - Edit Share

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>

Merge pull request #61976 from guits/cv-update-batch-unit-tests

ceph-volume: update lvm batch unit tests

Merge pull request #61985 from zdover23/wip-doc-2025-02-25-releases-squid-order

doc/releases: correct squid release order

Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge pull request #61788 from VallariAg/wip-nvmeof-teuthology-improve

qa/suite/nvmeof: Add asserts to scalability_test and extra logs in fio_test

Merge pull request #61984 from cbodley/wip-doc-rgw-path-style

doc/rgw: clarify path-style vs virtual-hosted-style access

Reviewed-by: Zac Dover <zac.dover@proton.me>

doc/releases: correct squid release order

Put the releases of Squid in descending order. This change alters the
order of the Squid releases so that it is the same as the order of the
other Ceph releases.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #61875 from rhcs-dashboard/update-bucket-details

mgr/dashboard: Update bucket details section after making bucket lifecycle changes

Reviewed-by: Aashish Sharma <aasharma@redhat.com>

mgr/dashboard: Add GKLM(KMIP) option to SSE-KMS dropdown Menu in the Dashboard UI

Fixes: https://tracker.ceph.com/issues/69490
Signed-off-by: Prachi Goel <prachi.goel@li-25d455cc-279f-11b2-a85c-b097dffa3f04.ibm.com>
Changes for GKLM dropdown UI:

1.Added option kmip in KMS dropdown in configuration page RGW
2.Made API dynamic for setting form data for Vault, kmip both for (S3 and
KMS)
3.Both Getdata and setdata API for RGW config page is modified

mgr/dashboard: SMB - Edit Share.

Fixes: https://tracker.ceph.com/issues/70094
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>