client: Various fixes to fix multi-fuse client

Provide various fixes in which size used in
multi-fuse client tests.

Fixes: https://tracker.ceph.com/issues/68431
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

qa: Convert and create tests for libcephfs fscrypt

Convert existing tests to use teuthology framework.
Create tests to test N>1 fscrypt clients

Fixes: https://tracker.ceph.com/issues/66577
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Update fscrypt_file when mds info is newer

Fixes: https://tracker.ceph.com/issues/68233
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

qa: Add tests of interopability of fscrypt between fuse/kernel

Fixes: https://tracker.ceph.com/issues/66577
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

ObjectCacher: handle nullptr hole case

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

ObjectCacher: On RetryRead, ensure "hole" value is used

On C_RetryRead, ensure "hole" value instead of pointer
to a pointer is used to allow populating vector of holes.

Fixes: https://tracker.ceph.com/issues/67659
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: When calling update_inode_file_size, provide correct size

Fixes: https://tracker.ceph.com/issues/67559
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Use effective_size in eof read

Fixes: https://tracker.ceph.com/issues/67347
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client/FSCrypt: securely erase crypto key

Fixes: https://tracker.ceph.com/issues/64136
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Implement status for fscrypt key status

Fixes: https://tracker.ceph.com/issues/64130
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Add busy case on key removal

Fixes: https://tracker.ceph.com/issues/64159
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

fuse: enable ioctl on dir for fscrypt

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

src/test/libcephfs: add test cases for fscrypt key removal busy case

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

qa: Added workunits for testing problem snippets on rmw workloads

Fixes: https://tracker.ceph.com/issues/66038
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

qa: Allow fscrypt testing on fuse

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: fix non-encrypted case in read_sync

Only append pbl to bl if encrypted case.

Fixes: https://tracker.ceph.com/issues/65964
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: fscrypt rmw fails when endoff end of block or file
Fscrypt rmw fails when end of a write lines up with end of
a block or end of the file.

Fixes: https://tracker.ceph.com/issues/65745
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: During lookup of fscrypt symlink, use target fscrypt info.

During a lookup of fscrypt enabled symlink, use target fscrypt
info. This must be used because enc key for each file is derived
from master_key+nonce.

Fixes: https://tracker.ceph.com/issues/65615
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Fix logic in need read start/end to account for fscrypt.

Fix the logic in need read start/end. We need to make sure that a
whole block is read when a rmw is issued, regardless if it starts
at offset 0 or not. Change size that may be read from where offset
starts to the whole fscrypt block.

Fixes: https://tracker.ceph.com/issues/64819
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: After a rebase, no lock is necessary.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

Client: fix O_TRUNC issue

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Ensure symlink plaintext is set

When creating symlink, ensure symlink_plain is set in fscrypt and
non-fscrypt cases.

Fixes: https://tracker.ceph.com/issues/64691
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: rmw doesn't get_cap_ref like it should. After rmw,
it put_cap_ref and it didn't happen. Issue with bool need_read.

Fixes: https://tracker.ceph.com/issues/64307
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

Populate add key return

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

test: Add test for semantics

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: On fscrypt enabled directories, only allow read/write to files
in dir when unlocked. Client should not be able to read/write the encrypted payload.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

fscrypt: Add tests for add key, remove key, set policy.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Allow setpolicy only on dir and when it is empty.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

Add multi user support on a unlock claim.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client,test,osdc: add beginnings of fscrypt support

Signed-off-by: Yehuda Sadeh <ysadehwe@ibm.com>

Merge pull request #64981 from zdover23/wip-doc-2025-08-12-cephfs-troubleshooting

doc/cephfs: edit troubleshooting.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #64930 from zdover23/wip-doc-2025-08-10-cephfs-troubleshooting-3

doc/cephfs: edit troubleshooting.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #64923 from zdover23/wip-doc-2025-08-09-cephfs-troubleshooting-3

doc/cephfs: edit troubleshooting.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #64922 from zdover23/wip-doc-2025-08-09-cephfs-troubleshooting-2

doc/cephfs: edit troubleshooting.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>

Merge pull request #64811 from ceph/fix-dependabot_dashboard_upgrades-main

dependabot: stop dashboard npm upgrade PRs

doc/cephfs: edit troubleshooting.rst

Edit the "Mount 5 Error" and "Mount 12 Error" sections of
doc/cephfs/troubleshooting.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>

Merge pull request #64982 from zdover23/wip-doc-2025-08-12-cephfs-troubleshooting-2

doc/cephfs: edit troubleshooting.rst

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #65008 from chanyoung/omap-fast-check-mappings

test/crimson/seastore/test_omap_manager: faster check mappings

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #65013 from Naveenaidu/wip-naveen-update-cephadm-docs

doc/man/cephadm: update `--orphan-initial-daemons` argument description

Merge pull request #63938 from tchaikov/wip-osd-recalc-decode

osd: recalculate coll_t::_str during decode() to fix stale values

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #61770 from rkachach/fix_issue_69484

Prometheus configuration generation improvements

Reviewed-by: nizamial09 <nia@redhat.com>

doc/man/cephadm: update --orphan-initial-daemons argument description

Signed-off-by: Naveen Naidu <naveen.naidu@ibm.com>

test/crimson/seastore/test_omap_manager: faster check mappings

Signed-off-by: Chanyoung Park <chaney.p@kakaoenterprise.com>

Merge pull request #64998 from chanyoung/monotonic

test/crimson/seastore/test_omap_manager: add monotonic inc UT

Reviewed-by: Samuel Just <sjust@redhat.com>

doc/cephfs: edit troubleshooting.rst

Edit the "Dynamic Debugging" section of doc/cephfs/troubleshooting.rst.

Signed-off-by: Zac Dover <zac.dover@proton.me>
doc/cephfs: improve dynamic debugging section

Signed-off-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #64976 from phlogistonjohn/jjm-container-pkgs

container: support building a container image with custom packages

Merge pull request #64942 from chanyoung/redesign_depth3_split

test/crimson/seastore/test_omap_manager: redesign depth3 split

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #64964 from edwinzrodriguez/ceph-wip-72454

test/librbd/fsx: initialize effective_size variable

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge PR #64821 into main

* refs/pull/64821/head:
.github: only run redmine-upkeep actions from ceph/ceph.git
script/redmine-upkeep: add transform to resolve merged issue
script/redmine-upkeep: set default filter name/priority
script/redmine-upkeep: raise exception for PUT failures
script/redmine-upkeep: finish transform after application
script/redmine-upkeep: indicate log location in comments
script/redmine-upkeep: check envvar to see if running as action
script/redmine-upkeep: bullet issue list
script/redmine-upkeep: add stronger note on upkeep-failed tag in failure message
script/redmine-upkeep: do not raise commennt if upkeep-failed already present
script/redmine-upkeep: correct filter out of upkeep-failed
.github/workflows: allow redmine-upkeep to write comments

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

.github: only run redmine-upkeep actions from ceph/ceph.git

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/redmine-upkeep: add transform to resolve merged issue

Few things:

- Add priority to transforms. Largely this is to have the "merged"
  transformation run first to update the "Merge Commit" field of the ticket
  before any other transform intends to look at that field. This avoids
  duplicating logic to set the Merge Commit field.

- Fix a bug where the github API cannot be trusted to indicate the Merge Commit
  for a PR. When the branch is rename or changed, the github backend clearly
  gets confused and gives the "HEAD" commit instead.

- Add new transform to resolve tickets that are merged to either Resolved or
  Pending Backport status.

* Note: filters on TAGS cannot be combined. There is some restructuring to deal with that.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #64938 from chanyoung/omap-binary-search

crimson/.../string_kv_node_layout: switch linear to binary search

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

script/redmine-upkeep: set default filter name/priority

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/redmine-upkeep: raise exception for PUT failures

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/redmine-upkeep: finish transform after application

Doing this at the end makes confusing error messages.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/redmine-upkeep: indicate log location in comments

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/redmine-upkeep: check envvar to see if running as action

This obviates --github-action in the workflow.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/redmine-upkeep: bullet issue list

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/redmine-upkeep: add stronger note on upkeep-failed tag in failure message

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/redmine-upkeep: do not raise commennt if upkeep-failed already present

If the script is run manually on an issue, don't leave a new comment if we
already have the issue tagged with upkeep-failed.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

script/redmine-upkeep: correct filter out of upkeep-failed

"!~" is the proper filter for "does not contain".

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

.github/workflows: allow redmine-upkeep to write comments

Fixes errors like:

    Posting a comment to GitHub PR #62992.
    GitHub API error posting comment to PR #62992: 401 Client Error: Unauthorized for url: https://api.github.com/repos/ceph/ceph/issues/62992/comments - Response: {"message":"Requires authentication","documentation_url":"https://docs.github.com/rest/issues/comments#create-an-issue-comment","status":"401"}

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

test/crimson/seastore/test_omap_manager: add monotonic inc UT

Signed-off-by: Chanyoung Park <chaney.p@kakaoenterprise.com>

Merge pull request #64501 from aainscow/ec_fixpack2_pr

Optimized Erasure Coding - Fixpack 2

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #64926 from chanyoung/faster-seastore-eph-dev

crimson/.../ephemeral: faster seastore eph dev

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #64861 from ronen-fr/wip-rf-pgdict

test/standalone/scrub: improve build_pg_dicts() performance

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #64253 from arm7star/rgw-openssl

rgw, common: OpenSSL provider support

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #64675 from chanyoung/fix-omap-balance

crimson/.../omap_btree_node_impl: pass pivot_idx for balancing

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #64575 from chanyoung/add-test-omap-update

test/crimson/seastore/test_omap_manager: add heavy update UT

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #64812 from Matan-B/wip-matanb-crimson-seastore-trans-intr

crimson/os/seastore: introduce with_repeat_trans_intr

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>

Merge pull request #64911 from rhcs-dashboard/dashboard-loading-state

mgr/dashboard: loading state for inventory card

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #64890 from cloudbehl/add-osd-ratio

prometheus: Add OSD full and nearfull ratio to prometheus

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #64753 from Hezko/nvmeof-cli-mtls-fix2

mgr/dashboard: fix nvmeof mtls

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #64767 from bluikko/doc-notifications-fixes-cephadm

doc/radosgw: Small fixes and improvements in notifications.rst, part 1

Reviewed-by: Yuval Lifshitz <ylifshit@ibm.com>
Reviewed-by: Zac Dover <zac.dover@proton.me>

Merge PR #64716 into main

* refs/pull/64716/head:

Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: Sachin Prabhu <sp@spui.uk>

container: pass CUSTOM_CEPH_REPO_URL thru container build script

Update the build.sh script such that the environment variable
CUSTOM_CEPH_REPO_URL is passed to the container build command
as a build argument. This allows a developer or system outside of the
ceph CI to build container images similar to those produced by the
CI but using custom RPM packages.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

container: add CUSTOM_CEPH_REPO_URL build argument

Add a CUSTOM_CEPH_REPO_URL build argument to the Containerfile.
This configures the container to consume ceph packages from the repo
file indicated by this URL. This allows constructing a container image
"the ceph way" without requiring the use of CI built packages.

Signed-off-by: John Mulligan <jmulligan@redhat.com>

test/librbd/fsx:: Initialize effective_size variable to prevent undefined behavior

Fixes: https://tracker.ceph.com/issues/72454
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>

Merge pull request #64952 from Matan-B/wip-matanb-rgw-fifo-ftbfs

neorados/cls/fifo/detail/fifo: include strtol.h

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

neorados/cls/fifo/detail/fifo: include strtol.h

https://github.com/ceph/ceph/commit/a2d26647c011274b61805f8ac17c3422e9b9b63c

ftbfs:
```
/home/jenkins-build/build/workspace/ceph-pull-requests/src/neorados/cls/fifo/detail/fifo.h:630:14: error: no member named 'parse' in namespace 'ceph'; did you mean 'pause'?
  630 |     auto n = ceph::parse<decltype(m.num)>(num);
      |              ^~~~~~~~~~~
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

Merge pull request #64773 from afreen23/freq-api-calls

mgr/dashboard: Stop rules api being polled on every page

Reviewed-by: Afreen Misbah <afreen@ibm.com>

Merge pull request #64905 from rhcs-dashboard/mem-leak

mgr/dashboard: fix memory leak in prometheus service

Reviewed-by: Afreen Misbah <afreen@ibm.com>

test/crimson/seastore/test_omap_manager: refactor list UT

Signed-off-by: Chanyoung Park <chaney.p@kakaoenterprise.com>

test/crimson/seastore/test_omap_manager: add innernode structural test

Signed-off-by: Chanyoung Park <chaney.p@kakaoenterprise.com>

Merge pull request #62915 from MaxKellermann/common_includes

common: include cleanup

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #61116 into main

* refs/pull/61116/head:

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #64822 into main

* refs/pull/64822/head:

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

mgr/dashboard: fix memory leak in prometheus service

Prometheus API calls in the Cluster Utilization call is subscribed in
the for loop multiple times but this is not properly unsubscribed. As we
stay in the dashboard page for longer time, it produces a significant
memory leak which eventually lags the UI. Attempting to fix it by
properly handling the subscription

Fixes: https://tracker.ceph.com/issues/72511
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: loading state for inventory card

show loading state when inventory details are being loaded. also don't
block the UI when inventories are not loaded.

Fixes: https://tracker.ceph.com/issues/72494
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #62488 from dparmar18/i70553

client: fix unmount hang after lookups

Reviewed-by: Venky Shankar <vshankar@redhat.com>

crimson/.../string_kv_node_layout: switch linear to binary search

Signed-off-by: Chanyoung Park <chaney.p@kakaoenterprise.com>

crimson/os/seastore: mkfs_managers() into coroutines

* txn is now created only once and not in each repeated
  iteration as before, with_repeat_trans_intr will reset the txn
  if needed.

* in order for us to return a seastar::future, we would have to handle
  each and every errorator. Instead, change the return value to
  errorator and let the caller handle the error.

* The lambda which is passed to with_trans_intr returns mkfs_iertr
  since this is an extended version of base_iertr.
  Introduce a mkfs_ertr for the reason above. Future commits would
  unify the new mkfs_ertr with seastore_mkfs_ertr.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore: cleanup final base_ertr users

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore: introduce with_repeat_trans_intr

Most of with_trans_intr use repeat_eagain so it makes sense to introduce
this wrapper and audit exisitng users (future PR).

Note: each repeated iteration should use a fresh txn if we failed
in-flight.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore/transaction: Cleanup journal_seq_t users

Seems like to logic is moved here:
```
  last_commit = start_seq;

  apply_backref_byseq(t.move_backref_entries(), start_seq);
  commit_backref_entries(std::move(backref_entries), start_seq);
```

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore/transaction_interruptor: include transaction.h

With the new design, only transaction_interruptor would include
transaction in order to use its methods (is_conflicted and future
commits)

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore/transaction: fix circular dependency

transaction_interruptor includes transaction and vice verca.
With the previous commits in this PR, we can not drop transaction
from including transaction_interruptor.

* move interrupt_cond_t to transaction_interruptor.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore/btree/fixed_kv_btree: use common base_ertr

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore/backref/btree_backref_manager: use common base_ertr

Signed-off-by: Matan Breizman <mbreizma@redhat.com>

crimson/os/seastore/object_data_handler: use base_iertr

Signed-off-by: Matan Breizman <mbreizma@redhat.com>