mgr/volumes: Add enctag to subvol

Add functionality to support enctag for subvols. This
will be useful for app or administrator to know which
master key to use.

Fixes: https://tracker.ceph.com/issues/69693
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Use new errno identifiers

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: use fscrypt headers provided by linux

Fixes: https://tracker.ceph.com/issues/68116
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Fix up a few things in read_sync path

Fix up a few things in read_sync path
1. File size may not be updated locally, do not check for trim read
2. Do not get_cap before RMW, each do_write takes care of having proper caps

Fixes: https://tracker.ceph.com/issues/69796
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Calculate len before prepare_data_read

Fixes: https://tracker.ceph.com/issues/69797
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: During fscrypt write, ensure we get Fr cap

During fscrypt write will require Fr cap. It's best to ensure
this requirement is handled at the cap level instead of at the mode
level. Otherwise, O_WRONLY flag won't be enforced.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

test/libcephfs: Add skips to tests where mount root is a subdir

When dir_prefix is a subdir, skip tests that expect
behavior of "/" to be on root of filesystem.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Use enc_name when linking a fscrypt enabled inode

Fixes: https://tracker.ceph.com/issues/64163
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Do not encrypt '.' or '..'

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Use symlink_plain for decrypted fscrypt name

Use symlink_plain for only the value of a decrypted fscrypt
dname.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: fscrypt last block

Support two edge cases in fscrypt last block.

1. When fscrypt last block is not the first block
2. Make sure to clean up SaferCond, allowing for successive
   truncates utilizing lastblock.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Do not decrypt bl on trim read

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: set symlink dest to proper value

Fixes: https://tracker.ceph.com/issues/69442
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

test: Add function policy populator for reuse in fscrypt tests.

Fixes: https://tracker.ceph.com/issues/69161
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: change conditional to check for is locked.

Fixes: https://tracker.ceph.com/issues/64137
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

fuse client, fscrypt, test: Implement and create tests for S_ENCRYPTED in inode i_flags

This PR adds test for S_ENCRYPTED bit in the i_flags field of Inode.
The test implements 2 quering methods: using FS_IOC_GETFLAGS and STATX_ATTR_ENCRYPTED

Fixes: https://tracker.ceph.com/issues/64129
Author: Igor Golikov <igolikov@ibm.com>
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Use correct sizes on write_success

Fixes: https://tracker.ceph.com/issues/69302
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: clear logical size on open(O_TRUNC)

Fixes: https://tracker.ceph.com/issues/65613
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

test, client: implement tests for not supported FALLOC ops, use policy to determine the padding size for encrypted filename

Adding test for not supported FALLOC ops on the encypted tree
Using policy to determine the padding length for encrypted file names and symlinks

Fixes: https://tracker.ceph.com/issues/64162
https://tracker.ceph.com/issues/64131
Author: Igor Golikov <igolikov@ibm.com>
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Add fscrypt last block

Add logic to support fscrypt last block. Includes sending
truncated last block data (decrypted->trunc->encrypted)
from client to mds. The server then writes the last block
on successful truncate.

Fixes: https://tracker.ceph.com/issues/69160
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Ensure file_cache cap is released.

Fixes: https://tracker.ceph.com/issues/68798
Fixes: https://tracker.ceph.com/issues/68831
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Various fixes to fix multi-fuse client

Provide various fixes in which size used in
multi-fuse client tests.

Fixes: https://tracker.ceph.com/issues/68431
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

qa: Convert and create tests for libcephfs fscrypt

Convert existing tests to use teuthology framework.
Create tests to test N>1 fscrypt clients

Fixes: https://tracker.ceph.com/issues/66577
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Update fscrypt_file when mds info is newer

Fixes: https://tracker.ceph.com/issues/68233
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

qa: Add tests of interopability of fscrypt between fuse/kernel

Fixes: https://tracker.ceph.com/issues/66577
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

ObjectCacher: handle nullptr hole case

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

ObjectCacher: On RetryRead, ensure "hole" value is used

On C_RetryRead, ensure "hole" value instead of pointer
to a pointer is used to allow populating vector of holes.

Fixes: https://tracker.ceph.com/issues/67659
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: When calling update_inode_file_size, provide correct size

Fixes: https://tracker.ceph.com/issues/67559
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Use effective_size in eof read

Fixes: https://tracker.ceph.com/issues/67347
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client/FSCrypt: securely erase crypto key

Fixes: https://tracker.ceph.com/issues/64136
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Implement status for fscrypt key status

Fixes: https://tracker.ceph.com/issues/64130
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Add busy case on key removal

Fixes: https://tracker.ceph.com/issues/64159
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

fuse: enable ioctl on dir for fscrypt

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

src/test/libcephfs: add test cases for fscrypt key removal busy case

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

qa: Added workunits for testing problem snippets on rmw workloads

Fixes: https://tracker.ceph.com/issues/66038
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

qa: Allow fscrypt testing on fuse

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: fix non-encrypted case in read_sync

Only append pbl to bl if encrypted case.

Fixes: https://tracker.ceph.com/issues/65964
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: fscrypt rmw fails when endoff end of block or file
Fscrypt rmw fails when end of a write lines up with end of
a block or end of the file.

Fixes: https://tracker.ceph.com/issues/65745
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: During lookup of fscrypt symlink, use target fscrypt info.

During a lookup of fscrypt enabled symlink, use target fscrypt
info. This must be used because enc key for each file is derived
from master_key+nonce.

Fixes: https://tracker.ceph.com/issues/65615
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Fix logic in need read start/end to account for fscrypt.

Fix the logic in need read start/end. We need to make sure that a
whole block is read when a rmw is issued, regardless if it starts
at offset 0 or not. Change size that may be read from where offset
starts to the whole fscrypt block.

Fixes: https://tracker.ceph.com/issues/64819
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: After a rebase, no lock is necessary.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

Client: fix O_TRUNC issue

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Ensure symlink plaintext is set

When creating symlink, ensure symlink_plain is set in fscrypt and
non-fscrypt cases.

Fixes: https://tracker.ceph.com/issues/64691
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: rmw doesn't get_cap_ref like it should. After rmw,
it put_cap_ref and it didn't happen. Issue with bool need_read.

Fixes: https://tracker.ceph.com/issues/64307
Signed-off-by: Christopher Hoffman <choffman@redhat.com>

Populate add key return

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

test: Add test for semantics

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: On fscrypt enabled directories, only allow read/write to files
in dir when unlocked. Client should not be able to read/write the encrypted payload.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

fscrypt: Add tests for add key, remove key, set policy.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client: Allow setpolicy only on dir and when it is empty.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

Add multi user support on a unlock claim.

Signed-off-by: Christopher Hoffman <choffman@redhat.com>

client,test,osdc: add beginnings of fscrypt support

Signed-off-by: Yehuda Sadeh <ysadehwe@ibm.com>

Merge pull request #66122 from cbodley/wip-install-deps-gcc13

install-deps: warn about gcc older than 13 for el9 and later

Reviewed-by: David Galloway <david.galloway@ibm.com>
Reviewed-by: Zack Cerza <zack@redhat.com>

Merge pull request #66124 from rhcs-dashboard/fix-svc-create-issue

mgr/dashboard: fix oauth2-service creation UI error

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>

Merge pull request #64292 from VallariAg/wip-nvmeof-ns-load-balancing

qa: Improve scalability test

mgr/dashboard: fix oauth2-service creation UI error

While creating the service without providing the allowlist domain, the
UI fails with an error which is logged in the mgr log

```
Nov 05 04:11:56 ceph-node-00 ceph-mgr[1587]: [dashboard ERROR frontend.error] (https://192.168.100.100:8443/#/services/(modal:create)): Cannot read properties of null (reading 'split')
                                              TypeError: Cannot read properties of null (reading 'split')
                                                 at ServiceFormComponent.onSubmit (https://192.168.100.100:8443/src_bootstrap_ts.js:31997:74)
                                                 at ServiceFormComponent_Template_cd_form_button_panel_submitActionEvent_60_listener (https://192.168.100.100:8443/src_bootstrap_ts.js:34168:83)
                                                 at executeListenerWithErrorHandling (https://192.168.100.100:8443/node_modules_angular_core_fesm2022_core_mjs.js:26276:12)
                                                 at Object.wrapListenerIn_markDirtyAndPreventDefault [as next] (https://192.168.100.100:8443/node_modules_angular_core_fesm2022_core_mjs.js:26308:18)
                                                 at SafeSubscriber.__tryOrUnsub (https://192.168.100.100:8443/default-node_modules_rxjs__esm2015_internal_AsyncSubject_js-node_modules_rxjs__esm2015_intern-7c6e1a.js:960:10)
                                                 at SafeSubscriber.next (https://192.168.100.100:8443/default-node_modules_rxjs__esm2015_internal_AsyncSubject_js-node_modules_rxjs__esm2015_intern-7c6e1a.js:900:14)
                                                 at Subscriber._next (https://192.168.100.100:8443/default-node_modules_rxjs__esm2015_internal_AsyncSubject_js-node_modules_rxjs__esm2015_intern-7c6e1a.js:847:22)
                                                 at Subscriber.next (https://192.168.100.100:8443/default-node_modules_rxjs__esm2015_internal_AsyncSubject_js-node_modules_rxjs__esm2015_intern-7c6e1a.js:824:12)
                                                 at EventEmitter_.next (https://192.168.100.100:8443/default-node_modules_rxjs__esm2015_internal_AsyncSubject_js-node_modules_rxjs__esm2015_intern-7c6e1a.js:604:17)
                                                 at EventEmitter_.emit (https://192.168.100.100:8443/node_modules_angular_core_fesm2022_core_mjs.js:7069:13)
```

Fixes: https://tracker.ceph.com/issues/73717
Signed-off-by: Nizamudeen A <nia@redhat.com>

Merge pull request #65717 from ronen-fr/wip-rf-evesrionhash

osd: hash function for eversion_t

Reviewed-by: Samuel Just <sjust@redhat.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Alex Ainscow <aainscow@uk.ibm.com>

Merge PR #66112 into main

* refs/pull/66112/head:
doc/radosgw: Fix frontends.rst typo

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #66115 into main

* refs/pull/66115/head:
src/common/options/global.yaml.in: Fix BlueStore capitalization

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge pull request #64122 from kamoltat/wip-ksirivad-fix-msg-v2

msg/async/ProtocolV2: Speed up connection logic when peer restarts & Added Netsplit Grace Period

install-deps: remove old logic for unsupported centos/rhel 8

Signed-off-by: Casey Bodley <cbodley@redhat.com>

install-deps: warn about gcc older than 13 for el9 and later

function ensure_decent_gcc_on_rh() is supposed to log a warning when it
finds the default gcc version is older than the gcc toolset version
installed by ceph.spec.in

> Your GCC is too old. Please run following command to add DTS to your environment:

however, this function is only called if the dts_ver variable is
initialized, so set it to 13 for centos/rocky 9 and later to match
ceph.spec.in

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #65872 from aainscow/tracker_73184

Relax scrub of shard sizes for upgraded EC pools

Merge PR #66114 into main

* refs/pull/66114/head:
Revert "Merge pull request #65940 from Tom-Sollers/fixing_blaum_roth_for_ec_profiles"

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #66075 from rhcs-dashboard/bootstrap-fixes

mgr/dashboard: fixes for quick-bootstrap script

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>
Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #64961 from edwinzrodriguez/ceph-wip-72446

btree: Safeguard delete_leaf_node to prevent dereferencing null nodes

src/common/options/global.yaml.in: Fix BlueStore capitalization

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Revert "Merge pull request #65940 from Tom-Sollers/fixing_blaum_roth_for_ec_profiles"

This reverts commit a22278724154bfcf225c6fe420bd305424559e00, reversing
changes made to cf11d9dd6e1caf3ea398766953b91fde3e078100.

This was not QA'd or reviewed by SME for mon code changes.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #66087 into main

* refs/pull/66087/head:
doc/foundation: Update associate member representative

Reviewed-by: Joseph Mundackal <jmundackal@bloomberg.net>

Merge PR #65886 into main

* refs/pull/65886/head:

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #65884 into main

* refs/pull/65884/head:

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>

Merge PR #65882 into main

* refs/pull/65882/head:

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge PR #65876 into main

* refs/pull/65876/head:

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

doc/radosgw: Fix frontends.rst typo

Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>

Merge pull request #66097 from cbodley/wip-cmake-system-qatlib

cmake: fix checks for WITH_SYSTEM_QATLIB

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #66101 from mrVectorz/main

doc: Update the old ceph.com/community/ links to ceph.io/en/news/blog/

Merge pull request #65693 from vshankar/wip-73278

qa: ignore cluster warning (evicting unresponsive ...) with tasks/mgr-osd-full

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>

doc: Update the old ceph.com/community/ links to ceph.io/en/news/blog/

Signed-off-by: Marc Methot <mb.methot@gmail.com>

Merge pull request #66001 from baum/nvmeof-client-timer

nvmeof: refactor beacon timer for exact frequency timing with drift correction

Merge pull request #66073 from benaryorg/crushtool-tries-docs-improvement

doc: detailed explanation of set_choose_tries

doc: detailed explanation of set_choose_tries

- specifically call the *crushtool* output a histogram
- include a surface explanation of how PG placement calculation works
- more info on `choose_total_tries`
- small but complete example for explanatory purposes
- that way people can follow along locally and test out things

Signed-off-by: benaryorg <binary@benary.org>

Merge pull request #66060 from dudy-duce/doc-svc-discovery-auth

doc/cephadm/services: Clarify service discovery auth in monitoring.rst

Merge pull request #65942 from samarahu/d4n-add-local-rgw-option

Reviewed-by: Jesse Williamson <jwilliamson@suse.de>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>

doc/cephadm/services: Clarify service discovery auth in monitoring.rst

Signed-off-by: Dudy <dudythegoat@proton.me>

cmake: fix checks for WITH_SYSTEM_QATLIB

commit 30681236678c7ee006a699b658233388b0f884c8 introduced the cmake
option WITH_SYSTEM_QATLIB, but the checks were based on nonexistent
variable WITH_SYSTEM_QAT

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #65568 from rhcs-dashboard/table-check-fix

mgr/dashboard: fix missing checkbox icon in table

Reviewed-by: Ankush Behl <cloudbehl@gmail.com>

Merge pull request #66082 from rhcs-dashboard/fix-cve

mgr/dashboard: bump validator package to address vulnerability

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #66016 from NitzanMordhai/wip-nitzan-suites-perf-ci-disable-mira118-machine

tasks/cbt_performance: Tolerate exceptions during performance data up…

Merge pull request #65763 from gardran/wip-gardran-kill-get-omap-keys

os: eliminate ObjectStore::get_omap_keys method

Merge pull request #66083 from rhcs-dashboard/add-to-mailmap

Add Abhishek Desai into githubmap, organizationmap, mailmap

Merge pull request #65948 from jzhu116-bloomberg/wip-73540

rgw/multisite: check the local bucket's versioning status when replicating deletion from remote

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>

Merge pull request #65842 from h3nryc0ding/main

rgw: beast add ssl hot-reload

Reviewed-by: Mark Kogan <mkogan@ibm.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #66049 from rhcs-dashboard/table-overview-redirect

mgr/dashboard: generic redirect link for a table item

Reviewed-by: Naman Munet <nmunet@redhat.com>

Merge pull request #66084 from rhcs-dashboard/fix-multisite-action-btn

mgr/dashboard: fix icon alignment in navigation header

Reviewed-by: Nizamudeen A <nia@redhat.com>

Merge pull request #65796 from anthonyeleven/caps

src/pybind/mgr/cephadm: Improve text in module.py

Merge pull request #66088 from rhcs-dashboard/fix-cypress-dep-graph

mgr/dashboard: fix cephadm e2e dependency graph failure

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>

mgr/dashboard: bump validator package to address vulnerability

fixes: https://tracker.ceph.com/issues/73657

Signed-off-by: Naman Munet <naman.munet@ibm.com>

mgr/dashboard: fix icon alignment in navigation header

Fixes: https://tracker.ceph.com/issues/73665
Signed-off-by: Naman Munet <naman.munet@ibm.com>

mgr/dashboard: fixes for quick-bootstrap script

- bump kcli to fedora42
- modify ip logic
- generate cephadm binary if it doesn't exist

Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: fix missing checkbox icon in table

Fixes: https://tracker.ceph.com/issues/72961
Signed-off-by: Nizamudeen A <nia@redhat.com>

mgr/dashboard: generic redirect link for a table item

Fixes: https://tracker.ceph.com/issues/73638
Signed-off-by: Nizamudeen A <nia@redhat.com>