Merge PR #36089 into master

* refs/pull/36089/head:
mds/OpenFileTable: add option that disables dirfrags prefetch
mds: use parent/d_name in cap_reconnect_t to speedup MDCache::open_ino()
client: encode inode's parent/d_name in cap_reconnect_t
mds: explicitly track files with client writeable ranges
mds: properly set CLIENTWRITEABLE flag for imported caps
mds: in-place update projected inode's client ranges
mds: only add inodes that clients want Frw or excl caps to open file table
mds/OpenFileTable: reduce anchor map lookup during prefetch
mds/OpenFileTable: merge dirfrag set into anchor map

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36887 into master

* refs/pull/36887/head:
qa: ignore expected mds failover message

Reviewed-by: Jeff Layton <jlayton@redhat.com>

Merge PR #36846 into master

* refs/pull/36846/head:
client: support getting ceph.dir.rsnaps vxattr
messages: fix decode version for snap_btime

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36777 into master

* refs/pull/36777/head:
osdc/Journaler: do not call onsafe->complete() if onsafe is 0

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36672 into master

* refs/pull/36672/head:
client: handle readdir reply without Fs cap

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36631 into master

* refs/pull/36631/head:
client: dump frag's repmap if dir_replicated is true

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36885 into master

* refs/pull/36885/head:
mon: ignore statfs requests on invalid poolid
qa: test for statfs on deleted file system
qa: move run_shell args to kwargs

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36881 from mhackett/wip-mhackett-47189

doc: document tuning of object gateway garbage collection

Merge pull request #36826 from agayev/fix-hmsmr-compilation

os/bluestore: Restore HM-SMR functionality on BlueStore

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #34653 from clwluvw/bucket-stats

rgw: Add bucket name to bucket stats error logging

Merge pull request #36857 from tchaikov/wip-crimson-conf

crimson/osd: support "dump_pgstate_history" command and update legacy conf vars

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

crimson/osd: improve the const correctness

it should suffice to pass a `const OSD&` to the ctor of `OsdStatusHook`

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: support "dump_pgstate_history" command

it is required by the "ceph_test_admin_socket_output" test.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/admin: make injected_args optional

this option for `injectargs` command is optional, so we should not
crash if it does not exist.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: do not check for experimental/filesystem anymore

seastar is only compatible with C++17 and C++20, so no need to check for
<experimental/filesystem> anymore. the C++ standard library should be
C++17 compatible.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: call config::update_legacy_vals()

so the legacy vals are populated with the changes received from conffile.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/config: make update_legacy_vals() public

so crimson can call it in `ConfigProxy::parse_config_files()` after calling
`get_config().parse_buffer()`. so the legacy vals are populated with the
changes received from conffile.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36891 from tchaikov/wip-less-warnings

crimson/mon,common: replace #warning with // TODO

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #36892 from tchaikov/wip-crimson-cleanups

crimson/osd: return EAGAIN if !can_serve_replica_read() and cleanups

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #36905 from sebastian-philipp/cephadm-pep8-test-migration

mgr/cephadm: PEP8tify test_migration.py

Reviewed-by: Adam King <adking@redhat.com>

Merge pull request #36676 from matthewoliver/cephadm_orch_ipv6

cephadm: auto wrap and unwrap ipv6 addresses

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #36896 from xxhdx1985126/wip-47133

crimson/pg: check acting set change before appending pglog

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/cephadm: PEP8tify test_migration.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #36877 from sebastian-philipp/mgr-module-fix-kwargs-cli-command

mgr/mgr_module.py: CLICommand: Fix parsing of kwargs arguments

Reviewed-by: Jan Fajerski <jfajerski@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36859 from tspmelo/wip-fix-onselect

mgr/dashboard: Fix stale table actions

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #36839 from dillaman/wip-46875

librbd: flush all incomplete in-flight IOs upon image close

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #36819 from vumrao/wip-vumrao-46663

mon/PGMap: add pg count for pools in the ceph df command

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

crimson/pg: check acting set change before appending pglog

Fixes: https://tracker.ceph.com/issues/47133
Signed-off-by: Xuehan Xu <xxhdx1985126@gmail.com>

Merge pull request #33852 from rosinL/wip-fix-enqueue-dead-qp-deadlock

msg/async/rdma: avoid enqueue_dead_qp deadlock

Reviewed-by: Peng Liu <liupeng37@baidu.com>

Merge pull request #36767 from tchaikov/wip-47061

test/osd/RadosModel: check retval of read op

Reviewed-by: Samuel Just <sjust@redhat.com>

crimson/osd: return EAGAIN if !can_serve_replica_read()

it's necessary for serving balanced read.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: check for primary out side of is_misdirected()

so we can check can_serve_replica_read() without checking primary again.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: encode OI with CEPH_FEATURES_ALL

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: use logger() instead of crimson::get_logger(...)

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: call PeeringState::get_peer_missing(peer)

as PeeringState::get_peer_missing(const pg_shard_t &peer) already checks
for the non-existent shard, and because `get_replica_recovery_order()`
never returns primary osd, it's safe to use
`PeeringState::get_peer_missing(peer)` instead of
`pg->get_peering_state().get_peer_missing().find(peer)`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: eliminate a local variable

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/osd/RadosModel: assure that tid seq is contiguous

before this change:

4:  finishing write tid 1 to gen81094472-1
4:  finishing write tid 2 to gen81094472-1
4:  finishing write tid 3 to gen81094472-1
4:  finishing write tid 5 to gen81094472-1
4:  finishing write tid 6 to gen81094472-1

after this change:

1:  finishing write tid 1 to gen81117124-1
1:  finishing write tid 2 to gen81117124-1
1:  finishing write tid 3 to gen81117124-1
1:  finishing write tid 4 to gen81117124-1
1:  finishing write tid 5 to gen81117124-1

more pleasant when reading the logging messages.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/osd/RadosModel: check retval of read op

* check racing read's return value
* check if racing read completes

Fixes: https://tracker.ceph.com/issues/47061
Signed-off-by: Kefu Chai <kchai@redhat.com>

test/osd/RadosModel: do not set done

we don't reuse TestOp, and `done` should have been reset in the
constructor.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/osd/RadosModel: use range-based for loop

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/osd/RadosModel: use in-class initiailization

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/osd: use stack for ObjectDesc::iterator::stack

better readability this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/mon: replace #warning with TODO

less distracting when fixing compiling error / warnings this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/admin_socket: replace #warning with TODO

less distracting when fixing compiling error / warnings this way

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36815 from sebastian-philipp/cephadm-pep8-module

mgr/cephadm: PEP8tify module.py

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

mgr/cephadm: PEP8tify module.py

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #36858 from adk3798/cephadm-47171

mgr/cephadm: Verify non-empty list in get_active_daemon functions

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #36849 from sebastian-philipp/cephadm-kick-when-monmap

mgr/cephadm: kick serve loop to trigger ceph.conf deployment

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #36753 from sebastian-philipp/mgr-cephadm-schedule-redeploy

mgr/cephadm: Make daemon actions asynchronous

Reviewed-by: Adam King <adking@redhat.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

mon: ignore statfs requests on invalid poolid

Otherwise, we'll see a SIGSEGV when trying to get the size in
get_pool_free_space.

Fixes: https://tracker.ceph.com/issues/47182
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: test for statfs on deleted file system

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: move run_shell args to kwargs

To avoid listing all run args.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35891 from changchengx/multi_stress_watch

test/multi_stress_watch: use internel API to connect cluster

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36868 from zhoufeng2021/removeunsed

msg/async: cleanup unused function local_deliver()

Reviewed-by: Kefu Chai <kchai@redhat.com>

test/osd/RadosModel: use insert_or_assign()

for better performance, and less repeatings.

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/osd/RadosModel: add more comments

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/osd/RadosModel: rename local variable

for better readability

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36888 from batrick/release-notes-union

doc: use union merge strategy for resolving conflicts in PendingRelea…

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36886 from batrick/librados-statfs-header

librados: remove unused header

Reviewed-by: Kefu Chai <kchai@redhat.com>

msg/async/rdma: avoid enqueue_dead_qp deadlock

In RDMADispatcher::handle_async_event, when case
IBV_EVENT_QP_LAST_WQE_REACHED matched, enqueue_dead_qp wants the lock,
result in deadlock here. Change enqueue_dead_qp to lockless.

Fixes: https://tracker.ceph.com/issues/44298
Signed-off-by: Chunsong Feng <fengchunsong@huawei.com>
Signed-off-by: luorixin <luorixin@huawei.com>

librados: remove unused headers

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc: use union merge strategy for resolving conflicts in PendingReleaseNotes

Generally, the conflict resolution performed manually is always to do a
union: automate this. This avoids pointless rebases of PRs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36701 into master

* refs/pull/36701/head:
client: make Client::open() pass proper cap mask to path_walk

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36641 into master

* refs/pull/36641/head:
mds : skip show_cache if level of log is lower than 7

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

qa: ignore expected mds failover message

Fixes: https://tracker.ceph.com/issues/47202
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #36882 into master

* refs/pull/36882/head:
qa/tests: Added "--force-priority"

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #36729 from changchengx/zero_malloc

crush: do not malloc for empty bucket and avoid member variables after memset() with 0

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36820 from tchaikov/wip-test-multi_stress_watch

test/multi_stress_watch: use condition_variable for level triggering

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #36832 from neha-ojha/wip-clean-temps

tools/osdmaptool.cc: add ability to clean_temps

Reviewed-By: Josh Durgin <jdurgin@redhat.com>

qa/tests: Added "--force-priority"

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

doc: document tuning of object gateway garbage collection
Fixes: https://tracker.ceph.com/issues/47189
Signed-off-by: mhackett <mhackett@redhat.com>

Merge pull request #36854 from idryomov/wip-krbd-default-pool-fix

rbd: fix default pool handling for krbd map/unmap

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/mgr_module.py: CLICommand: Fix parsing of kwargs arguments

Make parsing of keyword arguments a bit safer by
makeing sure the key is actually a valid parameter name

Fixes: https://tracker.ceph.com/issues/47185
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mon/PGMap: add pg count for pools in the ceph df command
Fixes: https://tracker.ceph.com/issues/46663
Signed-off-by: Vikhyat Umrao <vikhyat@redhat.com>

Merge pull request #36866 from yison/fix-core-dump-issue-of-sync-read

blk/spdk/NVMEDevice.cc: fix a core dump issue of sync read

Reviewed-by: Kefu Chai <kchai@redhat.com>

blk/spdk/NVMEDevice.cc: fix a core dump issue of sync read

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

msg/async: cleanup unused function local_deliver()

Signed-off-by: zhoufeng <zhoufeng2021@outlook.com>

Merge pull request #36802 from dillaman/wip-46916

librbd: expose internal plugin API to plugin libraries

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/cephadm: add test_daemon_check

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: Make daemon actions asynchronous

Fixes broken `ceph orch redeploy <current active mgr>`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: add scheduled_daemon_actions to HostCache

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: move fail_over from upgrade.py to MgrService

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: _wait_for_ok_to_stop: add missing f-prefix

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/cephadm: _create_daemon: remove usused parameter

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/orch: daemon redeploy: Fix missing default value

Fixes: https://tracker.ceph.com/issues/47035
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #36792 from sebastian-philipp/cephadm-pep8-inventory

mgr/cephadm: PEP8tify inventory.py

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #36848 from sebastian-philipp/cephadm-no-pep8-test_scheduling

mgr/cephadm: disable autopep8 for test_{spec,scheduling}.py

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #29489 from jan--f/cephfs-snap-schedule

pybind/snap_schedule: add initial interface

Merge pull request #34566 from X0Ken/master

os/bluestore: Support flock retry

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #36507 from matthewoliver/msgr_intercepter_step_enum

msgr: interceptor test step enum to ease readability

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #36762 from qwren/qat-doc-update

doc: the update for QAT Acceleration

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #35941 into master

* refs/pull/35941/head:
doc/mgr: Supplement missing keywords 'crash'

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

client: support getting ceph.dir.rsnaps vxattr

Fixes: https://tracker.ceph.com/issues/47168
Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>

messages: fix decode version for snap_btime

Signed-off-by: Yanhu Cao <gmayyyha@gmail.com>

mgr/cephadm: Verify non-empty list in get_active_daemon functions

The get_active_daemon functions for monitoring stack daemons
were just returning the first or last daemon in the given list
without checking the list actually contained any daemons

Fixes: https://tracker.ceph.com/issues/47171
Signed-off-by: Adam King <adking@redhat.com>

mgr/dashboard: Fix stale table actions

If we don't emit the select event, the action button can become stale and not
update when the data changes.

This fixes a regression introduced in 31e9bbb9b308277d2f89390d5ff62f13457d69cb

Fixes: https://tracker.ceph.com/issues/47175
Signed-off-by: Tiago Melo <tmelo@suse.com>

test/cli/osdmaptool/help.t: add clean-temps

Signed-off-by: Neha Ojha <nojha@redhat.com>

doc: add clean-temps to osdmaptool.rst

Signed-off-by: Neha Ojha <nojha@redhat.com>

tools/osdmaptool.cc: add ability to clean_temps

This is particularly useful for debugging purposes when clean_temps()
takes abnormally high amount of time due to flaws in crush rules etc.

Fixes: https://tracker.ceph.com/issues/47159
Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #36572 from dmick/wip-cephadm-doc

doc/mgr: Clean up orchestrator.rst (spell/punct, awkward wording)

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Varsha Rao <varao@redhat.com>