Merge PR #29321 into master

* refs/pull/29321/head:
Client: unlink dentry for inode with llref=0

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #29338 into master

* refs/pull/29338/head:
qa: ignore expected MDS_CLIENT_LATE_RELEASE warning

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #28825 into master

* refs/pull/28825/head:
qa: wait for kernel client death
qa: use hard_reset to reboot kclient

Reviewed-by: David Galloway <dgallowa@redhat.com>

Merge PR #29374 into master

* refs/pull/29374/head:
cephfs-shell: Fix flake8 line too long error
cephfs-shell: Fix flake8 line break after binary operator error
cephfs-shell: Fix flake8 continuation line over-indented for hanging indent error
cephfs-shell: Fix flake8 missing whitespace around arithmetic operator error
cephfs-shell: Fix multiple spaces after ':' flake8 error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #27780 from theanalyst/cls-timeindex-denc

cls: add timeindex types to ceph-dencoder

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29182 from dcasier/mgr-dashboard-ganesha-fix-cephfs-name

mgr/dashboard: ganesha: Specify the name of the filesystem (create_path)

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #29138 from Reperator/dashboard-pw-hash

mgr/dashboard: support setting password hashes

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

cephfs-shell: Fix flake8 line too long error

https://tracker.ceph.com/issues/40992
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fix flake8 line break after binary operator error

Fixes: https://tracker.ceph.com/issues/40992
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fix flake8 continuation line over-indented for hanging indent error

Fixes: https://tracker.ceph.com/issues/40992
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fix flake8 missing whitespace around arithmetic operator error

Fixes: https://tracker.ceph.com/issues/40992
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fix multiple spaces after ':' flake8 error

Fixes: https://tracker.ceph.com/issues/40992
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #29272 from votdev/issue_40925

mgr/dashboard: Various UI issues related to CephFS

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29216 from rafaelquint/configtest

mgr/dashboard: E2E test to verify Configuration editing functionality

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29324 from adk3798/typepages

mgr/dashboard: Explicitly type page variables

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #29371 from bk201/wip-40971

mgr/dashboard: fix MDS counter chart is not displayed

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #29341 from zhangbingyi1/master

test/librados: avoid residual crush rule after test case execution

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: fix MDS counter chart is not displayed

Unit of timetamp for performance counters is changed to nanoseconds [1].
Adapt this for CephFS MDS performance counters in backend.

[1]: https://github.com/ceph/ceph/pull/28882

Fixes: https://tracker.ceph.com/issues/40971
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

test/librados: avoid residual crush rule after test case execution
Fixes: http://tracker.ceph.com/issues/40970
Signed-off-by: Bingyi Zhang <zhangbingyi@umcloud.com>

Merge pull request #29348 from myoungwon/fix-wrong-class

osd: fix wrong arguments when dropping refcount

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29345 from majianpeng/bluestore-flush-waiter

os/bluestore: avoid unnecessary notify.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #29126 from xiexingguo/wip-clamp-ratio-to-0

mgr/progress: clamp pg recovery ratio to 0

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28012 from rouming/bootstrap-after-setuid

global/global_init: do first transport connection after setuid()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27985 from ifed01/wip-ifed-slow-omaps

tools/ceph-objectstore-tool: introduce list-slow-omap command.

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #29238 into master

* refs/pull/29238/head:
os/bluestore: it's better to erase spanning blob only once

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #29239 into master

* refs/pull/29239/head:
qa/tasks/ceph_manager.py: always use self.logger

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #29305 into master

* refs/pull/29305/head:
mgr: Reduce logging noise when handling commands

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29307 into master

* refs/pull/29307/head:
monmaptool: added --addv option to usage description

Reviewed-by: João Eduardo Luís <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29316 into master

* refs/pull/29316/head:
common,tools: make sure the destination buffer can handle the size of the string
src/tools: initialize variables before the goto statement
src/mount: check before dereference buf
src/crush: check before dereference out2
src/test: s/strcpy/strncpy

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29358 into master

* refs/pull/29358/head:
mgr/telemetry: salt osd ids too

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #29054 from dillaman/wip-40621

mgr/rbd_support: support scheduling long-running background operations

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #29361 from tchaikov/wip-qa-osd-pause

qa: fixes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

qa: wait for kernel client death

After sending the reboot command, we need to wait briefly for it to be
rebooted so that the kernel client doesn't voluntarily give up its Fb
cap.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use hard_reset to reboot kclient

power_off may allow the mounts to gracefully unmount. We don't want this if the
kclient is stuck or we desire the client to "disappear" and come back.

Fixes: http://tracker.ceph.com/issues/37681
Depends-on: https://github.com/ceph/teuthology/pull/1296
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28685 into master

* refs/pull/28685/head:
client: more precise CEPH_CLIENT_CAPS_PENDING_CAPSNAP
client: unify kicking cap flushes and kicking snapcap flushes
client: define helper function that sends flushsnap message
client: cleanup tracking of early kicked flushing caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28793 into master

* refs/pull/28793/head:
qa/cephfs: make run_shell() accept args as str too
qa/vstart_runner.py: ignores when source and destination are same
test_cephfs_shell: add a test for cd with arguments
test_cephfs_shell: add a test for cd with no arguments
test_cephfs_shell: group test methods
test_cephfs_shell: add a new method to return script output
test_cephfs_shell: allow running CephFS shell script
cephfs-shell: cd with no args shouldn't print an error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29064 into master

* refs/pull/29064/head:
mds: execute PurgeQueue on_error handler in finisher

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29302 into master

* refs/pull/29302/head:
cephfs: fix a memory leak

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29066 into master

* refs/pull/29066/head:
mds: Move inline functions with loops or switch statements to CInode.cc
mds: Reorganize class members in CInode header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29336 into master

* refs/pull/29336/head:
qa: wait for MDS to come back after removing it

Reviewed-by: Jos Collin <jcollin@redhat.com>

qa: ignore expected MDS_CLIENT_LATE_RELEASE warning

Fixes: http://tracker.ceph.com/issues/40968
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29359 from jmunhoz/wip-msite-az-tests

rgw: az: add archive zone tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard/qa: add more fields to report

to be in sync with #28847

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/rest/mgr-restful: whitelist OSDMAP_FLAGS

`rest/test-restful.sh` calls `test_mgr_rest_api.py`, which in turn
calls

```
    ('patch',  '/config/osd', {'pause': True}),
```

and rest module translates it to `ceph osd set key=pause`

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: az: add archive zone tests

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

mgr/telemetry: salt osd ids too

Better to fully obfuscate here.

This has a nice side-effect of assigning entity names that are *globally*
unique across the full telemetry data set, since the salts are unique and
sha1 is (sufficiently) collision-free.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #29330 into master

* refs/pull/29330/head:
mgr/telemetry: obscure entity_name with a salt

Reviewed-by: Dan Mick <dmick@redhat.com>

mgr/dashboard: Explicitly type page variables

Explicitly gives a type to page variables.
Helps make better use of Typescript's static type checking.
Undoes mistake from #29070.

Fixes: https://tracker.ceph.com/issues/40961
Signed-off-by: Adam King <adking@redhat.com>
Signed-off-by: Rafael Quintero <rquinter@redhat.com>

mgr: Reduce logging noise when handling commands

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #29237 from adk3798/usertest

mgr/dashboard: RGW user E2E Tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #29192 from ceph/dependabot/npm_and_yarn/src/pybind/mgr/dashboard/frontend/lodash-4.17.13

build(deps): bump lodash from 4.17.11 to 4.17.13 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Tiago Melo <tmelo@suse.com>

pybind/mgr: don't log exception when cannot find RBD task by id

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: fix wrong arguments when dropping refcount

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

Merge pull request #29198 from rhcs-dashboard/e2e-ci

mgr/dashboard: run e2e tests against prod build (jenkins job)

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29280 from rhcs-dashboard/bucket-tests

mgr/dashboard: RGW bucket creation when no placement target received

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Various UI issues related to CephFS

* Check if data is accessible, otherwise set grafanaId to undefined.
* Catch errors in 'Clients' tab and stop loading spinner. Display ViewCacheStatus Exception.
* Render the 'Enabled' column using the checkIcon cell template as other lists do.

Fixes: https://tracker.ceph.com/issues/40925
Signed-off-by: Volker Theile <vtheile@suse.com>

os/bluestore: avoid unnecessary notify.

Add parameter count waiting list for Onode::flush to avoid unnecessary
notify.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

common,tools: make sure the destination buffer can handle the size of the string

fix klocwork issues:
Width is not specified for 's' conversion specifier.
This can result in an overflow of the buffer
provided in argument 3 of a call to 'sscanf'

Signed-off-by: songweibin <song.weibin@zte.com.cn>

src/tools: initialize variables before the goto statement

CID 172143 (#15 of 15): Branch past initialization (PW.BRANCH_PAST_INITIALIZATION)
1. branch_past_initialization:
2. name_at_decl_position: variable "oid" (declared at line 220)
3. name_at_decl_position: variable "exkeys" (declared at line 241)
4. name_at_decl_position: variable "exvals" (declared at line 242)
5. name_at_decl_position: variable "safe" (declared at line 278)

Signed-off-by: songweibin <song.weibin@zte.com.cn>

src/mount: check before dereference buf

CID 180671 (#1 of 1): Dereference null return value (NULL_RETURNS)

Signed-off-by: songweibin <song.weibin@zte.com.cn>

src/crush: check before dereference out2

CID 174874 (#2 of 2): Dereference after null check (FORWARD_NULL)
30. var_deref_op: Dereference null pointer out2.

Signed-off-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #29264 from rjfd/wip-dashboard-fix-qa-rbd

mgr/dashboard: qa: fix RBD test when matching error strings

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Client: unlink dentry for inode with llref=0

when client get notification from MDS that a file has been deleted(via
getting CEPH_CAP_LINK_SHARED cap for inode with nlink = 0), if the client
hasnt touch the inode in the past, the ll_ref will be zero.

In previous code, we only call Client::unlink when ll_ref > 0, which is wrong
and will leave the dn in cache, keeping the caps and resulting the inode stays
in stray till the dn cache is dropped by kernel.

Under certain workload(write intensive and rotate intensive), this issue can cause
stray stacking to 100k+ and causeing huge space "leaking".

Fixes: http://tracker.ceph.com/issues/40960
Signed-off-by: Xiaoxi CHEN <xiaoxchen@ebay.com>

Merge pull request #28597 from markhpc/wip-bs-ind-cache

os/bluestore: Refactor Bluestore Caches

mds: execute PurgeQueue on_error handler in finisher

Fix: https://tracker.ceph.com/issues/40695
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>

Merge PR #28847 into master

* refs/pull/28847/head:
doc/mgr/telemetry: update
mgr/telemetry: move contact info to an 'ident' channel
mgr/telemetry: accept channel list to 'telemetry show'
mgr/telemetry: always generate new report for 'telemetry show'
mgr/telemetry: add 'device' channel and call out to devicehealth module
mgr/telemetry: add telemetry channel 'device'
mgr/telemetry: add separate channels

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge PR #29130 into master

* refs/pull/29130/head:
os/bluestore: Reduce unnecessary wakeups.
os/bluestore: Reduce unnecessary wakeups.

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #29221 into master

* refs/pull/29221/head:
mon/OSDMonitor: do not clobber pending xinfo during boot
osd/OSDMap: add is_dead() helper
mon/OSDMonitor: 'osd down ... --definitely-dead'
osd: send MOSDMarkMeDead when we are marked down
mon/OSDMonitor: implement MOSDMarkMeDead handling
mon/OSDMonitor: set dead_epoch for immediate failure (ECONNREFUSED)
osd/OSDMap: add dead_epoch to xinfo

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #29323 into master

* refs/pull/29323/head:
rocksdb: enable rocksdb_rmrange=true by default

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #29300 into master

* refs/pull/29300/head:
doc: fix youtube channel url
doc: fix urls in posix.rst

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28874 into master

* refs/pull/28874/head:
ceph-mds: dump all info of ceph_file_layout, InodeStoreBase, frag_info_t, inode_t

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29103 into master

* refs/pull/29103/head:
mgr/volumes: add `ceph fs subvolumegroup getpath` command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29166 into master

* refs/pull/29166/head:
mds: Reorganize class members in Capability header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge PR #29149 into master

* refs/pull/29149/head:
cephfs-shell: Fix flake8 E302 blank line error
cephfs-shell: Fix flake8 indentation error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28860 into master

* refs/pull/28860/head:
mds: Move scalar type initializations of CDir data members to its header
mds: Reorganize class members in CDir header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>

qa: wait for MDS to come back after removing it

Fixes: http://tracker.ceph.com/issues/40967
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mon/OSDMonitor: do not clobber pending xinfo during boot

If we have a pending xinfo change and also process a boot message, we
should not throw out the pending xinfo change.

This triggers when you mark an osd down and it sends both a MOSDMarkMeDead
and a new MOSDBoot message in quick succession: the first message sets
dead_epoch but the boot message processing clobbers it.

Signed-off-by: Sage Weil <sage@redhat.com>

osd/OSDMap: add is_dead() helper

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: 'osd down ... --definitely-dead'

Like 'osd down', but we also set dead_epoch.  Should only be used if we
know the process/host is already completely dead and unresponsive.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #29036 from mikechristie/rbd-nbd-add-nl-resize

rbd-nbd: add nl resize

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge PR #28955 into master

* refs/pull/28955/head:
doc: Update mount CephFS index

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29111 into master

* refs/pull/29111/head:
qa/mgr/progress: Update the test suite for progress module

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

osd: send MOSDMarkMeDead when we are marked down

Signed-off-by: Sage Weil <sage@redhat.com>

mon/OSDMonitor: implement MOSDMarkMeDead handling

Allow updates to the dead_epoch in osd_xinfo, so that the OSD can inform
us that they know they are down (as of a particular epoch).

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/telemetry: obscure entity_name with a salt

Signed-off-by: Sage Weil <sage@redhat.com>

pybind/mgr: handle duplicate rbd task commands

The 'ceph' CLI will duplicate commands within teuthology to test
the MONs idempotency. This shouldn't be required for the MGR module,
but we can keep a fixed set of completed tests to handle this
possible command replay.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

mds: Move inline functions with loops or switch statements to CInode.cc

Signed-off-by: Varsha Rao <varao@redhat.com>

mds: Reorganize class members in CInode header

Fixes: https://tracker.ceph.com/issues/40787
Signed-off-by: Varsha Rao <varao@redhat.com>

doc/mgr/telemetry: update

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/telemetry: move contact info to an 'ident' channel

Signed-off-by: Sage Weil <sage@redhat.com>

rocksdb: enable rocksdb_rmrange=true by default

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #29117 from tchaikov/wip-mgr-set_health_checks-unicode

mgr: check for unicode passed to "set_health_checks()"

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #29320 into master

* refs/pull/29320/head:
os/bluestore: remove unused arg to _get_deferred_op()

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #29282 from yuriw/wip-yuriw-upgrade-rwg-splits-master

qa/tests: added rgw into upgrade sequence to improve coverage - splits

os/bluestore: remove unused arg to _get_deferred_op()

Fixes: http://tracker.ceph.com/issues/40918
Signed-off-by: Sage Weil <sage@redhat.com>

qa: test case for new rbd background tasks

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr: rbd tasks now provide a dict to the progress events

This dict will include an origin key fixed to 'rbd_support' as well
as pool and image references.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr: mark progress events as failed if an error occurs

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr: support marking progress events as failed

The failed events can also include a failure message to indicate
the reason for the failure.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

pybind/mgr: new 'rbd task' background task management

Fixes: http://tracker.ceph.com/issues/40621
Signed-off-by: Jason Dillaman <dillaman@redhat.com>