pybind/rados: add get_id and get_name method in class Ioctx

Signed-off-by: Zheng Yin <zhengyin@cmss.chinamobile.com>

Merge pull request #29318 from tspmelo/wip-backport-component

script/ceph-backport.sh: Allow to set component label

Merge PR #29363 into master

* refs/pull/29363/head:
qa/suites/multimds/basic/tasks/ceph_test_snapshots: disable RECENT_CRASH
qa/suites/kcephfs/recovery/failover.yaml: disable RECENT_CRASH
qa/suites/fs/multifs/tasks/failover.yaml: disable RECENT_CRASH

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29185 into master

* refs/pull/29185/head:
os/bluestore: add tail length when caculating reserved buffer size
os/bluestore: record next_offs when offset is not sequential.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #29356 into master

* refs/pull/29356/head:
pybind/mgr/restful: use dict.items() for py3 compatibility

Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #29098 from liewegas/wip-unhexdump-c

src/script/unhexdump-C: script to reverse a 'hexdump -C' style hexdump

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #29394 from mkogan1/wip-rgw-bi-list-shard-id

rgw: add option to specify shard-id for "bi list" admin command

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge PR #29362 into master

* refs/pull/29362/head:
Update nautilus release for 14.2.2
Updated schedule with latest Nautilus release

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #28239 into master

* refs/pull/28239/head:
cephfs-shell: Add tox for testing with flake8

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #29386 from alfredodeza/wip-rm40987

ceph-volume  simple: better detection when 'type' file is not present

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

cephfs-shell: Add tox for testing with flake8

Fixes: https://tracker.ceph.com/issues/39947
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge PR #29331 into master

* refs/pull/29331/head:
common: add comment about pod memory requests/limits

Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

Merge pull request #29161 from valentinbajrami/improve-grep-regex

src/script/check_commands.sh: fix grep regex class range

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

rgw: add option to specify shard-id for "bi list" admin command

related to: https://tracker.ceph.com/issues/40700
Signed-off-by: Mark Kogan <mkogan@redhat.com>

src/script/check_commands.sh: fix grep regex class range

Correct the faulty regex class.

Signed-off-by: Valentin Bajrami <valentin.bajrami@gmail.com>

Merge pull request #28915 from sebastian-philipp/python-common-package-2

python-common:  Python common package

Reviewed-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #27336 from SUSE/wip-orchestrator-deepsea-iscsi-ganesha

mgr/deepsea: return ganesha and iscsi endpoint URLs

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #29387 from ceph/wip-ceph-volume-tests-no-dashboard

ceph-volume: when testing disable the dashboard

Merge pull request #29347 from smithfarm/wip-40974

rpm: improve ceph-mgr plugin package summaries

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #29346 from smithfarm/wip-40973

rpm: fdupes in SUSE builds to conform with packaging guidelines

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29367 from tchaikov/wip-doc-crush-rule

doc/rados/operations: crush_rule is a name

Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #29368 from tchaikov/wip-40618

qa/tasks/mgr/test_progress.py: s/ev/new_event/

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/suites/multimds/basic/tasks/ceph_test_snapshots: disable RECENT_CRASH

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #28804 into master

* refs/pull/28804/head:
mds: delay exporting directory whose pin value exceeds max rank id

Reviewed-by: Zheng Yan <zyan@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29339 into master

* refs/pull/29339/head:
client: don't report any vxattrs to listxattr

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: David Disseldorp <ddiss@suse.de>

Merge PR #29304 into master

* refs/pull/29304/head:
cephfs: avoid map been inserted by mistake

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29321 into master

* refs/pull/29321/head:
Client: unlink dentry for inode with llref=0

Reviewed-by: Zheng Yan <zyan@redhat.com>

ceph-volume: when testing disable the dashboard

ceph-ansible has started enabling the dashboard by
default as of https://github.com/ceph/ceph-ansible/pull/4268

Disabling the dashboard gets us around needing to run
the grafana_server role which is required for the dashboard
and not needed for ceph-volume functional tests.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #29338 into master

* refs/pull/29338/head:
qa: ignore expected MDS_CLIENT_LATE_RELEASE warning

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #28825 into master

* refs/pull/28825/head:
qa: wait for kernel client death
qa: use hard_reset to reboot kclient

Reviewed-by: David Galloway <dgallowa@redhat.com>

ceph-volume tests: ensure that better heuristics exist for objectstore detection

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume simple.activate better detect bluestore/filestore when type is not found

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Update nautilus release for 14.2.2

Nautilus v14.2.2 added to releases

Signed-off-by: JuanJose Galvez <juanjose.galvez@gmail.com>

Updated schedule with latest Nautilus release

Signed-off-by: JuanJose Galvez <juanjose.galvez@gmail.com>

Merge PR #29374 into master

* refs/pull/29374/head:
cephfs-shell: Fix flake8 line too long error
cephfs-shell: Fix flake8 line break after binary operator error
cephfs-shell: Fix flake8 continuation line over-indented for hanging indent error
cephfs-shell: Fix flake8 missing whitespace around arithmetic operator error
cephfs-shell: Fix multiple spaces after ':' flake8 error

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #27780 from theanalyst/cls-timeindex-denc

cls: add timeindex types to ceph-dencoder

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #29182 from dcasier/mgr-dashboard-ganesha-fix-cephfs-name

mgr/dashboard: ganesha: Specify the name of the filesystem (create_path)

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #29138 from Reperator/dashboard-pw-hash

mgr/dashboard: support setting password hashes

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

cephfs-shell: Fix flake8 line too long error

https://tracker.ceph.com/issues/40992
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fix flake8 line break after binary operator error

Fixes: https://tracker.ceph.com/issues/40992
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fix flake8 continuation line over-indented for hanging indent error

Fixes: https://tracker.ceph.com/issues/40992
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fix flake8 missing whitespace around arithmetic operator error

Fixes: https://tracker.ceph.com/issues/40992
Signed-off-by: Varsha Rao <varao@redhat.com>

cephfs-shell: Fix multiple spaces after ':' flake8 error

Fixes: https://tracker.ceph.com/issues/40992
Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #29272 from votdev/issue_40925

mgr/dashboard: Various UI issues related to CephFS

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29216 from rafaelquint/configtest

mgr/dashboard: E2E test to verify Configuration editing functionality

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #29324 from adk3798/typepages

mgr/dashboard: Explicitly type page variables

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #29371 from bk201/wip-40971

mgr/dashboard: fix MDS counter chart is not displayed

Reviewed-by: Ricardo Dias <rdias@suse.com>

pybind/mgr/restful: use dict.items() for py3 compatibility

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #29341 from zhangbingyi1/master

test/librados: avoid residual crush rule after test case execution

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: fix MDS counter chart is not displayed

Unit of timetamp for performance counters is changed to nanoseconds [1].
Adapt this for CephFS MDS performance counters in backend.

[1]: https://github.com/ceph/ceph/pull/28882

Fixes: https://tracker.ceph.com/issues/40971
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

test/librados: avoid residual crush rule after test case execution
Fixes: http://tracker.ceph.com/issues/40970
Signed-off-by: Bingyi Zhang <zhangbingyi@umcloud.com>

os/bluestore: add tail length when caculating reserved buffer size

Signed-off-by: Xiangyang Yu <penglaiyxy@gmail.com>

os/bluestore: record next_offs when offset is not sequential.

Signed-off-by: Xiangyang Yu <penglaiyxy@gmail.com>

Merge pull request #29348 from myoungwon/fix-wrong-class

osd: fix wrong arguments when dropping refcount

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #29345 from majianpeng/bluestore-flush-waiter

os/bluestore: avoid unnecessary notify.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #29126 from xiexingguo/wip-clamp-ratio-to-0

mgr/progress: clamp pg recovery ratio to 0

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #28012 from rouming/bootstrap-after-setuid

global/global_init: do first transport connection after setuid()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #27985 from ifed01/wip-ifed-slow-omaps

tools/ceph-objectstore-tool: introduce list-slow-omap command.

Reviewed-by: Sage Weil <sage@redhat.com>

qa/tasks/mgr/test_progress.py: s/ev/new_event/

as a follow-up fix for 5604ba4e

Fixes: http://tracker.ceph.com/issues/40618
Signed-off-by: Kefu Chai <kchai@redhat.com>

doc/rados/operations: crush_rule is a name

like
```
ceph osd pool set <pool-name> crush_rule <rule-name>
```
where `<rule-name>` is a string instead of a number.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #29238 into master

* refs/pull/29238/head:
os/bluestore: it's better to erase spanning blob only once

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #29239 into master

* refs/pull/29239/head:
qa/tasks/ceph_manager.py: always use self.logger

Reviewed-by: Samuel Just <sjust@redhat.com>

Merge PR #29305 into master

* refs/pull/29305/head:
mgr: Reduce logging noise when handling commands

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29307 into master

* refs/pull/29307/head:
monmaptool: added --addv option to usage description

Reviewed-by: João Eduardo Luís <joao@suse.de>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29316 into master

* refs/pull/29316/head:
common,tools: make sure the destination buffer can handle the size of the string
src/tools: initialize variables before the goto statement
src/mount: check before dereference buf
src/crush: check before dereference out2
src/test: s/strcpy/strncpy

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #29358 into master

* refs/pull/29358/head:
mgr/telemetry: salt osd ids too

Reviewed-by: Dan Mick <dmick@redhat.com>

Merge pull request #29054 from dillaman/wip-40621

mgr/rbd_support: support scheduling long-running background operations

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #29361 from tchaikov/wip-qa-osd-pause

qa: fixes

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

qa: wait for kernel client death

After sending the reboot command, we need to wait briefly for it to be
rebooted so that the kernel client doesn't voluntarily give up its Fb
cap.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use hard_reset to reboot kclient

power_off may allow the mounts to gracefully unmount. We don't want this if the
kclient is stuck or we desire the client to "disappear" and come back.

Fixes: http://tracker.ceph.com/issues/37681
Depends-on: https://github.com/ceph/teuthology/pull/1296
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

common: add comment about pod memory requests/limits

Written in private correspondence by Travis Nielson.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28685 into master

* refs/pull/28685/head:
client: more precise CEPH_CLIENT_CAPS_PENDING_CAPSNAP
client: unify kicking cap flushes and kicking snapcap flushes
client: define helper function that sends flushsnap message
client: cleanup tracking of early kicked flushing caps

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #28793 into master

* refs/pull/28793/head:
qa/cephfs: make run_shell() accept args as str too
qa/vstart_runner.py: ignores when source and destination are same
test_cephfs_shell: add a test for cd with arguments
test_cephfs_shell: add a test for cd with no arguments
test_cephfs_shell: group test methods
test_cephfs_shell: add a new method to return script output
test_cephfs_shell: allow running CephFS shell script
cephfs-shell: cd with no args shouldn't print an error message

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29064 into master

* refs/pull/29064/head:
mds: execute PurgeQueue on_error handler in finisher

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29302 into master

* refs/pull/29302/head:
cephfs: fix a memory leak

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29066 into master

* refs/pull/29066/head:
mds: Move inline functions with loops or switch statements to CInode.cc
mds: Reorganize class members in CInode header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #29336 into master

* refs/pull/29336/head:
qa: wait for MDS to come back after removing it

Reviewed-by: Jos Collin <jcollin@redhat.com>

qa/suites/kcephfs/recovery/failover.yaml: disable RECENT_CRASH

Signed-off-by: Sage Weil <sage@redhat.com>

qa: ignore expected MDS_CLIENT_LATE_RELEASE warning

Fixes: http://tracker.ceph.com/issues/40968
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa/suites/fs/multifs/tasks/failover.yaml: disable RECENT_CRASH

This test deliberately crashes daemons.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #29359 from jmunhoz/wip-msite-az-tests

rgw: az: add archive zone tests

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard/qa: add more fields to report

to be in sync with #28847

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/rest/mgr-restful: whitelist OSDMAP_FLAGS

`rest/test-restful.sh` calls `test_mgr_rest_api.py`, which in turn
calls

```
    ('patch',  '/config/osd', {'pause': True}),
```

and rest module translates it to `ceph osd set key=pause`

Signed-off-by: Kefu Chai <kchai@redhat.com>

rgw: az: add archive zone tests

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>

mgr/telemetry: salt osd ids too

Better to fully obfuscate here.

This has a nice side-effect of assigning entity names that are *globally*
unique across the full telemetry data set, since the salts are unique and
sha1 is (sufficiently) collision-free.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #29330 into master

* refs/pull/29330/head:
mgr/telemetry: obscure entity_name with a salt

Reviewed-by: Dan Mick <dmick@redhat.com>

mgr/dashboard: Explicitly type page variables

Explicitly gives a type to page variables.
Helps make better use of Typescript's static type checking.
Undoes mistake from #29070.

Fixes: https://tracker.ceph.com/issues/40961
Signed-off-by: Adam King <adking@redhat.com>
Signed-off-by: Rafael Quintero <rquinter@redhat.com>

mgr: Reduce logging noise when handling commands

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #29237 from adk3798/usertest

mgr/dashboard: RGW user E2E Tests

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #29192 from ceph/dependabot/npm_and_yarn/src/pybind/mgr/dashboard/frontend/lodash-4.17.13

build(deps): bump lodash from 4.17.11 to 4.17.13 in /src/pybind/mgr/dashboard/frontend

Reviewed-by: Tiago Melo <tmelo@suse.com>

pybind/mgr: don't log exception when cannot find RBD task by id

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

rpm: improve ceph-mgr plugin package summaries

Fixes: http://tracker.ceph.com/issues/40974
Signed-off-by: Nathan Cutler <ncutler@suse.com>

rpm: fdupes in SUSE builds to conform with packaging guidelines

Quoting from https://en.opensuse.org/openSUSE:Packaging_Conventions_RPM_Macros

"%fdupes is generally safe for use on /bin, /lib*, /usr (= %_prefix) and /sbin.
Do note however that files with same content but different ownership, when
hardlinked, will get the ownership of any of its content siblings."

Fixes: http://tracker.ceph.com/issues/40973
Signed-off-by: Nathan Cutler <ncutler@suse.com>

osd: fix wrong arguments when dropping refcount

Signed-off-by: Myoungwon Oh <omwmw@sk.com>

Merge pull request #29198 from rhcs-dashboard/e2e-ci

mgr/dashboard: run e2e tests against prod build (jenkins job)

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

cmake/modules/Distutils: do not add ${name}-clone if already added

if `distutils_install_module("foo" ...)` is called mutiple times with
different python version, `foo-clone` will be added multiple times as a
custom target. which is not allowed:

add_custom_target cannot create target "foo-clone" because another
target with the same name already exists.

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph.spec, debian: Add python-common to ceph-common

Co-authored-by: Tim Serong <tserong@suse.com>
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

client: don't report any vxattrs to listxattr

The convention with kernel filesystems is to not report vxattrs when
listxattr is called. Doing this can throw a wrench to archiving tools
that will attempt to restore files with xattrs intact, only to find
that some of them can't be stored.

Remove the code that prints out vxattrs in listxattr. With this we
also don't need the "hidden" flag in the vxattr definitions.

Also fix up the existing testcases to account for the change in
behavior.

Fixes: https://tracker.ceph.com/issues/40965
Signed-off-by: Jeff Layton <jlayton@redhat.com>

cmake: Add python-common

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Co-authored-by: Kefu Chai <kchai@redhat.com>

python-common: Move Drive Group tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>