mds: finish implementation of ephemeral pins

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 703e4b01fa83f81cda4fdc46b91f24a5e92f95f0)

mds: do string equality comparison

The string::find method would return true for ceph.dir.pin even for the
other ephemeral pin xattr names. For this reason, it was never possible
to actually turn ephemeral pins on!

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 21872bdb118c5eef4c2cfdadcc62737bb9471022)

mds: add ephemeral pinning for subtrees

This PR introduces inode xattrs export_ephemeral_random and
export_ephemeral_distributed which enables two different metadata
distribution strategies - the first being suitable for a more depthwise
scaling of metadata (height of the tree keeps increasing) and the latter
for horizontal scaling (many subtrees under a single parent).
export_ephemeral_distributed applies is not hierarchical. Any direct
descendant directory (i.e. a child directory) has an ephemeral export
pin applied to it according to a consistent hash of the child directory
inode number. export_ephemeral_distributed is hierarchical like
"export_pin". Any CDir loaded into the cache may be ephemerally pinned
to a random rank. Like "export_ephemeral_distributed", the random rank
is determined by a consistent hash.

The metadata distribution strategies are facilitated by using John
Lamping and Eric Veach's Jump Consistent Hashing as the consistent hash
algorithm. This hashing algorithm eliminates the need to store the data
structures representing the consistent hash cluster state and performs
as well as Akamai's original implementation providing a fairly uniform
distribution. This algorithm only works for distributed systems with
numbered buckets (nodes) arranged in ascending order and cluster resizes
does not produce any holes in the arrangement of nodes i.e (0, 1, 2, 3)
--[removing node 1]--> (0, 1, 2). CephFS satisfies these conditions as
the MDSs are arranged as numbered ranks and cluster modifications does
not produce any holes in the resulting arrangement of ranks.

Fixes: https://tracker.ceph.com/issues/41302
Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ced15ed7ef70ff832d4bebedecb89944276b0395)

Conflicts:
src/mds/MDSRank.cc

mds: trim pinned and empty subtrees

Before export (and ephemeral) pinned subtrees are stuck in cache
forever.

Add qa test for checking export pinned directories can be trimmed.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 5eea6761cc7e5950810cb0a625d49bd3f788183c)

mds: refactor remove_subtree

Search each map only once as necessary.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 747687f5dec8adc86abafb367275373bb44d96e9)

mds: allow export of pinned directory if empty

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 4749e5abb16a38b34ed63d11dbcd8bf3831b3f13)

mds: reduce subtree processing verbosity

and some mild loop refactoring.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c0271eea75eb83de2e38219f6dcc6597d41a5653)

mds: skip export of empty directories

Note: empty as in no cached sub-entries.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 570e6cfbcfc98f877a0e21cf536dfc826a4f9d15)

mds: remove frozen export pin from queue

The export already belongs on the node id so there's no need to keep it
in the queue.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 70ba356bbdd5c10979f9ff6a630de23d8304fb35)

mds: simplify for loop construction

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit d3158bf11ea1ea4934b12f48f6c373d912072c31)

mds: add debug messages for export queue processing

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 461fad8f807bdc01fbb19d79934573ee94d5712a)

qa: refactor _wait_subtree and _get_subtree

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit c0506230653b0c190d4793feb77ca197b45d85ee)

Conflicts:
qa/tasks/cephfs/mount.py

qa: use status from wait_for_daemons

Avoid an extra `fs dump` call.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit ddb18091a64b29bb69aafa2941cda01e3f749cf5)

qa: quietly print json output from asok commands

Pretty print output once. Use --format=json so the stdout on teuthology
is not pretty printed, taking hundreds of lines.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit f451c43aca58945ca5d4646bf057e009deb0a787)

Merge branch 'wip-46095-octopus' into octopus - from  https://github.com/ceph/ceph/pull/35685

octopus: mon: Warn when too many reads are repaired on an OSD

Reviewed-by: Neha Ojha <nojha@redhat.com>

test: flush_pg_stats() ignore OSDs that don't respond to getting sequence

This eliminates bogus errors in the logs and returned from flush_pg_stats()

Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 41322eaa62be0de20fdedb44d5817a3f1916ab5e)

mgr: Warn when too many reads are repaired on an OSD

Include test case
Configurable by setting mon_osd_warn_num_repaired (default 10)
Ignore new health warning with random eio injection test

Fixes: https://tracker.ceph.com/issues/41564
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 661996d4342c427209b1eae4b0247f8210a00fc3)

Merge pull request #35715 from dzafman/wip-46115

octopus: tools: Add statfs operation to ceph-objecstore-tool

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35550 from neha-ojha/wip-reduce-noise-octopus

octopus: qa/tasks/ceph_manager.py: dump more useful info before failing

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35136 from shyukri/wip-45209-octopus

octopus: monitoring: alert for pool fill up broken

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #35830 from guits/wip-46251-octopus

octopus: ceph-volume: add dmcrypt support in raw mode

Merge pull request #35753 from smithfarm/wip-wipe-prn-octopus

octopus: doc: PendingReleaseNotes: clean slate for 15.2.5

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge PR #35809 into octopus

* refs/pull/35809/head:
qa/tasks/vstart_runner.py: be python3 compatible
doc/dev/developer_guide: use python3 to launch vstart_runner.py
pybind/mgr/dashboard/run-backend-api-tests.sh: use python3 by default

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

15.2.4

ceph-volume: remove unused function

This function is never called in raw context, let's remove it.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 20e8286f5e1b79678b17659272daa1e28e763618)

ceph-volume: add raw testing coverage

This commit adds testing against `ceph-volume raw` subcommand.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 7bb811c3b6ec25ca34734131115cee531439f0ff)

ceph-volume: add dmcrypt support in raw mode

This commit adds the dmcrypt support in `ceph-volume raw` mode.

Note about `ceph-volume raw list` change:
Given `lsblk -J` (json output) option isn't available on all OS, I came up with
adding '--inverse' option to the existing command which allows us to get the
mapper devices list in that command output. Not listing root devices containing
partitions shouldn't have side effect since we are in `ceph-volume raw`
context.

example:
running `lsblk --paths --nodeps --output=NAME --noheadings` doesn't allow to
get the mapper list because the output is like following :

$ lsblk --paths --nodeps --output=NAME --noheadings
/dev/sda
/dev/sdb
/dev/sdc
/dev/sdd

the dmcrypt mappers are hidden because of the `--nodeps` given they are
displayed as a dependency.

$ lsblk --paths --output=NAME --noheadings
/dev/sda
|-/dev/mapper/ceph-3b52c90d-6548-407d-bde1-efd31809702f-sda-block-dmcrypt
`-/dev/mapper/ceph-3b52c90d-6548-407d-bde1-efd31809702f-sda-db-dmcrypt
/dev/sdb
/dev/sdc
/dev/sdd

adding `--inverse` is a trick to get around this issue, the counterpart is that
we can't list root devices if they contain at least one partition but this
shouldn't be an issue in `ceph-volume raw` context given we only deal with
raw devices.

Fixes: https://tracker.ceph.com/issues/45959
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 159c30d29f009c07e51a4bd7cd5ecea8cf6c8739)

qa/tasks/vstart_runner.py: be python3 compatible

differentiate `str` and `bytes` instances, and drop python2 support from
vstart_runner.py, as we've moved to python3 already

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit bedea85019076999a205c421bbf593ab8ab0641b)

doc/dev/developer_guide: use python3 to launch vstart_runner.py

as vstart_runner.py is compatible with python3, and so is the tests
under `qa/` directory.

and use `pip` to install teuthology, simpler this way.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c6e48ef0f024b4a3f4b457367544d773197933d4)

pybind/mgr/dashboard/run-backend-api-tests.sh: use python3 by default

since vstart_runner.py is now py3, let's use python3 by default

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d54f3ea43e05c2823c813e9687194b0b1fe15ffe)

Merge pull request #35807 from cbodley/wip-qa-rgw-ceph-octopus

octopus: qa/rgw: run external tests against ceph-octopus branch

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

octopus: qa/rgw: run external tests against ceph-octopus branch

Signed-off-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #35801 from yuriw/wip-yuriw-rgw-octopus

qa/test: fixed s3tests, changed to `force-branch: ceph-octopus`

Reviewed-by: Casey Bodley <cbodley@redhat.com>

qa/test: fixed s3tests, changed to `force-branch: ceph-octopus`

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #35780 from sebastian-philipp/octopus-qa-cephad-upgrade-no-rgw

octopus: qa/cephadm/upgrade: Exclude RGW for now

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #35774 from theanalyst/octopus-rgw-cors-fixes

octopus: rgw: sanitize newlines in s3 CORSConfiguration's ExposeHeader

Reviewed-By: Josh Durgin <jdurgin@redhat.com>
Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

qa/cephadm/upgrade: use quay.ceph.io

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit ab1d42c991037741064f35a9e881c7a45ef8b6c4)

qa/cephadm/upgrade: Exclude RGW for now

tasks/cephadm.py gained RGW support very recently and
I'm now facing a dilemman:

* Either we set the upgrade start to 15.2.4 and thus
  no longer upgrade from an old version, or
* Disable RGW upgrade for now.

I think doing both would be optinal, but for now, let's
disable RGW, in order to keep the coverage for everything
else.

Fixes: https://tracker.ceph.com/issues/46157
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
(cherry picked from commit 4c5b20f9168d2d7ea40f48702f1bc9fb999bbce3)

rgw: sanitize newlines in s3 CORSConfiguration's ExposeHeader

the values in the <ExposeHeader> element are sent back to clients in a
Access-Control-Expose-Headers response header. if the values are allowed
to have newlines in them, they can be used to inject arbitrary response
headers

this issue only affects s3, which gets these values from an xml document

in swift, they're given in the request header
X-Container-Meta-Access-Control-Expose-Headers, so the value itself
cannot contain newlines

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Reported-by: Adam Mohammed <amohammed@linode.com>

doc: PendingReleaseNotes: clean slate for 15.2.5

All of these Pending Release Notes have been included in the official
15.2.4 Release Notes, so keeping them in this file any longer would be
counterproductive.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #35625 from kshtsk/wip-46061-octopus

octopus: qa/tasks/cephadm: setup site based container registry

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #35517 from votdev/wip-45952-octopus

octopus: mgr/dashboard: Show labels in hosts page

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>

Merge pull request #35498 from ifed01/wip-ifed-hybrid-alloc-octopus

octopus: os/bluestore: introduce hybrid allocator

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

Merge pull request #35463 from bk201/wip-45836-octopus

octopus: monitoring: fixing some issues in RBD detail dashboard

Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35446 from smithfarm/wip-45682-octopus

octopus: os/bluestore: fix large (>2GB) writes when bluefs_buffered_io = true

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #35236 from smithfarm/wip-45676-octopus

octopus: qa/workunits: Remove redundant Xenial cmake3 requirements

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Brad Hubbard <bhubbard@redhat.com>

Merge pull request #34849 from rhcs-dashboard/wip-45346-octopus

octopus: mgr/dashboard: Asynchronous unique username validation for User Component

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #35718 from callithea/wip-46136-octopus

octopus: mgr/dashboard: fix autocomplete input backgrounds in chrome and firefox

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ishan Rai <ishanrai05@gmail.com>

Merge pull request #35135 from shyukri/wip-45557-octopus

octopus: mgr/dashboard: Prevent dashboard breakdown on bad pool selection

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #35706 from tspmelo/wip-45541-octopus

octopus: mgr/dashboard: Fix random E2E error in mgr-modules

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

Merge pull request #35693 from rhcs-dashboard/wip-45727-octopus

octopus: mgr/dashboard: grafana panels for rgw multisite sync performance

Reviewed-by: Ernesto Puertat <epuertat@redhat.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

mgr/dashboard: fix autocomplete input backgrounds in chrome and firefox

Fixes: https://tracker.ceph.com/issues/46109
Signed-off-by: Ishan Rai <ishanrai05@gmail.com>
(cherry picked from commit 01df08a49d7d06c7688cc1bcb5eac1fac402c5da)

tools: Add statfs operation to ceph-objecstore-tool

Fixes: https://tracker.ceph.com/issues/46064
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 19054ceb43bb2f56dadfe5462d546f2ee1de9263)

Merge pull request #35670 from kotreshhr/wip-46001-octopus

octopus: mgr/volumes: Add snapshot info command

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge pull request #35671 from kotreshhr/wip-45849-octopus

octopus: mgr/volumes: Create subvolume with isolated rados namespace

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35600 from smithfarm/wip-46013-octopus

octopus: qa/vstart_runner: update vstart_runner.LocalRemote.sh

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35555 from Vicente-Cheng/wip-45851-octopus

octopus: mds: flag backtrace scrub failures for new files as okay

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35554 from Vicente-Cheng/wip-45848-octopus

octopus: qa/tasks/vstart_runner.py: add def mountpoint parameter

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Xiubo Li <xiubli@redhat.com>

Merge pull request #35451 from smithfarm/wip-45846-octopus

octopus: tests: cephfs: run modprobe beforehand so that /sys/fs/fuse/connections is never absen

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>t

Merge pull request #35450 from smithfarm/wip-45845-octopus

octopus: src/client/fuse_ll: compatible with libfuse3.5 or higher

Reviewed-by: Xiubo Li <xiubli@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35449 from smithfarm/wip-45842-octopus

octopus: cephfs: fuse: add the '-d' option back for libfuse

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #34998 from smithfarm/wip-45476-octopus

octopus: cephfs-shell: Change tox testenv name to py3

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #35616 from Vicente-Cheng/wip-45888-octopus

octopus: qa/tasks/cephfs/test_scrub.py: use umount_wait to avoid possible ceph-fuse daemon stuck

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35448 from smithfarm/wip-45838-octopus

octopus: mds: cleanup uncommitted fragments before mds goes to active

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Fix random E2E error in mgr-modules

This test failed at random times when it tried to find the new value of pool_ids
in the balancer module.
This happened because the value of pool_ids is automatically reverted by ceph,
so in some situations when we tried to read the new value,
it was already reverted and failed.

Enhanced the tests to be able to use any text input, not only the ones with
empty default values.

Fixes: https://tracker.ceph.com/issues/45445
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit 08bb5a851f729b01c7cf69f662588903fd7c14bc)

Merge pull request #35268 from callithea/wip-45710-octopus

octopus: mgr/dashboard: redesign the login screen

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #35269 from callithea/wip-45368-octopus

octopus: mgr/dashboard: displaying Service detail inside table

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: grafana panels for rgw multisite sync performance

* RGW sync perf. counters are now exposed through grafana panels.
* Sync Performance tab is only shown if rgw realm is detected.
* Prometheus module: added metrics suitable for prometheus consumption (from existing ones, not replacing for backward compatibility).

Fixes: https://tracker.ceph.com/issues/45310
Signed-off-by: Alfonso Martínez <almartin@redhat.com>
(cherry picked from commit cf4ff7d2f03bc285a3fae3f27577333f11dab58a)

 Conflicts:
   src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-bucket-form/rgw-bucket-form.component.spec.ts
   src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-bucket-form/rgw-bucket-form.component.ts

   - Backport conflicts resolved in these files.

Merge pull request #35679 from tspmelo/wip-46111-octopus

octopus: mgr/dashboard: Fix e2e chromium binary validation

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Fix e2e chrome and chromium binaries verification

Fixes: https://tracker.ceph.com/issues/46110
Signed-off-by: Tiago Melo <tmelo@suse.com>
(cherry picked from commit e4b8d7eac352eb31fd17e420521aa2c1ce44453f)

Conflicts:
src/pybind/mgr/dashboard/run-frontend-e2e-tests.sh
There was an extra square bracket in octopus that caused the conflict.
This was manually fixed, since the commit that removed it will not be
backported.

Merge pull request #35243 from smithfarm/wip-45707-octopus

octopus: mgr/dashboard: Fix redirect after changing password

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/volumes: Create subvolume with isolated rados namespace

1. Add --namespace-isolated option to 'subvolume create' command
   to create subvolume in a separate RADOS namespace
2. Add "pool_namespace" field to 'subvolume info' command
   which displays the rados namespace if set else empty string

Fixes: https://tracker.ceph.com/issues/45289
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 7396880d4096b3a2a813b8ebcdcdf3082ee98b23)

mgr/volumes: Add snapshot info command

The following command is added

"ceph fs subvolume snapshot info <vol_name> <sub_name> <snap_name> [<group_name>]"

The output is in json format with following fields

    created_at: time of creation of snapshot in the format "YYYY-MM-DD HH:MM:SS:ffffff"
    data_pool: data pool the snapshot belongs to
    has_pending_clones: "yes" if snapshot clone is in progress otherwise "no"
    protected: "yes" if snapshot is protected otherwise "no"
    size: snapshot size in bytes

Fixes: https://tracker.ceph.com/issues/45237
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 171930ef773ee6630c76eb9581c725594fb4e884)

Merge pull request #35270 from callithea/wip-45232-octopus

octopus: mgr/dashboard: Show table details inside the datatable

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35452 from shyukri/wip-45783-octopus

octopus: ceph-volume: add and delete lvm tags in a single lvchange call.

mgr/dashboard: Asynchronous unique username validation for User Component

Implements an asynchronous validation for the username field in the Create User form which immediately display an error message if the username already exists.

Fixes: https://tracker.ceph.com/issues/36375
Signed-off-by: Nizamudeen <nia@redhat.com>
(cherry picked from commit c4dfe20c08ecb9e50b7cb7bef6eb603e04d56387)

mgr/dashboard: update background image in about modal box

Signed-off-by: Ishan Rai <ishanrai05@gmail.com>
(cherry picked from commit 80d5ad81fa740bd3b9f597632f546949700dc0ec)

mgr/dashboard: redesign the login screen

fixes: https://tracker.ceph.com/issues/35689
Signed-off-by: Ishan Rai <ishanrai05@gmail.com>
(cherry picked from commit 6b35508ef87cf5c94f6290e5752e36af3acab9bf)

Merge pull request #35444 from smithfarm/wip-45882-octopus

octopus: Objecter: don't attempt to read from non-primary on EC pools

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>

Merge pull request #35442 from smithfarm/wip-45775-octopus

octopus: osd: make "missing incremental map" a debug log message

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #35382 from jschmid1/wip-45880-octopus

octopus: osd: add --osdspec-affinity flag

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #35291 from callithea/wip-45738-octopus

octopus: mgr/dashboard: Proposed About Modal box

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #35152 from callithea/wip-45370-octopus

octopus: mgr/dashboard: ECP modal enhancement

Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #35151 from callithea/wip-45468-octopus

octopus: monitoring: Fix "10% OSDs down" alert description

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Patrick Seidensal <pnawracay@suse.com>

Merge pull request #35447 from smithfarm/wip-45773-octopus

octopus: cephfs: vstart_runner: set mounted to True at the end of mount()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #35256 from Vicente-Cheng/wip-45680-octopus

octopus: mgr/volumes: Fix subvolume create idempotency

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

fuse: update to newer FUSE_USE_VERSION

The build was failing for me against fuse-devel v3.9.1. The prototype
for fuse_ll_ioctl was wrong, as it was expecting the old-style one with
signed int args.

In newer libfuse versions, the prototype varies based on
FUSE_USE_VERSION. Update to a newer FUSE_USE_VERSION value to ensure
that we use the newer ioctl prototype. This also means that we need to
handle a new prototype for fuse_session_loop_mt as well.

While we're in here, move the definition of FUSE_USE_VERSION to
ceph_fuse.h so we have the definition in one place. This does mean we
need to reorganize the includes in a few places.

Fixes: https://tracker.ceph.com/issues/45866
Signed-off-by: Jeff Layton <jlayton@redhat.com>
(cherry picked from commit cfbc0fb48a9880dabe8f332e9bf67d81867ae198)

Merge pull request #35531 from bk201/wip-45921-octopus

octopus: mgr/dashboard: remove space after service name in the Hosts List table

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

qa/tasks/cephadm: setup site based container registry

Add containers registry mirror for 'docker.io' to the config
file /etc/containers/registries.conf

Since we need site based config, so each teuthology instance
could have own containers' mirror, store corresponding data
in /etc/teuthology.yaml under 'overrides/cephadm' task specific
section.

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
(cherry picked from commit d8edde0c53b80d01c3797c521c5adfc2ac0a99bb)

qa/tasks/vstart_runner: make remote.sh return str as default

To address the issue in https://github.com/ceph/teuthology/pull/1459.

Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 624944503e7c6aa246331f4d9f291cf714296380)

qa/tasks/cephfs/test_scrub.py: use umount_wait to avoid ceph-fuse stuck

If the ceph-fuse client need to flush the caps and does sync wait,
the umount() will just return successfully, then the netns container
will be destroyed and the network will not be reachable, but the
ceph-fuse daemon is still stucked and waiting for the flush caps ack.

This will cause the ceph-fuse daemon get stuck forever and if the
mds daemons get restarted, it will try to reconnect the clients,
but the stucked ceph-fuse daemnon won't reply to it, because it is
not reachable any more.

Fixes: https://tracker.ceph.com/issues/45665
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit a7423b2286154a4403e586b5148984649f988a03)

qa/cephfs: add FUSE module before running mount -t fusectl

Also, change timeout from 15 minutes to 30 seconds for the command
"mount -t fusectl xxx xxx" since 15 minutes is too much as per Zheng.

Fixes: https://tracker.ceph.com/issues/45304
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 2d7c32997b9e5db5a13f5ed6e117ac9f71dac370)

qa/tasks/cephfs/fuse_mount.py: by python3 compatible

pass `StringIO()` to capture stdout whose value will be interpreted as a
string later on

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit c5bd318de65866b4992314c2fb0e82090b4d2979)

qa/vstart_runner: update vstart_runner.LocalRemote.sh

Commit 9f6c764f10f replaces remote.run calls by remote.sh without
updating the definition of vstart_runner.LocalRemote.sh which breaks the
cephfs tests when executed locally.

Fixes: https://tracker.ceph.com/issues/44579
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 2cc0ee709c36eabe03311a00b72295da468bccf4)

Conflicts:
qa/tasks/vstart_runner.py
- trivial

Merge pull request #35364 from tchaikov/octopus-py3

octopus: migrate qa/ to Python3

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>

qa: add .teuthology_branch with `master` contents

This is follow up change for:

    teuthology-suite: automate -t argument default value
    https://github.com/ceph/teuthology/pull/1490

Signed-off-by: Kyr Shatskyy <kyrylo.shatskyy@suse.com>
(cherry picked from commit 07cc36dcd264b66a7f3a19687b336737a772c2f5)

qa/tasks/rbd_fio: unbreak after the conversion from StringIO

Fix a bad typo in commit db7ae8eff60a ("qa/tasks/rbd_fio: get rid of
StringIO for py3").

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit c3f4f1d6604749a51c552b868ef373f321139005)

qa/tasks/cephfs: add mount_wait() support to simplify the code

Mostly we should wait the mountpoint to get ready, especially for
the fuse mountpoint, sometimes it may take a few seconds to get
ready.

Fixes: https://tracker.ceph.com/issues/44044
Signed-off-by: Xiubo Li <xiubli@redhat.com>
(cherry picked from commit 5c24d91327f598ea9fee59ea87ddc6582d48bd96)

qa/tasks/cephfs: use StringIO for capturing str

if we don't expect non-utf8 in stdout, use StringIO instead of BytesIO.

see also d8d44ed1566b19eec055e07da2a0fed88fed4152

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit f10f86579684a0d1b8d99787b8e19709a64df7d6)

qa/rgw: partial revert of 'import with full path'

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit aa60edcec89b1538070b316c496e55bd0e15f966)