ceph-volume: fix the type mismatch, covert the tries and interval to int from string.
Fixes: https://tracker.ceph.com/issues/43186
Signed-off-by: dongdong tao <dongdong.tao@canonical.com>

Merge PR #31120 into master

* refs/pull/31120/head:
mds: Reorganize class members in MDSRank header

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: Cross sign button not working for some modals (#31783)

mgr/dashboard: Cross sign button not working for some modals

Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: sort monitors by open sessions correctly. (#31752)

mgr/dashboard: sort monitors by open sessions correctly.

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #31738 from ricardoasmarques/wip-41776

mgr/dashboard: Unable to set boolean values to false when default is true

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #31773 from idryomov/wip-krbd-snapshot-removal-test

qa: kernel.sh: update for read-only changes

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Don't use any xlf file when building the defaul… (#31727)

mgr/dashboard: Don't use any xlf file when building the default language

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge PR #31748 into master

* refs/pull/31748/head:
qa/standalone/test_ceph_daemon.sh: remove old vg before creating
qa/standalone/test_ceph_daemon.sh: sudo for untar
qa/standalone/test_ceph_daemon.sh: sudo for losetup etc
qa/standalone/test_ceph_daemon.sh: fix overwrites of temp files

Reviewed-by: Michael Fritch <mfritch@suse.com>

mgr/dashboard: Update dev documentation URL (#31724)

mgr/dashboard: Update dev documentation URL

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Use new ImageSpec class (#31622)

mgr/dashboard: Use new ImageSpec class

Reviewed-by: Kiefer Chang <kiefer.chang@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Dashboard can't handle self-signed cert on Graf… (#31129)

mgr/dashboard: Dashboard can't handle self-signed cert on Grafana API

Reviewed-by: Stephan Müller <smueller@suse.com>

mgr/dashboard: open files with UTF-8 encoding in Grafana check… (#31750)

mgr/dashboard: open files with UTF-8 encoding in Grafana checking script

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Cross sign button not working for some modals

Fixes: https://tracker.ceph.com/issues/42179
Signed-off-by: Ricardo Marques <rimarques@suse.com>

mds: Reorganize class members in MDSRank header

Fixes: https://tracker.ceph.com/issues/42465
Signed-off-by: Varsha Rao <varao@redhat.com>

qa/standalone/test_ceph_daemon.sh: remove old vg before creating

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/test_ceph_daemon.sh: sudo for untar

The deepsea.tgz tar contains actual device nodes for the OSD block devices
(not symlinks or files).  Must be root to untar.

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/test_ceph_daemon.sh: sudo for losetup etc

Signed-off-by: Sage Weil <sage@redhat.com>

qa/standalone/test_ceph_daemon.sh: fix overwrites of temp files

mktemp creates these files, so we have to pass --allow-overwrite (or
delete them after we get the unique name but before we write to them--this
is easier).

Broken by c7fe27a72a61d1345a66b8830fd17e7b922abd44

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31418 into master

* refs/pull/31418/head:
test: use distinct subvolume/group/snapshot names

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>

Merge PR #30553 into master

* refs/pull/30553/head:
qa/tasks: check if fs mounted in umount_wait

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31534 into master

* refs/pull/31534/head:
mds: fix invalid access of mdr->dn[0].back()

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31548 into master

* refs/pull/31548/head:
mds: properly evaluate unstable locks when evicting client

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #31531 into master

* refs/pull/31531/head:
cmake: link ceph-fuse against librt

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

qa: kernel.sh: update for read-only changes

Snapshot existence validation code was removed from krbd.  It was racy
and relied on having watch established for snapshots.

Fixes: https://tracker.ceph.com/issues/42916
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge PR #31745 into master

* refs/pull/31745/head:
qa/suites/rados/thrash-old-clients: skip TestClsRbd.mirror

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #31392 from dillaman/wip-42598

journal: fix flush by age and in-flight byte tracking

Reviewed-by: Mykola Golub <mgolub@suse.com>

mgr/dashboard: Use new ImageSpec class

This class should be used when dealing with RBD image specs.
It allows the creation of an ImageSpec given the names of the pool, image and
namespace. Alternatively you can also create one with an already existing
image spec string.

With it you keep the access to each individual component and can also convert it
to a well formated string.

Fixes: https://tracker.ceph.com/issues/42787
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: fix restored RBD image naming issue (#31590)

mgr/dashboard: fix restored RBD image naming issue

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #31758 from votdev/fix_typo

python-common: Fix typo in device type

Reviewed-by: Joshua Schmid <jschmid@suse.de>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Don't use any xlf file when building the default language

The build-i18n script no longer uses a xlf file when building the default
language. This means that we don't need to keep the messages.en-US.xlf in the repository anymore.

Fixes: https://tracker.ceph.com/issues/42693
Signed-off-by: Sebastian Krah <skrah@suse.com>

Merge pull request #31410 from jan--f/c-v-create-vgs-on-demand

ceph-volume: allow raw block devices everywhere

mgr/dashboard: Fixes removal of custom tags during pool edit (#31373)

mgr/dashboard: Fixes removal of custom tags during pool edit

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

python-common: Fix typo in device type

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #31751 from tchaikov/wip-run-tox.sh-python3

script/run_tox.sh: do not use python2 if we have python3

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

Merge pull request #31607 from tspmelo/wip-pool-pattern

mgr/dashboard: Add invalid pattern message for Pool name

Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: show "Rename" in header & button when renaming… (#31551)

mgr/dashboard: show "Rename" in header & button when renaming RBD snapshot

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: CephFS snapshot management UI (#30996)

mgr/dashboard: CephFS snapshot management UI

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31563 from tspmelo/wip-default-i18n

mgr/dashboard: Use default language when running "npm run build"

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

doc: update ceph-volume lvm prepare

Add option to pass raw physical devices everywhere, restructure a little
(bluestore section before filestore) and reword a few things.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

ceph-volume: make lvm report fields into constants

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

ceph-volume: api/lvm create or reuse a vg

This changes create_lv so one can pass the desired device and either a
VG with a name starting with ceph is re-used or a new one is created.
This commit also adds two new lvm primitives, making use of lvm's select
feature. The goal is to eventually avoid keeping a full list of lv's (or
vg's) around and query the lvm system as needed.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #31755 from tchaikov/wip-mgr-orch

mgr/orch: pass unicode string to ipaddress.ip_network()

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Joshua Schmid <jschmid@suse.de>

mgr/orch: pass unicode string to ipaddress.ip_network()

otherwise it complains like

>               raise e
E               AddressValueError: '10.1.1.10' does not appear to be an
IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead
of a unicode object?

orchestrator.py:94: AddressValueError

Signed-off-by: Kefu Chai <kchai@redhat.com>

pybind/mgr: add ipaddress to requirements.txt

ipaddress dependency was introduced in 16f281d2. because the tox testenv
is isolated from the host python env, the test env has no access to the
ipaddress python module installed by install-deps.sh. we have to install
ipaddress for python<3.3 for testing which uses `orchestrator.py`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Update dev documentation URL

Update the developer documentation URL because the
old one is only returning "403 Forbidden".

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: sort monitors by open sessions correctly.

Fixes: https://tracker.ceph.com/issues/42893
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

script/run_tox.sh: do not use python2 if we have python3

always use python3 for bootstrapping tox environment unless the tox-env
contains "py2" but not "py3"

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31574 from jan--f/c-v-refactor-get_devices

ceph-volume: make get_devices fs location independent

mgr/dashboard: open files with UTF-8 encoding in Grafana checking script

Fixes: https://tracker.ceph.com/issues/42892
Signed-off-by: Kiefer Chang <kiefer.chang@suse.com>

Merge PR #30698 into master

* refs/pull/30698/head:
common/options: Set bluestore min_alloc size to 4K

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #31638 into master

* refs/pull/31638/head:
mgr/pg_autoscaler: only generate target_* health warnings if targets set

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #31572 into master

* refs/pull/31572/head:
mgr/diskprediction_local: some debug output during predict (and self-test)

Reviewed-by: Kefu Chai <kchai@redhat.com>

qa/suites/rados/thrash-old-clients: skip TestClsRbd.mirror

Older versions have this test and fail it.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #31721 into master

* refs/pull/31721/head:
mgr/ssh: set UserKnownHostsFile

Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge pull request #31660 from ivancich/wip-add-some-const

rgw: add const correctness to some rest functions

Merge pull request #16715 from adamemerson/wip-I-Object!

osdc/Objecter: Boost.Asio (I object!)

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge PR #31672 into master

* refs/pull/31672/head:
mgr/orch: adapt update_mgr/mon funcs to new parsing
mgr/orch: improve commandline parsing

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Michael Fritch <mfritch@suse.com>

Merge PR #31679 into master

* refs/pull/31679/head:
ceph-daemon: add osd create test(s)

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/dashboard: unable to set bool values to false when default is true

Fixes: https://tracker.ceph.com/issues/41776
Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #31731 from varshar16/wip-doc-update-orchestrator_cli-url

mgr/orchestrator_cli: Update doc link in README

mgr/ssh: set UserKnownHostsFile

Set this to /dev/null, since we have StrictHostKeyChecking
off anyway.  This gets rid of messages to stderr every time we connect
to a new host.

Signed-off-by: Sage Weil <sage@redhat.com>

ceph-volume: refactor tests for refactored get_devices

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

ceph-volume: refactor get_devices, don't use os.path.realpath

Fixes: https://tracker.ceph.com/issues/42777
Signed-off-by: Jan Fajerski <jfajerski@suse.com>

journal: properly handle immediate overflow upon advancing object set

While the old set is being closed, additional IO can be queued up within
the old, closed objects while the in-flight IO settles. It's therefore
possible that the queued IO that is transferred from the old set to the
new set causes an immediate overflow of the new set.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: fix flush by age and in-flight byte tracking

The flush by age was always causing an immediate flush due to a
backwards comparison. Additionally, the in-flight byte tracker was
never decremented which caused premature closure of the journal
object.

Finally, there was a potential race condition between closing the
object and in-flight notification callbacks executing. Now we keep
the lock held for both closed and overflow callbacks to prevent
the small chance of a race.

Fixes: https://tracker.ceph.com/issues/42598
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

journal: simplify object recorder locker aggregation

Use move semantics and RAII to control the locking of the
per-object recorder lock.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #29399 from ofriedma/master

rgw: Fix expiration header does not return the earliest rule

Merge pull request #29937 from ofriedma/wip-ofriedma-days0

Add days0 to rgw lc

Merge pull request #31642 from trociny/wip-rbd-mirror-service-daemon-ns-support

rbd-mirror: add namespace support to service daemon

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #31627 from trociny/wip-pybind-rbd-snap_remove

pybind/rbd: provide snap remove flags

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #31608 from trociny/wip-mock-warnings

test/rbd_mirror: fix mock warnings

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #31606 from trociny/wip-test_rbd.py-tabs

test/pybind: inconsistent use of tabs and spaces in indentation

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: CephFS snapshot management UI

CephFS snapshots can now be created on a directory basis. Multiple
snapshots can be deleted at once.

Fixes: https://tracker.ceph.com/issues/41791
Signed-off-by: Volker Theile <vtheile@suse.com>
Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/orchestrator_cli: Update doc link in README

Update the orchestrator cli doc link to fix page not found error. Also remove
some unnecessary blank lines.

Signed-off-by: Varsha Rao <varao@redhat.com>

Merge pull request #31594 from smithfarm/wip-bri-regex-fix

backport-resolve-issue: narrow regular expression and read key/token from files

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #30336 from joscollin/wip-F40959-subvolume-uid-gid

mgr/volumes: uid, gid for subvolume create and subvolumegroup create commands

mgr/dashboard: show Rename in modal header & button when renaming RBD snapshot

Fixes: https://tracker.ceph.com/issues/42762
Signed-off-by: Alfonso Martínez <almartin@redhat.com>

Merge pull request #31546 from zjcmszh/execute

pybind/rados: add WriteOp::execute()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #30968 from majianpeng/osd-optimze-send_message_osd_cluster

osd: optimize send_message to peers

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #31550 from wido/issue-42761

systemd: Wait 5 seconds before attempting a restart of an OSD

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #31623 from badone/wip-tracker-42387-extend-admin-socket-timeout

common/admin_socket: Increase socket timeouts

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #31241 from zjcmszh/add_rados_create_pool2

pybind/rados/rados.pyx: improve Rados.create_pool()

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31617 from Devp00l/wip-42776

mgr/dashboard: Fix pool renaming functionality

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

ceph-daemon: add osd create test(s)

Signed-off-by: Michael Fritch <mfritch@suse.com>

Merge pull request #31515 from drunkard/master

doc: typo fix in doc/dev/dev_cluster_deployement.rst: s/hostanme/hostname/

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31541 from BlackLotus/master

added a remark to always use powers of two for pg_num

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31685 from liewegas/fix-mgr-shutdown

mgr: _exit(0) from signal handler even if we are standby

Merge pull request #31695 from liewegas/wip-cd-simpler-bootstrap

ceph-daemon: bootstrap: make --output-* args optional

mgr/orch: adapt update_mgr/mon funcs to new parsing

update_mgr and update_mon (and their correspinding orch-backend
implementation) are adapted to the new parsing parsing method
introduced in ff9093873.

Signed-off-by: Joshua Schmid <jschmid@suse.de>

mgr/dashboard: Remove title from sparkline tooltips (#31645)

mgr/dashboard: Remove title from sparkline tooltips

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #31689 from tchaikov/wip/workunits/cephtool/test.sh

qa/workunits/cephtool/test.sh: a handful fixes

Reviewed-by: Sage Weil <sweil@redhat.com>

Merge pull request #30397 from MahatiC/wip-ref-counting

librbd: add reference counting

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

qa/suites/rados: whitelist health warnings

in cephtool/test.sh, we

ceph fs set cephfs inline_data {1,0}

so the health check fails when the test ends, like

mon.a (mon.0) 3498 : cluster [WRN] Health check failed: 1 filesystem
with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED)" in
cluster log

so, before we remove the test, we need to whitelist this warning

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #31688 from smithfarm/wip-bci-clean-print

backport-create-issue: flush line before overprinting

Reviewed-by: Tiago Melo <tmelo@suse.com>

backport-resolve-issue: read Redmine key and GitHub token from file

Thanks to Tiago Melo for inspiring me to do this.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/workunits/cephtool/test.sh: set min_{bytes,pgs}_per_osd for reweight

otherwise ceph-mgr will refuse to perform reweight

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits/cephtool/test.sh: create pool with power-of-2 pg num

otherwise wait_for_health() fails like:

wait_for_health:  ceph health detail
HEALTH_WARN 1 pool(s) have non-power-of-two pg_num
[WRN] POOL_PG_NUM_NOT_POWER_OF_TWO: 1 pool(s) have non-power-of-two pg_num
    pool 'rbd' pg_num 10 is not a power of two
../qa/workunits/cephtool/../../standalone/ceph-helpers.sh:1613: wait_for_health:  return 1

the failure was found when testing test_mon_pg().

this behavior was introduced by 6e46b1c0e50ad377d5db058e0452b0d956b3fad6

Signed-off-by: Kefu Chai <kchai@redhat.com>

qa/workunits/cephtool/test.sh: use "ceph compact" in test_mon_deprecated_commands()

both "compact" and "scrub" are now cli command, so we cannot issue them
using "tell" anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

backport-resolve-issue: narrow regular expression

The regular expression used to extract tracker URLs from the PR body
was too generous. When there are two URLs on the same line and the
second is a tracker URl, the regex picked up both of them. For example,
the following string matched the regex:

"https://github.com/ceph/ceph/pull/26538, https://tracker.ceph.com/issues/41452"

This commit fixes the issue.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #31710 from votdev/ignore_ssh_config

mgr/ssh: Ignore ssh-config file

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>