mgr/dashboard: Improve RBD form

Remove pools with '/' in the name and add RBD name validation.

Fixes: http://tracker.ceph.com/issues/38303
Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Replace encodeURI for encodeURIComponent

Fixes: http://tracker.ceph.com/issues/38303
Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge PR #26469 into master

* refs/pull/26469/head:
common/ceph_timer: stop timer's thread when it is suspended

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #26470 into master

* refs/pull/26470/head:
mgr/MgrStandby: include addrs in metadata

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #26479 into master

* refs/pull/26479/head:
pybind: move FLAGS into Flag class

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #26485 into master

* refs/pull/26485/head:
qa/suites/upgrade/luminous-x: force clone v1 format for final rbd python test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #26452 from tchaikov/wip-crimson-send-known-addresses

crimson/osd: send known addresses at boot

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #26450 from s0nea/wip-dashboard-readonly-config-options-2

mgr/dashboard: disallow editing read-only config options (part 2)

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #25233 from p-na/wip-pna-qos-pr

mgr/dashboard: Add support for managing RBD QoS

Reviewed-by: Alfonso Martínez <almartin@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Render error in pool edit dialog

Fixes: http://tracker.ceph.com/issues/38004
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard: Add support for managing RBD QoS

Fixes: http://tracker.ceph.com/issues/36191
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #26405 from jtlayton/wip-jlayton-orch

Allow rook to scale the mon count

Reviewed-by: Noah Watkins <noahwatkins@gmail.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Travis Nielsen <tnielsen@redhat.com>

Merge pull request #23221 from baixueyu/wip-rgw-cleanup-regex

rgw: delete some unused code about std::regex

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

crimson/osd: send known addresses in MOSDBoot

otherwise, the peers will drop the connection at seeing unmatched
address in banner.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: populate crc settings

we should populate messenger related settings before starting
messengers.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: listen on v1 addresses

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26254 from tchaikov/wip-crimson-cache

crimson: cache osdmap using LRU cache

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #26378 from votdev/issue_38267

mgr/dashboard: Module dashboard.services.ganesha has several lint issues

Reviewed-by: Tatjana Dehler <tdehler@suse.com>

rgw: delete some unused code about std::regex

Signed-off-by: Xueyu Bai <baixueyu@inspur.com>

cmake: add PGMap.cc to crimson-common

otherwise we have failures like:

/usr/bin/ld: ../../../lib/libcrimson-common.a(Message.cc.o): in function
`PGMapDigest::~PGMapDigest()':
/var/ssd/ceph/src/mon/PGMap.h:37: undefined reference to `vtable for
PGMapDigest'
/usr/bin/ld: ../../../lib/libcrimson-common.a(Message.cc.o): in function
`PGMapDigest::~PGMapDigest()':

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: set_callback in ctor

there is no need to do so in `start()`.

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/common: lookup using unordered_map first

assuming the unordered_map in LRU cache is faster.

inspired by #25480 by Radoslaw Zarzynski

Signed-off-by: Kefu Chai <kchai@redhat.com>

crimson/osd: cache osdmaps using LRU cache

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #25570 from cbodley/wip-rgw-cls-bi-list-log-level

cls/rgw: raise debug level of bi_log_iterate_entries output

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #26491 from ceph/revert-25926-wip-rgw-lc-prefix

Revert "rgw: lifcycle: don't reject compound rules with empty prefix"

Revert "rgw: lifcycle: don't reject compound rules with empty prefix"

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #26399 from jan--f/c-v-fix-hashable-py3

ceph-volume: make Device hashable to allow set of Device list in py3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #26401 from jan--f/c-v-fix-siez-format-py3

ceph-volume: implement __format__ in Size to format sizes in py3

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #26403 from jan--f/c-v-no-iteritems-py3

ceph-volume: remove iteritems instances

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #26463 into master

* refs/pull/26463/head:
script/ptl-tool: allow invocation within a subdirectory

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #26272 from ivancich/wip-log-excl-lock-busy-info

rgw: when exclusive lock fails due existing lock, log add'l info

Reviewed-By: Abhishek Lekshmanan <abhishek@suse.com>

qa/suites/upgrade/luminous-x: force clone v1 format for final rbd python test

Fixes: https://tracker.ceph.com/issues/38346
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #25926 from linuxbox2/wip-rgw-lc-prefix

rgw: lifcycle: don't reject compound rules with empty prefix

Merge pull request #26386 from votdev/improve_dashboard_module_opts

mgr/dashboard: Set MODULE_OPTIONS types and defaults

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #26226 from trociny/wip-rbd-re-sparsify

rbd: online re-sparsify of images

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #26242 from p-na/wip-pna-osd-purge

mgr/dashboard: Implement OSD purge

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

pybind: move FLAGS into Flag class

so we have shorter names, and don't need to import ceph_argparse
only for accessing the flags

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26431 from wjwithagen/wjw-fix-src_os_bluestore_KernelDevice.cc

os/bluestore: Only use F_SET_FILE_RW_HINT when available

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: Implement OSD purge

Removes the `remove` functionality from the UI.

Fixes: http://tracker.ceph.com/issues/35811
Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

Merge pull request #26440 from xiexingguo/wip-balancer-week

mgr/balancer: restrict automatic balancing to specific weekdays

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #26425 from trociny/wip-38315

journal: fix potential race when closing object recorder

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/balancer: restrict automatic balancing to specific weekdays

We have implemented similar semantics for the scrub behaviour.
Let's try to keep pace with that.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge PR #26353 into master

* refs/pull/26353/head:
osd/PG: print get/put under debug_refs
osd: clear priv on heartbeat_reset

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/MgrStandby: include addrs in metadata

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: print get/put under debug_refs

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #24805 into master

* refs/pull/24805/head:
qa/suite: add dedup test
src/tools: fix compile error (master version issue)
src/tools: add stats (fixed objects,total objects)
src/tools: make room for cdc
src/tools: make enhacned stats and interface class
src/tools: set timelimit and add signal handler to check progress
src/tools: use the slice thing and make parallel (chunk_scrub)
src/test: add max-thread test in test_dedup_tool.sh
src/tools: use the slice thing and make parallel
src/test: add chunk-scrub test in test_dedup_tool.sh
src/tools: add chunk-scrub op in dedup tool
src/cls/cas: add has_chunk op
src/test: add test_dedup_tool.sh
src/tools: initial works for dedup tool

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26462 into master

* refs/pull/26462/head:
options: update some descriptions and see_also fields.

Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>
Reviewed-by: Sage Weil <sage@redhat.com>

common/ceph_timer: stop timer's thread when it is suspended

otherwise the client could hang when shutting down

Fixes: https://tracker.ceph.com/issues/37766
Signed-off-by: Peng Wang <wp389@126.com>

script/ptl-tool: allow invocation within a subdirectory

Make ptl-tool.py work from a subdirectory inside the repo (e.g., build/).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #26187 into master

* refs/pull/26187/head:
mgr/balancer: balance pools with same crush_rule in batch
osd/OSDMap: more improvements to upmap

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #26435 into master

* refs/pull/26435/head:
msg/async/ProtocolV2: assume peer has SERVER_NAUTILUS for encoding v2 frame

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #26275 from trociny/wip-mksnap-errmsg

rados: provide user with more meaningful error message

Reviewed-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #26454 from batrick/i37955-followup

ceph.in: use correct module for cmd flags

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26446 from tchaikov/wip-mgr-plugins-are-arch-indep

cmake,rpm,deb: install mgr plugins into /usr/share/ceph/mgr

Reviewed-by: Boris Ranto <branto@redhat.com>

Merge pull request #26449 from tchaikov/wip-monc-cleanup

mon/MonClient: drop my_addr

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

options: update some descriptions and see_also fields.

Signed-off-by: Mark Nelson <mnelson@redhat.com>

rpm: mark mgr plugins "noarch"

as they are architecture independent.

Signed-off-by: Kefu Chai <kchai@redhat.com>

debian: s|lib/ceph/mgr|share/ceph/mgr|

as the mgr plugins are installed into $prefix/share/ceph/mgr now.

Signed-off-by: Kefu Chai <kchai@redhat.com>

rpm: s|lib/ceph/mgr|share/ceph/mgr|

as the mgr plugins are installed into $prefix/share/ceph/mgr now.

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: rename CMAKE_INSTALL_{FULL_}?PKGLIBDIR

rename CMAKE_INSTALL_{FULL_}?PKGLIBDIR to
CEPH_INSTALL_{FULL_}?PKGLIBDIR. in order to differentiate them from the
paths not ended with "ceph".

Signed-off-by: Kefu Chai <kchai@redhat.com>

cmake: install mgr plugins into $prefix/share/ceph

as mgr plugins are architecture independent source files. see also
/usr/share/cmake-*/

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge PR #26455 into master

* refs/pull/26455/head:
qa/suites/upgrade/mimic-x/stress-split: drop pglog_hardlimit test
qa/suites/upgrade/mimic-x/stress-split: update for msgr2
qa/suites/upgrade/mimic-x/parallel: update for msgr v2

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #26416 from dzafman/wip-fix-test-loops

test: Limit loops waiting for force-backfill/force-recovery to happen

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #26448 into master

* refs/pull/26448/head:
osd: do not send peers really old maps
osd: build_incremental_map_msg: recover if we are missing an incremental
osd: build_incremental_map_msg: behave if we have latest full but not incremental

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge PR #26345 into master

* refs/pull/26345/head:
ceph-osd: be helpful about upgrade gate
mon: record 'min_mon_release' file and prevent startup if it's too old
mon/Monitor: do not join cluster that is >2 releases old
mon/Elector: respect monmap min_mon_release
mon: check min_mon_release during probe
mon/MonmapMonitor: increase min_mon_release when full quorum is upgraded
mon: maintain quorum_min_mon_release
mon/MonMapMonitor: set initial min_mon_release
monmaptool: add --set-min-mon-release
monmaptool: don't spew usage on any error
mon/MonMap: add min_mon_release field

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge PR #26456 into master

* refs/pull/26456/head:
mgr/PyModuleRegistry: avoid null deref from get_progress_events

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

qa/suites/upgrade/mimic-x/stress-split: drop pglog_hardlimit test

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/upgrade/mimic-x/stress-split: update for msgr2

Signed-off-by: Sage Weil <sage@redhat.com>

qa/suites/upgrade/mimic-x/parallel: update for msgr v2

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #26212 from linuxbox2/wip-rgw-master-nosuch

rgw: check for non-existent bucket in RGWGetACLs

osd: clear priv on heartbeat_reset

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #25739 into master

* refs/pull/25739/head:
mds: handle leftover reconnect information
client: split large reconnect into multiple messages
mds: update MClientReconnect encoding

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #26423 into master

* refs/pull/26423/head:
mgr/volumes: create/delete fs subvolumes
pybind/ceph_volume_client: allow volume_client to

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #26434 into master

* refs/pull/26434/head:
mds: return string_view for type
mds: reorg LogEvent def
mds: pass LogEvent as unique_ptr

Reviewed-by: Zheng Yan <zyan@redhat.com>

ceph.in: use correct module for cmd flags

argparse != ceph_argparse

Introduced-by: 19fbdb3ffde5bcb73792b1d822fe3f8d0407d490
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/PyModuleRegistry: avoid null deref from get_progress_events

active_modules may be null.

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #26445 into master

* refs/pull/26445/head:
mgr: hold GIL while generating a typed option value
mgr/ActivePyModules: do not touch PyFormatter without the GIL

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>

mgr/dashboard: check config option updatability also for bulk create

Fixes: https://tracker.ceph.com/issues/34528
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: add missing test case for 60bda9ae429

Fixes: https://tracker.ceph.com/issues/34528
Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mon/MonClient: drop my_addr

the only consumer of this member variable is MgrStandby, and MgrStandby
does have access to the messenger. so let it get the IP address
directly.

Signed-off-by: Kefu Chai <kchai@redhat.com>

osd: do not send peers really old maps

We may receive a message that sat in a queue for a while with a low
priority and is tagged with an older epoch.  Don't send a bunch of old
maps that we have already sent the peer.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: build_incremental_map_msg: recover if we are missing an incremental

If we are building a history and are missing an incremental, continue with
a full map in its place.

Generally speaking, an OSD should *always* have a full map stored, but
may not have the corresponding incremental for various reasons.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: build_incremental_map_msg: behave if we have latest full but not incremental

Fixes 47a3f63ef71522d6b9572e81674598dd3e19df9d and
follow-on 57a13adc8d0e34b4bb1a4022eacbb3de2636df53

Fixes: http://tracker.ceph.com/issues/38330
Signed-off-by: Sage Weil <sage@redhat.com>

mgr/dashboard: Set MODULE_OPTIONS types and defaults

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr: hold GIL while generating a typed option value

Drop the GIL while looking up the module and option value.  Retake
it before calling get_typed_option_value(), where we generate the
Python objects for the value.

Fixes: http://tracker.ceph.com/issues/38123
Fixes: http://tracker.ceph.com/issues/38204
Fixes: http://tracker.ceph.com/issues/38292
Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ActivePyModules: do not touch PyFormatter without the GIL

Signed-off-by: Sage Weil <sage@redhat.com>

cmake: move install(.. ./mgr) down to mgr

this helps to modularize the CMakeLists.txt

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #26437 from tspmelo/wip-npm-update-19-2

mgr/dashboard: Update npm packages

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

rbd: online re-sparsify of images

Signed-off-by: Mykola Golub <mgolub@suse.com>

pybind/rbd: add image sparsify method

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: online re-sparsify of images

Signed-off-by: Mykola Golub <mgolub@suse.com>

librados_test_stub: enable sparse_read

Although it actually does not do sparse read, enabling it makes
possible to link the same test code with both librados_test_stub
and real librados.

Signed-off-by: Mykola Golub <mgolub@suse.com>

librados_test_stub: add cls_cxx_replace

Signed-off-by: Mykola Golub <mgolub@suse.com>

cls/rbd: add sparsify method

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #26429 from rhcs-dashboard/fix-toast-notifications

mgr/dashboard: fix: toast notifications hiding utility menu

Reviewed-by: Sebastian Krah <skrah@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #26334 from votdev/fix_typo

mgr/dashboard: Fix typo in NoOrchesrtatorConfiguredException class name

Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Update prettier to 1.16

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Update npm packages

Signed-off-by: Tiago Melo <tmelo@suse.com>

mds: return string_view for type

To avoid unnecessary heap allocation.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: reorg LogEvent def

To follow code-style guidelines and protect some members.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>