Merge pull request #21005 from rjfd/wip-mgr-fix-error-handling

mgr: fixes python error handling

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #21024 from s0nea/wip-dashboard-return-features-list

mgr/dashboard: Convert the RBD feature names to a list of strings

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #21004 from s0nea/wip-dashboard-skip-datapool-testcase

mgr/dashboard: skip data pool testcase for none-bluestore clusters

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20873 from sebastian-philipp/dashboard_v2-simple-html-debug

mgr/dashboard: Add minimalistic browsable API

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21047 from rjfd/wip-mgr-smart-fix-py3

mgr/smart: fix python3 module loading

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #21027 from cbodley/wip-qa-swift-force-branch

qa/tasks/swift: add support for the "force-branch" configurable.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

mgr/dashboard: New API tests  for `perf_couters`

Added OSD and MDS tests.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Add minimalistic browsable API

Also provides a simple HTML form to POST
data to a `RESTController`'s `create()` method.

Also added ENABLE_BROWSABLE_API setting to the dashboard

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Renamed `types` as it clashes with std lib

This will lead to a name clash, when importing `types`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/smart: fix python3 module loading

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr: fixes python error handling

The current `handle_pyerror` function implementation relies in the
`traceback.format_exception_only` python function to format the
exception object. The problem is that this python function might also
raise an exception. This commit fixes it by enclosing that python
function call in try...catch block.

Fixes: http://tracker.ceph.com/issues/23406
Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #19439 from chardan/jfw-wip-14363-normalize-rm

rgw, mon: normalize delete/remove in admin console (cleanup 22444)

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #21011 from tspmelo/wip-submit-btn

mgr/dashboard: Add submit button component

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21032 from simransinghal/fixtypo

rgw: Fix typo in comment

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #21020 from trociny/wip-vstart-no-mon-config

stop.sh: use --no-mon-config when trying to unmap rbd devices

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21036 from xiexingguo/wip-wrong-tip

os/bluestore: s/bluefs_total/bluefs_free/

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #21007 from rjfd/wip-dashboard-fix-py3

mgr/dashboard: fix dashboard python 3 support

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #21006 from rjfd/wip-dashboard-fix-auth-py3

mgr/dashboard: fixed password generation in Auth controller

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20865 from sebastian-philipp/dashboard_v2_pool-create

mgr/dashboard: Add Pool-create to the backend

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: convert RBD features to a list of strings

Until now the API returns the features provided by an RBD as one comma
separated string. It's more usable for the frontend to receive a list
of strings instead.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

os/bluestore: s/bluefs_total/bluefs_free/

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

src: Fix typo

Replace "dont" with "don't". "Dont" is not same as "Do not" or "Don't".

Signed-off-by: Simran Singhal <singhalsimran0@gmail.com>

Merge pull request #21002 from smithfarm/wip-23437

build/ops: setup-virtualenv.sh: do not hardcode python binary

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add submit button component

This component is to be used inside a form.

It will render a submit button with the given label.
The button will disabled itself and show a loading icon when the user clicks it,
usually initiating a request to the server, and it will stay in that state
until the request is finished.

To indicate that the request failed, returning the button to the enable state,
you need to insert an error in the form with the 'cdSubmitButton' key.

It will also check if the form is valid, when clicking the button, and will
focus on the first invalid input.

Signed-off-by: Tiago Melo <tmelo@suse.com>

qa/tasks/swift: add support for the "force-branch" configurable.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #21013 from dzafman/wip-remove-bc

test: Replace bc command with printf command

Reviewed-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: Add Teuthology test for `/api/pool/_info`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Minor tweak to `DashboardTestCase._request`

Don't fall through silently.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Change `crush_rule` serialization of Pools from id to name

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Add `pool/_info` api endpoint to fill the create-pool dialog

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Add create() to Pool controller

Also:

* Added tests.
* Renamed `DashboardTest` to `PoolTest`.
* Added `delete()`.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Changed `@RESTController.args_from_json`

It now allows any keyword arguments. This is needed for ECP
and Pool creation.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Refactor `send_command` into CephService

* `CephService.send_command` is much easier to use.
* Refactored `CephFSClients.get` and `Dashboard.load_bufer`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #21008 from trociny/wip-23430

vstart: set osd_pool_default_erasure_code_profile in initial ceph.conf

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #20981 from cbodley/wip-qa-fix-ssl

qa: fix overrides for openssl_keys task

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #20841 from oritwas/wip-rgw-metadata

rgw: trim all spaces inside a metadata value

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #20842 from mkogan1/wip-rgw-23147

rgw: enable 'qlen' & 'qactive' performance counters

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #20846 from theanalyst/rgw/fix-empty-stats

rgw: radosgw-admin abort early for user stats for empty uids

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21021 from sebastian-philipp/dashboard-auth-qa-fix

qa/tasks/mgr/dashboard: Fix login expires too soon

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20972 from ricardoasmarques/wip-dimless-binary-directive

mgr/dashboard: Add DimlessBinaryDirective

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20971 from ricardoasmarques/wip-helper-component

mgr/dashboard: Add helper component

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add helper component

Signed-off-by: Ricardo Marques <rimarques@suse.com>

tools: setup-virtualenv.sh: improve error message

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: setup-virtualenv.sh: add usage function and --help option

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/tasks/mgr/dashboard: Fix login expires too soon

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

tools: setup-virtualenv.sh: check for mandatory argument

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: declare python version when calling setup-virtualenv.sh

This just makes the current Python 2.7 dependency explicit. Making
these statements optionally use Python 3 is left for later.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: setup-virtualenv.sh: do not hardcode python binary

. . . instead, take a command-line option that defaults to
"python2.7"

Fixes: http://tracker.ceph.com/issues/23437
Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: skip 'test_create_rbd_in_data_pool' without bluestore

Skip the test case 'test_create_rbd_in_data_pool' only if the cluster
does not support bluestore, otherwise run the test. The Ceph mgr
provides the information if the cluster supports bluestore or not, so
the test will be skipped depending on the current cluster.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: add 'osd metadata' command call

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Add DimlessBinaryDirective

Signed-off-by: Ricardo Marques <rimarques@suse.com>

stop.sh: use --no-mon-config when trying to unmap rbd devices

Otherwise if stop.sh is run for already stopped cluster it will get
stuck trying to get mon config.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21019 from majianpeng/fix-vstart

vstart: fix initial start when there is no ceph.conf

Reviewed-by: Mykola Golub <mgolub@suse.com>

vstart.sh: Avoiding ceph-conf error when create a new cluster.

This partly revert commit:f437598cfcabbd66c372bc8.
Before firstly creating a ceph cluster, there is no ceph.conf.
If specify a non-exist ceph.conf, ceph-conf will meet error.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

test: Replace bc command with printf command

Signed-off-by: David Zafman <dzafman@redhat.com>

vstart: set osd_pool_default_erasure_code_profile in initial ceph.conf

Setting it via `ceph config`, when the cluster is already built and
started is too late, because it is used when building the initial osd
map.

Fixes: https://tracker.ceph.com/issues/23430
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20986 from jecluis/wip-vstart

vstart: fix option (due to quotes) and allow disabling dashboard

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: fix dashboard python 3 support

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20933 from dzafman/wip-22902

src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad state machine event")

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa: fix overrides for openssl_keys task

teuthology looks for overrides by the class name, so overrides weren't
working after the task was renamed

Signed-off-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: fixed password generation in Auth controller

Fixes: http://tracker.ceph.com/issues/23404
Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20989 from ricardoasmarques/wip-npm-start

mgr/dashboard: Improve `npm start` script

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20847 from votdev/improve_auth_interceptor

mgr/dashboard: Improve auth interceptor.

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve auth interceptor.

In case of an error the response must be returned.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20970 from ashishkumsingh/doc-fix-resharding-formatting

doc: Fix dynamic resharding doc formatting

Reviewed-by: Jos Collin <jcollin@redhat.com>

vstart: allow disabling dashboard with option

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

mgr/dashboard: Add support for EiB-YiB on dimless.pipe

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add support for EiB-YiB on dimless-binary.pipe

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #20978 from votdev/improve_formatter

mgr/dashboard: Add toBytes() method to FormatterService

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20941 from dillaman/wip-dashboard-rbd

mgr/dashboard: additional fixes to block pages

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Implicit use of proxy.conf.json

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: ng serve should not delete fontend dist dir

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add toBytes() method to FormatterService.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20977 from smithfarm/wip-23424

cmake: add chrono to BOOST_COMPONENTS

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #20969 from smithfarm/wip-23238

osd: fix s390x build failure

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>

vstart: remove single-quotes from option value

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #20966 from dillaman/wip-23399

rbd: children list should support snapshot id optional

Reviewed-by: Mykola Golub <mgolub@suse.com>

cmake: add chrono to BOOST_COMPONENTS

According to https://github.com/ceph/ceph/commit/9535165149d16a2a944a81f4888d3ab0a7358027
Boost::chrono is a dependency of Boost::context and Boost::thread,
but Boost::chrono does not get built unless it is included here.

Fixes: http://tracker.ceph.com/issues/23424
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20958 from votdev/remove_useless_code

mgr/dashboard: Remove useless code

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

rbd: listing children now supports a snapshot id optional

Fixes: http://tracker.ceph.com/issues/23399
Signed-off-by: Jason Dillaman <dillaman@redhat.com>

osd: fix s390x build failure

46de629c3d0bfae5eba3f6b810808938b5a9af1f introduced a regression: on big-endian
machines, there was no matching function for call to 'eversion_t::encode(const
version_t&, ceph::bufferlist&) const' and 'eversion_t::decode(version_t&,
ceph::buffer::list::iterator&)'

Fixes: http://tracker.ceph.com/issues/23238
Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: Fixes dynamic-resharding doc formatting

Fixed the formatting errors in dynamicresharding.rst

Signed-off-by: Ashish Singh assingh@redhat.com

Merge pull request #20888 from majianpeng/fix-vstop

stop.sh: add ceph configure file location

Reviewed-by: Mykola Golub <mgolub@suse.com>

pybind/rbd: added 'set_snap_by_id' API method

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: added '(rbd_)snap_set_by_id' API methods

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

librbd: internally utilize snap id when setting snapshot

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

stop.sh: specify the ceph.conf path to avoid using wrong ceph.conf.

Also specify the ceph.conf path for ceph-conf command in vstart.sh.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #20876 from smithfarm/wip-doc-license

legal: remove doc license ambiguity

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>

Merge pull request #20898 from tspmelo/wip-nodejs-improvement

mgr/dashboard: remove node/npm system installation

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: Remove useless code.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20784 from theanalyst/docs/release-reorg

doc: reorganize releases

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #20909 from tchaikov/wip-23370

mgr/MgrClient: guard send_pgstats() with lock

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #20874 from sebastian-philipp/dashboard_v2-source-run-api-tests

mgr/dashboard: Allow sourcing `run-backend-api-tests.sh`

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20931 from ashitakasam/fix-typo

doc: s/deamon/daemon/

Reviewed-by: Jos Collin <jcollin@redhat.com>

doc: 'daemon' is misspelled in doc/cephfs/health-messages.rst and src/tools/rbd_recover_tool/README

Signed-off-by: ashitakasam <694240887@qq.com>

Merge remote-tracking branch 'gh/mimic-dev2'

Merge pull request #20944 from liewegas/wip-23339

osd: make scrub wait for ec read/modify/writes to apply

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

osd/ReplciatedBackend: do not op_applied on zeroed eversion_t

Signed-off-by: Sage Weil <sage@redhat.com>

osd/PG: wait for pushes and last_update_applied before requesting replica scrub

Wait on the primary for the EC backend to finish r/m/w cycles before we
ask the replicas to scrub.  Otherwise we will wait on the primary but the
replicas will scrub immediately and we'll get a mismatched result!

Signed-off-by: Sage Weil <sage@redhat.com>

osd/ECBackend: do not op_applied on zero at_version for dummy rollforward transaction

The dummy rollforward transactions have at_version=0'0; do not call
op_applied on those.

This ensures that the op_applied call

Signed-off-by: Sage Weil <sage@redhat.com>