ceph-volume lvm.activate error if no bluestore OSDs are found

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests/functional add 'activate all' for xenial/filestore/dmcrypt

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests/functional add 'activate all' for xenial/bluestore/dmcrypt

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests/functional add 'activate all' for centos7/filestore/dmcrypt

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests/functional add 'activate all' for centos7/bluestore/dmcrypt

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests/functional add 'activate all' in filestore playbook

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests/functional add 'activate all' in bluestore playbook

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests ensure activate all behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.activate warn when no osds are found with --all

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests add process.call unit tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.activate add 'activate all' functionality

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume process.call add conditional verbosity on failure

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume systemd.systemctl add helpers for daemon status

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.listing create a helper that lists osds

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvv.activate allo to configure the CLI with --all

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #21070 from ceph/wip-cv-vagrant-reload

ceph-volume improve robustness when reloading vms in tests

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20494 from wido/influx-time-fixes

mgr/influx: Various time fixes

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #21063 from guzhongyan/bs-cleanup

os/bluestore: remove unused variable.

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #21043 from wido/zabbix-osd-numpgs

mgr/zabbix: Send max, min and avg PGs of OSDs to Zabbix

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #21037 from guzhongyan/bf-format

os/bluestore: fix some code formatting

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #20951 from tchaikov/wip-FTBFS-bluestore-armhf

bluestore: fix build on armhf

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #21034 from rzarzynski/wip-bs-reorder-bluefs_extent_t

os/bluestore: reorder members of bluefs_extent_t for space efficiency

Reviewed-by: Igor Fedotov <ifedotov@suse.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #20900 from ifed01/wip-ifed-fix-interals-in-repair

os/bluestore: bluestore repair should use interval_set::union_insert

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #20774 from rishabh-d-dave/fix-tilde-expansion

config: expand tilde for ~/.ceph/$cluster.conf

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20558 from chardan/jfw-wip-show_compression_methods-22420

osd,compressor: Expose compression algorithms via MOSDBoot.

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20314 from ovh/bp-librados-nntk

librados: extend C API for so it accepts keys with NUL chars

Reviewed-by: Sage Weil <sage@redhat.com>

ceph-volume tests create a retry script for vagrant reload

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests use a reload script in simple functional tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests use a reload script in lvm functional tests

Signed-off-by: Alfredo Deza <adeza@redhat.com>

os/bluestore: remove unused variable.

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

Merge pull request #21005 from rjfd/wip-mgr-fix-error-handling

mgr: fixes python error handling

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tim Serong <tserong@suse.com>

Merge pull request #21024 from s0nea/wip-dashboard-return-features-list

mgr/dashboard: Convert the RBD feature names to a list of strings

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>

Merge pull request #21004 from s0nea/wip-dashboard-skip-datapool-testcase

mgr/dashboard: skip data pool testcase for none-bluestore clusters

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20873 from sebastian-philipp/dashboard_v2-simple-html-debug

mgr/dashboard: Add minimalistic browsable API

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21047 from rjfd/wip-mgr-smart-fix-py3

mgr/smart: fix python3 module loading

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #21027 from cbodley/wip-qa-swift-force-branch

qa/tasks/swift: add support for the "force-branch" configurable.

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

mgr/dashboard: New API tests  for `perf_couters`

Added OSD and MDS tests.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Add minimalistic browsable API

Also provides a simple HTML form to POST
data to a `RESTController`'s `create()` method.

Also added ENABLE_BROWSABLE_API setting to the dashboard

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Renamed `types` as it clashes with std lib

This will lead to a name clash, when importing `types`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/smart: fix python3 module loading

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr: fixes python error handling

The current `handle_pyerror` function implementation relies in the
`traceback.format_exception_only` python function to format the
exception object. The problem is that this python function might also
raise an exception. This commit fixes it by enclosing that python
function call in try...catch block.

Fixes: http://tracker.ceph.com/issues/23406
Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #19439 from chardan/jfw-wip-14363-normalize-rm

rgw, mon: normalize delete/remove in admin console (cleanup 22444)

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #21011 from tspmelo/wip-submit-btn

mgr/dashboard: Add submit button component

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #21032 from simransinghal/fixtypo

rgw: Fix typo in comment

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #21020 from trociny/wip-vstart-no-mon-config

stop.sh: use --no-mon-config when trying to unmap rbd devices

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/zabbix: Send max, min and avg PGs of OSDs to Zabbix

We already send the max, min and avg fill ratio of OSDs but
knowing the OSD with the highest amount of PGs is also useful.

This allows admins to create a trigger should it happen that there
is a OSD with too many PGs.

This could happen if a lot of OSDs fail and PGs start to move filling
up one or more OSDs with many PGs.

As PGs eat CPU and Memory people usually like to watch out for these
situations.

Signed-off-by: Wido den Hollander <wido@42on.com>

Merge pull request #21036 from xiexingguo/wip-wrong-tip

os/bluestore: s/bluefs_total/bluefs_free/

Reviewed-by: Igor Fedotov <ifedotov@mirantis.com>

Merge pull request #21007 from rjfd/wip-dashboard-fix-py3

mgr/dashboard: fix dashboard python 3 support

Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #21006 from rjfd/wip-dashboard-fix-auth-py3

mgr/dashboard: fixed password generation in Auth controller

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20865 from sebastian-philipp/dashboard_v2_pool-create

mgr/dashboard: Add Pool-create to the backend

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: convert RBD features to a list of strings

Until now the API returns the features provided by an RBD as one comma
separated string. It's more usable for the frontend to receive a list
of strings instead.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

os/bluestore: fix some code formatting

Signed-off-by: Gu Zhongyan <guzhongyan@360.cn>

os/bluestore: s/bluefs_total/bluefs_free/

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>

os/bluestore: reorder members of bluefs_extent_t to make it smaller.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

src: Fix typo

Replace "dont" with "don't". "Dont" is not same as "Do not" or "Don't".

Signed-off-by: Simran Singhal <singhalsimran0@gmail.com>

Merge pull request #21002 from smithfarm/wip-23437

build/ops: setup-virtualenv.sh: do not hardcode python binary

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Add submit button component

This component is to be used inside a form.

It will render a submit button with the given label.
The button will disabled itself and show a loading icon when the user clicks it,
usually initiating a request to the server, and it will stay in that state
until the request is finished.

To indicate that the request failed, returning the button to the enable state,
you need to insert an error in the form with the 'cdSubmitButton' key.

It will also check if the form is valid, when clicking the button, and will
focus on the first invalid input.

Signed-off-by: Tiago Melo <tmelo@suse.com>

qa/tasks/swift: add support for the "force-branch" configurable.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Merge pull request #21013 from dzafman/wip-remove-bc

test: Replace bc command with printf command

Reviewed-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: Add Teuthology test for `/api/pool/_info`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Minor tweak to `DashboardTestCase._request`

Don't fall through silently.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Change `crush_rule` serialization of Pools from id to name

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Add `pool/_info` api endpoint to fill the create-pool dialog

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Add create() to Pool controller

Also:

* Added tests.
* Renamed `DashboardTest` to `PoolTest`.
* Added `delete()`.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Changed `@RESTController.args_from_json`

It now allows any keyword arguments. This is needed for ECP
and Pool creation.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard: Refactor `send_command` into CephService

* `CephService.send_command` is much easier to use.
* Refactored `CephFSClients.get` and `Dashboard.load_bufer`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Merge pull request #21008 from trociny/wip-23430

vstart: set osd_pool_default_erasure_code_profile in initial ceph.conf

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <sebastian.wagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #20981 from cbodley/wip-qa-fix-ssl

qa: fix overrides for openssl_keys task

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #20841 from oritwas/wip-rgw-metadata

rgw: trim all spaces inside a metadata value

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>

Merge pull request #20842 from mkogan1/wip-rgw-23147

rgw: enable 'qlen' & 'qactive' performance counters

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

Merge pull request #20846 from theanalyst/rgw/fix-empty-stats

rgw: radosgw-admin abort early for user stats for empty uids

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #21021 from sebastian-philipp/dashboard-auth-qa-fix

qa/tasks/mgr/dashboard: Fix login expires too soon

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20972 from ricardoasmarques/wip-dimless-binary-directive

mgr/dashboard: Add DimlessBinaryDirective

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20971 from ricardoasmarques/wip-helper-component

mgr/dashboard: Add helper component

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add helper component

Signed-off-by: Ricardo Marques <rimarques@suse.com>

tools: setup-virtualenv.sh: improve error message

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: setup-virtualenv.sh: add usage function and --help option

Signed-off-by: Nathan Cutler <ncutler@suse.com>

qa/tasks/mgr/dashboard: Fix login expires too soon

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

tools: setup-virtualenv.sh: check for mandatory argument

Signed-off-by: Nathan Cutler <ncutler@suse.com>

build/ops: declare python version when calling setup-virtualenv.sh

This just makes the current Python 2.7 dependency explicit. Making
these statements optionally use Python 3 is left for later.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: setup-virtualenv.sh: do not hardcode python binary

. . . instead, take a command-line option that defaults to
"python2.7"

Fixes: http://tracker.ceph.com/issues/23437
Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: skip 'test_create_rbd_in_data_pool' without bluestore

Skip the test case 'test_create_rbd_in_data_pool' only if the cluster
does not support bluestore, otherwise run the test. The Ceph mgr
provides the information if the cluster supports bluestore or not, so
the test will be skipped depending on the current cluster.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: add 'osd metadata' command call

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Add DimlessBinaryDirective

Signed-off-by: Ricardo Marques <rimarques@suse.com>

stop.sh: use --no-mon-config when trying to unmap rbd devices

Otherwise if stop.sh is run for already stopped cluster it will get
stuck trying to get mon config.

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21019 from majianpeng/fix-vstart

vstart: fix initial start when there is no ceph.conf

Reviewed-by: Mykola Golub <mgolub@suse.com>

vstart.sh: Avoiding ceph-conf error when create a new cluster.

This partly revert commit:f437598cfcabbd66c372bc8.
Before firstly creating a ceph cluster, there is no ceph.conf.
If specify a non-exist ceph.conf, ceph-conf will meet error.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

config: replace '~' by '$home' in CEPH_CONF_FILE_DEFAULT

And expand "$home" to get the actual path to ceph.conf.

Fixes: http://tracker.ceph.com/issues/23215
Signed-off-by: Rishabh Dave <ridave@redhat.com>

test: Replace bc command with printf command

Signed-off-by: David Zafman <dzafman@redhat.com>

vstart: set osd_pool_default_erasure_code_profile in initial ceph.conf

Setting it via `ceph config`, when the cluster is already built and
started is too late, because it is used when building the initial osd
map.

Fixes: https://tracker.ceph.com/issues/23430
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #20986 from jecluis/wip-vstart

vstart: fix option (due to quotes) and allow disabling dashboard

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: fix dashboard python 3 support

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20933 from dzafman/wip-22902

src/osd/PG.cc: 6455: FAILED assert(0 == "we got a bad state machine event")

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>

qa: fix overrides for openssl_keys task

teuthology looks for overrides by the class name, so overrides weren't
working after the task was renamed

Signed-off-by: Casey Bodley <cbodley@redhat.com>

mgr/dashboard: fixed password generation in Auth controller

Fixes: http://tracker.ceph.com/issues/23404
Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #20989 from ricardoasmarques/wip-npm-start

mgr/dashboard: Improve `npm start` script

Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #20847 from votdev/improve_auth_interceptor

mgr/dashboard: Improve auth interceptor.

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard: Improve auth interceptor.

In case of an error the response must be returned.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #20970 from ashishkumsingh/doc-fix-resharding-formatting

doc: Fix dynamic resharding doc formatting

Reviewed-by: Jos Collin <jcollin@redhat.com>

vstart: allow disabling dashboard with option

Signed-off-by: Joao Eduardo Luis <joao@suse.de>