common/config: make debug_* regular options

Signed-off-by: Sage Weil <sage@redhat.com>

crush/CrushWrapper: get_full_location by name

Signed-off-by: Sage Weil <sage@redhat.com>

common/config: move parse_value() into Option

It is more reuable there.

Signed-off-by: Sage Weil <sage@redhat.com>

common/config: new get_val() variant

Signed-off-by: Sage Weil <sage@redhat.com>

common/config: record is_daemon

Signed-off-by: Sage Weil <sage@redhat.com>

common/config: find_option

Signed-off-by: Sage Weil <sage@redhat.com>

common: don't pass alt_def_args if we don't use them

Signed-off-by: Sage Weil <sage@redhat.com>

common/options: convert bool safe to a flags field

Signed-off-by: Sage Weil <sage@redhat.com>

mon/ConfigKeyService: make STORE_PREFIX public

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20681 from liewegas/wip-fork-vs-msgr

common,rbd-nbd: fix up prefork behavior vs AsyncMessenger singletons

Reviewed-by: Haomai Wang <haomai@xsky.com>

Merge pull request #17530 from smithfarm/wip-21264

tools: cleanup: rip out ceph-rest-api

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

Merge PR #20688 into wip-sage-testing-20180306.131906

* refs/pull/20688/head:
doc: Drop the output of pg query
doc: Update Monitoring OSDs and PGs

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Sage Weil <sage@redhat.com>

githubmap: update contributors

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #20718 from ceph/wip-rm23163

ceph-volume tests alleviate libvirt timeouts when reloading

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #20726 from b-ranto/wip-doc-upgrade

build-doc: Upgrade ceph python libraries

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20103 from openattic/wip-mgr-dashboard_v2

mgr/dashboard_v2: Initial submission of a web-based management UI (replacement for the existing dashboard)

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20659 from tchaikov/wip-msg/async/dpdk/launch-on-coreid

msg/async: execute on core specified by core_id not its index

Reviewed-by: shangfufei <shangfufei@inspur.com>
Reviewed-by: Haomai Wang <haomai@xsky.com>

tests: workunits: drop rest/test.py workunit

This workunit is only used by the two rest-api test cases that are also being removed.

Fixes: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>

tests: rados suite: drop rest-api test cases

Fixes: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20519 from gregsfortytwo/wip-22882-linger-locking

osdc/Objecter: take budgets across a LingerOp instead of on child Ops

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

osdc: create and use take_linger_budget for prefetching watch/notify budgets

We don't actually take any budget, as we don't want to hit the
total op limits (especially on eg rbd-mirror), but this prevents
our per-message budgeting from causing a deadlock.

Fixes: http://tracker.ceph.com/issues/22882
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Merge pull request #20517 from gregsfortytwo/wip-22114-beacon-blocking-2

mon: mark OSD beacons and pg_create messages as no_reply

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #20720 from ctrlaltdel/typo

doc: Fix typo s/applicatoin/application/

Reviewed-by: Jos Collin <jcollin@redhat.com>

Merge pull request #20535 from ifed01/wip-ifed-bring-fio

test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20708 from majianpeng/bluefs-bug-fix

os/bluestore: fix wrong usage for BlueFS::_allocate.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: fix wrong usage for BlueFS::_allocate.

In fact, function _allocate is atomic function which don't
allocate less than want space.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

os/bluestore/BlueFS: cleanup code for more readable.

And remove the useless judgement.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>

Merge pull request #20699 from yaozongyou/fix-potential-memory-leak

common: fix potential memory leak in HTMLFormatter

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20701 from tchaikov/wip-23212

os/bluestore: recalc_allocated() when decoding bluefs_fnode_t

Reviewed-by: Jianpeng Ma <jianpeng.ma@intel.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #20544 from liewegas/wip-min-kv

os/bluestore: kv_max -> kv_min

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #20713 from smithfarm/wip-spdk-python3

spdk: advance to upstream dc82989d

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #20131 from jcsp/wip-doc-encoding

doc/dev: add a brief guide to serialization

Reviewed-by: Kefu Chai <kchai@redhat.com>

build-doc: Upgrade ceph python libraries

If you have python ceph bindings installed on your system, the doc-build
script will fail since pip detects them and fails to install the updated
bindings in the virtualenv. This fixes the issue by adding --upgrade
flag to pip so that it overrides the system-installed libraries.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #20717 from b-ranto/wip-restful-doc

doc: Improve mgr/restful module documentation

Reviewed-by: John Spray <john.spray@redhat.com>

build/ops: deb: move python-jinja2 dependency to mgr

87399bea8321a7a9ab554085be3a3d5e0563f27f introduced an explicit dependency on
python-jinja2, but mistakenly as an overall build dependency instead of as a
runtime dependency of ceph-mgr as intended.

Fixes: http://tracker.ceph.com/issues/22457
Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc: PendingReleaseNotes: deprecate/drop ceph-rest-api

References: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>

tools: cleanup: rip out ceph-rest-api

Obsoleted by the mgr REST API.

Fixes: http://tracker.ceph.com/issues/21264
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #20542 from liewegas/wip-bluefs-buffered-io

common/options: bluefs_buffered_io=true by default

Reviewed-by: Mark Nelson <mnelson@redhat.com>

Merge pull request #20721 from idryomov/wip-krbd-parent-overlap-test

qa: krbd parent-overlap test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

test/fio: enable objectstore FIO plugin building without the need to install and build FIO source code

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

Merge pull request #20695 from mikechristie/wip-doc-mnc-update-iscsi-kernel

doc: update iSCSI upstream kernel to 4.16

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20663 from tchaikov/wip-ceph.in-py3

ceph.in: use a different variable for holding thrown exception

Reviewed-by: John Spray <john.spray@redhat.com>

Doc: Fix typo s/applicatoin/application/

Signed-off-by: Francois Deppierraz <francois@ctrlaltdel.ch>

Merge pull request #20714 from idryomov/wip-krbd-msgr-segments-test

qa: krbd msgr-segments test

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

os/bluestore: recalc_allocated() when decoding bluefs_fnode_t

Fixes: http://tracker.ceph.com/issues/23212
Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

doc: Improve mgr/restful module documentation

Fixes: https://tracker.ceph.com/issues/23219
Signed-off-by: Boris Ranto <branto@redhat.com>

qa: krbd parent-overlap test

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

mgr/dashboard_v2: added script to run backend API tests locally

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/suites/mgr/dashboard_v2: added suite yaml for testing dashboard_v2

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard_v2: implementation of backend API tests

We moved the dashboard_v2 test cases that required a real ceph cluster
to run into qa/tasks/mgr with the necessary adaptations.
These tests can now be run in teuthology using the cephfs_test_runner
task.

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr: move test initialization to setUpClass method

With this change, we avoid the disabling/enabling of the ceph-mgr module
being tested for each test function declared in each test case. Now
the ceph-mgr module being tested is disabled/enabled only once for each
test case.

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: Added smoke test.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard_v2: Add mgr-dashboard_v2 frontend unittests to `ctest`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard_v2: Add CMake target to build the frontend.

* Added new CMake flag `WITH_MGR_DASHBOARD_V2_FRONTEND`:
  Build the mgr/dashboard_v2 frontend using `npm install && npm run build`

* Set this flag to `OFF` when building packages.

* Removed creation of the frontend from `vstart.sh`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard_v2: Integrate dashboard_v2 into `make check`

Also:

* Disabled API tests in tox.
* No longer start a vstart.sh cluster in tox.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard_v2: Remove `Service` and `ServiceMeta` class

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard_v2: Remove `BaseControllerMeta` meta class

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard_v2: use global manager instance instead of injection

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: Fix OSD menu entry position

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard_v2: Auto reload cd-table

Now every used cd-table with a valid "fetch" function will auto reload
itself by default every 10 seconds. You can also identify the unique
property of the row now, by default the unique identifier is "id".

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard_v2: Fixed exception in rbd_mirroring

Fixed issue when rbd-mirroring is enabled without any Pools.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard_v2: Enable table header and footer on perfcounters

Enable the ability to filter and paginate the performance counter tables
to make it easier to find specific information quickly.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

mgr/dashboard_v2: Add support for $PREFIX config

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard_v2: Add links on dashboard

User will be able to navigate from dashboard to "Monitor" and "OSDs" pages

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard_v2: enable rbd-mirror perf counters

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard_v2: Updated reference documentation

Updated the dashboard chapter in the manual to reflect the
changes introduced by Dashboard v2 (e.g. username/password).

Updated description to include a high-level overview of the features
available.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

mgr/dashboard_v2: Adapt status datatable to default design of cd-table

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard_v2: add mirroring page

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard_v2: Add rbd_mirroring to `/api/summary`

Also:

* Made `get_daemons_and_pools` a function independant of `RbdMirroring`.
* Added tests

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard_v2: OSD pages

You can now list all osds and see details for each OSD.
The details can be viewed in tabs. You can view attributes, meta data
and histogram of the selected OSD.

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard_v2: Fix datatable header layout in configuration doc

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard_v2: Move `dashboard/toplevel` to a new controller

Details:

* Moved `/dashboard/toplevel` API endpoint to `/summary`.
* Moved testcase to `tests/test_summary.py`.
* Also renamed service in the frontend.

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard_v2: Add RBD mirroring functionality

This commit adds the RBD mirroring functionality of the first dashboard to
the dashboard_v2 REST API. That's why the code is basically based on the
code of the first dashboard. It was just brought into a different structure
to fit into the structure of dashboard_v2.

The new API endpoint can be found at http://<host>:<port>/api/rbdmirror

The commit adds also related tests.

Signed-off-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard_v2: add ceph configuration documentation page

Signed-off-by: Tiago Melo <tmelo@suse.com>

mgr/dashboard_v2: updated .gitignore

Signed-off-by: Ricardo Dias <rdias@suse.com>

install-deps: install dashboard_v2 development dependencies

Signed-off-by: Ricardo Dias <rdias@suse.com>
Signed-off-by: Kai Wagner <kwagner@suse.com>

debian/control: add dashboard_v2 development dependencies

Signed-off-by: Ricardo Dias <rdias@suse.com>

ceph.spec: added dashboard_v2 development and runtime dependencies

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: fix tox.ini to handle the vstart changes

vstart.sh script now enables and configures dashboard_v2 module, so
tox.ini script needs to handle that.

Signed-off-by: Ricardo Dias <rdias@suse.com>

vstart: mgr/dashboard_v2 enabling and configuration

Signed-off-by: Ricardo Dias <rdias@suse.com>

make-dist: Added dashboard_v2 frontend build files to dist tarball

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: Fix hosts columns width

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard_v2: Add missing 'aria-current' attribute to active breadcrumbs.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard_v2: Add missing i18n attributes

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard_v2: get rgw service list using CephService

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: get tcmu_runner service list using CephService

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: get rbd pool list using CephService

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: added CephService service class

This class provides methods to get the list of services registered in
the cluster, and also to get the list of pools by application type.

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: Service class base code

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard_v2: Data transform in k/v table

The key value table can transform the input data into key value objects
now, you don't have to care about that anymore.

Also the first column of the data used in cd-table will be sorted if
not set manual through the "sorts" array.

Signed-off-by: Stephan Müller <smueller@suse.com>

mgr/dashboard_v2: Also protect `/api`

Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

mgr/dashboard_v2: Fixed capitalization on login page

Changed "ceph" to "Ceph" in the welcome message.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

mgr/dashboard_v2: Improve empty row style

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard_v2: Reorder menu entries

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard_v2: Add footer and header to hosts table

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard_v2: Auto refresh hosts

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard_v2: Prevent double click on table refresh

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard_v2: Migrate cluster configuration

This commit adds the ability to retrieve and filter the cluster
configuration of Ceph through the RESTful API.

Parameters `level` and `service` can be added to filter the list.

Example:

    http://<host>:<port>/api/cluster_conf?level=basic&service=mds

Signed-off-by: Patrick Nawracay <pnawracay@suse.com>

mgr/dashboard_v2: Add iSCSI page

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard_v2: Use monospace font on logs

Signed-off-by: Ricardo Marques <rimarques@suse.com>