mgr/dashboard: K-v-table uses autoReload input

The "autoReload" input from the data table can now be used in the key
value table as well.

Signed-off-by: Stephan Müller <smueller@suse.com>

Merge pull request #21517 from rjfd/wip-dashboard-fix-rbd-auth

mgr/dashboard: rbd: add @AuthRequired to snapshots controller

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20869 from votdev/dashboard_rgw_ro

mgr/dashboard: Add RGW user and bucket lists (read-only)

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>

Merge pull request #21386 from votdev/api_interceptor

mgr/dashboard: Rename and refactor ApiInterceptorService class

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #21531 from LenzGr/mgr-dashboard-documentation

doc: updated the dashboard feature list

Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

doc: updated the dashboard feature list

Updated and improved the Mgr Dashboard feature list: added RBD management
features.

Signed-off-by: Lenz Grimmer <lgrimmer@suse.com>

Merge PR #21499 into master

* refs/pull/21499/head:
doc/cephfs: update kernel client quotas support info

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>

githubmap: update contributors

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21472 into master

* refs/pull/21472/head:
mds: optimize purge queue expire pos update

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21434 into master

* refs/pull/21434/head:
client: add client option descriptions

Reviewed-by: Nathan Cutler <ncutler@suse.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #21498 into master

* refs/pull/21498/head:
client: flush the mdlog in _fsync before waiting on unstable reqs

Reviewed-by: dongdong tao <tdd21151186@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge pull request #21530 from jan--f/mgr-prometheus-fix-osd-metadata

pybind/mgr/prometheus: fix creation of osd_metadata metric

Reviewed-by: Boris Ranto <branto@redhat.com>

pybind/mgr/prometheus: fix creation of osd_metadata metric

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

Merge pull request #21446 from smithfarm/wip-ctest-py3-fallback

cmake/mgr: use Python 3 virtualenv if mgr subinterpreter is Python 3

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21477 from liewegas/wip-kvstore-tool

ceph-kvstore-tool: make it a bit more friendly

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21488 from tchaikov/wip-i-choose-to-die-instead-of-leak

msg/async/EventKqueue: assert on OOM

Reviewed-by: Haomai Wang <haomai@xsky.com>

mgr/dashboard: Add ability to cancel notifications.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Do not show notifications for HTTP 409 errors.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Rename auth-interceptor and refactor it to display notifications for more errors than 401 and 500.

Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #21496 from ceph/wip-rm23678

ceph-volume: allow skipping systemd interactions on activate/create

Reviewed-by: Andrew Schoen <aschoen@redhat.com

mgr/dashboard: Modify RGW proxy response of errors to display a detailed message in the error notification.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Force datatable to hide the loading indicator in case of an error.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add RGW user and bucket lists (read-only)

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: rbd: add @AuthRequired to snapshots controller

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21466 from jcsp/wip-dashboard-cmake-all

cmake: include frontend build in 'all' target

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21497 from votdev/forceidentifier_datatable

mgr/dashboard: Add 'forceIdentifier' attribute to datatable

Reviewed-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #21469 from liewegas/wip-log-nomsgs

mon/LogMonitor: do not crash on log sub w/ no messages

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Joao Eduardo Luis <joao@suse.de>

Merge pull request #21467 from tchaikov/wip-perf-stats-ns-incarnation

include/ceph_features: fix OS_PERF_STAT_NS's incarnation

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #21459 from runsisi/wip-mgrc-register

mgr/MgrClient: service registration filtered by service name instead of daemon name

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #21344 from yangdongsheng/lock_timeout

rbd: support lock_timeout in rbd mapping

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

cmake/mgr: use Python 3 virtualenv if mgr subinterpreter is Python 3

Signed-off-by: Nathan Cutler <ncutler@suse.com>

mgr/dashboard: Force identifier for RBD datatable.

Signed-off-by: Volker Theile <vtheile@suse.com>

mgr/dashboard: Add 'forceIdentified' attribute to datatable

Signed-off-by: Volker Theile <vtheile@suse.com>

rbd: support lock_timeout in rbd mapping

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>

Merge pull request #21056 from hitoshikamei/rbdmap-notrim

rbd: add notrim option to rbd map

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #21490 from tchaikov/wip-tox

mgr/dashboard: create venv for running tox

Reviewed-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: create venv for running tox

the tox shipped with centos is 1.4.2, and is way too old. it does not
allow us to define the dependency using generative-envlist.

Signed-off-by: Kefu Chai <kchai@redhat.com>

tools/setup-virtualenv.sh: bump up the version of tox to 2.9.1

2.8.1 brings an important fix, see
https://github.com/tox-dev/tox/issues/515.

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21489 from thmour/thmour_parallel_create

ceph-volume: allow parallel creates

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #16608 into master

* refs/pull/16608/head:
qa: whitelist mds down wrn during cephfs testing
mds: add config to disable fragmentation
qa: add max_mds thrash test
qa: mds_thrash updates for new max_mds behavior
doc: update upgrade procedure and release notes
qa: add test for cluster resizing
qa: remove use of mds deactivate
cephfs: add new down/joinable fs flags
mds: evict all clients if last mds shutting down
cephfs: deprecate ceph mds deactivate
cephfs: kill allow_dirfrags
cephfs: Kill allow_multimds
cephfs: Change behavior of cluster_down flag
mon/FSCommands: Set extra MDS to standby
cephfs: Health check changes
mon/MDSMonitor: Remove command support for legacy syntax

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #21456 into master

* refs/pull/21456/head:
MDSMonitor: initialize new Filesystem epoch from pending

Reviewed-by: John Spray <john.spray@redhat.com>

Merge PR #21457 into master

* refs/pull/21457/head:
mds: unset deleted vars in shutdown_pass

Reviewed-by: Zheng Yan <zyan@redhat.com>

Merge PR #21458 into master

* refs/pull/21458/head:
MDSMonitor: cleanup and protect fsmap access

Reviewed-by: João Eduardo Luís <joao@suse.de>

Merge PR #21473 into master

* refs/pull/21473/head:
client: fix error operator precedence

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>
Reviewed-by: Zheng Yan <zyan@redhat.com>

client: add client option descriptions

Fixes: http://tracker.ceph.com/issues/22933
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: unset deleted vars in shutdown_pass

So future passes do not try to delete again.

Fixes: http://tracker.ceph.com/issues/23766
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #21480 from smithfarm/wip-backport-create-issue

scripts: new backport-create-issue script

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

script/backport-create-issue: enable script to be imported

Signed-off-by: Nathan Cutler <ncutler@suse.com>

doc/cephfs: update kernel client quotas support info

Signed-off-by: Luis Henriques <lhenriques@suse.com>

ceph-volume tests functional python tests for --no-systemd in activate

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.activate add --no-systemd flag to skip starting/enabling services

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.common add --no-systemd flag to skip enabling/starting services

Signed-off-by: Alfredo Deza <adeza@redhat.com>

client: flush the mdlog in _fsync before waiting on unstable reqs

Recently, we changed ganesha to call to ceph_ll_sync_inode after
issuing a setattr call, to ensure that the attribute changes makes it
to the backing store before returning.

While testing with this, I noticed that we'd occasionally see setattr
calls stall for a long time. Analysis showed that the client's _fsync
request was stalled waiting on an mdlog flush.

This patch adds a call to flush_mdlog_sync prior to waiting on unsafe
requests in _fsync and seems to resolve the issue.

Tracker: http://tracker.ceph.com/issues/23714
Signed-off-by: Jeff Layton <jlayton@redhat.com>

Merge pull request #21436 from trociny/wip-test-out_of_order

test/librbd: ensure OutOfOrder test has enough concurrent management ops

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21493 from trociny/wip-deep-copy-missing-return

librbd: missing 'return' in deep_copy::ObjectCopyRequest::send_read_object

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21385 from ricardoasmarques/wip-list-all-rbds

mgr/dashboard: RBD management (frontend)

Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #21451 from ceph/wip-rm23581

ceph-volume: automatic VDO detection

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

mgr/dashboard: RBD management

Signed-off-by: Ricardo Marques <rimarques@suse.com>

librbd: missing 'return' in deep_copy::ObjectCopyRequest::send_read_object

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #21383 from b-ranto/wip-mgr-rgw-metadata

prometheus: Implement rgw_metadata metric

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Jan Fajerski <jfajerski@suse.com>

mgr/dashboard: Add 'executing' cell template

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Configurable type for 'cd-submit-button'

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add 'delete-confirmation-modal' component

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Force focus on submit

Forcing focus on submit button will guarantee that all inputs will lose focus
before submission.

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Add get method to summary service

Signed-off-by: Ricardo Marques <rimarques@suse.com>

Merge pull request #21462 from ricardoasmarques/wip-backgroud-tasks-style

mgr/dashboard: Improve background tasks style

Reviewed-by: Laura Paduano <lpaduano@suse.com>

prometheus: Implement rgw_metadata metric

This maps rgw id to its hostname and ceph version running on the node.

Signed-off-by: Boris Ranto <branto@redhat.com>

Merge pull request #21443 from LenzGr/mgr-dashboard-documentation

doc: Improved dashboard documentation

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

ceph-volume: allow parallel creates

Currently the create code decides the vg_name "ceph-$cluster_fsid" as
the primary vg_name and creates a new name if this already exists.

If this code is run N times in parallel, the script will try to
create N times the vg with the name "ceph-$cluster_fsid" and it
will fail to create the N osds successfully.

Creating vgs with names like "ceph-$uuid4" lets our scripts to run
without any problems.

Signed-off-by: Theofilos Mouratidis <t.mour@cern.ch>

Merge pull request #21485 from tchaikov/wip-tox

test/dashboard: hardcode .coverage path to workaround tox bugs

Reviewed-by: Ricardo Dias <rdias@suse.com>

msg/async/EventKqueue: assert on OOM

Signed-off-by: Kefu Chai <kchai@redhat.com>

test/dashboard: hardcode .coverage path to workaround tox bugs

rhel/centos 7.3 comes with tox 1.4.2, which has a bunch of issues related
to environment variable substitution. for instance, see
- https://github.com/tox-dev/tox/issues/515
so, instead of having different coverage files for py3 and py2.7, it
would be simpler to just use the same path for coverage. this approach
works because pytest-cov plugin erases the coverage data file at the
beginning of test[1].

---
[1] https://pypi.org/project/pytest-cov/#coverage-data-file .

Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21474 from jecluis/mimic-mon-pending-releasenotes

PendingReleaseNotes: mention some monitor changes

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21428 from ivancich/wip-fix-existing-mclock-cost

osd: remove cost from mclock op queues; cost not handled well in dmclock

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21464 from LenzGr/fix-cmakelist

cmake: fix typo in status message

Reviewed-by: Volker Theile <vtheile@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #21418 from jecluis/wip-fix-mon-fs-caps

mon/AuthMonitor: don't validate `fs authorize` caps with `valid_caps()`

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #21450 from dzafman/wip-jq-compat

test: Use jq in a compatible way and for easier diff analysis

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa: whitelist mds down wrn during cephfs testing

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

script/backport-create-issue: explain why release_id is hard-coded

Signed-off-by: Nathan Cutler <ncutler@suse.com>

scripts: backport-create-issue (initial commit)

This is a standalone implementation of a script written by Loic Dachary.

Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #21483 from yuriw/wip-yuriw-crontab

qa/tests - one more typo fixed :(

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests - one more typo fixed :(

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #21482 from yuriw/wip-yuriw-crontab

qa/tests - Fixed typo in crontab entry

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests - Fixed typo in crontab entry

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #21454 from yuriw/wip-yuriw-crontab

qa/tests - added for the suites with subset be able to use 'testing' …

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

ceph-volume util.prepare preserve order on incoming mount flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests update (preserved) order on mount flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>

doc/dev/ceph-volume add VDO tag information

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests ensure non-duplicate flags in mount calls

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.prepare allow and dedupe extra flags for mounting

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.prepare detect vdo from device for mounting filestore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

mds: add config to disable fragmentation

For disaster situations or testing.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add max_mds thrash test

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: mds_thrash updates for new max_mds behavior

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

doc: update upgrade procedure and release notes

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add test for cluster resizing

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: remove use of mds deactivate

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

cephfs: add new down/joinable fs flags

Fixes: http://tracker.ceph.com/issues/20609
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mds: evict all clients if last mds shutting down

This allows all ranks to stop. If the administrator shuts down the cluster then
we should immediately shut down all client's I/O.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

cephfs: deprecate ceph mds deactivate

MDS deactivation is now handled by the max_mds parameter. Deprecate
ceph mds deactivate and note it to be removed in a future release.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>