mgr/dashboard: User permissions control (frontend)

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: controllers: fix endpoint return type when method returns `bytes`

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: controllers: fix controllers endpoints initialization order

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: test_cluster_conf: restore original value of mon_allow_pool_delete

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: hacking.rst: update controller access control instructions

Signed-off-by: Ricardo Dias <rdias@suse.com>

doc/mgr/dashboard: user and role management documentation

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: auth: return user permissions on login

Signed-off-by: Ricardo Dias <rdias@suse.com>

qa/tasks/mgr/dashboard: Adapted tests to work with new authentication system

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: configured security info for each controller

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: removed `@AuthRequired` decorator

Now all controllers require authentication by default.
No disable authentication, a boolean parameter can be passed
in the `@ApiController` decorator.

Fixes: http://tracker.ceph.com/issues/23796
Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: auth: remove single user authentication

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: pylint: updated list of files to verify

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: tests: local role-based authorization

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: load and cofigure authorization system

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: authmanager: authorization handling cherrypy tool

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: local role-based authorization system implementation

Signed-off-by: Ricardo Dias <rdias@suse.com>

mgr/dashboard: security: static scopes and permissions

Signed-off-by: Ricardo Dias <rdias@suse.com>

Merge pull request #22706 from joscollin/wip-doc-dashboardgrafana

doc/dashboard: fix formatting on Grafana instructions-2

Merge pull request #22694 from dillaman/wip-test-rbd-mirror

test/rbd-mirror: disable use of gtest-parallel

Reviewed-by: Erwan Velu <erwan@redhat.com>
Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #22591 from tchaikov/wip-gcc-8.1

cmake: disable -Werror-stringop-truncation for rocksdb

Reviewed-by: Casey Bodley <cbodley@redhat.com>

doc/dashboard: fix formatting on Grafana instructions

The whole list was in a code block, instead of just the snippets.

Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge PR #22691 into master

* refs/pull/22691/head:
os/bluestore: fix minor issues in bitmap logging output
os/bluestore: fix overlappedd allocation returned by bitmap allocator

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22682 from wwformat/fix-centos-install-deps

install-deps: check the exit status for the $builddepcmd

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #22532 into master

* refs/pull/22532/head:
qa/suites: Add supported-random-distro$ links.

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #22626 into master

* refs/pull/22626/head:
scripts: add ceph-common-dbg to debug packages

Reviewed-by: David Zafman <dzafman@redhat.com>

scripts: add ceph-common-dbg to debug packages

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

test/rbd-mirror: disable use of gtest-parallel

This test repeatedly deadlocks when run under in parallel.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22693 from tspmelo/fix-lint-codelyzer

mgr/dashboard: fix lint error caused by codelyzer update

Reviewed-by: Kefu Chai <kchai@redhat.com>

mgr/dashboard: fix lint error caused by codelyzer update

Signed-off-by: Tiago Melo <tmelo@suse.com>

Merge pull request #22544 from ifed01/wip-ifed-alloc-bench-fix

test/objectstore: fix random generator in allocator_bench.

Reviewed-by: Adam Kupczyk <akupczyk@redhat.com>

os/bluestore: fix minor issues in bitmap logging output

Signed-off-by: Igor Fedotov <ifedotov@suse.com>

os/bluestore: fix overlappedd allocation returned by bitmap allocator

Fixes: https://tracker.ceph.com/issues/24598
Signed-off-by: Igor Fedotov <ifedotov@suse.som>

install-deps: check the exit status for the $builddepcmd

in some case, the $builddepcmd will failed without any "error:" output.
so we should check the exit status to handle it.

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

Merge PR #22479 into master

* refs/pull/22479/head:
mgr/devicehealth: add self-test
mgr/devicehealth: make config options instance attributes
mgr/devicehealth: active -> enable_monitoring
mgr/mgr_module: update docstring for get()
mgr/devicehealth: fix 'device get-health-metrics' when no metrics are stored
mgr/devicehealth: add some scraping infrastructure
osd: optionally query smart for a single devid
mgr/ActivePyModules: add get for 'device <devid>'
osd: key smart results by unique device id
mgr: enable devicehealth by default
mgr/devicehealth: rename old smart module to devicehealth

Merge PR #22572 into master

* refs/pull/22572/head:
osd: update clone_overlap even the clone have been evicted

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #22576 into master

* refs/pull/22576/head:
common/obj_bencher.cc: faster object name generation

Reviewed-by: Erwan Velu <erwan@redhat.com>

Merge PR #22588 into master

* refs/pull/22588/head:
mgr/iostat: column_width should be int not float

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Mohamad Gebai <mgebai@suse.com>

githubmap: update contributors

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #22615 into master

* refs/pull/22615/head:
common: fix enum redeclaration

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #22620 into master

* refs/pull/22620/head:
common/WorkQueue: Before set_ioprio it must do create/try_create.

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>

Merge PR #22634 into master

* refs/pull/22634/head:
mgr, pybind/mgr: pass inbuf (ceph -i <file>) to modules

Reviewed-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: add self-test

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: make config options instance attributes

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: active -> enable_monitoring

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/mgr_module: update docstring for get()

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: fix 'device get-health-metrics' when no metrics are stored

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: add some scraping infrastructure

- command to fetch smart info
- command to scrape a device and store the metrics
- command to scrape all devices (and store)
- command to dump stored metrics
- purging of old metrics

This is based on code originally written by Yaarit.

Signed-off-by: Yaarit Hatuka yaarithatuka@gmail.com
Signed-off-by: Sage Weil <sage@redhat.com>

osd: optionally query smart for a single devid

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/ActivePyModules: add get for 'device <devid>'

This is slightly different than the usual pattern because it is
parameterized.  I want to avoid fetching *all* devices if we don't need
it.

Signed-off-by: Sage Weil <sage@redhat.com>

osd: key smart results by unique device id

This is more useful than the current local device name.

Clean up some formatting.

Signed-off-by: Sage Weil <sage@redhat.com>

mgr: enable devicehealth by default

Signed-off-by: Sage Weil <sage@redhat.com>

mgr/devicehealth: rename old smart module to devicehealth

Let's avoid "SMART" since it's misleading (it refers specifically to ATA).

Signed-off-by: Sage Weil <sage@redhat.com>

Merge pull request #22676 from trociny/wip-24637

librbd: fix crash when opening nonexistent snapshot

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22636 from tchaikov/wip-cmake-add_npm_command

cmake: add "add_npm_command()" command

Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Willem Jan Withagen <wjw@digiware.nl>

Merge PR #22486 into master

* refs/pull/22486/head:
mds: avoid traversing all dirfrags when trying to get wrlocks

Reviewed-by: Zheng Yan <zyan@redhat.com>

mgr, pybind/mgr: pass inbuf (ceph -i <file>) to modules

Modules may wish to receive bulk data; allow it

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #22674 from yuriw/wip-yuriw-perf-distro-master

qa/tests - added all supported distro to the mix, …

librbd: fix crash when opening nonexistent snapshot

Fixes: http://tracker.ceph.com/issues/24637
Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge PR #22578 into master

* refs/pull/22578/head:
os/bluestore: provide a rough estimate for bitmap allocator

Reviewed-by: Adam Kupczyk <akucpzyk@redhat.com>

githubmap: update contributors

Signed-off-by: Sage Weil <sage@redhat.com>

Merge PR #22586 into master

* refs/pull/22586/head:
global/signal_handler: one less frame of context
global/signal_handler: write crash dumps to /var/lib/ceph/crash/$uuid/
common/ceph_context: add "assert" and "abort" asok commands
common/assert: record assert info in g_assert_* globals
common: add crash_dir option
common/assert: get rid of duplicate log dump from assert handler
common/BackTrace: add dump()
log: do not discard recent after dumping it
debian,rpm: /var/lib/ceph/crash

Merge PR #22610 into master

* refs/pull/22610/head:
os/bluestore: fix length overflow.

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

Merge PR #22630 into master

* refs/pull/22630/head:
osd/OSDMap.cc: move sanity checks to the proper place
osd/OSDMap.cc: remove pg_upmap/pg_upmap_items too if osd is gone

Reviewed-by: Sage Weil <sage@redhat.com>

Merge PR #22661 into master

* refs/pull/22661/head:
mgr: remove unused [get|set]_store_json fns

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Dan Mick <dmick@redhat.com>

githubmap: update contributors

Signed-off-by: Sage Weil <sage@redhat.com>

qa/tests - added all supported distro to the mix, so we can track stats across diff OSes

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

osd: update clone_overlap even the clone have been evicted

the clone_overlap is difference of range between head and clones.
if the clone have been evicted, the clone_overlap is still exist in the
snapset, so we should update the clone_overlap to make it sense.

Signed-off-by: Yunchuan Wen <yunchuan.wen@kylin-cloud.com>

Merge pull request #22665 from idryomov/wip-krbd-lvcreate-args

qa: krbd_msgr_segments.t: filter lvcreate output

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #22217 from tianshan/fix_dir_suggest_changes

rgw: fix index update in dir_suggest_changes

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>

Merge pull request #22344 from neurodrone/add_usage_log_entry_dencoder

cls/rgw: ready rgw_usage_log_entry for extraction via ceph-dencoder

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22410 from mkogan1/wip-rgw-t23379

rgw: remove expired entries from the cache

Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #22440 from yehudasa/wip-24117

cls/rgw: don't assert in decode_list_index_key()

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>

Merge pull request #22249 from cbodley/wip-qa-rgw-verify-tasks

qa/rgw: reorganize verify tasks

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22601 from tspmelo/wip-confirmation-modal

mgr/dashboard: Add shared Confirmation Modal

Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22639 from tspmelo/wip-node-version

mgr/dashboard: Update node version

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #22654 from dmick/wip-mgr-store-doc

mgr/mgr_module.py: fix doc for set_store/set_store_json

Reviewed-by: John Spray <john.spray@redhat.com>

Merge pull request #22666 from yuriw/wip-yuriw-crontab

qa/tests - added luminous-p2p suite to the schedule

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

qa/tests - added luminous-p2p suite to the schedule
(Merge after https://github.com/ceph/ceph/pull/22474 is merged)

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

Merge pull request #22536 from yuriw/wip-yuriw-distros-master

qa/tests: changed disto symlink to point to new way using supported OS'es

mgr/mgr_module.py: add 'delete a key' doc comment for set_store

Signed-off-by: Dan Mick <dan.mick@redhat.com>

Merge pull request #22664 from neha-ojha/wip-fix-choose-acting

osd/PG: restrict async_recovery_targets to up osds

Reviewed-by: David Zafman <dzafman@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>

Merge pull request #22432 from tianshan/fix_RGWSyncTraceNode_crash

rgw: fix RGWSyncTraceNode crash in reload

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>

qa: krbd_msgr_segments.t: filter lvcreate output

Some versions of lvm emit a log message

  Using default stripesize 64.00 KiB.

which fails the test.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

osd/PG: restrict async_recovery_targets to up osds

When an osd that is part of the acting set and not the up set, gets chosen
as an async_recovery_target, it gets removed from the acting set. Since this
osd is no longer in the up or acting set, it is classified as a stray in
the next peering cycle. This results in choose_acting() looping between two
proposed acting sets.

To avoid this, we will only choose up osds as async_recovery_targets.

Signed-off-by: Neha Ojha <nojha@redhat.com>

Merge pull request #22054 from theanalyst/wip-civetweb-url-fixes

civetweb: update for url validation fixes

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #22470 from votdev/feature_24436

mgr/dashboard: Replace RGW proxy controller

Reviewed-by: Patrick Nawracay <pnawracay@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

Merge pull request #22658 from joscollin/wip-doc-dashboard-numbers

doc: Fixed the wrong numbers in mgr/dashboard.rst

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>

Merge pull request #22657 from ceph/wip-doc-grafana-formatting

doc/dashboard: fix formatting on Grafana instructions

Reviewed-by: Jos Collin <jcollin@redhat.com>

mgr: remove unused [get|set]_store_json fns

These were little used helpers, now totally unused!

Signed-off-by: John Spray <john.spray@redhat.com>

mgr/dashboard: Replace RGW proxy controller

Fixes: http://tracker.ceph.com/issues/24436
To fully support the role based authentication/authorization system it is necessary to replace the RGW proxy controller by separate controllers for RGW user and bucket.

Signed-off-by: Volker Theile <vtheile@suse.com>

doc: Fixed the wrong numbers in mgr/dashboard.rst

Signed-off-by: Jos Collin <jcollin@redhat.com>

doc/dashboard: fix formatting on Grafana instructions

The whole list was in a code block, instead of just the snippets.

Signed-off-by: John Spray <john.spray@redhat.com>

civetweb: update for url validation fixes

The new civetweb 1.10 version in mimic and later is strict on control characters
being url encoded, making url validation more relaxed and passing these through
to rgw where the requisite url validation is done.

Fixes: http://tracker.ceph.com/issues/24158
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #22608 from dillaman/wip-24410

librbd: namespace create/remove/list support

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #22628 from dmick/wip-cli-path

src/ceph.in: dev mode: add build path to beginning of PATH, not end

Merge PR #22585 into master

* refs/pull/22585/head:
osd: fix getting osd maps on initial osd startup

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #22647 from yuriw/wip-yuriw-teuthology-master

qa/tests - added all supported distro

Merge pull request #22648 from idryomov/wip-krbd-bump-lock-timeout

qa: krbd_exclusive_option.sh: bump lock_timeout to 60 seconds

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc/rbd: update man page

Signed-off-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #20094 from rjfd/wip-msgr2-docs

doc/dev/msgr2.rst: update of the banner and authentication phases