mgr/dashboard: Improve error message handling

The current implementation of the HTTP interceptor does not use the error description in all cases.

Fixes: https://tracker.ceph.com/issues/36190
Signed-off-by: Volker Theile <vtheile@suse.com>

Merge pull request #24350 from ricardoasmarques/wip-json-decode

mgr/dashboard: Check content-type before decode json response

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #24213 from familyuu/mgr

mgr/dashboard: Display logged in user

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Ricardo Marques <rimarques@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #20645 from rishabh-d-dave/fix-doc-add-subcommand-application-manpages

ceph: describe application subcommand in ceph man page

Reviewed-by: Lenz Grimmer <lgrimmer@suse.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24117 from trociny/wip-35915

mgr: create shell OSD performance query class

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>

mgr: module to test osd_perf_query methods

Signed-off-by: Mykola Golub <mgolub@suse.com>

mgr: add osd_perf_query methods to ActivePyModules

Signed-off-by: Mykola Golub <mgolub@suse.com>

mgr: create shell OSD performance query class

Signed-off-by: Mykola Golub <mgolub@suse.com>

Merge pull request #24380 from smithfarm/wip-opensuse-15

qa/distros: add openSUSE Leap 42.3 and 15.0

Reviewed-by: Yuri Weinstein <yweins@redhat.com>

Merge pull request #24399 from alfredodeza/wip-rm36246

ceph-volume util.encryption don't push stderr to terminal

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24394 from alfredodeza/wip-rm24795

ceph-volume lvm.prepare update help to indicate partitions are needed, not devices

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

ceph: describe application subcommand in ceph man page

Add the summary and syntax for all the subcommands under the subcommand
"ceph osd pool application" to ceph the manual page.

Signed-off-by: Rishabh Dave <ridave@redhat.com>

ceph-volume util.encryption don't push stderr to terminal when cryptsetup fails to find device

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume lvm.prepare update help to indicate partitions are needed, not devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #24348 from trociny/wip-config-pool-mirror

 rbd-mirror: use pool level config overrides

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr/dashboard: Delete white space beform 'Sign'

Signed-off-by: guodan <guodan1@lenovo.com>

mgr/dashboard: Delete the old logout component

Signed-off-by: guodan1 <guodan1@lenovo.com>

mgr/dashboard: Display logged in user

Signed-off-by: guodan1 <guodan1@lenovo.com>

Merge PR #24367 into master

* refs/pull/24367/head:
doc: Fix typo error on cephfs/fuse/

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge PR #24362 into master

* refs/pull/24362/head:
doc: fix broken fstab url in cephfs/fuse

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24276 from dillaman/wip-36183

osdc/Objecter: possible race condition with connection reset

Reviewed-by: Greg Farnum gfarnum@redhat.com

Merge pull request #24262 from cernceph/dvanders_creating

osd: add creating to pg_string_state

Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: xie xingguo <xie.xingguo@zte.com.cn>

Merge pull request #24304 from tchaikov/wip-monclient-remove-unused-label

mon/MonClient: use scoped_guard instead of goto

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #24372 from alfredodeza/wip-rm36247

ceph-volume: skip processing devices that don't exist when scanning system disks

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

qa/distros: add openSUSE Leap 42.3 and 15.0

References: https://tracker.ceph.com/issues/35927
Signed-off-by: Nathan Cutler <ncutler@suse.com>

Merge pull request #24180 from colletj/wip-35914-mmgrconfigure-osd-query

mgr: update MMgrConfigure message to include optional OSD perf queries

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

mgr: add optional OSD perf queries to MMgrConfigure message

Fixes: https://tracker.ceph.com/issues/35914
Signed-off-by: Julien Collet <julien.collet@cern.ch>

ceph-volume util.disk when there are no devices mapped, skip to the next one

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.util verify devices that don't exist don't break get_devices

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #24349 from ricardoasmarques/wip-ex-message-py3

mgr/dashboard: Exception.message doesn't exist on Python 3

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

doc: Fix typo error on cephfs/fuse/

Fixes: http://tracker.ceph.com/issues/36180
Signed-off-by: Karun Josy <kjosy@redhat.com>

Merge pull request #24266 from theanalyst/doc/releases/13.2.2

doc: release notes for 13.2.2 mimic

Reviewed-By: Nathan Cutler <nculter@suse.com>

Merge pull request #24354 from idryomov/wip-krbd-kfsx-fancy-striping

qa/suites/krbd: more fsx tests

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

doc: fix broken fstab url in cephfs/fuse

Fixes: http://tracker.ceph.com/issues/36286
Signed-off-by: Jos Collin <jcollin@redhat.com>

Merge pull request #24291 from tchaikov/wip-23964

erasure-code: add clay codes

Reviewed-by: Myna Vajha <mynaramana@gmail.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #24256 from dzafman/wip-36170

osd: Remove old bft= which has been superceded by backfill=

Reviewed-by: Neha Ojha <nojha@redhat.com>

Merge pull request #24147 from sebastian-philipp/orchestrator-debuggability

mgr/orchestrator: Improve debuggability

Reviewed-by: John Spray <john.spray@redhat.com>

Merge PR #24346 into master

* refs/pull/24346/head:
doc: misc fixes cephfs-shell

Reviewed-by: Pavani Rajula <rpavani1998@gmail.com>
Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24340 from tchaikov/wip-36260

qa/workunits/mon/test_mon_config_key.py: bump up the size limit

Merge PR #21566 into master

* refs/pull/21566/head:
test: add test for mds drop cache command
mds: command to trim mds cache and client caps
mds: implement journal flush as asynchronous context execution
mds: cleanup some asok commands

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>

Merge pull request #24355 from alfredodeza/wip-rm36249

ceph-volume:  activate option --auto-detect-objectstore respects --no-systemd

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge PR #24044 into master

* refs/pull/24044/head:
qa: tolerate failed rank while waiting for state

Reviewed-by: Venky Shankar <vshankar@redhat.com>

Merge pull request #23674 from wjwithagen/wjw-wip-ceph-volume-zfs

ceph-volume: zfs, the initial first submit.

Reviewed-by: Alfredo Deza <adeza@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge PR #24275 into master

* refs/pull/24275/head:
qa: add timeout to cleaning up workunit sandbox
qa: add timeout to kclient umount
qa: do not cleanup sandbox on error
qa: use default timeout in fs workunits

Reviewed-by: Nathan Cutler <ncutler@suse.com>

ceph-volume lvm.activate pass no_systemd args when detecting objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.lvm verify activate works with --auto-detect-objectstore

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume: zfs, the inital first submit.

Does nothing other than invoce the ceph-volume-zfs modules when
called with 'ceph-volume zfs

Like:
> ceph-volume zfs list

Use ZFS as the underlying technology for OSDs

--verbose   Increase the verbosity level

['list']
-->  TypeError: main() takes exactly 2 arguments (1 given)
Exit 1

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>

qa/suites/krbd: more fsx tests

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

test/librbd/fsx: don't use time() for seeding

When starting multiple concurrent jobs with rbd_fsx task, all jobs get
the same seed.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>

Merge pull request #24073 from AntonOks/master

doc: added some Civetweb configuration options

Reviewed-by: Casey Bodley <cbodley@redhat.com>

Merge pull request #24343 from jecluis/wip-dashboard-fix-oa

mgr/dashboard: fix reference to oA

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #24237 from votdev/ignore_idea_dir

.gitignore: Ignore .idea directory

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

mgr/dashboard: Check content-type before decode json response

Signed-off-by: Ricardo Marques <rimarques@suse.com>

mgr/dashboard: Exception.message doesn't exist on Python 3

Signed-off-by: Ricardo Marques <rimarques@suse.com>

doc: misc fixes cephfs-shell

Signed-off-by: Jos Collin <jcollin@redhat.com>

qa: add timeout to cleaning up workunit sandbox

If there is a bug preventing rm from completing, the workunit will get stuck.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add timeout to kclient umount

Otherwise QA sits forever waiting for the kclient to umount when there is a
problem.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: do not cleanup sandbox on error

Otherwise the command will hang if the mount is broken.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: use default timeout in fs workunits

Six hours is unnecessarily long.

Fixes: http://tracker.ceph.com/issues/36184
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

mgr/dashboard: fix reference to oA

Signed-off-by: Joao Eduardo Luis <joao@suse.de>

rbd-mirror: use pool level config overrides

Signed-off-by: Mykola Golub <mgolub@suse.com>

librbd: add rbd mirror tunables to pool level config overrides

Signed-off-by: Mykola Golub <mgolub@suse.com>

qa/workunits/mon/test_mon_config_key.py: bump up the size limit

in b38b8e980cb477ab2b0f320ab51eaa0c0fec7da6, we changed the upper
limit of size of `config key` 's value to 64k, so we need to update
the test accordingly.

Fixes: http://tracker.ceph.com/issues/36260
Signed-off-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24302 from hydro-b/fix-rbd-nbd-docu

doc: add "--timeout" option to rbd-nbd

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: songweibin <song.weibin@zte.com.cn>

Merge pull request #24289 from ukernel/wip-msg-clear-payload

msg: clear message middle when clearing encoded message buffer

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24301 from tchaikov/wip-async-connect-timeout

msg/async: keep connection alive only actually sending

Reviewed-by: Ricardo Dias <rdias@suse.com>

Merge pull request #24278 from wjwithagen/wjw-fix-inline_variant

common: fix missing include boost/noncopyable.hpp

Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24263 from tchaikov/wip-drop-lock-policy

auth,common: use ceph::mutex instead of LockMutex

Reviewed-by: Sage Weil <sage@redhat.com>

Merge pull request #24336 from alfredodeza/wip-rm36251

ceph-volume add new ceph-handlers role from ceph-ansible

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

doc: add "--timeout" option to rbd-nbd

Fixes: https://tracker.ceph.com/issues/22333
Signed-off-by: Stefan Kooman <stefan@bit.nl>

ceph-volume tests.functional add new ceph-handlers role from ceph-ansible

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #24330 from alfredodeza/wip-rm36098

ceph-volume:  lsblk can fail to find PARTLABEL, must fallback to blkid

Reviewed-by: Andrew Schoen <aschoen@redhat.com>

Merge pull request #24315 from tchaikov/wip-jenkins-does-not-read-package-desc

install-deps: do not specify unknown options

Reviewed-by: Alfredo Deza <adeza@redhat.com>

test: add test for mds drop cache command

Fixes: http://tracker.ceph.com/issues/23362
Signed-off-by: Venky Shankar <vshankar@redhat.com>

mds: command to trim mds cache and client caps

With this command, the MDS would request clients to release
caps followed by trimming its own cache and a journal flush.
The command accepts a timeout to wait for clients to respond
to session recall and flush messages.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Signed-off-by: Rishabh Dave <ridave@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>

mds: implement journal flush as asynchronous context execution

Signed-off-by: Venky Shankar <vshankar@redhat.com>

mds: cleanup some asok commands

Signed-off-by: Rishabh Dave <ridave@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>

install-deps: do not specify unknown options

per apt.conf(5), and
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=550564,
"Acquire::Translation" is not documented and will be deprecated. it
would be better to use "Acquire::Languages" solely. and it's consistent
with https://github.com/ceph/ceph-build/pull/1137 .

Signed-off-by: Kefu Chai <kchai@redhat.com>

ceph-volume simple.scan better detection for ceph-disk data devices

Uses the new ``CephDiskDevice`` to look into PARTLABEL from both lsblk
(the default) falling back to blkid, which in some cases has the right
value when lsblk fails.

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.util verify CephDiskDevice behavior

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume tests.conftest add blkid patching to device_info fixture

Signed-off-by: Alfredo Deza <adeza@redhat.com>

ceph-volume util.device add a ceph-disk device object to detect types

Signed-off-by: Alfredo Deza <adeza@redhat.com>

Merge pull request #24300 from trociny/wip-36236

test/rbd_mirror: race in WaitingOnLeaderReleaseLeader

Reviewed-by: Jason Dillaman <dillaman@redhat.com>

Merge pull request #23950 from ceph/wip-rgw-dpp-metasync

rgw: log refactoring for meta sync

Reviewed-by: Casey Bodley <cbodley@redhat.com>

mon/MonClient: use scoped_guard instead of goto

also silences the unused label warning. as "out_shutdown" is not used
anymore after f35e10f4849440f0583d021e23a83563dd3b1607 .

Signed-off-by: Kefu Chai <kchai@redhat.com>

auth: switch from Mutex to ceph::mutex

to be aligned with our direction to use std::mutex or its wrapper. see
https://trello.com/c/aUSezBRH/365-cephmutex-etc-for-release-builds

Signed-off-by: Kefu Chai <kchai@redhat.com>

auth,common: switch to ceph::mutex, etc

in this change, along with LockPolicy, src/common/lock_* are completely
removed.

instead of using LockPolicy based template specialization of LockMutex,
etc, it would be simpler if we can just rely on WITH_SEASTAR
preprocessor macro to tell if we are compiling code for crimson or not.
but please bear in mind, we cannot link against the plain libceph-common
in crimson anymore.

Signed-off-by: Kefu Chai <kchai@redhat.com>

common/ceph_mutex: add shared_mutex to `ceph` namespace

for the sake of completeness, also pave the road to ditch
SharedMutex<LockPolicy>. please note, in code used by crimson, we are
not using shared_mutex or SharedMutex yet. so we are not adding
the dummy shared_mutex for WITH_SEASTAR.

Signed-off-by: Kefu Chai <kchai@redhat.com>

msg/async: keep connection alive only actually sending

When connection stuck into odd state, we need to let connection timeout.
If send_message could update last_active, it won't play the role as
expected.
So we move last_active updated to the place actually should.

Signed-off-by: Haomai Wang <haomai@xsky.com>
Signed-off-by: Kefu Chai <kchai@redhat.com>

test/rbd_mirror: race in WaitingOnLeaderReleaseLeader

Fixes: http://tracker.ceph.com/issues/36236
Signed-off-by: Mykola Golub <mgolub@suse.com>

qa/suites/rados/thrash-erasure-code: add clay 4+2

Signed-off-by: Sage Weil <sage@redhat.com>

erasure-code: add clay codes

Introducing Clay codes to ceph. These codes are optimal
in terms of network, disk traffic utilized during repair of a lost chunk.
This feature also provides gradual increase in network, disk traffic with
respect to the number of lost chunks.

Authors: Myna, Elita.
Fixes: http://tracker.ceph.com/issues/19278
Signed-off-by: Myna V <mynaramana@gmail.com>

Merge pull request #24193 from rhcs-dashboard/coverage-disabled-in-api-tests

mgr/dashboard: disable backend tests coverage

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Sebastian Wagner <swagner@suse.com>

Merge pull request #24250 from jecluis/wip-mon-increase-ck-default

mon/config-key: increase max key entry size

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24200 from jecluis/wip-dashboard-fix-ac

mgr/dashboard: no side-effects on failed user creation

Reviewed-by: Ricardo Dias <rdias@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #23351 from tspmelo/wip-rbd-trash

mgr/dashboard: Add support for RBD Trash

Reviewed-by: Sebastian Wagner <swagner@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

Merge pull request #24293 from dillaman/wip-36220

librbd: object copy state machine might dereference a deleted object

Reviewed-by: Mykola Golub <mgolub@suse.com>

Merge pull request #22659 from theanalyst/cmake-pybind-ssl

build: mgr: check for python's ssl version linkage

Reviewed-by: Tim Serong <tserong@suse.com>
Reviewed-by: John Spray <john.spray@redhat.com>
Reviewed-by: Kefu Chai <kchai@redhat.com>

Merge pull request #24243 from smithfarm/wip-bci-rate-limit

tools: backport-create-issue: rate-limit to avoid seeming like a spammer

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>

Merge pull request #23131 from smithfarm/wip-15100

qa: add test that builds example librados programs

Reviewed-by: Brad Hubbard <bhubbard@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Jesse Williamson <jwilliamson@suse.de>